Look at what happens when viruses meet any previously unexposed population, such as Native Americans, or measles in the Yanomami. The virus rips through them, killing large percentages of the population. Read some children's books from the early C20th - as I child I could not understand the fear around colds become pneumonia, measles, mumps, TB, smallpox etc in the children's books I was reading. As I got older I realised it was because they didn't have vaccinations and whilst the advent of antibiotics helped eventually with TB and bacterial (but not viral) pneumonia, the viruses behind measles etc were big killers until vaccines came along. Even when the infections didn't kill, they left people with lifelong disabilities - I know people who survived polio and German Measles in the womb, leaving them deaf, blind, unable to walk etc.
Vaccines are one of our only defences against viruses, and are also important for bacterial infections, particularly given the massive coming disaster of widespread antimicrobial resistance.
A vaccine is giving your body a tiny dose of a virus/bacterium, so that your body learns what it is and how to fight back, without becoming overwhelmed. This has 2 effects: 1. If you are exposed to the virus in the wild (which will be a much bigger dose, which would be hard to fight) then your body is primed to fight back immediately, 2. If enough people in the population are vaccinated then it will be hard for the virus to pass from person to person, and that will reduce (in best cases eliminate) the chance of people being exposed to the virus in the first place. If you'd prefer to take your chances with a full dose of a deadly virus in the wild, then good luck to you!
As you say, encroaching on wild habitats increases the chance of people being exposed to viruses that cross the species barrier, as Covid did, leading to SARS, MERS, Covid, Ebola etc. Whilst we should be working across the world on reducing human impact on the natural world, we should simultaneously be protecting people with vaccines, because there are (1) already so many circulating dangerous viruses and (2) pragmatically, will be more in the future, due to habitat encroachment, and the impact of climate change, as we will be exposed to more and more illnesses that are endemic to hotter areas.