I suspect the OP is not talking about people who can't drive for any reason or people who don't need to drive, but people who won't learn simply because they don't want to and who cadge lifts off other people.
A friend's ex was a "CBA to learn to drive" type. He was a school caretaker, and one year he was called out to go back to the school late at night to switch the Christmas tree lights off. Being rural there was no public transport and taxis were as rare as hen's teeth, so my friend had to get her little girl out of bed to drive him to the school to switch the lights off.
My DH, who was a good driver, hated driving so much that he once walked an 8 mile round trip to the dentist rather than drive. It got to the point where he basically forgot how to drive and nearly knocked a pedestrian over on a zebra crossing. I have sinced changed the car and he has never driven my car. By now he wouldn't be a safe driver, yet I still resent him for just giving up and me being the only driver in the household.
We live in an area where public transport isn't that frequent and is often unreliable so we do need a car. Fortunately, there are more taxis than there used to be, but it is nothing like living in a city.
I do wish that the smug "I don't drive, I just use public transport" posters could be a little more understanding of people who live in more rural areas and who don't have the luxury of frequent public transport or readily available taxis.