Well, I don't drive. DH doesn't drive either. We both can drive and have valid driving licences.
Part of the reason we don't drive is because we don't own a car. But, if we wanted to, we could buy a car, we've just chosen not to.
We haven't bought a car, because it is cheaper not to, and we can manage fine without one.
Lots of people don't drive because they don't want to. Sometimes they don't even go through the initial stage of learning to drive. Because, you know, for whatever reason, they don't want/need to be a car driver.
It doesn't make them some sort of lesser person. It is fine not to drive, and not to own a car. It can actually be a choice. Driving is not a necessity. "Can't drive" implies you would if you could. But plenty of people just don't, and it is irrelevant whether they can or not, because it doesn't restrict their life.
I know a few people who don't drive and have never learned/passed their test. They are, in fact, the people who I can always rely on not to be late, not to get lost, never to let you down. They aren't restricted at all, always manage to get anywhere they want to, don't rely on others for lifts, are totally independent.