I don't think much about it.
I grew up on a farm and my parents paid for driving lessons from when I was 17. However, I proved not to take to it naturally, and didn't pass till I had left home, and then couldn't afford to run a car, so apart from occasions when I was back home and my parents insisted I drive to keep my hand in, I barely drove till I was in my late 20s and could finally afford a car - and when I did, I took a refresher lesson and a motorway lesson, which really helped my confidence. But I still cycled and walked a lot (I don't cycle now - I've lost my nerves on the road.) I still use the train if it's an option for where I'm going. But I do appreciate having the choice.
I have a cousin who didn't learn to drive till their late 40s when they moved out of London. I've got a friend who just didn't replace his car when his last one failed. He lives and works in a city centre and walks a lot, but he has also hired a car to go away for a weekend, to a wedding or something. I've a friend who gave up learning to drive through nerves, so she's always lived in cities - but she's an academic, and universities are often in cities with good public transport. She's really great with finding obscure bus routes and does a lot of hiking on places you'd think might need a car. Another friend who doesn't drive could just never afford to learn. She too lives in a city centre.
I don't judge them for not driving - they manage their lives accordingly, and it's just a fact about them, like being vegetarian or liking ballet or something. We usually meet somewhere on a railway line - or if I go to visit, I usually insist we go out somewhere local, but that they can't usually get to, because they might as well make the most of having access to a car and driver for once.
I'm usually happy to give people lifts, because I don’t often have to do it. Where I would get judgemental would be with someone who takes the piss and expects it - and then I'm judging them for being a CF, not for not driving.
I assume most of my other friends can drive, but in some cases, I have no idea. Colleagues who walk to work, I know for some it's a choice because they live close - but I don't know if it always is - they might not all have access to a car, whether or not they ever learnt. I do assume most people in the office drive, but I don't outside of work, at evening and exercise classes, unless I have seen them drive, or they have mentioned it.