Neither myself nor my dh drive + we have two school age dc's. My dh did pass his test though (back in the 80's at the age of 20!), but has never wanted to drive since.
We both use public transport + taxis, + have deliberately picked a very central area to live in.
It doesn't hamper our lifestyle at all, + we don't rely on other people. We combine buses, taxis, trains + cycling which works for us.
I definitely don't find my dh unattractive because of not driving, + if anybody questions him on it he replies that he earns more than enough to pay for transport any time we need it. To be honest I've never been stranded anywhere in bad weather with dc's or late at night, unable to afford a taxi!
If he's offered lifts he always turns them down, as I do because we've always made our own arrangements!
I've heard it said that people see male non-drivers as "wimpish". That certainly doesn't apply to my dh, he's head of his department at work + if his staff need back up on anything, he'll go straight to the top + sort it out for them. He hasn't lost an argument with Senior Management yet, to be honest he's not a person that many would argue with!!