I also have four young children and DH doesn't drive. Well, he has a motorbike, but that isn't much use for family life.
To be honest, I really, really resent it. I feel like I can't go away or let my hair down because what if there is an emergency? He can hardly pack four kids on a bike or even take a normal cab, as there are too many of us. We live abroad and I often take kids to visit grandparents alone. As we can't afford to leave car at the airport, I am constantly relying on people to drive my car and then come pick us up again. Even for his job, if he needs to go away from the office, he has to wait until one of the drivers are available.
And of course, if we ever get a night out, I have to drive. And sometimes, I would just really like to have a drink. Esp to his work functions which are 20 miles away and I don't know anyone.
All four kids were c-sections and I had to rely on others to bring us home, and run DS to his daily therapy sessions. I also broke my ankle whilst pregnant with #4. That was a nightmare and we lost friends over having to be so dependent.
Silly little things like just after DD was born (she was 10 weeks old), I had to drive 1000 miles to attend his brothers wedding. A big thunderstorm wrecked all of our garden toys, and as I was heavily pregnant and could not lift anything, plus I couldn't get it in the car plus all the kids, I had to ring around and beg someone else to collect our stuff and take it to the tip.
When DS was born 17 months ago, we kept our five seater car for him to learn to drive, in addition to the 7 seater. We couldn't afford it but people offered to take him out driving and so it would be cheaper than lessons. I just sold it last week, as he never even sat behind the wheel.
I think if you have a small family, and your partner is ok with it, then I guess it is fine. You can move your family around easily in an emergency. But with a large family, it really is a must. I hate having to ask people to put themselves out for something very simple.
God knows what it will be like when they are all teenagers!