DS, I agree that he needs to have sole care of them. You need to have a day out, and leave them with him so that they HAVE to muddle along together and he has to find ways of dealing with them.
I find it really hard to respect men who don't get involved with their own kids at all. Well, I don't respect them.
My dh:
is home at about 5.30pm and when he comes in despite a long cycle ride home he just has a glass of water then gets on and plays with ds. After dinner we get ds ready for bed and bath and we do this between us but DH is ALWAYS involved fully. DH reads ds his bedtime stories most nights. We go up in turns 'checking' on ds while he drops off.
DH and I take it in turns to be with ds at things like doctor's appointments and we time our holidays so that we both have time off to look after ds in the school holidays.
DH takes ds to children's parties if I can't or don't fancy it.
He takes him to the park at the weekend and swimming every weekend, taking responsibility for organising ds' swimming lessons and takes him for a lunch out afterwards.
That's what i expect from him as a father to be honest. We both decided to have a child and we both do the work. I know some men don't have every evening available to be at home but there is really no excuse for doing as little as your DH. I wouldn't accept that not only because it shows little respect for you but also because that doesn't allow for much of a relationship between your dh and the kids, does it?