I have been through this situation OP, almost exactly. My DH did not want me to terminate number 4, that was never on the cards, but we both panicked about how we would cope.
When DC4 was born we had a 6, 5, 22month old. Everyone thought we were mental and took great delight in telling me how hard it was going to be.
I had school runs for the bigger two twice a day and yes mornings were hectic, but honestly no more hectic than before she arrived oddly, mornings with three are hectic enough aren't they :) Honestly with things like running the home, being out the house on time, spending time with the DC, DC4 made no difference (maybe because I got a sling and carried her every where, often latched on, and got on with things that way?)
The real difference with 4 DC is things like the car, if you don't already have a 7 seater.. get one. We needed a double buggy so had to sell single and push one child in a double solo until she arrived (a few odd looks on that one!) We never had expensive holidays, so that wasn't an issue for us, but could be a thing to think about. Get used to camping :D
When we tested for DC 4 with such young children already I was crapping myself about how we would cope with her but honestly day to day stuff is no harder than it was before.
You need to give your DH a swift kick up the arse to do more though, I find lists work well :D I have a permanent list written on the fridge that I update every day of things DH needs to do, he wont think for himself but will do anything asked, it works for us.
Hope it all works out for you :)