So he's out of the house by 9.30,doesn't return til 7pm and you still cannot stand having that small amount of time with him ? It sounds very much that the problem is you're angry being with him because you no longer love him and his presence is irritating.
Trying to banish a man from his own home because you find him irritating is completely unreasonable and he must feel extremely hurt. You've basically blamed your mental health problems on his presence, you're saying to him if you don't get out of here I'll stay unhinged.
He doesn't get back from work til 7pm,you're home all day,so of course childcare is your responsibility and as your child is in nursery such a lot, it seems he has the harder job.
Your husband has been accommodating your moods because he clearly loves you and you are obviously suffering post natal depression, you do realise that don't you?
As for his inability to be on time it's clearly not out of laziness,he works til 7 and helps with childcare on his weekends. Folk are mentioning adhd, it is very likely he may have some form of ocd ,which means certain rituals are taking a great deal of time before he can exit the house, ive witnessed this happen firsthand. People with ocd can be very good at disguising their behaviours and coping with integrating them into routines, way such a way that it can be a long time before others can see what's actually going on. Perhaps missing flights, being late for dinners (so that they may be cancelled) and you're not next to germy folk ,maybe a factor,especially post covid
.
It seems to me that you are both having issues and rather than make rash decisions, you need to sit and talk,but you need to approach from a point of finding out why he is like he is,rather than demanding action,because otherwise he won't open up to you.