The way I see it is do you want to look back in 10years and say I was tired and lonely for most of my 20/30s
No wonder you are tired I have a 2 yr old and work 3 days a week and am zonked and my partner is very supportive.
I do think it is important for people to be individuals in a relationship and have outside interests but not to the detriment to the family unit.
Your husband's work sounds like a deal breaker to me, not having a firm time they would be home that they could meet 95% of the time would just be a huge no.
Case in point my partner is on a half day wfh today and was meant to finish 15mins ago and my patience is now thin as its my time he is taking from.
I seriously would put my little one in nursery full time with his wages and use the time to build up yourself for a while. Or take some time off work whilst your children are in school/nursery
At the end of the day if you split up what would change, chances are you would do everything you do now with less disposable income.
I would treat it as he is obviously not concerned about my happiness that I would put that to the front of my interests. Organise babysitters instead of relying on him, take him out of the equation. Treat yourself as a single parent and madate that one day per weekend is family time, he puts himself first you should do the same (even if just means self care time)