Walking out on you doesn't sound good. But what you say about him feeling helpless to make your life easier also sounds like you have been able to talk about things a bit and that he would like to be supportive.
It's a really difficult stage of family life - 2yo & 3yo, plus you have moving on top of that. To an extent, you can't get rid of the fact that kids that age are really hard, relentless work. It DOES pass, but you will have to get through this winter. Things go in phases - you might have a couple of months until things hit a better patch - or it could take a year or two. That's going to feel a long time to get through, but in terms of a lifetime or a childhood it's actually shortish and it will eventually fade in to memory.
Actually one thing I learnt from DS1's school - but which might work earlier - is to very explicitly talk about values and behaviour expectations with the children. They might be keener to 'do the right thing' if there is an explicit, simple set of rules. And even if not, I've found that having a simple list of things that I can point to that are being done well/badly helps me to both give good praise and also to keep a bit more calm & rational when things are difficult. DS2's nursery has a set of 'golden rules' - things like 'we share our toys', 'we use indoor voices', 'we listen when the ladies ask us to help'. DS1's school has a more values driven set - eg Our Values Are Love, Endeavour, Respect...etc I have adopted the school list and stuck it on the wall at home (plus added one for my own purposes). I point out the list frequently and try and pick up when I can see them demonstrating this stuff successfully, maybe give them a sticker on tummy in reward etc. Sounds rather bureaucratic written down, but I'd observe that kids (and me) like the structure, repetition & clarity of this.
Lots of people say to try and have regular 'date night' or similar with DH. We do this now that kids are a bit bigger. At 2 & 3 yo it was unrealistic for us - babysitting v expensive and DS2 was an appalling sleeper so difficult to leave with someone else and I was often too tired to enjoy going out. But we used to try and do an at-home 'date' evening (glass of wine & DVD, no housework/evening jobs) every few days. Maybe generating a bank of 'nice evenings' with DH would free up a bit more emotional capacity for moans/rants on other evenings?
Sorry this so long. Hope it doesn't sound like I have all the answers - I definitely don't - but just know that this was such a hard stage and now that we are starting to move past it I really feel for people who are still there.