I'm always a bit reluctant to comment on threads like this because I don't have kids, so obviously haven't lived through the experience, so my apologies in advance if I'm off the mark with anything.
A few things struck me though here. Firstly is there a chance that your husband depressed? Or is he just experiencing parental regret? It's a taboo subject isn't it, people who have kids are never meant to admit to possibly regretting their choice, but if it is the case, you absolutely need to know.
It sounds like an awful situation for you to be in OP, having to struggle on like this in what I imagine is feeling like a very unloving and emotionally and physically unsupportive relationship - you're doing all the home labour with 2 tiny ones, all on your own whilst your husband is literally only there in body. It would be less lonely and less stressful and less emotionally taxing if you were going it alone, I'd imagine. If your husband is depressed, that's maybe something that can be addressed, but would take wilingnesss from his side to acknowledge and then make steps to tackle.
Secondly, this will start affecting the children. Daddy is about when he's at home, but not really. He's not really that interested in them, is he? Are they going to grow up feeling like an annoyance that he'd rather weren't there?
From what I can gather, you've tried to speak with him about this and you get nowhere. It sounds very much like he doesn't want to have to bother with family life, I can only imagine how lonely a place you must be in right now, for yourself and the 2 little ones, and if it doesn't improve this will be felt ever more acutely by the little ones as they get older.
I agree with previous posters who suggested going to a supportive family members home with the kids. Or even a friend, if the situation would allow. And take as long as you can to have a bit of space away from him. Not to see if he suddenly wakes up and realises he needs to sort himself out otherwise he's going to lose his family, but just for yourself. I think you need time away from him to think about what would be best for you and the kids. There's no guarantee your husband will make sustained changes, and at the moment he's not even acknowledged that he really needs to step up. It sounds like he's broken your trust - you can't rely on him to be a parent in the very real sense of the word, and there's something quite frightening and destructive about that.
Thirdly, there's 4 of you. The needs of 1 person do not trump the needs of 3 others, especially when 2 of those others are very little people relying on both parents for all that parenting entails, and the 3rd person is you, the woman he married, and who he is meant to love and cherish. You must be feeling absolutely knackered right now, physically from lack of sleep, and emotionally... just utterly drained.
I think you need time away from him to explore your options. What happens when you go back to work and you stay with him, what does that picture look like? What does the picture look like if you leave?
Wishing you and your family well, OP.