He shouldn't be being pissy with you for being asked to occasionally support the kids, and he knew you came with kids when he married you. This is a marriage and a family and it shouldn't come down to the numbers of who contributes what, whether financial or non-financial. If he's going to be living with you and the kids then he needs to be treating the unit as a family which means things don't always balance out - if it's constantly him thinking that you are solely responsible for the kids that's a pretty horrible relationship in my view. I would prefer to live separately personally. I could also completely understand it if someone in your husband's position didn't want to take this on himself, but at the end of the day he married you!
However, I am not clear that the contributions here are as equal as you are making them out and actually think he has a point. You not recognizing quite how unbalanced this actually is that might be making this issue worse. Yes you paid more financially into the house, but in doing that you got a higher ownership share. He's building his ownership share back up by paying 100% of the mortgage [hopefully your deal means that it stops when it gets to 50:50 ownership or it's a very bad deal for you!]. You and your two kids are living mortgage/rent free and presumably expect to even when the amount he has paid in mortgage exceeds the amount you paid in deposit. To me that easily balances out the bigger deposit you paid, and plus presumably if he had been buying on his own the big deposit wouldn't have been needed in the first place because it would have been a smaller house.
You're working part time because of your children. He's covering the household bills. I don't know how that balances out, but I would assume that means he's paying at least 50% of bills plus groceries, and there's one of him and three of you. Everything else you talk about having to cover is all costs that are either directly your costs or related to your kids. Who pays for the luxuries in life? Holidays, meals out etc?
If he doesn't consider this a true family unit and therefore doesn't count contributions made to the kids when he's considering whether contributions are balanced or not (and see above that I personally wouldn't be ok with that), how is he not contributing massively more than you are? What non-financial contributions are you making that benefit him? I assume you're picking up more of the domestic stuff and cooking, but how much are you doing for him as opposed to for you and the kids.
In my view your issue isn't that he's unfairly not taking into account your true contributions (including non-financial) but that he doesn't see contributions that you make that relate to the kids as part of the package. The contributions you are making to the relationship are massively unbalanced and objectively this relationship almost certainly benefits you much more than him.
But all that said, it shouldn't be about that. This is part and parcel of being in a relationship with someone who has resident children. Relationships shouldn't be about totting up who contributes what providing that there's a reasonable balance of free time and personal spending money whatsoever. Imagine if you didn't have kids, both worked similar hours but he was in a much higher paying job. It would be the same point - there almost certainly wouldn't be enough of a non-financial contribution you would be making to balance out his financial one (say he hires a cleaner and cook!), but that happens all the time because relationships shouldn't be about a balance sheet of who gets what.