I'd say he's depriving them of more financial security, a nicer house, better holidays... even down to smaller things like some treats with the weekly shop.
The bugger doesn't seem to give two shits about his family.
This, right here, shows the double standards already mentioned.
If he were a woman, and OP a bloke, there us no way he would have been described as not giving a shit about his family.
And on re reading OP's posts, I don't see anything to justify describing him this way.
We simply don't have enough information here. Is he earning a lot less than OP? Or just a bit less?
Either way, OP herself said the money is not the issue, just that it would be nice. And having a parent being there for you is worth way more than extra treats in a weekly shop.
OP, you met him and started a family in the space of 5 years, a bit of a fast paced relationship.
I suspect you now have time to evaluate what you have, and it isn't what you thought it was. If you think you want something different then leave sooner rather than later, your DC will settle into a new life OK, and it'll be fair to your DP to find someone more compatible.