What did you learn about relationships when you were growing up?. Did you see similar yourself at home between your mum and dad?.
What do you get out of this relationship now?.
Why is he not working?.
I think your children wonder of you why you are still with their abuser of a dad. And no, your kids won't be devastated (why use that word too?) when you leave him, more like relieved for their own selves. You are showing them also that currently at least, his treatment of you and in turn them, is still acceptable to you.
Your relationship with them is perhaps good for now but going forward it may not be if you choose to remain with him. They could well ask you why you put him before them? You are still choosing this life for your own self and subjecting them to this from their dad at the same time, why?. Would you want your children to be in a relationship like this, no you would not. And its not good enough for you either.
Asserting yourself in front of an abuser will not work and is a dangerous tactic. He knows what you want and does not care about you at all, all he cares about is having you around to abuse and getting his own needs to abuse you met. He is angry because he is abusive, not because he is angry. He does this because he can. He won't make any aspect of you leaving him at all easy because he is abusive and likes having this level of power and control over you. He has trained you and put a lot of work into getting you to become this totally subserviant and compliant to him.
If you have not already done so I would urge you to contact Womens Aid and the domestic violence unit at your local Police station. Coercive control is now a crime and the police do take this far more seriously.
This is who he is; an abuser so how can you be helped into leaving your abuser?. He won't change and what he does works for him. He has you precisely where he wants you. He targeted you and also in all likelihood started to ramp up the power and control against you soon after you met. Marriage and childbirth were two further flashpoints to further ramp up the control against you and now your children. It was indeed a bad day for you when your paths crossed.