By the way I am myself in a difficult relationship with a man who likes things just so and has issues controlling his temper. And I am a chronic people pleaser so it has taken me a long time to stand up for myself. But ultimately I could not take what his outbursts of anger were doing to our daughter. I gave him an ultimatum that I would leave if he didn't seem help. He is now accessing therapy and his behaviour has been much better. As I suspected, he CAN control it -as he does at work, in public etc - he just felt like his home environment was "his space" where he could let loose his frustrations and issues. It isn't. It's OURS. It's my daughter's home, it's her safe space, and he knows I will 100% prioritise her wellbeing over his. For me I had been putting up with it for years, I had no clue how much I had compromised myself to accommodate him until she was born and suddenly what had been good enough for me was categorically not good enough for her. Try to stop seeing yourself as a victim - you are an adult with agency; your kids are absolutely helpless and you are the ONLY person who can blow up this awful dynamic. Currently it is the worst of all worlds - their environment is unsafe all the time because it is dictated by their neglectful, abusive father; their mother is walking on eggshells and becoming increasingly subservient and mentally ill; they have no sanctuary.
Worst case, you leave him and he gets some time alone with them. Horrifying thought, but at the moment he gets time alone with them anyway, you are "not allowed" to correct any of the things in their environment that endanger them, you are not or you cannot protect them. They are not safe just because you are there. He won't let you make them safe.
At least if you leave they will be for at least some of the time (probably most of the time) with you, in a safe, stable, non-abusive environment where they have a sanctuary and can heal. When they are with their dad it is out of your hands, but every. Single. Time they report anything less than safe to you you can compile it as evidence, report to SS, and make a case for reduced/supervised contact.
It's really all you can do. Would you rather be abused all the time with ineffective mitigation from your mother (as someone else characterised it, "dad hurts them and mum lets him") or be exposed to your abused only some of the time with a safe home to go back to and get help from a mother who is much more effective because she had her own home and is emotionally and financially separate from the abuser and can thus fight and advocate for you without fear for herself?
Give your children the best YOU can. You can't control him, you can't make up for him. You can only protect them from Jim as much as you can, and you'll do that much better away from him.