TBH, from a selfish perspective, it's better FOR YOU if you don't have contact with him, or if you must, then totally on your terms. Like me, if he starts up, I put the phone down. He won't call me, it costs him a fortune to do so, and he will text for me to call him back.
Usually he asks to speak to DS, so I call the number and hand the phone to DS. DS hands me the phone after X asks to speak to me and I click the END button. Job done!
My X never really directed any abuse at DS, only really at me. Had that been different, it'd make my decision much easier.
How old are your DC again? ARE they unsettled by seeing him? Are YOU unsettled having contact with him, even for their sake? If the answer to either of those questions is yes, then you need to - at least - take a break for a while, to gather your own strength, for your DC to do the same and for you all to heal.
You can not move on if you are constantly in contact with men like this. I have been booting poor old JK up the behind recently for this very same reason, she is trapped in her life, sad and lonely, and this is because she has maintained contact, hanging on to the fact that X is a decent bloke and needs to see her DS.
He ISN'T a decent bloke, and actually DS and HER need some decent time to get away from him, out of his clutches and to a space that is all theirs, before they let that poor excuse for a pair of balls anywhere near them.
If your XH - even if only for a second - vented the fury and foul abuse on them that he heaped on you, then GAME OVER tbh, he ought to have thought of that. Stop contact immediately and tell EVERYONE why.