I'm so sorry you are going through this. I could have written your post, almost word for word, albeit many years ago. I felt the same as you, my worries and concerns were the same, my exH's behaviour, dare I say it, was almost a carbon copy of yours, with a few despicable twists thrown in here and there. I couldn't see the wood from the trees at the time (my DC were both under 3). My exH was so arrogant, he actually thought it was ok to write to my solicitor, telling him he wasn't performing his job 'legally' with respect to our children, with regard to [insert which ever Children's Act he chose to quote at the time]. He was an intelligent professional, but Law was not his subject. My solicitor was the owner/partner in the practice and said he had never encountered anyone behaving in such a bizarre manner before, as it was clear he had a) no knowledge of such legalities and b) was actually discrediting himself by contacting the 'other side's' legal representative. He kept sacking his own solicitors (I suspect because they were telling him things he simply wouldn't accept - he despised being told what to do), however mine was an absolute rock and I know he dropped some charges for me at times because of the sheer amount of expense my exH was creating.
I'm out the other side of the horror of this type of family break up, and I also remarried 10 years down the line. Sadly, it didn't work out for very different reasons, however we have remained on friendly terms for years, and proves not everyone behaves like a maniac when relationships break down. My DC are now early 20's and have been high achievers at Uni. They are both now living with their DP's independently in their own homes, and are extremely emotionally well balanced individuals.
I almost moved to the other side of the country when we divorced, as I just wanted to run away from him and any association to him and the life we once had, but I forced myself to think of my DC and their lives, and they still deserved to have access (and an untarnished image of their dad) where I could manage to deal with it. So I stayed locally, and managed to co-parent with many, many difficult incidents being dealt with by my solicitor even post-divorce. I bit my lip and the DC knew almost nothing of their dad's behaviour, however as adults, I eventually explained what had gone on, and although saddened by this, they have processed it and I have always managed to maintain to the kids that whatever he did to me, was not a reflection of his love for them. It killed me to 'support' him in this manner, but I did it for them. He always supported the DC financially; it was just me that was the target of his financial and emotional spite, all, looking back, to justify his own affair and behaviour. Yes, he cheated on the OW, and it later transpired that he was dabbling with drugs, too, which may have accounted for the mood swings I had to deal with. He without doubt had a mid life crisis, and quite a severe one at that.
In a nutshell, I am now semi retired, financially secure, I have a lovely home, and potter about with my dogs and cat, stress free, answerable to nobody. The freedom and peace of mind that gives me, is something I could simply never have envisioned during my grief. To think that one person, could have caused me all that utter distress seems unfathomable now. I was perfectly content BEFORE I met him, and so it reasoned that I could be perfectly content AFTER he had gone - it sometimes takes many years to realise this, but one day you will. Wishing you and your children all the best, OP.