Get yourself organized, finances in order, etc and leave - or better yet, get him to leave!
Harsh as it is to face, everyone is talking sense on here. He is gaslighting and has no respect for you at all, while being massively inappropriate as a boss, it’s quite predatory given the power balance/her assumed vulnerability.
Don’t let him minimize this.
You say he’s done this before? Write down a timeline - or start one - of these events. How does it look? What would you advise a friend in that situation?
I have been there, it took me ages to end it, but when I did it wasn’t nearly as awful as I imagined - and the relief I felt!
Enlist good family/friends if you can, makes all the difference and will help to curb any future CF behavior about the split if he knows other people are aware. He’s thriving on you doing nothing/no one else knowing. Do people generally think he’s a great guy/husband?
Kids will be far better off in the long run.
So sorry OP. It’s an awful situation to be in. But it’s not your fault and you will be so much better without him.