I agree you need to protect the assets.
You have evidence he is a lying self serving manipulator, despite what you thought he was. Put nothing past him, now the mask has slipped and the stranger is revealed.
In addition to trying to save his reputation, he will be trying to come out with as much advantage as possible, including financial.
The joint account contains the ability to pay the bills. I would say taking half out of it to protect it from being emptied would be the pragmatic approach.
Then maybe start a separate account for it, purely for bills.
And see a solicitor. You and your children have rights, and he has responsibilities.
He will probably start shouting he wants 50/50 re the children. This is generally done as a poke to a woman's most vulnerable fears rather than fatherly concern.
If he's going to her, she will not be wanting another woman's children in her home, taking his interest and attention.
If he has to find somewhere for himself, he'll soon find young children are cramping his style. And may not be able to afford somewhere suitable.
But get legal and financial advice. Hopefully Dad can arrange that.
Tell your midwife and the rest of the team. If you understandably don't want him there - tell them. The only person who has the right to be there is you. And they have heard and seen EVERYTHING. Sounds like your brilliant family are stepping up there, too.
Make sure his parents / siblings know exactly what he has done, from your point of view. His take on the tale will be a very different colour and they may fall for it. They will probably support him as they don't want to lose touch with the children.
And because I have a nasty streak, I'd suggest messaging his line manager, explaining the situation, and naming her.
He will be frantically peddling to save his reputation there, too. He will of course be needing LM support in what is going to become a very stressful time for him, so it would be only kind ...