Bagpuss, you must be hurting like hell. And angry. So angry. My heart really goes out to you. gather as much RL support around you as possible. And yes, tell people what's happened as other people's shock and disgust will help give you resolve and feel protected.
There is absolutely no shame on you - though I know that you will feel it. The shame is all his.
BUT more importantly, before heading off anywhere, I would advise you to gather some strength and see to some major practical issues first. You need to speak to a solicitor and find out where you stand re money. Your husband has proved himself to be a master manipulator and consummate liar. Now that he's been found out, he is not suddenly going to change, even if he deigns to shed a few crocodile tears. In fact he may turn nasty - he has already tried to blame you for his behaviour after all.
Where has he been keeping the money he has been using to pay these women all these years? Does he have a secret/separate sole account or credit card that you do not know about?
I'd be interested to know what a solicitor says about how a divorce judge would take into account the thousands of pounds of family money he must have squandered over the years.
Set up a sole account in your name, if you do not already have one, and
if you have any joint savings that you have access to without needing his signature then I would certainly transfer them into your sole account and fast. This is not illegal. It is joint money and you can transfer it anywhere you like, as can he.
Is the house in joint names? How much debt do you have?
Get together as many past bank/savings/investments statements as you can and keep them somewhere safe. Copies of his payslips.
Make a note now, of everything that he has told you so far. Write it all down, even the grisly bits, because you may not remember it all later and it may come in useful if he tries to change his story or as evidence.
Keep any texts or emails he sends you.
TBH, i would not consider going anywhere right now. I would stay in your house and get your Husband to leave. Being in France won't actually solve anything and might give him the opportunity to organise affairs to his advantage.
Ask him to leave the family home at once. Do let him take a key with him. If he wishes to see the children he can arrange a time and place with you or through a third party if you are not up to communicating with him.
Will think of other things. Big big hugs to you.