I understand that you are in shock, and still processing what has happened. This is why all your posts are mainly about him and the relationship.
You need to start turning your thoughts away from him, and onto the practical and financial stuff.
Here is a list:
As others have said, start documenting all the times he is an unfit parent. You will need this evidence if he tries to claim 50/50 custody to avoid paying maintenance.
Normally, I would say never let him back into the house except to collect his stuff, but in this case it may work to your advantage to insist he only sees the children in your house under your supervision, as this is evidence that you can't trust him to be alone with them. This may be needed if it ever goes to court.
See a solicitor and start the ball rolling with divorce.
Put in a claim for child maintenance via the CMS.
You say you already have full details of his finances - make copies in case he changes passwords etc. to deny you access. You want evidence of his earnings, savings, pensions, and value of assets that you have each bought, e.g. cars, big home improvements.
This is needed in case he tries to cheat you and the children out of a fair financial settlement.
Then look around for some practical help - a babysitter? Play date swaps? You need to find a way to get a bit of time away from your children, so that you can have a good cry or punch a few pillows or just sleep.