Ouch. I don't care if it's a registry (yes, that's what we call it), registration, registrar's or whatever.
OP, you're up shit creek without a paddle at the moment. You could be sent on your way with your children and happy memories, and he keeps the rest. You'll get a bit of child maintenance but I'd bet he'd find a way to make it a fiver a week each.
Do NOT book a wedding and tell him (or trick him) to turn up. There's every chance he's done this deliberately to keep any financial assets to himself.
So, now you're awake and you realise your mistakes, you need to talk to him. Once. Is there someone else who can look after the children while you do that?
Tell him you know how vulnerable your position is and you want it putting right. A quick, no-nonsense wedding, almost immediately. Your name on the deeds of the house, see a solicitor together in the coming week. Your contribution to the family recognised. Or.
Or you find out how to accommodate your children and yourself, and how to support them, and crack on with it.
If he does comply with the marriage and finances... still, get yourself a job and never, ever get into this position again.
I wish you luck.