Domestic abusers win contact with their kids every day in the UK. The bar to stop them having contact is extremely high. The bar to remove kids entirely is even higher.
He dodged it when you met it, but you really should have been savvy enough to read between the lines instead of having kids with him as he is clearly a dangerous man. It’s done now so you have to do everything you can to protect your kids.
He isn’t on the birth certificate so has no rights. It’s time to stop contact entirely. Let him fight for contact because you’ll find out everything during that process. He probably won’t bother though.
He is a dangerous abuser whose child was permanently removed. You don’t need to know the full details; that’s enough. Stop all contact, raise your kids in a safe environment and get on with your life. Any day dreams you’re having about a family life with him are pie in the sky. You don’t want that. Walk away.
Some advice for the future as well; I’m a single mum, and have been since my kids were 6 months old and 2 years old. I started dating again when my kids were 10 and 12. Online dating sites are full of men who don’t see their kids, and everyone one of them will say it wasn’t their fault, they had a crazy ex, the kids are too far away (because the man moved away from them), they might mentioned a volatile relationship. It’s all a massive red flag. All of it. If you meet a man who doesn’t see his kids, do not continue talking to him or dating him. On the balance of probability, not seeing their kids means they are not a good man. You need to be more savvy. This advice won’t be needed for years but please keep it in mind. Do not date another man who doesn’t see his kids.