You're 18. You feel grown up, but there's a lot which is different in the big world, and you are on the cusp of having to grow up very fast.
There are somethings which social services find unforgivable. I know of a couple who broke the skull of one of their children. All of their kids were removed. They kept having more children who were all taken away at birth. No second chances or attempts to change, and rightly so. Allegations of sexual abuse of children is a similar situation.
You're 18, how old is he? How long have you known each other? For most people an older man and a 16/17/18 year old is suspicious in itself.
He may still be on his best behaviour around you.
You say you have information about his criminal record and have seen the court documents. Fair enough.
Where did you hear the story about his ex from? Just from him? Its very possible that if you went to his ex you'd get a very different story.
Go to the meeting with your mum if you have a good relationship with her and open mind.
Ask them nicely what they're worried about and listen.
Social services don't gain anything from taking your child away.
Then you probably have one of the biggest decisions of your life to make.
Do you put this child first and do everything in your power to parent this child safely at a large sacrifice to yourself.
Moving away is scary, but potentially a chance to make new friends and gain life skills and get support.
Something has happened with your boyfriend that has social services very worried for your child's safety.
Their agenda is clear, protect your child (even at your expense, they will put your child's safety over your emotional well being and rightly so).
His agenda is not clear. His relationship with you, his status in the community, his own gain. He has far more reasons to lie than they do.
I'm sorry for what your going through, I'm sorry that life is likely about to make you grow up too fast.
You need to give up Happily Ever After with your boyfriend and your child, and learning that happily ever after doesn't exist is something most adults have ongoing struggles with.
You need to make a decision and own it, knowing that what ever choice you make, it will hurt (a lot).