OP, you are in a really difficult position.
SS won't generally get involved in matters around custody of children because it's a civil matter. They won't tell you what to do either. They have to be really careful about how involved they get. And often I think they can't intervene preventatively unless there's a very high threshold of risk met.
So you can feel like you are on your own and powerless. But you aren't.
What you can do very much depends onwhat the level of risk is and what level of proof there is. You probably can't share that here. If it's such that no reasonable parent would introduce a child to that parent, you can expect your ex not to do that. If they continue to expose your children to unacceptable risk, you can take legal advice and ultimately stop your children from spending time with them unsupervised. It feels awful and wrong to protect your children from their other parent, and it feels like social services would be doing it if it was necessary, but that's not how things work (and that's not to denigrate social services, just that it doesn't work quite how you assume it does).
If your ex's partner has some kind of history of child abuse, they are likely to be predatory and manipulative. Your ex will be subject to this and may not be able to make good decisions. The fact that they are just ignoring yourvalid concerns doesn't reflect well on their thinking.
If there is some kind of mistaken identity they will hopefully be able to prove it's a different person. If there is some kind of error or mistake which means an innocent person has been branded a risk to children, that's awful. But I would suggest getting some support to deal with this - there may be charities which can help. You are not wrong to try to ensure your children aren't at risk of abuse, and putting in some temporary protections while you find out more is sensible and loving.