"It wasn't broken" doesn't make everything OK. It wasn't broken THIS time. If there were ever to be a NEXT time then things would probably be far, far, far worse. NEXT time he might permanently injure or even kill you and/or your DD. Social Services cannot take that risk, so they have acted.
Sorry, but with every post you make it is becoming very much clearer why they have done this. They haven't done it for fun, they have done it to protect your DD because you are not taking all of the actions available to you to do it yourself.
This man is a serious danger to both you and your child. Social Services have not got that wrong, they have got it right.
Get the non-molestation order. Call the police. Call them again and again every single time he appears. Call Women's Aid. Look seriously into moving to a new address that you will keep permanently secret from him and any mutual acquaintances you have. That may be a place in a refuge, or you may qualify for emergency housing from your local authority, perhaps with the backing of your social worker etc. Not sure what the system is there, but ask about it. Then ACT!!!
I think he remains a danger to you even though your DD is no longer in the house. I would also bet that he is a danger to others around him too, especially women and children.