Emmie I am very sorry this is happening to you and your dd.
Please, please do not lose your daughter because of your arse hole ex. I am sure he is very scary but please remember he is a shitty coward. Look at who he is victimizing, a four year old girl and her mum! What an utter shit bag. Please get as angry as you can about HIM and what he has done to your dd and to you. Do not waste any energy or anger on social services who have acted in your dd's best interests.
My son is adopted, he was almost as old as your dd when he came to us. His birth mum could not prioritize his needs over the needs of his birth father. I do not believe that is you, I think you can do it, you can break free. I hope so.
Please follow all the advice on here.
Do not miniimisze what happened, look at the language you use, as others have pointed out. She did not hurt her back, he hurt it, please do not say 'it wasn't broken' as if that somehow makes it seem slightly better, of course it could have been worse, he could have broken her back or even killed her! Who knows what some people are capable of, and even if an adult does not intend to kill a child, children are so very fragile compared to adults!
Be honest with social services, say you really, really want to work this out and be able to live with your dd and care for her.
In the future, if you get her back, move away and start a new life. You will be able to look back on this awful, terrible time and know you came through it. SHE needs you, but not the you who does not know how to handle things, or the you who is trying to appease her ex or whatever.
And believe me I have no idea how awful your ex has been to you both - and I totally understand you are scared of him.
But please find that primal part of you which will fight against him and fight for your dd.
I have no idea how tough all of this has been for you, you are a victim too, he is the arse hole who is keeping you from your dd. Please do all you can.
I love my son and I am so delighted to parent him, but I am sad his birth mum could not prioritize him enough to keep on caring for him as his mum. I believe you will have that strength, but you are going to have to dig deep to do this, please find it in you to ring Women's Aid, speak to Social Services and get your girl back and then keep her safe, because that is really the number one job of a mum. 