I think you need to break this down into more manageable chunks. You sound a little overwhelmed (unsurprisingly) at the moment. Congratulations on your baby BTW and I hope you are managing to enjoy life with your little one despite the grief being caused by your X.
First of all, did you have to fill out a DASH form when the police attended the incident? The officers would have filled it in themselves, but asked you questions like "are you very frightened?", "what are you most frightened of - e.g. injury?", "is there conflict over child contact", "are you separated or in the process of separating?"
If you filled in one of these, there will be a record of a domestic incident involving you and your X, which will qualify you for legal aid. If you didn't fill in a DASH form, go back to the police (call 101) and ask for the incident number relating to this event. It will come in handy. While you're at it, I'd ask to speak to someone about your concerns - the more documented evidence you have about fears for your safety and that of your child, the better placed you will be to protect yourself.
In your shoes, I would then also contact social services. As you have left your X and contact is being supervised, my guess is that they won't actually do anything other than tell you to carry on doing what you're doing, but again you can ask to have your concerns recorded - it all presents a documented history, which counts for a lot more than if you and your X go to court and only then do allegations of abuse come out (which just makes it appear as if you're making it up out of spite). They may, however, say they have concerns about your X having unsupervised contact, and if they are prepared to back that up in document form, this may carry some weight if your X took you to court for unsupervised contact.
Another thing I would do is refuse to deal with your X other than by text or email. Keep everything in writing so you can use it as evidence if need be. If he becomes in anyway threatening or abusive, tell him to stop. If he continues, you will have your grounds to get the police involved under stalking and harassment laws, despite the fact that you have to have some form of contact with him because of child contact.
Is your X on the birth certificate? If he isn't, then he doesn't have PR. If he doesn't have PR he has no rights to access. He can easily get PR if he goes to court, of course, but until he gets it (assuming he could be bothered with the hassle and expense) he cannot force you to handover your child for contact.