You need to get legal advice if you haven't already. I don't know where you are based but in Scotland if the father is on the birth cert then it is really difficult to remove parental rights, I think you have to prove no contact and no way of contacting the father for over a year (financial contribution counts as contact) or danger to child etc - hard to prove.
What can be done is a minute of agreement for your seperation and you can have a paragraph about how you are the decision making parent who will consult him if need be and whatever visit arrangements you want (eg none til older then supervised). He has to sign this as well and it is binding but still open to challenge later on. Then you would have a better idea of where you stand.
I have an athens password for research but if you don't have access to pubmed and the like you can still google and get some papers to read in full, they may be slightly older though with the newer available as abstracts. I also read up on attachment theory and have done a lot of work on early brain development.
apt.rcpsych.org/content/13/2/79.full
This is one to start you off but I must point out that only you can do what's best for your child and you have to follow your head as well as your heart. You know whether your child is safe or not, you know if you would tolerate this behaviour around your child from anyone else or if this person is getting a free pass simply because he is a relative. When it comes to courts you need a good family solicitor and if that's where you think things are headed then start talking to a couple and get some advice then choose one you can work with and clearly explain your concerns. Courts care about the law not the latest psychological research.
A common element in past research is comparing single parents to nuclear families and then throwing in a bit of attachment disorder for good measure. This was not always done honestly and was presented in a political manner. The most obvious negative is that single parents can be less well off than couples but as single parents start to outnumber nuclear families the guidelines are being redrawn. There are now children raised by either sex single parents, gay couples, split couples, fwb and single parents who used donors. These kids will be just fine as long as the people who do raise them don't mess with their heads and scare them on a regular basis.
Attachment in a nutshell;
(stolen from a cheesy usa site)
Attachment behavior ?Any behavior that an infant uses to seek and maintain contact with and elicit a response from the caregiver. These behaviors include crying, searching, grasping, following, smiling, reaching, and vocalizing.
Insecure attachment ?Develops when a primary caregiver does not consistently respond in ways that are warm, affectionate, and sensitive to a baby's needs.
Secure attachment ?Usually develops when the primary caregiver is sensitive to the infant's behavior and is emotionally and physically available to the child.
Children should not be denied a family member imo but should be protected from instability during extremely vital early developmental years and if that instability is coming from a family member then it needs to be managed. Grandpas, uncles and friends can be male role models too!
Hope that helps, dismounting my high horse now ;)