OP I had to post as I'm concerned you aren't getting accurate information from other posters. It's a long post so bear with me!!
I am a social worker and have been for 12 years and have also worked in the family courts for many many years including having involvement in cases such as yours.
Your ex can apply to have his name on your sons birth certificate and unfortunately if he is proven to be your sons father that will happen but that's a matter of parentage and doesn't automatically mean he has "rights" to your son.
If he applies to the family court for access, the fact he hasn't had contact for over 3 years will without doubt not be in his favour. It is likely that the court will order assessments by CAFCASS and if there are any risks, possibly Children's Services.
OP please don't be concerned about this as they will just want to know why your ex wants contact as if he is genuinely prioritising the needs of your son and if there are risks. Even if he is allowed contact this could start off as supervised and will be done at your sons pace.
This is clearly very emotional for you OP and I also have personal experience of this and what I would say is try and separate the issues you have with your ex with the fact he wants contact with your son (easier said than done I do understand).
What the courts will be looking for from you is that you are not being deliberately obstructive in trying to stop contact because it's painful for you.
Of course there's a chance your ex may be totally unsuitable and a risk to your son but that will come out in assessments.
If this is not the case it is possible that some men who don't always start off as good fathers can change and prioritise their children's needs; I've seen it happen (both professionally and personally).
Whatever the outcome part of your role as your sons mother is explaining what's happened to your son as he gets older but not letting your own issues/fears cloud that or believe me your son won't thank you for it when he's older.
Lastly please don't worry about your ex getting custody of your son, it's not going to happen unless you are proven to be a huge risk to him and your ex turns out to be father of the year!
You will get through this I promise. Also if you need independent advice the Family Rights Group are amazing.
Good luck and I hope everything works out for you and your son 