If you are raising him you ARE his dad. But he has a biological father too. His bio dad was not able to raise him because he was not able to put the son's needs above his own.
Our son. was adopted at 3 and his biological parents were not able to put his needs above their own.
Although he knew when he came to us, that he had had another family (actually two because he was fostered) it would be easy for him to forget his past as he has been with us almost two and a half years -almost half his life.
It is important to me he always knows his story, in age appropriate details, no surprises, even the bad stuff.
His life cannot be built on a lie because if so, when it is found out, it could undermine all he has learnt about the world.
Would you like to adopt your son legally? Either way I suggest you talk to your partner. Find some useful info on line about the importance of no secrets and no big reveal. Make sure she knows her ex is no threat to you, just part of your son's make up.
I strongly suggest you get a session of appropriate counselling in how to tell, then tell your boy, with his mum, followed at a suitable time by telling his brother. Even if a baby just make sure it is all normal family life.
A key thing is you chose your son, when you chose his mum, or chose to be his parent when you met his mum and became a family. But you need the right language for this and to keep it simple age appropriate.
I think it would be helpful for your son to see pictures of his bio dad too and for you to ensure there is no animosity/jealousy from you in this. This man is part of your partners past but part of your sons generic make up.
IMHO it is VERY important neither you nor his mum bad mouth his bio dad! Because his bio dad is part of his identity.
It is sad bio dad could not cope but it is reality, and another part of reality is you are dad too, dad who is there, dad who does things for him, even if you are not genetically related to him.
Just as I am not genetically related to my son but I am still mum, my Dh is dad, our birth dd is sister and our son is our son.
Having said all this of course if you split up from his mum, ever, you remain dad, so adopted or not I do hope this is a life time commitment, as all parenting is.