Well and truly, his father was a horrible man.
Hope he is in a better place now though 
I'm not sure how it could be affecting him, he was born with no father in the picture then after a few months my DP was there. It was explained to him that DP would be moving in etc, we done plenty of meetings before he moved in. DS has called him dad since he could talk but some days uses his name so he is aware that he isn't his actual dad but has never asked or enquired about his proper dad, none of the family on that side care either. I'm not even sure they know he is born but they are quite... selfish people and don't take kindly to babies born outside of marriage so it's unlikely that even if he were here, they'd care about him.
Thankfully DP has a huge family on his side who absolutely adore him, FIL loves him always spending money on him and he usually stays over at his house once a month. BIL takes him out when he can and again takes him to messy play, has taken him swimming. He is completely adored which I'm grateful for as my family are non existent.
Despite what I have said on here I have no detachment issues with DS. I have distanced myself from everybody before the ADs (I was being selfish I know) but I genuinely love spending time with him, exploring a new jigsaw, reading him a bed time story, helping him with numbers. He is a lovely wee boy when he is away from his siblings (again making me further believe its jealousy related) as I didn't really prepare him much, it was just sprung upon him and then the next one! It doesn't take a genius to work out that would of been hard, confusing and he just wanted the same attention.
I no longer think he needs evaluation from anybody else (my own parenting style was the problem, as was 2 babies in 2 years) I am embarrassed by my comments of take him away, again this is when I was going through a termination and was mentally at the lowest point of my life. I fucking hate myself for even getting pregnant. I'm not going to go into more detail on that. Like I said I have since been offered respite care (offered only last month) and I refused (a sign that things are better because I genuinely do not want him away from me) I couldn't imagine anything worse. That would confuse him even more, he would feel even more pushed out and that hurts 
It might not seem like it to some but I am trying. I've had to deal with a new baby who ended up in hospital, a termination, the passing of my GM, and now working. It's not hard to work out that I am a bit stressed and overwhelmed.
We are genuinely going to stick to giving him more time (once school starts, there will be 2 hours before getting DD and always the weekend)
But it's more about making him aware that we are there for him, that HE IS a part of this family and that no matter what he can come to us. And just keeping with playing with him when I/DP can, or using his mood board or trying to take him out.