Your son doesn't love his dad. He's just smiling away the way babies do. He barely knows his dad.
You think you love this man - think long and hard about why you still love a man who speaks to you like that, and doesn't care about his son. That should have killed your love stone dead.
This is such a hard time for you and you're obviously very young. I really think you need to ditch this loser, and concentrate on your son.
How long til your boy has a tantrum as a toddler and his "dad" shouts at him "I never bloody wanted this, didn't want you"?
Because I expect that's what is in your future.
OK, so it was an accidental pregnancy and it's fine that your boyfriend didn't want to go ahead. Many people would choose abortion. He can't FORCE himself to love a child he doesn't want.
But the boy is his responsibility now. You both had an accidental pregnancy - abortion is not a guaranteed outcome, this is the risk he chose to take in having sex.
So the LEAST he should do us behave well towards the child and shut the fuck up having a go at you. Who the fuck does he think he is telling you it's your problem you chose to keep him? He doesn't love you. Why do you think you still love him?
Oh and a little guess he isn't paying maintenance 
Ditch the loser.
Enjoy your son.