Your bf clearly didn't learn anything about baby/child care with his first dc and he's not going to learn from his second because he's convinced that he's got nothing to learn.
From what you've said, your baby is at serious risk of suffering physical harm at the hands of your bf.
His lack of appropriate care and concern for your baby, together with his history of violence and inability to control his anger when the baby cries, would suggest that he is unfit to be left in sole charge of any infant .
The welfare and wellbeing of your baby is of paramount importance and I have no hesitation in advising you to get this man out your home asap with the assistance of the police if necessary.
It would seem that your bf has an informal arrangement with the dm of his older dc whereby any contact he has with the child is supervised by his parents.
You are under no legal or moral obligation to facilitate any such arrangement and, after he has left your home and should he wish to see your dc, I would strongly urge you to insist that any contact takes place in your presence in the supervised environment of an approved contact centre.
The only moral issue that should be of concern to you is the need to safeguard and protect your dc from harm.
Until such time as your stbxbf attends parenting and anger management courses any 'relationship' he has with his dc must, of necessity, be subject to stringent conditions.
I suspect that once this tosser impregnates his next gf he won't be arsed to maintain any contact with your dc.
I'm at a loss to understand why any woman would want to have a relationship, let alone have a child, with such a man. Were his deficiencies not apparent to you before you became pg?