Right. He's now assaulted you and he's escalating.
Honey, you need to realise he is potentially a serious danger to the baby. He has gone off the rails since the birth, he has now hurt you, he has taken the baby away from you. Unfortunately domestic violence can peak when a new baby arrives and the younger the child the higher the risk of the abusive partner hurting them. If you possibly can, contact your father to come and get you and the children. If your father can bring any other large friends with him, so much the better.
You can, as a pp said, slip your midwife a note tomorrow that says 'i am being abused and threatened, he has become violent and assaulted me, I am afraid for the baby. Please help me'. The midwife SHOULD then find a reason to take you into another room alone, and if he kicks off or tries to prevent you then he's creating evidence right in public in front of people. But you would do better to ring your dad right now and get the hell out of there.
If he has a mental health issue with times of clarity then during the clarity he will understand and support that you HAVE to keep the children safe, and that means away from him. He will want what's best for his child. If he is an abuser playing mind games then there will be no clarity, just more mind games, and it will be all about him. In which case you need to be out of the house.
Report the assault to the police (tomorrow, when you're well away from him) that he pushed you, took your phone, took the baby, has made multiple threats, refuses to let you see professionals alone. You also need your midwife and health visitor in the picture. They will help you organise what comes next. Step one: get the hell out of there.