Unfortunately you have had a baby with an abusive man. Don't ever forget this, don't ever trust him, he is the enemy. His behaviour is not normal or decent, he will use your child to further abuse you both as long as he is able. Plan yours lives accordingly.
Go back and live in the house, speak to a lawyer see if you can get a non molestation order against him to keep him away. Document all his behaviour (detailed diary, texts etc, get it in writing if you can), the verbal abuse, the upcoming financial abuse (trust me it's coming), and anything else he throws your way. It will help in court. Put house on market for sale as soon as you can.
And if I were you after the sale, id move as far away from him as possible, make it very difficult for him to see baby. I'm sure plenty on here will disagree with me on this, and none of them have had the heart break of seeing their child used as a pawn of abuse. You can move wherever you like within the country with a good reason, job opportunities, cheaper housing, support etc.
Claim child support through the CMS. Look up grey rock, always use it. Educate yourself on how to manage your situation, plenty of advice on mumsnet. Always centre your child's wellbeing in any communication with him. Be aware anything you do or say could be used by him against you, so be careful. Get on and plan your lives without him in it.