You both own the house. He is the father to your DD and the baby you are expecting. It doesn't matter one jot that you are not married, as he is the father to your DC. He does not get to kick you out of the house, it wouldn't matter if he were a multi-millionaire and you were completely broke, no family court would have you and your children made homeless while he, a man on his own, lives in the jointly owned family home.
Unless you have a very real fear that he could turn violent, please do not leave the family home for even a night or two. He can stay at his mum's if he doesn't want to be in the same house as you, but just as he can't kick you out, unless he is physically abusive to you or your DD, you can't kick him out either. You can however, tell him he is sleeping in the spare room if you have one, or the sofa if you don't. If possible have your mum, or a good friend come and stay with you for a few nights (you can share your double bed with them). I doubt very much that your (hopefully you choose to be) ex would cause much of a rumpus in front of a witness, but if he does, you would have a witness.
If you know that he does have a physically violent personality then do take your DD to your mum's tonight, but (and you and other mumnetters might think this is ott, but it isn't) please go via your local police station and tell them that you are taking your DCren away from your and their home, because of your ex's intimidation, and threats, if he has made any. If you have any horrible texts etc from your ex please keep them and take screen shots of them, and if possible forward them to someone you trust as well! There needs to be proof that you didn't leave the family home because you wanted to.
OP, I feel so sorry for you and your DD, but please do not just accept that you have to do what your ex says, please stand your ground, but again on the proviso that it isn't dangerous to do so. If your ex has calmed down, and or doesn't want the hassle of him having to move out, pay maintenance etc, and if you don't want him to go (for some reason), please lay down some strict rules of your own, including him never raising his voice in front of your DD, never verbally abusing you, ever, and him being willing to go to counselling with you. 💐