Oh my god OP, I am so worried for you. This is clear cut financial abuse and coercive control.
I came on to write that you should leave now before the baby is born so that you won’t have such a fight over custody if you already live somewhere else permanently and are separated from him.
You sound like someone who has been abused, I’m so sorry to say that to you, but the way you are justifying yourself for perfectly normal things and talking about earning his trust (why is it you who has to earn the trust??)
I am relieved to then read this from you:
I just worry that once the baby's born, he may use my financial vulnerability and fragile mental state (which, although not an issue past or present will, udging by comments here, be an inevitable consequence of trying to stick with this) against me should we ever have to fight over custody.
And also that you think he wants the baby more than you.
Listen to yourself. The real you is still in there. You know what he’s like. Don’t stay in the vague hope it will get better. He will suck that last bit of resolve and straight thinking out of you.
I wonder that he is reducing his hours and forcing you to go back to work so that he can say you are equally caring for the baby, to give him a better chance at custody???
Please move back to your family now. One thing you’ve got in your favour here is no joint mortgage. You are not tied because of that. You can leave tomorrow and just pay rent elsewhere or to your family.
Sending hugs and 