This must be exceptionally hard for you and I wanted to say that you do not deserve this and there is nothing you've done or could do to change any of this.
I don't know what's going in his head. I suppose the worst case is that he's a complete psychopath who has no feelings for you, baby, or anyone else and is taking pleasure in messing with you and enjoying causing you this pain.
The best case scenario is... what? That he does love you but he's got some deep seated issues causing him to repeatedly run from relationships, commitment and responsibility with seemingly no care about the absolute trail of destruction he leaves behind him?
While I can understand you may feel like you want to be there for him and help him through it, I would really consider the implicit message you'd be sending to your child by repeatedly accepting this. There's a good chance he will mess with your child's head too but even if he doesn't, they will grow up thinking this is normal in a relationship.
It's not normal by the way. This is not the way a loving husband or partner treats the woman who birthed his child four weeks ago.
Instinctively I want to tell you that his actions show he doesn't care about you but I don't know if that's always helpful. It's so hard to understand how they can lie and seem so convincing and so when someone tells you that, it can be easy to write off what they're saying as you think they just don't understand.
So instead I'll say that even if we give him the benefit of the doubt and entertain the best case scenario, it's simply not a healthy relationship and not one you can fix. He is (at best) extremely damaged and nobody can fix him except himself and years of hard work (years of work for himself by himself, not a grand gesture or big manipulative emotional statement).
Instead please focus on healing. Other posters will be better with the practical side but I just want to be another voice to tell you that this isn't acceptable. For as long as you let him, he will continue this push/pull with you until he takes every last bit of happiness and confidence from you.
If we are giving him that benefit of the doubt, perhaps having a baby has brought up some childhood trauma and he's now playing out dysfunctional relationship patterns.
You'll notice that even giving him the benefit of the doubt, it's not sounding like a great situation for anyone, let alone for a child to be raised in.
Now consider that he may not deserve that benefit of the doubt and he could actually be anywhere along the scale from emotionally damaged to complete psychopath.
Please take control of this and choose your own sanity for the sake of you and your baby. There is nothing down this path for you but pain.