She is taking the piss! They are having an emotional affair that needs to stop now. It would appear he is using her as a back up plan and she is trying to undermine you. It is disrespectful to you and your child. I think you are going to need to put your foot down on this one and make it clear it's not acceptable.
This is what I would do in your shoes:
Once baby is in bed sit down with your DH and tell him he is being disrespectful. You want him to stop acting this way and break off contact. You are his partner and worthy of respect, this woman is not respectful towards his family (you and baby ) and he should not tolerate that.
You will not stand for it and will take action if necessary.
If he doesn't take you seriously I would suggest you ask him whether he would be happy as a weekend dad as that's where he is heading.
He is a father now with responsibilities. Time to grow up.
Stand firm. If he calls you a nutter then tell him very calmly you won't get drawn into playing mind games by questioning your own judgement. You are in the right, end of. Take it or leave it.
Remind him she is his ex for a reason!
Respect is what is missing here. He needs to know you have standards and won't lower yourself into fighting with another woman over him.
Don't speak to ex. That will add fuel to her fire. Don't FGS look on Facebook then she won't get a reaction. Sound like she thrives on drama, don't give her oxygen.
Stand up to him and let him know you mean it. But be warned it may not go how you want, just be mindful of that. At least then you've got your answer.
Good luck.