these things take time, strength and patience.
Whether your boyfriend wants a relationship with his ex or not, you will have to accept that he’s got one. It might not be a sexual relationship, he may not love her, but she is the mother of his young child and they are still a family; Even if not a “conventional” one. This will ALWAYS be the case.
The choice you’ve got to make for yourself is whether you are prepared to wait and stay the course, to be properly introduced into that family.
His ex will be disruptive, I’m assuming she is a young mum, she’s got a baby, she’s on her own, she will be needing support and the father of her child is seeing someone else. She’s letting him have some contact with the baby, but at the moment it will be on her terms. She needs space and time to gain trust and confidence - that’s perfectly reasonable (even if some of her behaviour hasn’t been).
Try being nice to people about it. Try encouraging your boyfriend to go and spend time with the baby, encourage him to keep the peace and keep working through it. Throwing down ultimatums at this point isn’t going to help anyone, and if he doesn’t feel like he can bend to your wishes then you’ll be the one left bruised by it.
On a positive note, if you can get things right then you have got the potential to be part of the family and the future may be bright. I’ve been a step mum for 8 years and have an amazing bond as I’ve had the contact and input for 7 years. But it will take time, strength and patience .... you decide.