It sounds like a whole massive mess with some responsibility (or lack of it) on each side.
Where you go from here is the important thing.
Rage is normal, and rage without a clear direction festers, unless you deal with it, not sure how you deal with it, we all do things differently, I tend to talk about it until I'm so bored the rage is replaced by total disinterest. I'm not sure your total rage should be directed at this woman, or your messed up ex, or even yourself.
Your ex may be messed up, maybe genuinely feeling shitty (rightly so) after all, he and you made a baby whilst you were not together, when things were uncertain, and it takes two to make a baby.
You're all human beings, and therefore prone to major fuck ups without being 'evil', I think now you need to focus on the future, the future which includes putting why you're in this situation to bed, and building some form of workable relationship with your ex (which would include him not badgering you, giving you puppy eyes, or dredging up his regret every time) and if he's with this other woman, trying to foster a polite agreement whereby your child is able to see both parents. She's not a demon.
You deserve respect and probably apologies, respect you can get, the best apology comes from actions. Tell you ex what you expect, be prepared to compromise and I hope you can find a workable future where rage is a thing of the past.