You are being emotionally (and psychologically) abused.
Of course you can add leaving to your options here. It will take a bit of planning, and it is hard to contemplate such a move when you are heading in a different direction with him (or so you believe) - with one child and a baby on the way.
When it comes to the option of leaving, you have the advantage of having a job, and it appears you have family of your own nearby. I think you should consider the option very seriously. Your alternative is more of the same for the rest of your lives together.
His 'apology' was no such thing. And miserable list of excuses that it was, you had to wring it out of him after an argument.
He should not be allowed to join the family holiday. Your mum should write him a strongly worded letter uninviting him and telling him that she is taking notes every time he opens his mouth.
He is testing everybody to see what he can get away with. It is giving him an ego boost to see how he can get everyone walking on eggshells in his presence.
He is probably also trying to make you dread family gatherings so much that you will decide not to go. He will thus isolate you.
He is creating a cycle of abuse here:
He humiliated you, then ordered you to 'stop nagging and shut up'. Then you finally got him to address your complaints, but instead of an apology he gave petty excuses and a good example of 'hoovering' - mentioning he was looking forward to the new baby.
Hoovering is an attempt to suck you back in after an incident.
In the remark about the baby he is also subtly reminding you of your vulnerability here (you are pregnant). He thinks he has you where he wants you - dependent on him and reluctant to confront him.