Blimey!!!!!!!!!
First of all, WELL DONE -- for telling him it's over, and sticking to it (despite fluctuations; it's hard when you can't bear to see him upset).
When you see him upset, just remind yourself that a reptilian part of his brain, which he doesn't control, actually sees you as a worthless whore who deserves to die. 🤐
Hearing problems, eh. Hahahahaaahaaa, he doesn't even care how pathetic he sounds as long as he's putting the boot into you. Oh dear! That is really scraping the barrel.
He thinks I haven't given him a chance to change
What's become clear is that what he really wants is for you to give him a chance to carry on just as usual, but with a delusional gap during which you have another baby with him. Thank goodness that's not going to happen! Well done for escaping a very cruel fate.
What happens next is going to be tough, but not as tough as spending another (potentially) 60-odd years with this child-man.
Everyone knows the gossips and idiots at work, and it sounds as though your STBXH (oh that is delightful to type, at last!!!) and his ex-colleagues will be recognised as such. Honestly, the worse the abuse he heaps on you, the more the folk who actually matter will take it with a pinch of salt. You mentioned that your boss was aware of problems, and helpful. Can you forewarn her that he is planning a hate campaign? (Not that you need to; it just might put your mind at rest.)
The willy scene is just bizarre and I hope it will give you something to hold onto, excuse the mental image!! 🤣 I think the weeks/months ahead are going to be so difficult that it is useful for you to have a quick reference point to remind you just how bad things are. He's given that to you on a plate. (Another weird mental image there, sorry)
I don't know where you are, but I live in Portugal so if you are in that country and find yourself stuck, PM me. We have plenty of room for you and DD! 
Good luck with the rest of your holiday. If I were you, I'd be on the phone to my mum. 💗 You need her kindness now more than ever.