Ok OP, you've got the weekend coming up. Time to tackle this. How about you both go out for a quiet drink somewhere lovely, or have a nice meal indoors if you have to be in. Once he's chilled and happy, tell him how much you love and value him, and appreciate the life you've built together.
Tell him you are deeply concerned for his welfare, in that you can see a sad but unfortunately likely possibility of him losing everything that he and you have put so much effort in to create. That should get him listening.
Keep calm through the whole of this conversation.
Explain that we live in a time where if something goes 'wrong' with a female colleague, he can lose his living, reputation, ability to provide, literally overnight. Workplace gossip has and does ruin careers. His references will be shot to shit for any future ambitions.
Say you totally get how nice it is to receive attention - I mean, who wouldn't (being such a handsome dude and all, yes lay it on a bit thick, remember you are currently at war here after all!)
But for you - having now had calm time to think about it - the line was irretrievably crossed by rescinding the Saturday night invitation to you. Tell him you were SO happy he decided for himself he didn't want to go, as you didn't want to fall out over it. That he's one of the good ones and you adore him for it.
Finally, just say the level of going out, phone calls etc has sadly - for your personal level of comfort and secure happiness in the relationship - gone beyond the line. The thought of him losing absolutely everything you've built together makes you sad, and of course extremely costly for you both.
Say you'll leave it for him to mull over in his own mind, but you needed him to be aware of what he is on the brink of losing at every level.
Don't make it a conversation about her, people listen more closely when you are talking about their own interests.
Then change the subject to something more cheery and drag him home to bed 😀 (I did say this was war, you gotta stand your ground 😉)