He said he didn't want to appear rude
This translates to he was mow concerned with her feelings than with yours.
You mention he is trustworthy. Sorry but your posts after that suggest he is anything but that and he is starting on the script.
Blaming you for being annoyed (translatyion: calling him out on it),
Insisting nothing going on,
Now angry and sulking and not going on your holiday.
This wasn't out of the blue OP, conversation history proves that. I bet he has met up with her.
His next steps will be:
Telling you that you are paranoid/irrational/over thinking/too sensitive
Telling you is isn't happy, never has been
Pushing you away so he can use this as an excuse to see her and then blame you for being overbearing/not understanding him
He absolutely knows what he has done and how it looks.
The fact he is so angry and not doing everything in his power to console you, limit the damage he has done to you and prove to you nothing has gone is the action of a guilty man.
I know for a fact if my DP did something that hurt my feelings, he'd be fixing it and reassuring me by his actions, however ridiculous he may think i'm being, innocent people would be horrified and would be fixing it.
If I were you id' be telling him exactly how it looks and how his reaction looks. And based on that you think he is guilty as hell. And you are going on the holiday on your own (or taking a friend) and if all this is so innocent he needs to start acting that way and fixing the massive great big mess he has caused by messaging another woman.