"It’s an empty house owned by a friend that’s that been renovated yet so it’s a shithole and not somewhere u would choose to stay."
"I completely accept he’s struggling and that the pain is bad"
Sometimes, when chronic pain is severe, people can shore-up their coping mechanisms by turning in on themselves. Could it be that this empty house is allowing him to do that, to a degree?
What is he using to control the pain? Could he be self-medicating with cannabis (or other non-prescribed drugs) and if so, would that be something that he would expect you to 'allow'?
If this action in going there, and the way he took his leave, is out of the norm for him, I would say this is the behavior of someone at the end of their tether, who perhaps needs more support and sympathy, not less.
I wouldn't read too much into him going to the pub either. That's fairly common escapist behavior, when people don't know how else to get by.
Talk to him. It may be previous posters are correct and an affair is looming, but I really don't think he sounds as if he's in that position, so talk without accusation. It sounds far more likely that he is close to the (his, personal) edge.