Have you told him that you are concerned?
What you've listed are, ok I'll be honest, all signs of an affair.
BUT there are other explanations, so don't let your imagination run away with you.
Is it the same day every week he goes to the pub? If so, next week, say "I have arranged for X to babysit so I'll join you tonight." His reaction will tell you loads!
Pop into the gym when he's supposed to be there.
Same to golf.