"My issue is I wouldn't go on holiday without him or our kids. If I was going to to abroad on holiday... he and they would be my first choice travelling companions."
Hmmmm, well you see I think that's really your problem, not his. I luffs DH and our DDs loads, but sometimes I am very happy to not be with them, and do something on my own/with friends.
DH has always gone away for a few days, here and there, with his mates (usually involving a lot of golf - yawn). I've never had a problem with it, as he works bleddy hard and luffs his golf.
Conversely, he's never had a problem with me spending any time away from the family home (he gets to watch crappy TV and sleep on the sofa, which is a bit like camping for him).
The fact your DH has made a new friend, and organises to do stuff with him isn't wrong. It's good and healthy to have other interests and friends outside of your marriage FGS.
I will never get these marriages where the couple have to do everything together, all the time. I would hate that, too stifling by far plus, it's usually because they don't actually have any other friends
Sounds to me like you need to improve your social circle OH + organise a nice holiday for you as a family.
You need to stop all this passive/aggressive sulking, resentment and 'Woe is me attitude' as it's not going to help matters at all, and down that path only lies misery and bitterness.