How old are your children?
This sounds really hard for you OP as you do seem to have an element of thinking is this your fault he uses escorts, well no it's not, only he can control his actions and behaviours.
He does sound like he has a very unhealthy relationship with drugs and an alcoholic if once he can't Just gave a few beers and cone home, the need to carry on fir more alcohol (and drugs and sex workers) i would suggest means he is addicted to those things.
Why have you not told your close family to help support you through this? Is it because you know they will advise you to leave and you are not ready to do that yet ? Because you are embarrassed ???( you really shouldn't be, all this is on him)
You do need to make plans to leave him, he will not change, he has had ample opportunity to get counselling for his addictions and opted not to.
He is not family orientated if he is showing his wife, the mother of his kids, such a disrespect and hurting you so much. Just because he used to do his nans shopping doesn't make him a good person, he just did a weekly good deed.
He is a liar and a cheat and potentially an addict.
You need to remove your children from this toxic relationship because they need to learn what healthy relationships look like, and that is not being scared of your DH going on a night out and having to tell him what to do, that's toxic.
Can he go on holiday, well yes of course, it's not your job to tell him what to do, he is cheating here so of course he will chest their, but he is an adult. If you need to manage him, you need to leave him.
I do hope everything works out ok for you and the children, sorry but you all deserve better.