I'm trying to look at this from both sides. You need to separate the 'lads holiday' and your caravan holiday to do this, as I think there is another whole thread to be made about him going away 'with the lads' every year at 56!
So:
He told you he was going away Easter weekend. Fair enough 🤷♀️
You booked a caravan away for yourself and your DS, and invited a friend with their DC. Again, fair enough 🤷♀️
He realised his date mix-up and you asked him to come to the caravan with you, your friend and several DC.
He said no, you think this is unreasonable.
Looking at JUST THIS issue, he isn't being unreasonable really I don't think. He's 56, probably working all week, and you've asked him to go away in a caravan with you, your friend and several young children, none of whom are his. Objectively, that would be my idea of hell on Earth! It's probably his too. It would be completely different if it were just you and your DS of course...but it isn't. There are other people in the mix.
Your disappointment is, of course, real and valid...because you know he's ALSO going away the following weekend. But that's a separate issue.
I'd be more concerned about his 'lads holidays' and his cheating if I were you. You say you don't really trust him. This is your AIBU, not the caravan holiday. Don't try to force or guilt him into going with you...it will sour your weekend away and you'll not enjoy it...and neither will your friend or the DC. Go, have fun, don't talk about him at all!
When you get back, that's time for you to sort out the real issue of trust (or lack of!)
💐💐