If you know it's going to be hell, don't do it.
Our autistic DS wasn't diagnosed until junior school and has lots of insight, so I'm not sure if our experience will help. He hates change and can't do unexpected changes at all, but he actually is ok with carefully planned, low key holidays with plenty of warnings. A holiday is a regular occurrence, so even though they are not frequent, he accepts that they are part of "normal" if that makes sense. Similar with days out - think season tickets, repetition is key.
AI holidays work brilliantly for us. He can eat broadly the same thing in the same place every day, which makes him so happy. I don't have to cook, and we can just be by the pool doing puzzles or reading so very little stress. Basically we just turn into slobs on holiday! Not winning any medals, I know, but it works for us.
For days out, this summer we are doing a mix of stuff he likes (massive roller coasters and season tickets to his favourite sport), stuff with just DD, and one or two things that he won't love but DD will, so we are pushing him a little on that. However it's within a careful framework that he'll do something really fun the day before, and maybe ice cream after. We usually have one parent prepared to leave early with DS while the other takes their time with DD so she can get the full experience. If we stay away anywhere we book a Premier Inn.
In terms of fairness we write a family list every holiday of stuff people want to do. Then we pick things off it so everyone gets 2 of their top 3 or something. Fair is making sure everyone gets the same number of their top things in, not making sure everyone goes to exactly the same number of things. (When mine were little they tended to write very modest requests like going swimming, or eating ice cream in the park, which helped!)