Great post by @JustSomebodyThatIUsedToKnow
However - out of term time we really get a ramp up of things
That's because at school, there's a structure, every day has a timetable and he knows what's happening and when.
Without that, he's totally lost, has no idea what's happening and it's anxiety central.
Let him structure his holiday time with you, you can veto anything that conflicts with other family interests, but sit together and try to arrange his days so he knows what's happening. He could draw picture cards if he wants to.
Use now and next terminology, especially if you have to alter the structure for him. Now, we will eat breakfast, next, we will go for a walk Halfway through the walk, prepare him for the next thing. Never assume he can pick up on hints or inferences, he has to have very plain, clear instructions. No "we might do" always we will do x then we will do y.
Once you've got the hang of that, start building in some flexibility, take it slowly, but there will be days where a walk doesn't follow breakfast, so prepare him in advance for that. Usually, we'll have breakfast then go for a walk, but the weather may be too bad or siblings may need to be elsewhere then, so what we'll do then is stay indoors and do X or see to the siblings, then the next thing will be back in his routine.