Different situation, but I have experience of something similar(ish).
When XH and I first separated, we continued to live in the same house for a while for financial reasons. He said he wanted 50/50 but didn't actually do that for several months. Started off doing alternate weekends, but I did most care the rest of the week. DC had access to both of us. So that probably sounds like quite a big difference from your situation, OP. The similarity is that DC2 (now a teenager) was 7 at the time. We're waiting for an assessment appointment but I now strongly suspect ADHD. Has struggled with emotional regulation and change/transitions for a while, other mental health issues, but for a long time attributed this to age and then the separation and divorce, lockdown, etc.
Anyway, first year we were separated, I took DC on holiday then XH took them. My DC1 told me afterwards that there had been some really challenging moments (obviously my words there, not how DC1 worded it!). It was probably a shock to the system for XH. He wasn't used to 24/7 solo parenting. I'm glad I don't know too much about it.
If you're on amicable terms (XH and I were not) then it's definitely worth the conversation about him spending more time with your daughter ahead of a holiday and talking about how you'll keep in touch, if DD will be able to call you, and anything else specific on your mind.
However, as you're not really going to be able to prevent him taking her away, you may just have to brace yourself for it and accept that your DD might have some challenging days, your XH might not be ready for it, but perhaps just ripping the plaster off is the way it's going to be.
From what DC1 tells me, I think DC2 is still more likely to have difficult days when with XH than when with me, but DC2 is genuinely happy to spend time with XH and looks forward to their holidays. There were a couple of years after the first holiday when XH didn't take the DC anywhere due to COVID, so DC2 did mature a bit before subsequent holidays.
I still always feel a bit uneasy when they go away with XH. But that's just part of being a separated parent and not always being there.
Good luck with talking to your XH about it all.