I do understand the guilt because I've experienced it myself.
One of mine is 21 and the other is nearly 19. Both are at uni. After doing the "dream family holiday" after GCSE/A-level year a couple of years back and really enjoying it we invited them both on a similar trip last summer (US road trip) even though the dream holiday was supposed to have been the last big one because it was expensive., because I knew that soon enough they'd both be off doing their own thing, and well, they were quite good company on the previous road trip. I paid for all of it so they got a very good deal in my view! (Both also went on their own holiday with friends/girlfriend separately.)
Well, while it was a thoroughly enjoyable holiday and we all had a great time, and it was lovely to have 2 more adults to plan things with and to have new experiences with, this time round it was noticeable how much more they had the urge to do things THEIR way. eg choose specific restaurants etc. Which caused an issue at times. I mean, due to our ages, DH and I are most likely never going to go back to that part of the US ever again, whereas they have their whole life ahead of them to go again and choose specific places where they want to go to, so dammit, I wanted to go to the restaurants I wanted to go to!
And I guess a small part of me thought "well, we ARE paying, so why SHOULDN'T it be our choice?" So sometimes there were sometimes clashes of opinion over things. Always respectful, it was just a difference in preference, but that's just what comes with being a young adult, and living away from home. They want to do things their way. And that's exactly as it should be, and a natural progression with young adults growing up. But it was apparent by the end of that holiday that there was going to be a natural "parting of the ways" in holidaying from then on. Which means the guilt has gone!
So this summer I've booked a cruise for just DH and me (cos it's a special anniversary) and it's an adults only cruise too so I can't bloody wait! Dh and I had a few days on our own in Rome a couple of years ago and it was just unbelievably nice to be just me and him for a change without having the responsibiltiy for anyone else's happiness. Later in the summer I've booked a cottage by the sea in this country for a week and planning on doing lots of walking, pub lunches, and generally chilling. It has a second bedroom and whoever wants to tag along is welcome, but at the moment they both say it's too much of a boring holiday and don't want to come. Can't say I blame them at their age but they may change their mind if they want a change of scene and a promise of meals out with someone else paying.
So I feel both DH and I have turned the corner with the guilty feelings and are looking forward to a new chapter with it being just us on holidays from now on. It has to stop somewhere. I'm not saying that's it forever, but I can't see us having an "annual family holiday" from now on as a routine thing. Apart from anything else, it's getting harder to time everything so that everyone is free at the same time.
Book the couple holidays, OP, and enjoy them!!