Every year we holiday for one week in the UK without grandparents and have done for 20 years - just me and husband and whichever children we had at the time.
However, in addition to this, every few years, my parents have paid for a holiday with all my siblings, siblings in law, kids and niblings - as a treat and sometimes to mark a big birthday. This tends to be in a big shared house near a beach in France. We've all studied French and use the language wherever we end up. We pay for travel, ferry, food, passports, have a kitty for drinks etc and they pay the accommodation because they like us all being together and we'd not all afford to be able to go abroad otherwise. We have cheap days out to the beach and museums and all have a great time.
When there's a family wedding away in the UK we try to stay in the same hotel to spend time together away. We've invited my parents on weekends and half terms away in the UK like music festivals and Butlins but they've declined! My MIL would hate our holidays as she doesn't like travelling, doing lots of activities (we like going kayaking, theme parks, abseiling, swimming, crazy golf, climbing hiking, exploring old castles), drinking, board games, sand, "foreign food", "other languages" etc. She has moaned that she never gets invited anywhere but we invite her to everything - she just likes to say no!
Last time we went on a big trip with my parents (that they paid for and planned for a big birthday) she moaned "no one asked me" "would you like to go?" "no, never, but it's nice to be asked" "do you want to come with us to Butlins next year?" "god no" "do you want to come with us to Tenby in the summer, we could take two cars" "ooo no, I wouldn't want to go for a week". You get the picture.
We are lucky that we are all close. We have never used my parents as babysitters on the holidays away with them. There are so many of us that in more recent years we tend to all do our own thing in the day or maybe split half and half (sometimes there's something the teens and younger adults want to do and the little kids go with the older adults to something more age appropriate) and then all come together for a big shared meal and games at the house.