If you genuinely don't enjoy it, and neither do the kids, it's a waste.
If it's hard work, but you enjoy it and you're making different memories rather than 7 years of same-old-same-old, it's worth it.
I have found it useful to lower/change your expectations of what a holiday is. The first holiday we had with a baby/toddler went badly, and I think it's because we were still judging holidays on how relaxing they are. To many of us a holiday means relaxation. But a holiday with small kids is never going to be as relaxing as being at home with your usual routine.
So rather than thinking "holidays are for relaxing" you have to think "holidays are for doing/seeing something different". Your kids can have different experiences too and you can have fun seeing them do new things.
Seeing your toddlers splash in shallow water in a nice beach destination is something you might not be able to do back home in the UK. Or watching a 4 year old doing his first skiing lesson and marveling at how much more naturally it comes to him than it does to you. Dragging a whiny toddler up the Empire State Building or through the streets of Venice? Not as fun as it was pre-baby but still more interesting than dragging a whiny toddler round Debenhams back at home.
It depends if you enjoy holidays because you have the travel bug and you love the feeling of being abroad/away from home, or if you only enjoy holidays for the relaxation factor. If it's the latter, it's pretty pointless and you might as well stay at home.
Finally, no-one likes to admit it but there's always the smugness of having more glamorous family photos to post on Facebook...