I don't find holidays with a child relaxing. I've also never been a beach/hiking/camping kind of person - give me ancient cities to explore, museums, galleries, and lovely little restaurants, please! Can do quite a bit of that with a breastfeeding baby who'll nap in a sling, but not much at all of it with a toddler who wants to be running, if you don't mind... so we don't really holiday with our 2 year old.
However ee fly back (from the east coast USA to the UK) to see my parents/extended family, have done it when DS was 6 months old and a year old. It was MUCH easier with a babe-in-arms (constant breastfeeding, boring, tiring but not impossible) than with a trying-to-walk-and-easily-bored one year old.
We're doing it again next summer (will have 3 yr old DS and a 6 month old baby), and I hope DS will be easier as he'll be more into the in-flight cartoons, Ipad games, etc. It's tiring being in England because the children have jet lag so naps etc are all over the place, we have jet lag but no way of lying in to make up for it, they wake more in the night, and it's all generally pretty tough on the sleep. Absolutely worth it to see people we love, but tough that just as we all adjust to the timings after 2 weeks away, we come back and have to do it all over again!
We haven't gone on any holidays here, yet. I think a bucket-and-spadey time might well be on the horizon at some point next summer, we have lovely beaches a couple of hours' drive from here. We've done day trips to beaches this spring, but DS wasn't keen at all. Next year he might be more into it. At the moment a trip to the local outdoor swimming pool and a look at the digger up the street are about as much excitement as he needs.
I tell myself that Venice can wait, Rome will still be there when the kids are teenagers who want to visit friends, etc :)