I'm a professional traveller, and within 1 month of being back at work after mat leave I took my baby to the Bahamas with me. I love showing them the world - the baby is now nearly old enough that she will be allowed to travel with me to work unaccompanied and we are already planning her birthday trip, where she we go through the airport on her own for the first time and come with me to work. Costa Rica is top of my picks for that trip but I am at the mercy of where work wants me to go, so it could be anywhere, but it will be exciting nonetheless!
I think you need to take into consideration what is feasible and what is likely to be enjoyable with a child. Things to consider would include health risks (you can't get malaria tablets down a toddler, for example,) how you will transport a child around (babies in snowy places can be hard to get around - deep snow no good with a pram, slippery ground - especially on a hill - terrifying with a baby in a backpack,) whether your child is a picky eater and how you will feed them in a tricky place, if they are likely to pick up a bug (OH used to go down with a tummy bug everywhere we went. He's got better with time, but the idea of 2 children explosively posing everywhere doesn't fill me with delight. It was bad enough being on a boat with OH and his dodgy tummy!)
Things that haven't held us back include time zone differences - we've cheerfully crossed 8 time zones just for a long weekend when I've taken kids to work with me! I just worked out what things we could do late at night while they were wide awake instead of sleeping. Long flights aren't a bother either if you go about them with the right mindset. Take enough gadgetry and kids will watch cartoons with a headset on quite happily.
Travelling with kids is a different experience and great fun. When we were in Italy they were treated like royalty and I was treated like some kind of superhuman for giving birth to them and then taking them to Italy. I'd been to Italy loads before then, but the treatment I got as a mother with kids with me was completely different to the treatment I got before. Same in the far east. You have to temper your expectations though. Few young kids will sit happily through a long meal, or in a bar on an evening if there is nothing to entertain them. This isn't a problem - we had a fabulous holiday last year and spent every evening sitting at a quiet beach bar sipping cocktails and playing cards or dominoes with them. You get to go to places you would never consider on your own - we found ourselves in the Trick Eye Museum in Hong Kong, I'd have never gone in there on my own but with kids it was amazingly funny.
We've travelled extensively as a family. We've been round Europe countless times, the Middle East, the Far East, Central America, Australia. We have Africa and South America still to tick off, but I have a list and each year I see what would be feasible and aim for that. This year I am comfortable that a child who falls off a sailing boat won't immediately drown, so we are going on a boat round the eastern Med. I'd wanted to go to Thailand but we can't all get the same time off work/school so we have to tailor our holiday to the time we do have. I look forward to all my time spent travelling with the kids.