I've been to two destination weddings with a baby; one child-free and one (allegedly) child-friendly, and they were both by far the most stressful weddings I've ever been to!
Sounds like you've now decided not to go, but if the bride and groom haven't thought plans through yet, for the sake of other guests, as well as their own enjoyment, they should either organise decent childcare that parents and children will be comfortable with, or make the wedding properly child-friendly.
Here's what could happen otherwise!
Child-free wedding. Bride and groom hired an entire boutique hotel for a few days. Other days were family-friendly, but they (understandably) wanted the ceremony and meal/speeches to be child-free. The hotel organised "professional nannies", who were meant to set up in one of the largest suites for the day. Turned out to be two women of a certain age who didn't speak a word of English (despite the entire wedding party coming from the UK/US), didn't bring any activities, and seemed to think their remit extended as far as knitting in the corner and making sure no kids left the room. The suite didn't materialise either, was a cramped, stifling room for ~10 kids aged 4 months to 8 yrs. Some parents took one look and didn't even attempt to leave their kids for the ceremony. We managed a couple of hours, as it was his main nap time, he was a great sleeper, and we'd written down and mimed his sleep times. When we checked on him though, he looked jet-lagged and blatantly hadn't been put down in a quiet corner for a sleep at all, and the remaining parents also removed their children at that point. Throughout the entire reception parents were therefore tag-teaming it to sit in their bedroom with the kids, missing either food or speeches. Thick walls also meant baby monitors didn't work, so that wasn't an option.
Child-friendly wedding. We could have asked a family member to babysit for this one, but the couple, who already had school-age kids, kept going on about how child-friendly it was and how we should bring him, so we did... All the school-age kids had a great time with the entertainer etc, but it was a total nightmare for a 1-year-old. Down a narrow spiral staircase so we couldn't bring the buggy in, no high chairs, no baby-friendly food, no changing facilities, nowhere to get away from the very noisy entertainer/disco, no outdoor space. We had to take turns jiggling him on our knees for hours.
Spent the following 24hrs of the holiday recovering in both cases!