I would try to find somewhere where you don't have to prepay nanny before you go, but have the service available if you want it. You may choose not to leave your DC once you see the set up, or you may only want to do it on odd days, or you may find that your baby is no bother and therefore you are quite happy to have him/her with you the whole time.
We went away when DS was 3mo, 8mo and 10mo. We found we didn't feel the need to leave him with anyone as we could sit and read by the pool or whatever while he slept in the shade, as the heat made him nap well!
Once DS was a toddler, we started using kids club. At 18mo he could use a kids club with a nanny, so got 1-1 care. Last month at 26mo, he could go to toddler creche (and actually ended up with 1-1 care as the hotel was very quiet!) Both were 2 hour sessions, and we didn't need any longer away from him to recharge our batteries. We didn't use it every day, and we trialled it for a short time at first to see how he got on.
MCTH/thisisyesterday - you think that a 5.5mo is better off left at home with other people while parents are away for a week/fortnight than being cared for just an hour or two a day by a qualified nanny but being with parents for the remaining 22-23 hours? And you think the OP is being unreasonable? Strange world you inhabit!
OP don't let anyone guilt trip you on this.