I have taken them on several uk holidays by myself and we always have a fairly (!) good time. We usually just go and stay in a cottage somewhere for a week.
When we have gone abroad we usually take my brother or a mate of mine with no kids.
I have just booked a week in ibiza for just myself and the dc in August. Was excited...but now just thinking i am crazy - and everyone there will look at me with pity and think im crazy too??!!