Actually this has happened to me, twice. I must have that kind of face.
The first time, me and DH were invited over for dinner by a couple we knew, all normal, we'd been to the house before with other people - this time we are ushered into a different room done out like a kind of exotic tent, bean bags, candles, soft lighting etc. It took us a while to twig on (in fact it was when the bloke handed me a photo of the wife in bondage gear) DH and I were mortified, but giggling when we made a hasty retreat.
The second time I met up with some old friends who told me their marriage had been on the verge of collapse when they had found out about the 'lifestyle' and they were just checking to see if I was remotely interested as they both were. Not predatory, not 'nasty', just a grown-up conversation.
Fine, if you feel you don't want anything to do with it - but yeah, over-reacting. You don't have to be involved, but last time I looked it was legal to ask.
And by the way, there are swingers in 'Benidorm'. Just in case you hadn't noticed.