Yes, it's crazy how some people think:
Johnny Foreigner coming here = immigrant. Immigration is bad!
Us going to foreign country = expat. Expats are good because, well, we're British and obviously superior, so any country would welcome us with open arms!
It reminds me of the crappy short-lived soap, Eldorado... where the characters were exactly the type of entitled morons who'd have voted for Brexit and been aghast that this had implications for their lives in Spain.