I think the problem, here, to an extent, is the feeling that it's about US. It isn't. It's about the experience and feelings of the local people.
Let's be honest, if you live and work in a touristy area, you'll mutter about 'bloody tourists'. Doesn't matter who it is or which country you are in. They are on holiday, they have all the time in the world and they don't really appreciate that the pretty little village they are gawping at is home.
It's not about who colonised who, generally, or even about stereotypes, it's about reaching a point where you just want to go down to the beach near your house when the weather is good and have enough room to sit on it.
People know perfectly well that tourism is a big part (the only part, in some places) of their economy. It doesn't make it any easier to live with. I really don't like the 'well, if it wasn't for my money' brigade. People with that attitude tend to be the least understanding of the simple fact that people who live in these places still need to be able to do the stuff people do in their home towns and villages.
If you ask people what pisses them off about tourists, it won't really be what their country did. It will be how it takes three times as long to get to work in the season. It will be how there's nowhere to park. It will be about the assumption that you have as much time as they do. It will be about your kids finding used condoms on the street, feeling less comfortable going out at night, being kept awake at night.
It's what happens when suddenly, in peak season, your sleepy little town/village/island quadruples in numbers of people. And for some places, like the Canaries, there's no respite any more. It's year-round.
SOME people make a lot of money. Most people just work ridiculously hard in a notoriously poorly paid industry with very long hours.
It's not your fault. You just want a nice holiday. But it's hard work for the locals, even if they aren't working in the industry. People living in tourist areas in the UK find it hard, and it's a short season, here and most visitors speak the language and know how to drive here (well, in theory). But even 'staycationers' who have only gone 200 miles up the road seem to think the 'rules' are different in a place that caters for tourists. That somehow, because you are on holiday, it's OK to do things you wouldn't do at home. That effect is magnified abroad.