"What the fuck is it with monoglot English speakers? It's like you're actively offended by the existence of other languages."
Erm--the general rule that "it can, depending on the circumstances, be rude to use a language that's only shared by some of the people in the room" actually applies in other countries too, believe it or not.
I live in Japan and if I am part of a group where some people don't speak English, I would avoid using it, even to another English speaker, if it is possible.
If there is another English speaker present who doesn't speak Japanese well then it can't be avoided, obviously. And speaking English to a small child because you are raising your child bilingually is generally felt to be acceptable too.
But as a rule, you do what you can to try to avoid shutting part of the group out of the conversation. It feels dismissive of the other part of the group, for a start, and can also feel like the linguistic equivalent of whispering to someone when you are in company. That is the rule in most countries I have been to, NOT just in English speaking countries.