Thing is, most people talking normally do so much more quietly than those having a phone conversation. It is an interesting psychological fact that we all tend to speak more loudly on phones and video links.
It is possible to ameliorate this and force yourself to talk more quietly on the phone. Try it; you may be surprised to find you can still be heard over the phone link. And every so often, on a train, for instance, you may spot someone who has learned to do this. Hats off, indeed, if you are one of those intelligent people!
Most people, though, do simply shout when on the phone or video link, which it is why it is so annoying. My response, in public places when I get annoyed, is quietly to ask the shouting person to speak more quietly, and if they do not, ask them to do so at the same volume at which they are shouting. Of course this can escalate. But, well, so far things have not got violent. And two people shouting at each other is no worse than one person shouting into a phone.
As for using speakers instead of headphones, well on trains at least you can call on one of the railway byelaws (Byelaw 7, (1), (ii) explicitly forbids this unless prior written permission has been gained from the train Operator). Elsewhere there may be similar laws or rules.
In a workplace or other restroom, you may need to argue the case. Good luck! It may help that you can be sure you are in the right, pace one or two of the idiots posting here. Of course people should be considerate of those who may not want to have noise imposed on them, be that noise shouting, video soundtrack or the inconsiderate one's own choice of music.
(Always awful music, ime, played by inconsiderates. I suppose undeveloped ethical sensibility marries with undeveloped aesthetic sensibility; I have yet to hear, say, a Gardiner Bach cantata or an Ashkenazi Mozart piano concerto played by a selfish person in a public place to the annoyance of others. Ed Sheeran, Sam Fender, say? -- A different matter. )