I think sometimes we seem to have more experiences with nasty people if we look like a target somehow (eg. a bit shy, vulnerable-looking, quiet, etc.).
I say that because I used to be one of those people. :) I was always going home from work in floods of tears from someone or other getting at me, and the more I went out of my way to be nice... the worse they seemed to get.
My experience of the world started changing as I gradually learned to be more assertive. Before when someone yelled at me, I would be in shock and go off somewhere and cry. Now if someone yells at me, I will glare at them, keep my voice low, but say firmly "Do not yell at me. That is unacceptable."
If they continue to do so I will take further action depending on the circumstances. If I can walk away I walk away. If its at work or someone else in a work setting I will pointedly ask for their name (or look at their name badge), and will say "If you continue this aggressive behaviour I am going to report you."
Nobody at work (or pretty much anywhere) yells at me anymore. ;)