So clipped if someone said they wore a certain item as their religion required it and were getting abuse for it, would you tell them to "toughen up" or conform?
Or if a natural red head said they were getting abuse for their hair colour, it's their choice not to dye their hair so they should "toughen up"?
What about someone who is overweight? Lose weight or "toughen up"?
Someone with extremely pale skin? Fake tan or "toughen up"?
Wears glasses? Change to contacts or "toughen up"?
Just because there are people who think they are entitled to yell abuse doesn't mean that the rest of us should do what they want. Being quiet and "taking the knocks" just allows them to carry on. Changing so as not to upset them makes them think they are right.
Saying "you've invited it" is victim blaming, whether you like it or not.
It is equivalent to saying that wearing a short skirt invites rape.
It is equivalent to saying being openly gay invites attacks.
It is equivalent to saying being black in a predominantly white area invites attack.
After all, all the above know that there are idiots who think they have the right to abuse those they see as different. Obviously the girl in the short skirt, the openly gay person, and the black person should all take responsibility for the abuse they get. They choose to put themselves in danger right?