I agree with Latara that these men are bullies, and so pick on anyone they perceive to be vulnerable. That is not to say that the women they target have brought it on themselves in any way - but that it really has not much to do with their attractiveness. These creeps make themselves feel powerful by taunting women.
I'd love to read more about bullying and feminism, google just brings me guff about feminists being bullys, so if anyone has any suggestions it would be appreciated. It strikes me that bullying is so often used to keep us in our place, crabs-in-a-bucket style.