Yes, it is MEN who are the problem. Regardless of their personalities/grooming habits/dress sense/inner identity/religion.
I've always felt that a big part of it is that kids for a number of years now receive a lot of educational content to the effect that it is discriminatory or racist to use the habits or tendencies of a group to make predictions or any kind of judgement at all about individuals.
It's been a talking point in progressive circles, things like racial profiling of terrorists etc, for 25 years, with progressives coming down pretty hard against people who would say its ok. But typically you see this going beyond them saying this is not ok - typically they also deny that it works.
Anyone who has worked in the police, or certain parts of the military, or for that matter in marketing or many other sectors, knows this is false. You can make predictions based on group category profiles and they are often very effective ways of making decisions or focusing effort. And that also means you can even improve your predictions about individuals to some extent, so long as you realise that you could also be completely and totally wrong about the individual.
Noe of that means it's moral to treat people on that basis, that's a different question. But for some reason, the ideas have collapsed - unethical and untrue have the same value.
I suspect this habit of thinking has a big part to play when people think about women's safety.