I take your points, but this goes back to my earlier point about absolute thinking - i.e. all men are responsible for male violence, all Muslims are responsible for the misogynistic actions of a minority.
The only men that should face condemnation are those that commit the sexual crimes, and I don't believe they should be granted impunity due to some misplaced notion of political correctness or the views of ignorant people that would tar all with the same brush.
If their actions do bring scrutiny to certain cultural aspects of Islam then it's them, the perpetrators, that should be held responsible for this, not the victims that should be scared to speak up.
But it's an unevenly applied logic tbh because the slightest mention of 'not all men' invokes derision on here and nobody is saying that we shouldn't address male violence for fear of men all being tarred with the same brush. In fact, it's far more common to see the view that all men are responsible for calling out the bad uns and that 'if you're not helping, you're part of the problem' etc.
So why is it suddenly different when the demographic/protected characteristic is religion/cultural practices rather than sex? I honestly think it goes back to my previous quote about how we're conditioned not to see the oppressed as oppressors when in reality they can simultaneously be both.