Daily, no-one is saying all men are rapists. Of course they aren't.
The problem is that a non-negligible number are, and we women cannot tell the difference. Men do not come conveniently marked as "safe", "bit of a cad but not actually a rapist", "full on rapist."
According to criminal psychologist David Lisak (who's done extensive research on date rape on American campuses), about 5% of men will admit (that's right, his methodology is based on what the men themselves will admit to) to being rapists. With an average of 6 victims each. Approximately 1 in 4 women has been the victim of rape or serious sexual assault.
I was alone with a man who had gone to great lengths to stress how secluded the location was, and that he wanted to have sex with me. I had repeatedly said I didn't want sex. He had repeatedly said he did (i.e. ignored my boundaries). If one adopts a Bayesian approach to belief formation, I'd started with prior odds that the chance of him being a rapist was 1 in 20 (which, I repeat, is small but certainly not non-negligible). His behaviour had led me to increase my odds to much higher than 1 in 20, because he kept pushing my boundaries and stressing the fact that there was no-one in earshot.