It's probably impossible (at this time) to undo the knot of the many and various influences which are probably working at the same time, i.e., what is biology-based, what is cultural in a wider sense, and what is caused by more narrowly-defined social norms about how men are allowed to behave in various societies and who it is who is supposed to be responsible for that behaviour.
In some cultures, most probably, it has been women, ultimately, who have been held responsible for most types of rapes because women are expected to gatekeep sex and so the way women behave or dress etc. is put under more scrutiny, while "boys will be boys" is often the excuse in some form or another. This is still very much the norm in some sub-cultures and in some countries, too, which means that men don't necessarily expect negative consequences from their non-consensual behaviour.
I'd look at general violent crime statistics and sexual violent crime statistics to begin with. Men are more violent overall than women, and not just when it comes to sexual crimes. So men use physical violence more as a way of solving what they see as problems or as a form of satisfying their wants or needs.
But most men are not violent, so clearly whatever is driving this is not unavoidable or immutable, and a small percentage of women are also physically violent. Average strength differences matter here, too, and is one reason why angry women are less likely to punch a man who has angered her as he might punch back much harder. Words are women's weapons, more likely,
The differences in the rates of sexual paraphilias and sexual violence are also hard to parse, because girls even today are not brought up with the kind of entitlement to sex as many boys are. Female incels exist, probably in quite large numbers, but they don't gather online to hate on the whole male sex or to demand that the government gives them boyfriends or toyboys. I think this is because they don't think they are entitled to have dates or sex.
But there's probably a biological component to this, too, though once again, most men don't practise exhibitionism or voyeurism either. Some argue that female strippers are exhibitionists, but it's difficult to tell when they are paid for it if that would be one of the reasons they strip. I doubt it, actually.
It's so easy to forget how recent all the changes in how we view rape are and I try to remind myself of that, because many of the implicit values in our societies are still from the older era where how a woman dressed and how many sexual partners she had had affected the outcome of a rape trial where she was the victim of rape. If those values actually change then we would be in a better position to evaluate the question for these sex differences.
Though I'd also argue that if women could easily force men to have sex they would still have the concern of a possible unwanted pregnancy and the possibility that the men will physically fight back etc. Abstract desire is very different from actual deeds here.