It isn't sexuality, but sexuality plus power, in the same way racism is prejudice plus power, rather than being equal in all directions. Lesbians don't have the power men have or conditioning to take what they want.
Men feel and behave (and are told by society) that they have an entitlement to female bodies, that our bodies are theirs to judge and they know they can easily over power us.
When our local high school made the toilets unisex, it was the boys mocking any girl who "pissed like a horse" while they got away with pissing like a faulty, unhygienic sprinklers. That is a power dynamic, the girls don't have the same power to mock actually gross behaviours, while the boys set the standard for female bodily functions and girls with healthy strong pelvic floors stop drinking to avoid the toilets.
There are also a number of patriarchal religions, telling women the onus is on them to stop their bodies being seen. This hasn't completely gone from any society, even non religious women know they'll be held responsible for their own vulnerabilities to assault.
It will be a long time before we reset that as society, and doing away with safe places will just set that back, it erodes boundaries.