Sorry for the long post but I do not think you are right, op.
I have four brothers and no sisters.
Two of my brothers never shared a punch in their lives (now in their 40s).
The other two got in a few scraps when they were teens/young adults, but neither they nor the other lads got into serious harm.
One of those two was followed home by three lads who jumped him and beat him up, leaving him bruised and bloody but intact - the scariest incident involving my brothers.
I was raped by a friend when I was 15, and by a stranger when I was 17.
I was followed home from where I got out of a taxi when I was 23 (a distance of 50 metres) - I wasn't quick enough getting into my apartment building and the prick broke my hand by slamming the door on it, then he knocked me to the floor and tried to assault me but I screamed with all my power and hit him across the face with my keys. He took my bag and ran off.
Another time, I was mugged and sexually assaulted while walking home from a pub, at 9pm.
My friends and I have been groped so many times while out, I couldn't try to give you a ballpark number. I didn't dress evocatively, but I'm small and busty and my friends would tell me I have a sweet, innocent look about me. Maybe that was it?
However, most recently, now 38, quite frumpy for my years and wearing a face mask, I was followed from the supermarket by a man who would not take the hint, even when I sternly told him I wasn't interested. I had to take a circular route around my home because I was afraid of him finding out where I live. Eventually, I approached a random group of people and asked them to pretend they knew me as the fella behind was freaking me out. They kindly obliged and waited until he gave up and wandered off. This was at 6 in the evening.
I don't like going out alone anywhere, at any time of day. Im hyper-aware of my surroundings and my skin prickles when I feel someone walking behind me. I've talked with my brothers about this and they can't fathom it.
All these attacks, bar one or two non-threatening but uninvited gropes in the ladies room, were by men. Women do not make me fear for my life or safety.