Burke, you seem to be struggling with this so let me try a different approach.
It isn't all about you.
Really.
As a woman, I'm statistically more likely to be attacked by a man than by a woman. I am more likely to be sexually assaulted than a man, and that assault will almost certainly be by a man. If I am assaulted, I will be asked what steps I took to avoid the assault, and asked why I put myself in a situation where I might get assaulted.
So, while I know most men are great, I know also that this is a country with a well-documented undercurrent of rape culture and victim blaming. I know that statistically I am more am risk from men than I am from women, and I'm more at risk than men.
So yes, if it's late at night and you're the only other person in my train carriage, I'll move carriage. If you're waking down the street behind me on a deserted street, I'll cross the road. Like many women, I have been sexually assaulted. I have no desire to repeat the experience.
Because my evidence-based desire for self-preservation is now important to me than your desire not to feel like women are afraid of men in certain circumstances, and whatever detriment you feel stems from that.
Like many women, I have been sexually assaulted. I have no desire to repeat the experience. So until men stop being statistically more likely to attack women, I'll be avoiding 'risky' behaviours and situation.