I have several friends who are women and 6 feet tall. None of them have ever been mistaken for a man.
I have however been mistaken for a man in ladies toilets and elsewhere on more than one occasion.
I've said this on Mumsnet multiple times. Last time I was mistaken for a man in a ladies loo, a lady said to me, excuse me this is the ladies. I looked up in order to say to her, I am a lady but as soon as she saw my face she said 'oh sorry love'. I didn't need to say a single word.
That was it. There was no bullying, she didn't eject me. I'm not very feminine looking, if you put me in one of those sex changing photo filters, I look the same.
However, if you take a single second to actually look at me, it's obviously i am a woman from my height, facial bone structure, the way I walk etc. That's why the women in the toilets could correct herself so very quickly, because in the vast vast majority of cases we can all tell.
The incidences of women being bullied out of toilets must be vanishingly rare indeed, and even if that does happen very very rarely it is a small price to pay to stop women being abused by men in their safe spaces.