You seem transfixed on this belief that we find these men a 'threat' and that's our only issue. Let's say your daughter is the best female runner in the world, she has spent her entire life training, sacrificing and giving her all to become the best athlete to ever win the gold medal. But she doesn't, not because another motivated female runner pips her to the post but because a mediocre male runner 'transitions', using his male muscle mass, his male stamina, his male strength to beat her. But it's okay because he's no 'threat' to her. Is that fair?
Your daughter goes out one night and is violently gang raped by a group of males - she makes the brave decision to report it and will undergo a series of invasive, internal examinations. She's now, in that moment, frightened to death of men and requests the examiner to be a woman. But oh no, in walks the 'female' nurse, who is clearly not female but identifies as such and there is nothing your daughter can do about it. So, in her panic, she refuses her examination, retracts her statements and her rapists walk free. Is that okay?
It's not just about us being able to piss in our own toilets, the more we give away the more they will take.
It doesn't matter if there are a minority of these men in women's spaces, if even just ONE removes the dignity of a woman, the chances of a woman, the right to a female space from a woman then it is one too many.