All men are more at risk of man-on-man violence than women are.
And lots of subgroups of men face higer risks than the average man.
Do you want to let them all into the ladies for their own safety? Or is it just the men who claim they are women you think shoudl get extra safety, even through outside their own minds they are no more like we are than any other man is?
You need to realise the only reason it has ever been suggested these men need to be treated differently is that they claimed the word "woman" to label their own self image.
That's all it is. All it ever was. The surgery and all that, for the few who even do, comes afterwards. It's a symptom of their feeling like a woman, not the thing that supposedly makes them "a woman".
Their claim of womanhood is not something that can be empirally, objectively detected. It can't be verified. It can't even, as you yourself I think already said, be described. We have no idea if the feeling that one trans "woman" feels very strongly makes him a "woman" is the same feeling another trans "woman" has, or something totally different that two of them both happened to have projected onto the female half of the species.
All we know is that whatever it is, it's not the same thing as being female and I have never ever ever heard a reasonable explanatiion for why we are supposed to be treating such men as if this fact of their minds is somehow completely the same thing as and interchangeable with the fact of our bodies.