This is the rule used by some sadly.
A prisoner found guilty of murder secured in the highest security male prison in the uk, was afforded protections from other inmates because they feared rape. It wasn't clear whether they meant that because they wanted to be viewed as a woman men would want to have sex with them, forcibly, that was my interpretation of it, in the absence of any other explanation tbh.
The prisoner wore their own, non-gendered, clothes, but had grown long hair and nails. Maybe theybfeot this made them vulnerable to rape idk, but the prisoner was carefully treated separately to everyone else in there and give ven privileges such as roaming free around the prison when other cellmates were all locked up normally during those hours. This requires costly additional security measures.
They got out and womens hostels would not accept them as a result of them 'not passing' sufficiently well as female.
What difference does this make? I've known men that could carry off as female at a fancy dress party. As in, seen them dressed as such for a party but was not any leap to see how they could do this in reality, depends on the physique combined with voice and mannerisms.
Some that have high voices and more typically female gendered mannerisms can still be and want to remain very much male happily. Its just not feasible as a reason.
On the basis of 'passing' which is horrible, I think very few would actually be accepted as actually female within an all woman environment.