I'm sure it does exist, and a percentage of trans people will genuinely be suffering from dysphoria, but at the end of the day, they are still the sex they were born and women should still be entitled to single-sex spaces that are actually based on gender, not sex.
I know two transwomen who transitioned way back in the 70s. Both are attracted to men and would be no risk at all to women. In fact, I'd have no personal problem sharing female spaces with them, but other women who don't know them, might not, whether for reasons of privacy or because they may worry about their safety around them, and that should absolutely be respected.
We can't just say, oh well these particular men pass fine, so they should be allowed to break the rules, We just can't.