What you are now trying to do is conflate a philosophical argument with a real life one.
Philosophically, all female people SHOULD be able to use the female single sex space.
In reality, those female people have deliberately taken the decision to take testosterone which gives them male body cues that may cause some female people distress. Male people can do very little to disguise their male body cues even after puberty has been blocked.
So there is no symmetry to your argument.
That is why the point about female people who have taken testosterone and gained male body is a fallacious argument. There is no symmetry despite some activists desperately trying to make it symmetrical.