If there was genuine evidence that it can actually be possible for a "male" brain to exist in a "female" body and vice versa (which there isn't - what you have got is a wide spectrum of brains, which have a lot of overlap and are impossible to define as "male" or "female" unless you use sexist stereotypes as a baseline, and which can exist in either male or female bodies) then that would certainly be enough to have some kind of reasonable legal status as having their chosen Gender if there were some kind of externally verifiable test that can detect and reliably measure someone's gender. I do not believe such a thing could possibly exist without accepting and codifying sexism at a deeply abhorent level so I am not advocating for it to happen.
However, that Gender still does exist in the body of the sex that it factually is. And a lot of the time when things are different for male and female people, it is because of the body not because of the brain. If the brain is practically, legally and emotionally female but the body is male then that male body still needs to be kept out of female spaces, and the female brain inside that body would need to accept that - obviously provision of 3rd space unisex options is important for such individuals. There is no surgery or drug treatment even theoretically that could make a male body into a female body, it will only ever be a male body that has undergone surgery or drug treatment.
The concept of TWAW and TMAM requires for this to be true from the moment someone first declares themselves to be Trans. Being trans is not a status that is earned after sufficient time and money has been spent on achieving a body that "passes". Any definitions for what makes it OK for a male to be considered female which require certain treatments to have been completed creates a pressure to railroad someone towards those treatments, even if they aren't in the best interests of the individual, so any such criteria are intrinsically unethical.
I could imagine a utopia in which all patriarchy and sexism has been eliminated, sexual predation has been eliminated, there is no more male violence against women, rape is unheard of. No one would touch someone's body without consent, it would just be "not a thing" - like trying to pick someone else's nose, no one would even try, the idea of trying would provoke disgust in anyone who you asked to do it. No employers ever make decisions about employment or promotion with the slightest amount of unconscious bias about a particular person being potentially more likely to need maternity leave, and no one struggles to get back into their professional career after a break for child rearing (obviously both male and female parents being equally likely to take such a break). In such a utopia perhaps there would be no more need for all-female spaces to protect female people. Even prisons could be fully mixed sex throughout because even criminals (of which there would be a lot fewer, if all male violence issues have been resolved) have sufficient deep respect for women that women are no more vulnerable than men. As are sometimes depicted in futureistic sci-fi tv programmes, men and women just get changed together and are spectacularly unbothered by naked bodies of the opposite sex around them, no one feels vulnerable because of this and there is no sexual banter triggered by it. In such a utopia, then yes perhaps a "man" could be considered a "woman" - insomuch as the words have any meaning really - socially in such a utopia there would be so little differences between the sexes that the words would be pretty meaningless - but you would still need a word in medicine for "the kind of body that is organised around developing a womb and ovaries (even if they didn't develop quite as standard)" and "the kind of body that is organised around producing sperm (even if the organs didn't develop quite as standard)" and it still wouldn't be possible for someone born with the first type of body to become someone who has the second type or vice versa - and perhaps that would be just a matter between the individual in question and their intended co-parent if they were trying to have a child (imagine the consultation "ah ha - I have discovered why you are having so much trouble Mr & Mrs Smith - it turns out you are both producing sperm!") and their medical caregivers (especially for diseases like heart problems and cancers which manifest differently in male and female bodies). But on a day to day level - the standard for me to change my mind would be "when there is no longer any point reserving anything just for female people because all disadvantages encountered by female people have been 100% effectively eliminated and there is no longer any advantage to being male in any circumstance".
Except that standard could never be reached in sport, not without genetically engineering the entire species.