What everyone has said already.
The point is that female and male are reproductive categories. Sex determines who is a man or woman, not culturally coded "gendered" personality traits or behaviour. Your brain can't be in the "wrong body" anymore than you liver can.
What a brain scan shows means absolutely fuck all to the organs of the reproductive system.
Remember the research on brain scans of London taxi drivers, which changed after they passed the Knowledge? I like it, because it reminds us of how the environment affects the appearance of the brain. Frankly, the idea of a "taxi driver brain" makes more sense than a "brain sex," given the cultural influences/oppressive nature of gender, confounding variables like mental health issues, sexual orientation, etc.
We found that compared with bus drivers, taxi drivers had greater gray matter volume in mid-posterior hippocampi and less volume in anterior hippocampi.
www.ncbi.nlm.nih.gov/pubmed/17024677
With the new research, scientists can definitively say that London taxi drivers not only have larger-than-average memory centers in their brains, but also that their intensive training is responsible for the growth.
www.scientificamerican.com/article/london-taxi-memory/
If the argument is that brain scans are what determine completely unrelated factors, does that mean we should now determine who is a taxi driver based on their MRI? Or can we keep going by things like driver's and taxi licenses before we step in a cab?