The brain thing all seems like a decades long No True Scotsman Fallacy.
For the uninitiated, the fallacy goes like this:
Person 1: no Scotsman puts salt on his porridge.
Person 2: Yerwot? My cousin Dan has salted his porridge every day for thirty years.
Person 1: Your cousin is it? Well, notrueScotsman puts salt on his porridge!
Possibly followed by:he should bugger back off to England with you!
Instead of questioning the original assumption in response to new information, fault is just found with later information to justify why it doesn't actually count.
At some point after some study or other, we developed some observations on how to distinguish a male brain and a female brain.
Male brains had this feature. Female brains had that feature. This was used to justify (in learned tones) why women were under-represented in well paid professions in various popsci magazines. Our brains, see? 
Then, someone points out that some male humans may have a neurological feature after all and it's:
"No true male brain has that feature. Bugger off, you're a woman!"
All seems rather daft. We all trace our origins back to the fusion of one egg cell and one sperm cell. 99.982% of human babies born were conceived with an XX or XY karyotype, by the way. And there has been no significant correlation between people with intersex conditions and people who identify as transgender, so let's not discuss that. People with variations of sexual development have said to keep them out of it!
So there you are, with a single cell that resulted from the fusion of the X egg and a Y sperm, rapidly undergoing cell division. Every cell is an undifferentiated, i.e. unspecialised cell, but each is XY in the DNA in the cell nucleus. Then as the weeks progress, cell differentiation starts happening, which means specialised cells, like bone, skin, muscle, pancreas andneuralcells start forming. Every single one carrying the same DNA instructions as that first cell, but following different sections of it.
Of course the male fetus has a male brain! Where would a female brain have come from? No-one would talk of a male baby having female kidneys if his kidneys were smaller than usual for a boy, would they?