All of that is the true.
It would indeed be possible, in theory, to measure multiple neurological functions of the brain A-Z, find what the average for each function was for male and females, match them to each other and call one the average female brain and the other the average male brain.
But we can't. Because none of the averages match up with each other. Somebody who has an average female brain part A is no more likely to have an average female brain part B, or C, or D and so on. And at any point, under the right conditions, the individual with average female brain part A can experience a life event which turn out that average female brain part into an extreme male brain part. Which part are we going to pick to define the brain as female? A, C, U, Z? None of them correlate to each other so you can't make an average of the different parts.
If an average female brain was like average female height, height would work like this:
There would be no correlation in an individual between the size of different body parts. It would be entirely normal and common to be five foot eight inches tall, but have size one feet, the world's smallest torso but an exceptionally long neck, while other people who were five foot eight would have size fifteen feet, tiny legs and an enormous torso. It would no longer be possible to calculate stature from leg length, or spine length, because nothing would be related to each other. Because no individual's body parts would match to any other body part. And also, people would routinely go to bed one night and wake up the next to find their legs had grown six inches in the night but their torso had shrunk two inches. They might never change back, or they might change back at some random point.
Who in that situation has masculine legs? Who has feminine legs? If no body part correlates to each other or to overall height? It just doesn't make any sense as a concept.
Particularly when we do use the words male and female to refer to things that match with each other 99% of the time, genes, gametes and gonads.
No neurological function has any great amount of accuracy in identifying sexual dimporhism and they don't match to each other.