I don't have a black belt in neuroscience😏but my understanding is that babies' brains are not blank slates, they are actually full of complex interconnections, and as the baby grows, some of those interconnections are strengthened [e.g. I'm guessing: the ones to do with walking upright, and talking]
and less or not used interconnections weaken and maybe even atrophy.
So those 'pink' and 'blue' messages that babies get from day one - as has been shown in experiments where the same baby was interacted with differently, depending on whether it was said to be a boy or a girl - reinforce pathways in the brain.
I wouldn't be at all surprised if they found differences between adult men's and women's brains, given the differences in how boys and girls are brought up, as GriseldaandMike says.
Some boys and girls resist the stereotyping more than others, and have experiences that are more related to the opposite sex in childhood - tomboys like me who played with Meccano sets and Airfix kits and learnt skills - and possibly brain connections? - that enabled me to work in a techie field as an adult.
As I write that, I realise that it may be less acceptable for boys to crossover like that in childhood, and have 'non-boy' experiences, because of the lower status of 'feminine' things. Perhaps that's why there is a greater spread of 'gender presentation' in women than in men, some of us are womanly or girly and some of us are not, whereas there isn't such a wide spread of difference in dress etc amongst men.
And perhaps - this is just speculation - fewer diverse interconnections in adult men's brains?
But that's not the same as saying there is a 'male brain' and a 'female brain' and that there can be mix-ups in the assembly area resulting in a male body getting a female brain or vice versa.