"Also remember that the studies of trans people's brains are tiny. "
Same double-take for me as well!
As well as being small sample size, one study I looked at that claimed to find trans closer to their identified wotsit than their actual also made the classic error of using the SAME set of subjects as training set and test set. Basically, take a smallish set of normal males and females. Measure the size of a whole bunch of brain regions, and pick the one (or combination of measurements), that best distinguishes the males in your sample from the females (that's the training of your wonderful male or female brain classifier). They then used that on the brains of trans folks, and claimed that TIMs had brains more like females, and so on.
Except you FIRST have to show that your wonderful male or female brain identifier WORKS on non-trans brains other than the ones you used to train it, that it does what it says on the tin in the first place.Only then does the result on trans brains possibly mean something.
With small datasets and measure enough stuff, you're very likely to find stuff that just by chance can distinguish your particular set of males from your particular set of females. And if you add OTHER data (like your trans test set or even a non-trans test set), then it is quite probable that it will tend to lie in between the extremes of the groups you trained on (just since the measure was specifically chosen to separate those particular groups anyway!).
For these reasons, I tend to stick to the recent larger samples work that seems to be showing that actually there isn't a meaningful categorization of brains into male-type and female-type. Although unfortunately the New Scientist report on this work took it is a 'gender is non-binary' result. Although at least based on that criterion we are almost all non-binary in terms of our 'brain-region-gender', so why bother with the concept at all?