I agree there are biological differences between boys and girls, just as there are between children of different sizes, different ethnic heritages, those with disabilities or chronic illnesses and those who aren't, etc.
My concern is that so much emphasis is placed on the biological, "innate" differences between the sexes that isn't based on fact. It's what people "feel," but even how people feel is bound to be based on their own conditioning.
Although it might sound outlandish now, it was quite common a century ago for people to firmly believe that biological differences between black and white people meant each possessed differing innate characteristics (e.g. White people more intelligent, less violent, better leaders and organisers and Black people less intelligent, more musical/rhythmic, achieve more by being directed, etc.) People genuinely believed these things, but their believe did not make these things true.
Similarly, biological difference was often used as a means of marginalising and excluding women again, without any hard science to prove it. Too much education would damage their smaller, more fragile brains. Too much self-determination would damage their reproductive functions. Even now you still get some people who genuinely believe that because women have monthly hormonal fluctuations that their mental capabilities are inferior to men.
I think it is important to base scientific arguments on scientific fact and point out where there is evidence that something is socially conditioned, not innate. Otherwise, biology will always be used as a tool to oppress and exclude women, just as it has been used in a similar way against people of colour, Jewish people, disabled people, etc.