Okay Ralf, let's talk neuroscience. From what I understand (admittedly I am a theoretical physicist rather than a biologist, so this is from reading people like Cordelia Fine and Lise Elliot, not from reading the original research papers - I don't have a subscription to pub med sci), where there are measurable differences in population means between men and women (a) it still doesn't answer the nature versus nurture question because the brain is incredibly plastic in the first five years of development, and there are numerous psychological studies showing that adults treat even very small babies differently depending on what they believe to be the baby's sex (in these experiments the colour of the baby grow is deliberately chosen not to give the adult any reliable indication of the child's sex); and (b) even where there are differences in the means, the d values for the populations are incredibly small.
What the small d values (difference in means divided by square root of product of standard deviations) indicate is that, for instance if you have a small difference between the average performance on a maths test at age 8, say, you might well still have 45% of girls out performing the average boy. The two distributions overlap more than they differ. Now think about the sciences that depend on doing maths - physics and engineering (and biology and neuroscience for that matter - you need to understand stats well enough to understand the difference between population statistics and sample statistics for a start, something that a lot of pop science article writers clearly don't). Do you want a publisher saying "I know, let's publish 'The big bumper book of science for boys'?" Because I don't!
Going back to the question I and others have asked: who will be harmed if those labels are taken away? I can tell you who does get harmed by the presence of those labels: women like me and Fuzzpig who go through our whole childhood, adolescence and adult years being told by a substantial proportion of adults who should damn well know better that we can't be proper girls/women because we like the wrong things. And equally (in fact, probably for once more seriously) boys/men who like the wrong things. Because our society is much more forgiving of non-gender-conforming women than it is of non-gender-conforming men (pace usual experiences whereby you have to struggle every fucking inch of the way to do something society doesn't approve of you doing, like mechanical engineering).
There are publishers out there doing excellent work. I've just persuaded a friend of mine to get into climbing (and by extension her DS, who is the same age as mine). She got a book out of the library with probably equal numbers of women and men climbers, including an Indonesian woman speed climbing champion who climbs in a hijab.