Isn't the main similarity the oppression aspect?
If men and women were treated exactly the same, and there was no such thing as patriarchal control, there wouldn't be such a problem.
The same with racism. If it didn't exist between blacks and whites, the power dynamic, which in this case allows a white person to identify into the oppressed group, wouldn't exist.
It's less to do with proving the actual, physical differences, than understanding how the differences manifest in a power dynamic.
Everyone is extremely aware of the power balance in racism. It's an incredibly prevalent topic. Hence the outrage of people like Rachel Dolezal.
And although the power dynamic between men and women is, of course, recognised, it's not got the traction that should have, and it's in the interests of half the population to keep it going. By normalising it.