If we had clothing for men and women that was identical in every way, except men wore shirts with four buttons had shirts with five, cross-dressers would want more than anything to have the shirt with five. We don't know why."
So he's answering the question "if things weren't coded feminine or masculine, would we still have crossdressers?" by saying "if we have just one thing coded feminine or masculine, crossdressers would still want the feminine-coded thing."
How does that in any way shape or form address the question properly?
How can you refute the idea that if we didn't have X, then people would not do Y by saying even if all we have is a tiny little bit of X, then people would still do Y.
Well, yeah, they would. We're asking if we have no X, not even a tiny remnant, what would you attach that paraphilia to?
I guess the female reproductive system and the roles and abilities arising from them. But you'd have no way to emulate them in a way that would signal your cross-sex identity to others. Unless you were full on pretending to be pregnant. Which would rightly elicit mental health concerns, as it does when women pretend to be pregnant.