I don't think the anti-GC argument comes from there, to be honest. Yes, to be sure, there are many shades between biologically female and biologically male. Yet, most people are grouped at the poles.
The anti-GC, pro-trans argument comes from a point of kindness and pragmatism. That is, it recognises that, transwomen suffer, and their suffering is a great deal more than the suffering of those who, collectively, claim to be 'oppressed' by transwomen. Most of those haven't ever been affected by a transwomen. A large number haven't even seen a transwoman, and known her as such.
We are in the strange - some would say outright perverse - situation of people arguing, ferociously, that the tiny few transwomen there are threaten us, in our toilets, changing rooms, and in our sports, because we're so fragile as natal women, and those possessing (or who once possessed) penises and testicles, are so powerful.
Go figure. I think, perhaps, we haven't seen off these transwomen and their supporters because we haven't burst into tears and lamented about how threatening they are to us ... despite our outnumbering them by 80 or more to one ... and that's why the powers-that-be haven't listened to us.
We need to cry some more. That's the answer!