Okay, I'll give this a go in the spirit of "giving a sympathetic hearing to your oponents". Apologies in advance - this will be long, and somewhat esoteric.
I think the fundamental issue is one David Hume pointed to back in the 18th century - what he called the problem of "natural kinds." We know that human beings love to classify things, and that classifying things has practical consequences. The question is whether this classification relates to how nature is or whether it reflects what we're interested in. For instance take the category "kinds of animals humans eat". In the UK, this doesn't include horses and dogs; in other parts of the world it does. But most of us would balk at including a cabbage in the list of "kinds of animals humans eat".
Now at this point you might say "well, science tells us the answer." But the problem is it doesn't, necessarily. There are two issues.
One is the somewhat esoteric issue of philosophical scepticism - if all our knowledge is mediated by what we choose to find interesting to look at (kind of confirmation bias on steroids) how do we know that we've identified the real natural kinds that are "out there" (and what does "out there" mean, and how do we get at it, independent of our experience, which is coloured and tainted by our preconceptions).
The other issue is a practical one. You sometimes end up with competing ways of doing the classification. Pre Darwin, biologists and "natural historians" relied on structural similarities/differences between animals to draw up the boundaries between species - Linnaean classification. Post Darwin, a lot of biologists suggest instead using evolutionary history to say where the boundaries lie. This difference of opinion lives on today and isn't as straightforward as you might think to resolve. Take, for e.g., one criterion which is "two animals are the same species if they can mate and produce fertile offspring." This becomes less-than-straightforward because of, say, ring species (e.g certain types of bird species round the edge of an ocean basin - A can interbreed with neighbouring species B, B with C, C with D... until we get all the way round the ring and find D can't interbreed with A). Because of this sort of complication, the argument about whether to classify based on structural "shape" or evolutionary heritage still rumbles on in biology today.
So if classification isn't straightforward even in the scientific world, what happens when we throw in the social world? Well, at that point all hell breaks lose. Social constructionists have loads of examples to point to - for instance from the history of psychology, where the decision to classify a certain set of behaviours in a patient as a "mental illness" often tells you more about the social and cultural background of the psychologists making the classification than it does about the patient ("hysteria" for example).
Now, you might think the hard sciences were immune from this, but in fact the history of science is littered with ideas that at the time seemed to have considerable explanatory value but then fell by the wayside (phlogiston theory, or the caloric theory of heat for instance). The trouble is it only seems possible to make the judgement about which ideas are good ones and which ideas turn out to be dead ends after the fact - there is no magic bullet of scientific methodology which will guarantee that the theory you're working on now will turn out to be one of the good ones.
Once you accept that you can't "see the joints" between natural kinds, they become up for grabs - especially when we're talking not about the scientific implications (how do I decide what a woman is in order to decide which subset of people to ask to donate eggs for IVF treatment for infertile women) but about the social and legal implications (all women shortlists, STEM scholarships etc.)
So I think the argument goes "There are no clear-cut criteria for identifying natural kinds - and that includes binary sex distinctions (hence TRA's emphasis on appropriating intersex conditions). And in any case, supposedly natural kinds are often socially-based categories of interest which tell us as much about social beliefs as they do about scientific fact. This being so, why shouldn't we define "woman" not according to our narrow and imperfect scientific interpretation of the word (science which comes loaded with a tacit and unexamined bias towards social conservatism), but instead in terms of other considerations such as creating a more tolerant, inclusive and open society.
Of course, as numerous posters have pointed out, in the real world you're going to go bankrupt pretty soon as a livestock farmer if you don't know which animals to breed with which other animals - both in terms of species and in terms of sex. But I think the old-fashioned sceptical/new version post-modern argument against the realism of natural kinds can be very seductive if you haven't actually run bang into the real-world consequences of sexism and realised that those are entirely driven by people correctly recognising your actual biological sex and treating you according to their prejudices about your actual biological sex.
Apologies for the slightly fatuous aphorisms:
"A boy is a boy for all his life..." ("Biologically" based societies.)
"A girl is a boy till she becomes a wife..." (Bacha Posh in Afghanistan)
"A girl is a boy if she's celibate for life..." (Sworn Virgins, Albania).
For the post-modernist, there is no answer as to which of these is right.