"They would need to argue male and female were also wishy-washy concepts"
They did that, with bells on.
Their stance is effectively sex has always included gender ID within its definition, and if it didn't, then it does now as things have moved on, and if you think they haven't, where have you been? This is all settled, can't possibly think why you think otherwise.
I'm paraphrasing massively but that's the gist of some of their arguments.
One of the points made I think initially in the 1st round but it was mentioned again today - when Scotgov declined to amend the census act to specify sex to include gender identity, because the minister said at the time they didn't want to conflate sex with gender/gender ID, apparently that was because sex was already (settled) as conflating sex & gender & includes gender identity within the definition of sex.
And I hate to point this out, as I know it's been a means to push back on self ID, but the acceptance of legal sex as the settled agreed remit of the notion of sex, is ultimately what was used to undermine arguments that sex, for the sex Q in the census, was always and should continue to be considered biological sex. Because the concession was already made that legal sex was what they census Q was asking all along. Which since the GRA 2004 act includes male people who obtain the legal fiction that they have 'acquired' a sex different from that on their original birth certificate.
The point was made that sex isn't immutable because it was accepted that sex can be changed via a GRC.