the conflation of sex and gender seems mad to me.
Of course it is. But it's what a lot of people want more than anything in the world. I suggest picking up Helen Joyce's Trans, which takes you through the whole history of this.
And of course you're transphobic. Everything is transphobic. And that list was from, so this won't even remotely cover latest developments.
Mind you even if not conflated, separate sex and gender doesn't make sense. Looping back to this:
it's probably fair enough we recognise them as their assumed sex.
What does that mean? In what circumstances might we want to "recognise" someone's sex, but not care about their actual sex?
That's pretty much a paradox. If we don't care about actual sex, then we shouldn't be caring about sex in the first place.
It would have to be something truly "gendered" in the feminist sense, where there were different expectations of men and women, but to no good reason. Like wearing different clothes, or women not being permitted in a certain profession.
In which case the correct solution is to remove that sex-based gendering altogether, not just allow some people to cross the gender line.
But if the line remains, and we think it's good that the line remains, then the denominator has to remain sex. In sports, privacy, safety, etc.
That paradox is what made sure I never really accepted "gender" in the first place - I couldn't think of a single case where I should care about someone's "gender".
Maybe pronouns are the sole fuzzy area - something seemingly inconsequential as a politeness. (And you don't need sexed pronouns - some languages lack them). But even then, they're a slippery slope, as per Barracker's essay. And even if you're prepared to do it as a politeness, the moment that politeness is demanded, everyone should refuse to comply on principle, IMO.