My issue with gender being the crucial category, is that it's apparently: fluid, discernible only to the individual themselves, and is entirely undefined/unrecognisable.
Someone may wear jeans, or lipstick, have hair long or short, work as an astronaut or a midwife- none of which accurately signifies their gender. It's so nebulous, a combination of various markers (lipstick, long hair and a handbag- but they are wearing trainers?!)
Sex is pretty clear and definitive. It has implications. There are vanishingly few people whose sex is not easily discernible- a very small number of them have a sex differentiation disorder, which is a medical issue usually requiring treatment.
I am happy to accept people as whatever gender they wish to tell me they are. I'm not terribly interested. They can present however they like. It's like being goth, emo or punk. We don't collect statistics on the oppression of Goths, punks, emos...
Their sex however matters- it matters medically, it matters in statistics, it matters in prisons, changing rooms.