Using sex-accurate pronouns to refer to somebody is often defended by it being factual reality. Even though the pronoun is a 'name' that the person does not want to be called by.
This calls to mind 'Yid' being used as a derogatory term for a Jew. Something that has been defended by "that's what they are" and "that's what they call themselves". Which is true. The word for Jew in Yiddish is 'Yid'.
Why is one acceptable but not the other?
For full disclosure, I am a gender critical Jew from a Yiddish-speaking background (though I don't speak it).