Well their isn't any difference from their own sex but Sex and Gender are 2 separate things.
Indeed. Which is why where sex is relevant it takes priority over gender. Toilets, sports, prisons, women's equity opportunities, medicine, etc
Unfortunately, the only situations in which gender is relevant are either completely internal (ie how the individual believes about themselves) and in stereotypes.
We do not accept that stereotypes should control our lives, nor do we accept that an individual's beliefs should be imposed on others.