Women's Rights and Trans Rights ARE in direct conflict with regards to women-only spaces.
This is not uncommon. Gay/Women's Rights and Freedom of Religion are also, often, in direct conflict. For example, women and gay men cannot become Catholic priests, bishops or the Pope. In this case, religious 'Rights' were priotised over those of women and gay people.
One trans woman in a women's prison, or women's changing room, or women's refuge, or women's Olympic 100m final, makes it mixed sex. The large majority of women do not want men (even if they identify as women) in these spaces. Include men means excluding these women. Why should men's rights be prioritised over the rights of women's and girls to well-being, fairness and safety in their own spaces?
Trans women aren't being excluded because they are trans, they're being excluded because their sex is male. Trans women were born male, and will always be male. Sex cannot be changed.
Trans women are men. The recent Supreme Court ruling is very clear that women's spaces MUST exclude all trans women. If a public space is advertised as, or has a sign saying 'Women' or 'Female', then the Equality Act 2010 requires this to mean only people whose sex is female.
This Judgement is a very good thing. I would divorce someone who prioritises men's rights to perform their sexual fetish in public, over the rights of women and girls to feel and be safe and comfortable in the public sphere.