Here's the whole text of para 61, carefully cut by GLP:
61. Whether different treatment is also less favourable treatment is, therefore, a qualitative question. In a case where the provision of separate lavatories labelled male and female was materially similar in terms of the extent of the provision, location, and so on, I consider there would, in principle, be scope for a strong argument that a rule or practice that permitted trans women to use the “female” lavatory but required other biological men to use the male lavatory would comprise different but not less favourable treatment on grounds of sex. However, the circumstances of the case would be decisive. (For the purposes of the EA 2010 the lavatory would be mixed-sex, but for the purposes of the Claimants’ submission in this case it would still be labelled “women”.)
This doesn't seem great - it would be deeply confusing for a woman to use a lavatory labelled "women" and then find men were allowed in - but the judgement is clear that this will not do if what the service provider needs to do is provide single-sex facilities.