The existence of the single sex ponds is a derogation from the Act - permitted sex discrimination for the purpose of a legitimate and proportionate aim ('LAPA' - usually about safety, decency and/or fairness).
A transwoman with male legal sex (no GRC), if excluded from the ladies' pond, cannot claim sex discrimination because of the derogation, and cannot claim discrimination on the basis of gender reassignment because he's treated the same as the other men. He can however claim indirect discrimination on the basis that he doesn't have a single sex pond to swim in like the other men (GR discrimination) or like the women (sex discrimination).
A TW with a GRC has female legal sex for discrimination purposes (assessed as relative disadvantage, not non-identical outcomes). Excluding him from the ladies' pond is direct GR discrimination.
So they get you coming and going. Stonewall build on this by teaching that excluding TWs is always illegal discrimination, that it could only become legal if there is a LAPA (the LAPA exception applies to all forms of discrimination) but that this virtually never arises (the bar is too high).
They never ever suggest that the answer would be to avoid disadvantaging TWs in the first place (eg provide a TW's pond). And they ignore indirect discrimination against Jewish women who no longer have anywhere to swim at all (because TWs remain men for real life purposes). Obviously, letting Jewish women swim would be a LAPA, but they are outranked by TWs in Stonewall-Land. I suppose they should get their own pond. 😡
I apologise for the above being so long (and not guaranteed to be correct). I have been trying to work out what single clarifying amendment to the Act would cut this Gordian knot, but it's hard. Because the Act is trying to do something that goes against all logic, create a space that's single sex and mixed sex simultaneously.
(You could - not unreasonably - point out that if excluding men fulfils a LAPA, then that LAPA can only be fully met by excluding all men, including TWs, who are men. But that's fighting talk!)