The problem we've got, which was quietly put in place with the GRA, is the idea that by saying he's a woman, a man becomes female, with the deliberately slippery 'treated as a woman for all purposes' wording. So if that's true, it's a single sex space even with him in it, as long as he has decided he's female. I'm not even sure he has to make this declaration public.
What we are seeing now is the collision between this as a idea on a piece of paper and real life, where it turns out that a man saying he's a woman is just a man saying he's a woman, people know he's a man, and they won't be forced to pretend otherwise. We were always going to end up here, at tribunals and in court when that legal fiction met cold, hard reality.
After all, the difference between male and female isn't trivial. The ability to grow new humans isn't inconsequential. It's not some silly thing that can be hand waved away. We've got centuries of evidence to show that women are restricted, controlled, punished economically, physically harmed and killed for it.