It is not a comparable situation. You are right. Yet we keep seeing this very basic error being made. The difference is one of legitimate and illegitmate discrimination. And actually equality too.
People who are same sex attracted (or to both sexes) were subject to illegitimate discrimination when people declared they should not access toilets of the same sex. For clarity: male people accessing male toilets if they were a homosexual male. They were not asking for special treatment under any safeguarding principles or indeed, in life. They were asking, and rightfully so, for EQUAL treatment and equal protection.
Male people demanding access to female single sex spaces are demanding ADDITIONAL accommodations be made because of their gender identity. This is not a demand for EQUALITY.
It was always wrong to deny a person who was homosexual or bisexual access to a facility that matched their sex when that discrimination was based on their sexual orientation.
For instance, one group of male people have no extra rights over other male people in demanding that they have privacy involving being completely separated from other male people in a facility designated as single sex for male people. Again, singling out same sex or both sex attracted people to be excluded, would be illegitimate discrimination.
This is, of course, not the case for excluding all male bodies from female single sex spaces, except for male children under a particular age who would require care from a female person.
It is NOT wrong to exclude male people from a female single sex space because they are male. There is a necessary form of discrimination that is used for calculating safeguarding risks which is based on sexed bodies. The only 'negative' impact it has is to exclude one sex from a space designated as single sex for safety purposes. This discrimination has been the basis of sex segregated spaces since those spaces became available to public life. Despite the deception that some male people used to gain access and this deception being wedge to leverage a false argument that female people 'never had toilets without male people accessing them'.
Male bodies are excluded from female single sex spaces also on the grounds of privacy and dignity. Not just safety.
Homosexual male people were wrongfully discriminated against based on no statistical evidence at all, just prejudice. This has, rightfully, been prevented with law. Because it was statistically inaccurate.
The fact remains that the comparator should start with 'does one group of male people have a lower risk profile compared to ALL other male people in the UK' ?
The answer for gay male people is 'no' and it was likely always no. Besides, they were not seeking unequal treatment.
If you are arguing that any male person over about 8 should be included in any single sex female space, you should need to provide evidence that the group of male people you are advocating for have not just a lower risk of committing sex crime than all other male people in the UK. You also need to provide evidence that they commit sex crime at the same or lower rate than all female people in the UK.
It is NOT wrong to exclude male people from a female single sex space because they are male. The protected characteristic is SEX not GENDER in this instance.