we had quietly accepted old school transsexuals using women's spaces (though many choose not to) and not see them as perverts/fetishists for living the way they want.
I think it may be truer to say that we had resigned ourselves to experience some men using women's spaces out of fear of the potential for male aggressive retaliation if we objected.
However, the Equality Act 2010 does give us the legal right to object to any perceived male in any of our communal spaces or services, etc via Schedule 3, section 27 [single sex services] subsection 6:
(6) The condition is that—
(a) the service is provided for, or is likely to be used by, two or more persons at the same time, and
(b) the circumstances are such that a person of one sex might reasonably object to the presence of a person of the opposite sex.
[See also section 26 [separate services for the sexes] and
section 28 [gender reassignment]]
www.legislation.gov.uk/ukpga/2010/15/schedule/3