I think it all gets messy. Just thinking about this issue:
If a man loses his penis in a terrible accident, should he now use the women's toilets and changing rooms because he is no longer a man?
If an old school mtf post-op transsexual later regrets surgery and wishes to re-identify as a man, should that person use the men's toilets, even though they had elective surgery to try to become a woman? If 'yes', why is that?
If that person then used the women's facilities, would women have the right to object? If yes, how would women know they had that right?
Does this all come down to the feelings in a man's head? Is a woman just a man without a penis? (A deficient man.)
What if a mtf transitioner really really wants to have surgery but can't, because that person has a medical condition which means that such surgery would be very dangerous for them? The hormones they have been taking mean that their penis is dysfunctional anyway so they couldn't rape anyone - should they be an exception to the 'no penises' rule?
What if the above hypothetical transwoman dies on the operating table because they thought it a risk worth taking. Would that be the fault of women for taking such a hard line and insisting on surgery before allowing access to their spaces? Or would that be the fault of women for not taking a hard enough line by allowing post-op transwomen into their spaces? Is this the fault of women at all, or is this the fault of transwomen demanding access to women's spaces in the first place?
When, exactly, do women have the right to say no?