It's also important to remember we are talking about different men and different demands.
I'm sure a handful of relatively old school transsexuals and possibly transvestite did use women's bogs and were either not clocked or quietly ignored.
However, the evolution and convergence of transexualism and transvestisism from personal secrets into political and social identities, and with that what I'd call the agressive/colonialist trans identifying men, has increased both the number and diversity of men who wish to use women's facilities.
And these new men, the ones who call themselves transgender and see their existence as a political statement as well as a practical fact, don't just hope for their presence in the wrong places to be quietly ignored as a courtesy. They want an absolute right to be there and for women to know they have that right, that power over us and know that we can't object or do anything about it. They see women asserting our own boundaries and rights as a challenge to them and they want to win.
That is a massive change.
So the TRActivists arguing that sex doesn't matter now based on what might or might not have been accepted or at least ignored 30 plus years ago for a completely different group of men (some of whom of course were very problematic towards women and women's spaces even then) are at best ignorant and at worst down right deceitful.