If someone is behaving in a pervy manner in a toilet or changing room of course you can get them removed
Assuming that you are an adult, you recognise the pervyness before it's too late, and there's someone in authority nearby to do the removing. Seriously, do all the public loos you use have an attendant present? The ones I frequent don't.
I was sexually-assaulted at school aged EIGHT, because two older boys waltzed in to the mixed changing room and I did not realise that they meant me harm until they had cornered me and it was far too late to escape or scream. You have to remember that CHILDREN also use toilets and changing rooms and may not have parents with them (e.g. at school). Children also need simple rules to follow to protect themselves, such as "if you see a man or teen boy(1) in the toilet, scream and run". This is in direct opposition to Stonewall/UWE/etc advice of "if you think someone's in the wrong loo, relax, they know better than you do".
This isn't a "moral panic", it's a reasoned fear that comes from recognising that males use their penises as weapons against females and force us to risk pregnancy. This is looking at Yaniv and Jess Bradley and Karen White and asking "how is an eight-year-old meant to know the difference between those predators and Rose of Dawn or Miranda Yardley or Seven Hex?" (the last two of whom have asked themselves similar questions and stopped using women's facilities 
). This is accepting that there is a risk associated with letting unvetted male-bodied people into female spaces, asking "how many girls do we allow to be harmed as collateral damage for transwomen's validation?", and concluding that the only acceptable answer is zero.
I've concluded just now that "moral panic" is one of those "thought-terminating cliche" things we talked about last year, with the purpose of silencing a very specific target group, namely women concerned about safeguarding. It does this by the mechanism of shaming the target through the slurs of social conservatism in the word "moral", so very effective against social liberals as many gender non-conforming women are, and irrationality in the word "panic", so very effective against women, who know the root meaning of "hysteria" and are sick of being branded irrational.
We should always question the motives of those who would shame us and silence us for being concerned about safeguarding.
(1) I'm all in favour of boys under eightish using the ladies if they aren't with their dads. They are too small to threaten us and are at risk from men if they use the gents.