Either people are allowed to wear what they like or they aren't.
No, it's really not that simple - much as "free speech" isn't an either/or situation.
You can wear (and say) what you want within the laws of decency, not inciting racial (or other) hatred, not committing sexual crimes which include for example exposing yourself, or for example behaving in a threatening way or deliberately scaring people (which could be dealt with under various laws like breach of the peace).
You don't get to behave in ways like these and then say it's fine because you can "wear what you want". If you used what you were wearing to effect your crime, then no it's not OK, or legal.
We have courts of laws to argue these details and ascertain the law in each case. We don't say "you can wear what you want and that's clear-cut so everything's fine".
Being suspicious of, and scared by, someone running around in a gimp suit, jumping out at people and wanking in public is a normal and reasonable reaction. It's not somehow mean to poor Mr gimp to say he can't do that.
And what does it have to do with feminism? Women can't "wear what they want" either. Go out naked, you'll be arrested. Wear a t-shirt calling for an ethnic group to be eliminated, you'll be arrested. Rightly so. It is utter bollocks to think feminism is about being cool with women - or apparently, anyone - doing exactly what they want.