Clothes send signals about how you want to be seen by others. A bald, goateed, heavily tattooed man in a wife-beater is infinitely more scary than a man in a dress, whether his legs are muscly or scrawny (or tattooed).
Maybe the tattooed guy is actually a nice harmless chap. (Most punks I knew back in the day looked scary but turned out to be really sweet, unusual people.) In an ideal world, sometime in the distant future, a woman will draw no conclusions one way or the other about the wife-beater guy. But that is then and this is now. So why does he choose these clothes that have that stereotypical information attached to it? At what point does the information become communication? Maybe, if he wants to re-assure women (and others) about his character when he's out and about (at home he can wear what he likes... unless his wife/girlfriend doesn't like it), he'd be better with a an ordinary T-shirt or even a tailored shirt.
So, guy in a long dress at an awards ceremony: no problem. That's a very specific, limited social situation. Guy in the chemist's or supermarket in a dress: bound to draw attention too... but we don't know what message he is trying to send; all we have is the information that is stereotypically associated with a man in a dress in a public place, which leaves women, in particular (because it is their dress he chose to adopt), wondering what his purpose is in wearing the dress.
So yes, suggestionsplease, we aren't mindreaders. Which is precisely the point. If a man is concerned to respect women, their dignity, their boundaries, he will not dress in public places in a way that carries either the information "Likes physical violence, including against women" even if he isn't a violent person who abuses women. If he does wear these clothes, then he must take the consequences, including women avoiding him.
Analogously, the public dress-wearer has crossed a boundary. What other boundaries is he going to cross, or try to? He would need to actively present himself as harmless, respecting of women, etc., in other ways, in his talk and demeanour (think Quentin Crisp: he didn't wear dresses but no-one could feel intimidated by him) for women not to think, 'Eyup, thas somat not reet 'ere', and to react accordingly—retreating, being evasive, trying not to draw attention to themselves looking at him, disappearing quietly from the scene, etc. And he should know that we will respond this way. It's up to him to correct anything about the information we get that he doesn't intend to communicate. It's not up to us to accommodate him.