My exact kind of conversation seems to have broken out! So, I'm fine with men in somewhat feminine clothes, and very much un-fond of men who're macho. Was always a goth so I'm used to seeing men in floaty things trying to look romantic and pretty and men in panda eyeliner - both are fine. I love pretty men in pretty clothes and very much prefer a look that's more aiming for pretty and less aiming for manly man. Singer of one of my favorite bands wears skirts sometimes and it doesn't bother me because it doesn't come across as a fetish thing, he's just a bit fey and very goth.
It's not even about whether or not a look is sexualized, because like people were saying, the 80s glam rock look (and the 70s version too) was extremely sexualized, but not in a way that gave the impression that the wearer wanted to be thought of as a woman for sexual reasons. That's the thing that sets off the spidey senses, I think, the awareness that a. this person wants onlookers to think of them as "woman" and b. what they mean by "woman" is basically "fuckbot", for whichever variety of that they prefer (porny or trying to look like a little girl both register as having the same sexualized intent), plus c. the reason they're doing this in public is to force a reaction from randoms. That's a violation of other people's boundaries and bad manners in the same way that walking your partner down the street in a submissive's collar and leash is. It's involving people in your fetish without their consent, and daring them to challenge you on it so you can make a huge fuss.
There's also the "that looks terrible on you" element, and that particularly applies to the mutton dressed as lamb category where people would also side-eye a woman the same age dressed the same way. The same "wow that's not appropriate" reaction applies tenfold for people who adopt a look that's both childish and sexualized - there's no way to dress a large adult man like a female toddler and not have that look disturbing.