When we are discussing small children who might reasonably wear a dinosaur costume, I think that is different territory altogether from older children and adults.
For the latter, I think most people understand that there are cultural differences in what is considered masculine and feminine clothing and also that these can change over time. In many societies with limited ability to create complex garments, men and women's clothes are very similar, though they might be decorated differently.
To some extent differences also reflect the differences in the male and female body, it's why many women who wear trousers (which are gender neutral in most of the west) don't wear men's trousers. They don't fit right. You get into a similar problem with men who try and wear women's clothing, it usually fits poorly and may be unflattering to the male body and the clothing. It's been tailored with women's bodies in mind. As for people trying to say that this isn't so, I really hope you aren't the same people who insist it's pretty easy to tell a woman from a man because they are built differently, right down to the skeletal level.
But the question that people are uncomfortable with all round I suspect is whether a man or older boy who is wearing a dress is generally doing so just because he is really just attracted to dresses, or because he is attracted to appearing like a woman, or appearing wearing women's clothing (whether that is a dress or a something else.) And pointing to all the gender benders of the 70s and 80s, or even Kurt Cobain, is not helpful because they were all looking to dress like women, that was the whole point of gender bending. Maybe some were trying to really subvert gender, but mostly I don't really think so. Playing with gender isn't the same as wanting to be rid of it.
The question about the date is really, would you be uncomfortable if your date seemed to want to appear as a woman. Which may be why also wearing make-up that appears as the sort a woman might wear crosses the line - it seems to make it clear that it's not just about the dress but the whole package "woman".