Here is a study done on faces
https://www.sciencedirect.com/science/article/pii/S004269892200133X
Our results indicate that facial structures with full information on the texture and color of the skin are correctly classified as to their sex by most of the participants (98.4 % for Exp. 1 and 94.6 % for Exp. 2). If we do not consider versions 3 and 5 (close to the androgyne version 4), which contain a certain degree of sex ambiguity and only consider the less ambiguous versions (1 and 2 for male faces, and 6 and 7 for female faces), the accuracy approaches the ceiling (99.9 % for Exp.1, and 99.1 % for Exp.2). This is in line with previous research which observed that natural faces, devoid of any cultural signs of sex, are generally correctly categorized into their sex
I would think that results over 90% indicate a correct assumption that most of the population can accurately identify the sex of humans from faces. However, note the difference between people correctly identifying male versions of faces vs female.
The study mentions other papers as well.
I have a link stashed away that show that babies can recognise male and female faces too.
I am really not sure when people started to believe that their inability to tell the difference between male and female people through observation and hearing voices was the norm. The research says otherwise.
I also suggest readers should look at the documentation behind facial feminisation surgery (many providers will describe what gets done). The changes they make even to narrow the difference between the top lip and nose and the tilt of the eye. But the obvious ones are skeletal. Brow, jaw and cheek.
I think either some people cannot correctly identify people’s sex so they believe others can’t as well. Or that perhaps due to being constantly told that humans cannot correctly identify another human’s sex from extreme transgender activists, that some people actually believe this.
Either way, the evidence shows that the vast majority of people can correctly identify people’s sex from observation.