The reason I brought up biological definitions rather than appearance is because it seems to me like in the modern age, appearance is much harder to judge. We seem to be moving away from gender/sex norms, which while I think we can all agree it is a good thing, does mean that we can no longer rely on things like ‘men wear trousers’ and ‘women wear skirts’ as ways of identifying people’s sex. With changing ways of dressing, ways of applying makeup, people who just happen to look ‘male’ even when they are female and vice versa, and of course transgender people (regardless of whether we agree with them, we need to factor in their existence) means that as far as I’m aware, observation is no longer a reliable way of distinguishing, even without factoring things like intersex people.
This is only a problem if you insist on using stereotyped dress and cosmetic norms as your sole means of judging between male and female. There are plenty of innate (biological) indicators of sex that don't involve testing someone's chromosomes or examining their genitals. Height, voice pitch, shoulder and hip width, breasts, facial hair, hand size, jaw shape. Gait is an excellent indicator - pelvis configuration is different in males and females and results in gait patterns that are distinctive even when most other physical features are obscured.
Based on what I know about single sex spaces and their importance for protecting women (please correct me if I get something wrong), the last thing we would want to do is prevent women who need them from entering because they don’t look ‘female’ enough
This is topsy-turvy thinking. If people entering a single-sex space can be confident that only members of one sex will be there, there is no need to police anyone's external appearance. Whether or not someone looks "female enough" ceases to be an issue if you can be certain that everyone present is in fact female.