This is a great post from AskBasil in 2014 about changing rooms, but which I think is very relevant:
I think it's partly the male gaze, that sense of intrusion by a stranger who has been socialised to appraise and rate me in terms of my sexual acceptability. I resent it in mixed changing areas (same way I resent men flicking their eyes up and down my body when they talk to me), but as all of us do, I adapt to it and find ways to avoid the intrusion and accept that it's part of being in a mixed changing area. You wouldn't be in the same mind frame if you thought you were in a same sex area, so the need to suddenly start adapting the way you do in the company of men, would be a huge intrusion.
Women's space is different from mixed - ie male - space.
Men are trained to take up space, to own space. If we look at how they sit, stand, walk, where they feel dominant they take up more space and where they do not feel dominant (like being with senior men, older men, richer men etc.) they take up less space. This is probably also true of women. When we are in spaces with each other, we are more comfortable in those spaces as we feel we have an equal right to occupy them. We move differently in them.
When we're with men, we shrink ourselves, we keep out of their way, we defer to them and they don't even notice we're doing it (and actually most of the time we don't either, it's so internalised that it's automatic). When men aren't around, we relax in the space.
Knowing there may be someone around who calls himself a woman but has a penis and has been socialised to appraise me, would mean that there is one less space in the world where I'm allowed to feel safe, comfortable and to own the space on an equal basis with the other users of the space.