The possible compromise is easy, create a third mixed sex space or service.
Inebriati has it.
The underlying principle does need drawing forth though, if you want to resolve the issue to your own satisfaction inside your own mind, OP.
Set aside the idea of "right" and "wrong" for a moment. Everyone thinks "I am right and you are wrong". It's a basic fundamental human tendency.
Reframe it as something else, so you can understand what the underlying issue is and see how to resolve it. And take the sexually dimorphic reality of bodies out of it a moment. Focus on the mind, just for now.
Each side has a psychological need to feel as though they are physically safe. And each side also has a psychological need to feel as though they're not under psychological attack. Everyone has these needs, they're universal human needs.
Both sides feel psychologically unsafe because we're in an open warfare situation over the words. One side gives precedence to the feeling of physical safety, and the other gives precedence to the feeling of psychological safety.
We're effectively stuck in an increasingly polarised argument over whether physical needs or psychological needs take precedence. One's position in this argument is ultimately determined by the sexed reality of one's body.
Until one steps back, and realises that's not a question of which side is right. It's a question of how everyone's tandem streams of psychological and physiological needs can best be accommodated.
It only feels like it's a tricky and complex and nuanced problem because you're asking yourself the wrong question, OP.
And the only reason we haven't got a third, mixed-sex option available already is because people's brains lie to them about what their psychological needs are, and the my behaviour means I am a woman lie is being indulged on a mass scale. Same as it always has been.
The middle ground is the patriarchal status quo. We won't smash it by finding a way to cling to it. And if we don't defend our own needs, they won't do it for us. So we hold the line on single-sex space, and wait for them to fixate on something else.