It's not 'simply called' sex-based. They are sex based.
Yes, we would rather there not be segregation. Unfortunately, sometimes it's necessary. In facilities where women are extra vulnerable - sleeping, sick, naked, running away from an abuser or in therapy, they are necessary.
imposing the labels of "man" and "woman" on people against their will' - they are not imposed. Gravity isn't imposed on anyone, it simply is. There is male & female for the purposes of reproduction which works in binary in human beings and most animals. This binary is observed not just at birth - even before birth we can tell if a baby is male or female.
If you don't like this, perhaps you need to have a go at Mother Nature, God, or some other entity.
You may think it's limiting to view humans in this way, but it's fact. Despite it's simplicity, there is enormous range and freedom to acknowledging people in this way. There is biology....and the rest is personality which is incredibly varied.
But apparently people can't just be free to use masculine, feminine, or gender neutral labels and pronouns for some reason
There is a range of opinions on this. Not everyone opposes pronouns, but most here do resist pronouns being imposed onto us or by persistently encouraging people to introduce themselves with pronouns. Many people think they are freeing themselves by using gender pronouns, but actually, all is serves is to reinforce and acknowledge often regressive and limiting 1950s stereotypes of what men & women are meant to be like. Women in particular have suffered from the view that women are inherently interested in dressing-up, high-heels and babies because it goes along with other things which are particularly unpleasant, such as women are dim, air-headed, and not to be taken seriously. THAT, is where we think we're all going back to with this.
Many people are are gender neutral most of the time, until something comes along to remind them of their biological sex. Lets take you and me for example. When I lift my hand to put the kettle on - I grab the handle, pour myself a cup of tea and look out of the window as the tea bag stews. Do you think that in that moment I am aware that I am 'a woman'? No. Do I feel 'feminine'. No. Do I feel 'masculine'? No. Every day life is filled with completely mundane moments where we are unaware of how we feel, or how we feel has no label on it. The way I make myself a drink is likely to feel exactly the same way you feel when you make a drink.
When I open the door, I don't feel 'feminine'. When I go down the stairs, I don't feel 'feminine'. These moments don't feel gendered. Most of my day is filled with ungendered moments, yet if I were to buy into the idea that I have to use pronouns, it would signify that I somehow feel 'feminine' most of the time. I'm certainly a woman, but I don't feel or I'm not aware of said feminity.
We don't see 'male' (man) 'female' (woman) as labels that have been imposed upon us. They are simply referring to our biological sex. We're ok with that because biology is important. It has a big influence on our lives. It very much informs the discrimination we face as women - men abuse, assault, and kill women not because of what those women feel they are on the inside or by have a feminine soul. An abuser will abuse based on biology - we are physically weaker than men, as a whole, and some unpleasant men fully take advantage of this. When a predatory man stalks a woman down a dark lane, he gives zero fucks about what she feels like on the inside, what her pronouns are or what her identity is - what he wants is her female body and the feeling of power over her. When some men start getting shouty, pointing their finger in a woman's face, towering over her, this is behaviour they simply wouldn't display to another man, especially a bigger man. They do this with women because they know it's unlikely they'll get a punch from her, and if they do, it's not going to hurt the same way as if they were to physically intimidate a man.
One can identify as we wish to ourselves, but we cannot identify ourselves out of the way that men will see us, which is very much biology based, not matter what they claim on Twitter.
I can understand the attraction of the non-binary. I think many people identify as non-binary because society has invested so much into gender. Young women in particular must understandably look at what they think means 'woman' and it probably feels totally alien to them - the pink, the beauty vloggers, the high heels, even the mannerisms that are expected, such as not sitting in a relaxed manner and always sitting with your legs crossed or closed. Then they look at what they think men are like and feel, well no, I'm not a 'lad' either. If you don't feel able to identify with gender stereotypes, you are in very good company here. A lot of women on this forum are like that I would say. How many non-binary people do you know in real life, as opposed to non-binary people? Would you surprise you to know that most people are non-binary, by your own definitions? They neither fully identity with male nor female gender stereotypes. The type of women you will find on this forum in particular are exactly the type of women you would have been friends with when they were young. NO! You might think. NEVER! But I doubt that. In fact, I'd put money on the fact that you probably have a lot in common with many women here, whatever your identity is.