It's hard isn't it?
At what point does someone's belief about other people become so unreasonable it can no longer be accomodated?
It's like some religions with strict rules about sex segregation. Society says "you can live your beliefs and you are welcome to do everything you can to avoid the other sex on your own account. However, we will not accomodate you by restricting the freedom of the opposite sex so you do not ever encounter or interact with them. If that means your ability to function within society is compromised, that is your choice to make."
Trans people have very strong beliefs about what type of personalities align to each sex. They have every right to believe that. However, most people do not believe personalities align to sex, and many people find that suggestion reductive and sexist. So eventually, there has to be a point we say no. No, trans people are not in reality some third, or other sex, or mentally aligned with the oppoiste sex, even if they believe they are.
I personally believe that point should have been recongised a long time ago, when (some parts of) society first suggested it was reasonable that a man could live "as a woman" if he adopted certain types of dress and mannerism. I believe that was a function of a more sexist cultural background and enshrining it in laws today is anachronistic.
I also believe that if the main protagonists back then had been women pushing to be accepted as men rather than vice versa, it would never have gained traction. I think that if the popular image of so called "sex change" surgery had been a woman made into a "man" by attaching a non-functioning phallus rather than a man made into a "woman" by removing the penis and creating a hole, men would never have accepted it as valid.
Nevertheless, we are where we are. As things stand, for whatever reason, society is currently accomodating the belief some people have that their personality or self image is incompatible with others of the same sex. But that's not the same as treating the belief as real. So to what degree does society say "we understand that being with others of your sex is distressing for you so we will build extra provisions for you" and to what degree does society say "we understand that being with others of your sex is distressing for you, but that is something you will have to find a way to live with"?