If Jane then insists on using the women's toilets, on the grounds that everyone believes that Jane is a woman, and you're not one of the women privileged enough to be able to use a mixed sex space and kindly continue enabling Jane's personal fiction for their happiness, is there skin off your nose then?
When you have to state that you cannot use a mixed sex space, the reason being that Jane is actually male, knowing what is likely to happen to you, is there skin off your nose then?
If you don't dare speak out and just can't use toilets at work any more - or go swimming because Jane's joined your local swimming club, or go to your lesbian group because Jane's joined it and is keen to enforce that male people are lesbians too and for the attendees to express their openness to overcoming their genital preference, any skin off your nose yet?
This is the issue. It's the gateway drug. Its all very nice and polite to comply and enable a male person in this way, and yes it stops rows and bad feeling and tears and tantrums and threats and all hell breaking loose. But people do this thinking it's a social contract in which they pretend politely up to a point, and the person they are pretending for does not really believe they're a woman, and will not push their personal fiction to the point of harming women.
In a casual social situation I am not going to start WW3 by pointedly misgendering a stranger, particularly if they're nice to chat to and I don't want to offend them. But I will as far as I possibly can just avoid the pronoun they don't want, and I won't actively lie to them as they would like, because I'm not going to say Hail Marys I don't believe in or agree that this other person gets to control me and my perceptions. And I'm very, very aware that this nice person may be one of the ones who then takes that inch I give via pronouns and starts demanding the next mile.
It's not easy. But being nice and polite and kind and afraid is what got women into this unholy mess.