I tend to find that if i am uncomfortable with the current occupants of a single sex space I either turn around and come back later, stay put behind the curtain or door or pack up and get out as quick as possible.
I don't find that all spaces are permanently closed to me, just not always desirable to use, and to be honest its far more likely to be the sanitary conditions i find objectionable than the clientele, but the same principle applies.
It might not always been convenient or my number one choice, but that is just one of the compromises that have to be made when using public facilities, no matter your religion, or if male, female, trans or cis, public spaces are just that.
I would like to answer that with one word.
Bollocks.
Under law, if you are in hospital that is legally defined as 'private'.space not 'public' despite it being publically run and owned.
This is because if you are in hospital you do not have the ability to just shuffle off if you want. The same goes for certain other spaces that are deemed public. You are supposed to enjoy the same privacy you would enjoy at home. If someone is making you feel uncomfortable, hospitals have a duty of care to respect that because sex is protected characteristic. There is a caveat about care givers, but this is based on need. Since other patients are not in anyway connected to this then they fall outside that caveat. If gender suddenly trumps this current protection then that is a removal of current rights and is unacceptable.
Even in places which do not have the same legal definition, there is a right for women to know this because of things like religious beliefs. If a woman shares a space with a man she could be discriminated against in certain circumstances which could put her at risk.
If women are not being given an alternative they are being discriminated against.
The ability to know if you are being discriminated against, also rests on you being given information. If this information is withheld you are being discriminated against by an institution.
Under law public toilets still are 'private' in respect of sexual indecency. If you allow self identification you make it more difficult to define and therefore bring a successful prosecution if it did happen. It therefore weakens the law and protects for women in a very real way because gate keeping which enshrined safe guarding by the mere process is removed.
So yes it matters. And this idea of just 'getting on with it' even if we don't like it, rather misses the point about why current protects are in place and why women are concerned about how people could exploit intended proposals to their detriment.
The fact that women are not being allowed to debate this or feel too intimidated to stand up to trans activists, is an erosion of women's power to express their discrimination.