Speaknowords I believe that's the direction we're headed, but it needs to be sensibly managed and take into account high-risk situations. For example, refuges are there to provide, well, refuge from abuse which is often gendered abuse - a woman who has been raped/abused by a man in a gendered way may not feel safe sharing a refuge with men or receiving care from them; however, a man who has been raped or assaulted by his male partner may feel the same way - what should be done for him? At the moment, there is no provision for such men (not to mention precious little for women). Ideally, there would be a wide range of choices; but given the lack of funding, I would say there should be a large number of women-only refuges, a smaller number of men-only refuges, and also some individual facilities for people who don't feel instinctively safer with people of their own gender, women with older male children who might not be eligible to stay with them in a woman-only refuge, etc etc. Until gender is not such a significant factor in domestic abuse, and as long as funding is limited, pragmatism needs to be observed, whilst we continue to expand the options in a way that accommodates progress and unusual situations.
As for prison, I think that should be looked at in a pretty major way in various senses. I think prison should be there only for dangerous crimes, and other options looked at for other types of crime, as it is in Scandinavia. Obviously those convicted for violent/sexual crimes should be highly isolated and monitored, regardless of gender. Rape in male prisons is at absolutely horrifying levels - I personally don't see why we are happy to let that stand just because it's men raping men rather than men raping women. So it would surely be a better thing for criminals to be segregated based on the severity of their crimes, their mental health needs, the best course of action for their rehabilitation etc, rather than by gender - and for prisoner safety to be a real priority, heavily invested in (rather than completely disregarded in male prisons, and blithely assumed in female prisons, as is currently the case).
Hospitals, yes, I think they should be genderless as default, but patient choice should be observed in terms of caregivers, e.g. if a man wants his testicles felt by a male member of staff rather than a woman, or a woman wants a female doctor to deliver her baby, this should be accommodated wherever possible and practical - again, I'd like to think over time it will cease to be an issue as people increasingly realise what matters is that you get treated by an expert, not what junk the expert has in their trunk. I don't really see why it would make a difference to you who was in the beds around you on a mixed ward - if you want privacy, pull the curtains.
The difference between the examples you give and Girl Guides is (a) Girl Guides is totally optional - you don't get to choose whether you need refuge/go to prison/are in hospital. These are essential services, which must perforce change in line with society's changing mores - happily, these are shifting ever further (if glacially) into a more openminded, tolerant direction. But forcing people who are already vulnerable into situations they are uncomfortable with is not my agenda, not because I necessarily think their feelings of discomfort are necessarily reasonable, but because it is cruel.
and (b), Girl Guides is a children's organisation. The children are, cheesy though it may be to say, our future. It is a good thing for places where children socialise to be forward-looking, tolerant and to assume the best in children, rather than paranoically trying to 'protect' them from each other. Obviously, common sense safety measures should be in place; but by and large, children need protecting from adults, not other children. And the best way to protect children from all kinds of abuse from anyone is to raise them with confidence in their own integrity and trust in their caregivers, so they feel confident to assert and protect their boundaries and to seek help from a trusted adult if those boundaries are not being respected. Abusers thrive on secrecy and shame, be they adults or other children.