The trouble is, I have no doubt that the overwhelming majority of "civilians", if well informed about the subject, would agree with the gender critical position.
Even the ones who are currently happy to say that trans women are women because they haven't really given much thought to things like prisons and sports.
Their real position is probably, "Yes, trans women are women, because that's how they identify. Oh, well, not when it comes to sports, obviously, they shouldn't be allowed to compete against female athletes, that's not fair. Prisons? Surely they wouldn't put male rapists in women's prisons, anyway, trans women can't even commit rape if they don't have a penis, can they? Oh, they do have penises? Well how are they women then? No, of course a rapist with a penis shouldn't be in a women's prison, that's insane. I don't see the issue with rape crisis groups to be honest, can't there be separate groups for women and for trans people? They don't want that? OK but what the women want matters too. Toilets and changing rooms though, I think that's OK as long as they don't still have a penis. What's that? How do we tell whether they have a penis or not before they walk in? Hmm. I hadn't really thought about that. So are there any situations in which they should be treated as women? Not really, no. Just pronouns, I suppose. It's polite to use people's preferred pronouns, isn't it? Unless they're a rapist, of course. I'm not calling a rapist "she", don't be absurd."
But you have to force the conversation to get them to think through these things. And it is mostly not on their radar.
That's how Labour is getting away with this nonsense. Because "civilians" have a million and one other things to worry about, they don't care about trans people either way and they aren't devoting any head space to this issue.