From my own experiences which I've mentioned upthread.
I would guess that there are a few different reasons that different people decide they cannot support the current demands from various trans orgs.
We all know what they all are. Safeguarding, safety, stats, fairness etc.
For me I was ??? when it first started being discussed on here years ago. Because it obviously just didn't make any sense.
And a large part of why it doesn't make any sense, for me, is that I don't have this internal sense. I feel like a person. Not female/ womanly etc. I always have.
According to stonewall this makes me trans, and an awful lot of other people too. Especially a lot of feminists who found feminism young because they felt like people and kept getting treated in all these odd ways because of their sex.
But instead of saying oh that's a good point. We can work together, you are natural allies.
We get TOLD we are cis. Get TOLD we don't get it. Get TOLD we are bigots.
Even though by the terms of the people telling us this, many of us are in fact trans.
So why are female 'agender' people's views rejected, repugnant.
That makes no sense at all.
Unless it's not about being trans at all. It's all about something else.
(I don't ID as trans or agender but according to stonewall etc that's what I am).
Additionally our insights into why pubescent girls might find a non binary identity appealing are discarded as hate. Even though we were once there ourselves...
Also how people feel inside is irrelevant to the reason we have sex segregated stuff (that's because of bodies and behaviour).
So why it's used as a reason for any changes to law or established social norms around sex segregation is beyond me as well.