I think a lot of people (male and female) genuinely have no idea about the lengths to which abusive men will go to control and manipulate women.
The population of trans people is very small. The population of men is very big.
When a refuge has a blanket policy to exclude biological men, it is not (just) because women inside the refuge may be traumatized by the presence of a trans woman, (although this is also legitimate). It is also because they are aware of the ways that manipulative and abusive men will attempt to gain access to women and children inside the refuge.
I think thatany people, particularly young people, who want to be inclusive think that everybody is kind and well-meaning. And that things go wrong because of ignorance or misunderstanding (or evil, intolerant bigots). I've had several conversations with left-wing activists (I am one myself) who genuinely think that in a socialist utopia there would be no crime and no need for eg prisons, because everyone will get a good education and enough food, so there will be no trauma and no exploitation.
That translates into policies in the real world that are based in fantasies about human behaviour and are not robust to rare, edge-case behaviour that is designed to be exploitative and harmful.
When you point out how the policy can be exploited, you are told 'it will never happen' because the behaviour is a rare one and seems outside the parameters of what a reasonable person would reasonably do.
That really fails vulnerable people, especially in settings like refuges, where by definition the women who need to use them are already directly linked to unreasonable, edge-case men, who will go way way way beyond what seems reasonable to keep their victims under their control.