The problem with saying "female only spaces" means that while it excludes trans women, it includes trans men which actually makes it easier for men to access female spaces.
A friend of mine is trans, he lives his life as a man as he has done for the past decade, has full beard, deep gravelly voice, has been taking hormones, has had top surgery but has not had full bottom surgery - his tubes were tied years ago but full bottom surgery was deemed too risky due to some other health issues he is getting under control.
In the situation where "you use the spaces of your biological sex" and people are not allowed to self identify (if a trans woman can't self identify and use the women's then he can't self identify to use the mens) he would then have to use the women's spaces, despite his very clear male presentation, because he is still biologically a woman with female reproductive organs and genitalia.
This could then lead to a situation where you have people entering "female only areas" who are presenting as male and are biologically female but this then sets a precedent where "males" are routinely entering the female only spaces. The problem comes because there is no way to differentiate between trans men who are using the spaces because they have been told they have to use the space aligned to their biological sex and cis-men who want to enter the female spaces for nefarious reasons.
So all we are doing is turning the issue from "men may be presenting as trans women to assault women in female spaces" to "men no longer need to present as trans women to assault women in female spaces, they just have to say they're a trans man" instead.
Is it a scenario that is likely to be happening to everyone on a daily basis? Not for most of us but my friend would be getting questioned on a daily basis.