Right, but you said that "some objections can veer into transphobic territory".
First of all, there appears to be no clear definition of what transphobic actually is. Trans activists seem to think an awful lot of things are transphobic, including saying things such as "trans women are male". If you take that to its logical conclusion, it seems some of them think the discussion of biological sex is transphobic. That can't be right. So I'm very uncomfortable with people's concerns being dismissed as "transphobic" when no one can agree on what actually is transphobic and what isn't.
Secondly, in this case, there is no evidence that any of the transphobic statements were actually made. We only have one person's word for it. One person who is clearly unreliable.
Thirdly, if the alleged transphobic comments were made, and we only have one person's word for it that they were, the situation came about precisely because women's need for single sex changing spaces was ignored in the first place, for the benefit of one male individual, and because we are living in some bizarre universe where women are actually expected to give reasons for why they don't want to share communal changing spaces with penises.
If "no is a complete sentence" were actually accepted in relation to women's rights and their bodily autonomy, the conversation wouldn't need to get as far as anyone saying anything transphobic. It would begin and end with women saying no.