I am not afraid of transwomen, and personally, don't care if they use the same facilities as I do. I've used mens on many occasions when the queue for the ladies has been ridiculous (theatre interval anyone?).
I think what makes me uncomfortable, without getting all won't somebody think of the children, is the fact there will be small children and minors.
I have two DD's. They are still young, but at that point where they are capable of of using the toilet by themselves, and on that cusp of independence when they don't want or need me accompanying them.
Currently, I know there will be women only in the toilet. So I can send them in a cafe loo without having to pack up everything and go with them.
However, if transwomen were allowed in, I'd not be happy. I don't want my prepubescent girls catching a flash of penis as they wee (because that, in any other environment would be a sex crime), or conscious as developing females that there are biological males close by while they perform bodily tasks.
Even as a teen, I remember going with friends and discussing the intricacies of our newly acquired periods. Wouldn't have done that unless we knew it was women only.
So that is my main reason for being against it.