OK, could you maybe use your imagination just a tiny bit?
Imagine you had a husband who actually took your daughters out to places without you being there. What happens when they need to use the toilet?
It seems to me that the options are, he takes them in the men's, he lets them go in the women's on their own, he takes them into an accessible toilet, or he lets them use an accessible toilet on their own. Let's not get into the moral implications of using an accessible toilet when you do not have a disability, for the time being.
If he takes them into the men's, they may have to walk past men who are using urinals in order to get to the cubicles. Do you really want your daughter of any age to accidentally get an eyeful of a stranger's penis? They may be confronted by men who, quite reasonably, don't want little girls in their toilets. The toilets themselves may be less clean than in the women's.
If he lets them use the women's, he has to let them go on their own, because he can't go in there.
If he takes them into an accessible toilet, he can keep an eye on them but there will come a time - probably not much beyond the age of 6 - when they absolutely don't want their dad in the cubicle with them while they use the toilet.
If he lets them use an accessible toilet on their own, he can still only stand outside, the only difference is that he can be sure no one else is in there.
What options do you think he should pick at what ages, and why?
What about if you had a son and took him out without his dad?