I don't have any daughters but I have a niece, and I have to admit we are quite protective of her - I allow my streetwise 13-yr-old DS1 to take the bus/train into town on his own [he is met at the other end], but we always make sure DN (aged 15) is accompanied).
I wouldn't insist on accompanying her into the ladies' toilets, but I would normally be close by, waiting outside. (And it wouldn't particularly bother me if there was a boy in there with his mum - even if it made her feel uncomfortable, I think the mum's worries about his safety must come first.
It's a risk analysis thing - boils down to my assessment of the risk of them being approached, what I think they could do about it if they were approached [depends to some extent on the individual and their capabilities], the level of harm that could ostensibly result, and the measures I can take to control the risk.
I know you can't sweep away risks altogether and kids have to learn how to deal with risks, but we need to keep in mind their capabilities.
And yes I know there are lots of other risks, and I try to deal with them appropriately too.
I do think men's toilets are dodgier than women's, and having taken the odd wrong turning in a pub or restaurant, ime they're usually scuzzier [looks down nose].