"Why does this remind me of Felix telling women they can't do this, that and the other, mustn't speculate, etc."
Both men and women complain about this, not just women.
"Women can absolutely accuse men who are violating their boundaries in their single sex space, without consent, as being predators."
You can, but you're both factually and morally wrong to do so. Just as people could accuse you of being a baby killer for getting an abortion. You might not appreciate that, but people can say it. You don't seem to understand what a predator is. If a man is in the women's toilets, who has no intention of harming anyone, does their business and leaves, they are, factually, not a predator or a criminal. A predator has to have predatory intent and action. You can be uncomfortable with whatever you want, but it doesn't make you're accusations true.
Using the word consent also comes off as manipulative. Since people being in a public place isn't actually a violation of anyone's personal rights. And if this is just about comfort and dignity, and not just about safety, then why is it okay for women to use the men's room? Men are even more exposed than women in there, because they have urinals, not just cubicles. It's all the more easy for men to be spied on when they urinate. And that does happen.