Grimarse - that your wife and daughter might be violent and abusive is irrelevant. We all know that women can be as vicious and abusive as the worst sort of man.
That does not translate into widespread fear of womenkind by men.
Men don't walk alone along a street at night and see a woman - or group of women - and get that automatic jolt of fear and uncertainty. Men don't walk past groups of (sober) women in broad daylight and automatically speed up and put their head down, for fear of drawing the group's attention and comment.
Unknown men cause fear in women, in a way that unknown women don't cause fear in men.
Women only cause fear in men, other women, and children, on an individual level. If they're a known entity - and have proved themselves to be violent or abusive - for example your 'teacher' wife or 'bully' daughter.
Unless I know a woman is a bully, I have no reason to fear her. I don't need to know anything about a man, to fear a random one that I might meet in X circumstance.
This fear is caused by nothing more or less than what I know men - as a group - are capable of (even though my personal experience of men has generally been extremely positive).