I didn't think I'd find myself typing this....but I'm not sure I agree with you, Datun 
And they have no real idea quite how many men are like it.
And how many men have it bubbling just below the surface. But tend not to let it show, because of customs and protocols
I think men are also very very aware of what other men are like. Don't forget that adult men have also had a lifetime of navigating their way through dealing with other men. They do know what it is like to be on the receiving end of male violence (I would bet that most men have been involved in some kind of scuffle, even if it wasn't a full fist fight). They recognise alpha males; if they are an alpha male they usually know it - and to some degree exploit it.
They also see and hear how men behave when women aren't there. I've changed my opinion on a number of men who I had thought were more or less decent chaps when DH has told me a sprinkling of the things they say in the changing room, or when I've seen the sort of messages that they send out to other groups of men.
For better or worse, this is why men who care about women (even if it's only "their" women that they care about, such as wife / daughter / mum / sister) recognise instantly the danger that women can be in if men are allowed into their spaces. I've found that men peak much faster than women when you say "but transwomen don't need to have any surgery at all in order to go into the ladies changing room at the swimming pool", because they know the potential for abuse that scenario creates