I completely agree with Oldman. Every man knows that if you push another man far enough, he will probably hit you. That's human nature and it keeps things (relatively) civil.
The problem comes precisely because the majority of men are very reluctant to hit women.
If a man gets aggressive with me, the chances are I will get aggressive back. He knows, I know this. It acts as a mutual detterant to violence.
If a woman gets aggressive with me, there is no real way I would consider being violent towards her, therefore there is no detterant and the framework is one sided.