It's a very fair point. The whole contract of law is, 'you don't take the law into your own hands and seek your own justice because you can trust in the fairness and effectiveness of the law of the land'.
If it's no longer fair or effective, the contract is broken and yes, people will inevitably seek their own justice.