There's every chance that a man who is violent to his wife will also be violent to his children, given time. You made sure that they never suffered from his violence- you didn't ruin their childhood, you did all you could to protect it.
If his family think that violent abuse is 'a bit of fun', then there's no guarantee that they would have protected your children's safety either.
Do you feel that it would be helpful for your children to know that you left because of the violence? They are old enough to understand. The violence, and your husbands's appalling treatment of you, are the things that broke up the family- you leaving with the children meant that you were limiting the harm they may have faced.
And if your husband continued to be a high earner, there was nothing to stop him continuing to use that to provide for the children even after you divorced. He withdrew that support, not you.