I cannot imagine how difficult this is for you.
I have two big sons and if either of them EVER raise a hand to me, or my daughters, it would be completely game over, such is my non existent tolerance of male violence.
No man has laid a hand on me and no man will ever remain sharing my home after doing so.
Your other children have more right to a peaceful home, than he has to bully you in his.
It is very sad, but your first obligation is to maintain a peaceful safe space for your family.
His behaviour means he can no longer share the home.
Very sad, but actions have consequences.
He needs to learn that.
Hopefully the reality of his actions will help him.
You are doing the right thing insisting that your other children are protected.
Many years ago a school pal of mine had a bully of a brother who played senior rugby and his parents were very proud of his success.
He often gave this friend a slap or a pinch and her parents used to say, "don't annoy him, don't wind him up, you know what he is like etc", when she complained.
This is more than 40 years ago.
She worked very hard at school and got an excellent offer after her results.
She chose to go away to Uni.
We only realised what she had endured when she asked her friends to take boxes of her possessions as she was clearing her family home of anything she cared about.
She headed off to college and NEVER return to the family home again. EVER.
Her parents were so shocked, disbelieving, outraged, angry, heartbroken and finally devastated as it became clear to them that she was very serious.
She was very open to people, that her brother had bullied and hurt her throughout her childhood and she would NEVER set foot in the family home again. EVER.
She never spoke to her brother again, she refused completely his efforts to apologise. He did try.
Primarily though she blamed her parents for not protecting her and for allowing her to be a punching bag for her brother.
She believed her parents failed them both, but only she had physically suffered.
She would only very very rarely see her parents, they were never invited to her home, they never met her children, their only grandchildren.
They weren't bad people, just people who thought more of him and his sporting success.
They apologised I believe several times years ago, but it never moved her.
As they aged she saw very little of them and she wasn't involved at all when they died.
She has made a great success of her life both professionally and privately, but I have no doubt that it has been very painful.
Too many parents think "rough housing" among children is acceptable.
I have had zero tolerance always.
Over the years I have told the above story to a few people who were tolerating it between their sons, as if that is somehow more acceptable.
It has given them pause for thought.
As parents we need to be very mindful that children can have a vastly different view of their chilhood with the same home.