Mother to an adult son here with a daughter who has suffered at his hands over the years.
Stop thinking of him as that little boy who was lovely. All children grow up into adults and it sounds like your son has passed that threshold now. He is confident enough to behave that way in your home knowing he can get away with it. My son was a lovely little boy but as an adult I cannot say he is lovely and I was in denial for a long time. I accept the facts as they are now and not as I want them to be.
Before long he will be fronting up to you and your husband. Are either of you prepared to protect yourselves should he rear up towards either of you? Do you have the strength between you? An angry/aggressive 21yr old male is very powerful physically and you'd be surprised how intimidating that is to manage in the moment. The shouting, banging around against walls and furniture, the panic coursing through your veins trying to protect your other children. That sounds similar to what your daughter experienced with the shouting, door-pushing, aggression, physical assault. It's a surreal, traumatising experience and alway stays with you.
I WISH I had protected my daughter from the psychological and physical violence she suffered at the hands of her older brother from the beginning, rather than several episodes later, then perhaps she wouldn't be in counselling now to help her deal with the trauma. We took exactly the same stance as you are now, often. In the end we had to make that break with him and I was forced to call the police to have him removed. My daughter took time to trust me and her father again as she didn't know if we would protect her in the way she needed us to in the future.
Violence is violence regardless of relationship and this event WILL stay with your daughter. If it becomes a regular occurrence then she may begin to lower her expectations of future partners and end up accepting similar behaviour from other males. Our daughter has sailed dangerously close to this situation and all we can do is support her financially with counselling and with regular reaffirmation of our commitment to her.
She is far from being recovered. She has achieved academically but in relationships it is a very different story and we have traced it right back to the toxic relationship with her brother but more importantly our lack of decisive action right from the beginning.
The role as a mother often means taking difficult decisions to protect all of your children. I am broken-hearted that the lovely boy I raised grew into someone I never imagined but I will forever hold myself responsible for the trauma my daughter experienced at the hands of her brother because I was the only person who could have protected her from it in the first place.
Good luck.