I'm now very nearly NC with my brother and low contact with my mum. Like you, I just couldn't do it anymore. My brother has MH issues and gets angry very quickly, shouting at everyone, including my DDs, and when I started to see that they were afraid of him I decided that enough was enough. My mum used to expect DH and me to look out for him when she was away and to have him stay with us sometimes. It was literally like having a third DC, except for the fact that he's now 50.
There's also the fact that we had a traumatic childhood, our F was abusive (SA) towards DSis and me, and our brother joined in with that too, although he was a victim too. It's been increasingly hard to be around him at all.
DSis and I remain very close, however, and our DCs are similar ages and love being together.
So for me it hasn't been possible to go NC with my brother whilst maintaining a close relationship with my mum. She pays lip service to accepting how hard it is for me to maintain contact with him, but she constantly talks about him, and makes me feel guilty.
Your parents will find it difficult to understand, but it sounds like you have a good enough relationship with them to move past it. My relationship with my mum wasn't like that sadly. You need to make the right decision for yourself. 