This makes me grateful that my abusive F is dead. He inflicted SA on my DSis and me and allowed others to do that to us too. He didn't abuse my DB but he allowed others to abuse him, and encouraged him to join in.
My DM didn't know about that, I do believe her. But she is guilty of gaslighting. She says she had no indication that there was anything going on, but there were indications but when we mention them, she says she doesn't remember that. (She's devastated about it and cries if we mention it, and asks us not to spoil her time with her DGC.
She accepts that our F smacked us too hard, and maintains that she stopped him. Whereas she actually didn't, and smacked us herself. She admits that she caned my DB at 10 because he couldn't work out a simple sum. She'd told us about it before, and that it had haunted her for years, but I'd assumed that he must have done something really bad, whereas it was only about his Maths homework. That really shocked me.
Thankfully she doesn't joke about it, but she rewrites history. When I'm having difficulties with my DDs she'll say, 'My 3 never did that', as if to make out that she must have been such a good parent. (I've never quite had the nerve to ask her, 'Then why did you snack us so often?')
She was guilty of neglect, other people thought it at the time. She was too busy with her work and emotionally distant, which is why we were never able to tell her what was going on.
I have a lot of resentment when she interferes in my parenting and my relationship with her isn't great, but she has a good relationship with my DDs. (I've learned to be more assertive when she oversteps the mark.)
In your case, OP, your F sounds downright horrible and I really would consider reducing contact completely. (I realise that it isn't easy, with them living so close, though.)
I considered that where my DM is concerned but she's 80 and my DDs love her, and tbh so do I really. I know that she was actually a victim of EA herself and of coercive and financial control.
I'm sorry that you've had this to cope with. You may well find that he backs off now you're standing up to him, and your DS has seen his true colours. 