My DM is a nightmare. I restrict contact. I certainly wouldn't allow my DC in her company unsupervised. If my DH was effectively forcing me and the children into contact I would be furious.
You describe your DH as selfish because he doesn't contact her to arrange regular visits etc. No, that's self preservation. She is abusive, it rubbed off on him. You know that. Let him stay away from her.
Don't tell him he is selfish for restricting his contact with someone who abuses him. Is that maybe a bit of projection there from you? You've stayed with him even though he has been abusive to you therefore you think he should stay with her even though she has been abusive to him? Also, when your DD says she doesn't want to spend time alone with her abusive grandmother, listen! Don't send her there!
You seem to have trouble saying "No, I will not allow you to do this" to abusers. He can separate himself from her (just like you could separate from him if necessary). You can choose not to serve up your child to an abusive person just so you, the adult, can avoid being on the receiving end of a strop.
It is you that's being selfish OP. You are sacrificing them to protect yourself from MIL having a tantrum at you. Human shields. Stop doing that. That said, DH should be stopping you from letting DD anywhere near her, and telling you straight not to arrange stuff with her.