Sorry for the delay, OP. It's because something terrible already happened, and he said that was it, and then went back to them again.
In lots of ways, I think you might find it useful to think of him as an addict and them as the addictive substance. It's about deciding what part of his behaviour is acceptable to you. You can't actually do anything with him, you can only decide what you're going to do, but you must follow through.
Examples.
You can see them but I don't want them mentioned in my home.
You can see them but you can't come home here in a bad mood because of them.
The children can't see them.
You must go for anger management counselling.
If these rules are broken, then the marriage is over. But that just can't be a threat, it's a boundary, and only you yourself can apply it. You're the only person who can decide what you can and cannot accept in your home.
Your boundary might be more simple - you tell him that the DC are not seeing them, and you won't ever be discussing it further. If he tries to discuss it, you leave the room. (Like ignoring a drunk person.) You refuse to engage in any conversations about his parents. Leave the room. Go out. Don't argue. (Don't be a passenger in a car with him btw, you need to be able to pull over and get out of the car.)
If he complains that you won't talk, just say you will never change your mind so there is no point.
If he is in a bad mood around the house because of them, go out and take the DC with you. Or play a game with them in the other room, or bake or something light hearted. Make sure his mood is not affecting any body else's behaviour. Plan your life regardless of what's going on with him.
He is being completely unreasonable and you need to withdraw with love and compassion. Don't get sucked into his drama. Give your reasons for them not seeing the children and never vary. Don't listen, don't support, don't sympathise, don't engage. Don't be an enabler. It's your life too, OP, not just his.