I was wondering if you had dependency issues, which is why his mother wouldn't want you in her house. Because, for a grandmother not to want to see her grandchildren, given the chance, at Christmas seemed really strange.
I also wondered if your partner was blaming you for the relationship breakdown due to something you had done for example, got drunk and kicked off.
If it is as you say then you need to maintain distance from your ex. Do you have contact arrangements in place? If so, why hasn't Christmas been discussed and pre arranged?
He's your ex and has no say on when you see your own parents at Christmas. He has no say on what you do at all. He doesn't get to dictate to you as it's no concern of his.
If Christmas hasn't been arranged, then arrange it with him. There is no need to tell him what your plans are on Christmas Eve as it's absolutely nothing whatsoever to do with him. Go and see your parents.
If he doesn't want to see his children on Christmas day, that's sad for the children and doesn't say much for him as a parent. Let him get on with it.
As for his deranged alcoholic mother. I wouldn't want her near my children as she sounds thoroughly toxic. Be thankful she doesn't want you in the house and why in god's name you'd want to go there for Christmas is beyond me. It would be like spending the day with a nest of boozing vipers. The atmosphere would be dreadful and it would ruin the day for your children. Block her number and have nothing to do with her.