Oh I could allow it to continue, quite easily, no problem there because he doesn't bother, at all.
I wouldn't have to do anything, just stand back and watch him not bother.
However, I would (if he bothered to ask) tell him that No, he was not seeing them Christmas Day.
My children have (as many others) spent weeks dreaming of Christmas, preparing for Christmas, and our Christmas Day is extremely busy.
For me to allow him the opportunity to see them Christmas Day then I would have to disrupt their day.
I would have to stop them doing things they have been looking forward to.
And then I would have to hang around waiting with them for him not to turn up, or would you suggest I lie to them about why we were not going to their Grandparents houses?
I told XP's mother (as she was the only one who bothered with sporadic contact ~ when there was a birthday party of someone's she could parade the children I have been bringing up) only recently, that it is not my door the blame is to be laid at, it is his.
He hasn't made the effort, and as such, it's going to take a little bit more effort than a phone call if he wants to see them.
Yes, the resident parent does have a responsibility to help the NRP have a relationship with their children, but to aid this, not manufacture it and the one person who has the real responsibility there is the NRP.
I'm sick and tired of seeing 'oh, he's their Dad, he has rights' posts. No. He doesn't.
And lastly, if I were to allow my XP to see my children Christmas Day then I would NOt be doing the best by them, but the best by him, because Dd wants nothing more to do with him and DS has no idea who the hell he is.
(I realise my post is not indicative of all NRP's and that not all RPs are the mother, I am just writing wrt the OP's situation, and my experience)