As many PP's have said, this is a hard no from me.
YANBU. There's nothing wrong with wanting a small Christmas family gathering. It absolutely will change the dynamic if a stranger is there-even if your DB does know him.
As far as the "you're selfish/mean-spirited/uncharitable/unkind" remarks go-why? Why does OP have to open her door and a place at her table to a man she's never met and will likely never meet again-for her DB who can't be bothered to play host to his own guest himself? Instead, DB turns the tables on OP and makes her responsible for his friend being alone on Christmas-when it's actually his fault. Not hers.
Why is it that when people, especially women, try and set boundaries they're labelled as being "unkind?" Even if their reasons are valid and understandable, there's still a message of "be kind" thrown about because there's nothing worse than a woman being seen to be unkind.
As for all the "I would always host all the waifs and strays of the community"-great, good for you. This post wasn't about you and what you would do, but about whether or not, given OP's circumstances and reasoning, she was being unreasonable. If you're happy to have people you've never met before over for dinner, great. OP clearly is not. Everyone's comfort levels are different. I personally would hate having a stranger over for Christmas dinner as I like it to be a strictly family time and I want to slob around, eating all the food without worrying what someone else is going to think.
Some people take a "more the merrier" approach and that is fine, but why is that approach seen as the only "good" approach? Why is it bad to want a small, intimate gathering of family?