Yup.
A bloke we knew had a habit of appearing with his kids. A couple of them were fine and would play with my DC, but one rang rings around his father who always missed that magic moment when you could say to the kid, 'Er, enough of that, how about you do xyz instead...' The bloke himself was going through a sticky patch, so he required tea, sympathy and advice. And we'd done a lot of that. And a lot of tidying up after his naughty kid.
One afternoon DH took all our kids off to the cinema. Result! THREE hours of freedom staring me in the face. No sooner had my arse touched my chair than I heard them coming down the hill. I though, 'No I can't, not again, really...' and legged it up the stairs, while the doorbell rang and the letter box clattered as the kids looked through it and the father said, 'Looks like they're not in, kids...'
I felt a total cow, but if I'd opened the door and said, 'Not now, busy', I think the bloke would have taken it badly.
Usually, I will answer the door, but I have honed the knack of blocking it 'to keep the dog in' and 'being busy'. If I do invite callers in, they are genuinely welcome and can stay for an hour.