Sure, I don't think anyone's arguing that it's a completely risk free choice. The house might catch fire.
But if you strap them into a carseat & head off to Sainsbury's, the car might crash. If only you'd had your shopping delivered.
Or if you go round to your friend's house for coffee & so your dc can play with hers, a madman might break in & slaughter you all with an axe. If only you'd had coffee with your other friend who didn't have an axeman scheduled to call at her house today!
Much of the heeby-jeebies about this one stems from 'But you don't need to go for a drink. BAD mother.' (if you don't think about your dc after you put them to bed, that's bad parenting...)
I think it's fine to think 'no, that's not a risk, however slight, I'm happy with - I wouldn't be able to relax & enjoy my mulled wine anyway'. Fair enough.
The 'well it's against the law, well OK it's not against the law but Social Services will come & tell you off, well OK not that either but but but Bad Parenting... ' routine - that's just a bit silly.