"What exactly do people think might have happened that would be so catastrophic??"
Um, a car crash perhaps - what were the DCs to do if their DF didn't return home? Or the 3yo choking on something - the Df couldn't have got home in time to save him/her. Or some other accident.
Yes, it's unlikely but that doesn't stop it happening! A&E and the morgues are full of people who had unlikely things happen to them.
That the OP's husband was just popping out for bread, not an emergency suggests it might be something he's in a habit of doing, and that increases the chances of something awful happening as a result of his negligence.
This kind of thinking annoys me tbh. People say things like "we took ourselves to school aged 5, it didn't hurt us". Well yes, you lived to tell the tale. My friend lost two young siblings on two separate occasions to RTAs on the way to school.
Allowing very young DCs to be unsupervised raises the risk of something happening to them, fairly obvious, isn't it?! Yes DCs should be encouraged to be independent, but the OP's example sounds like this was about the Dad's needs / laziness in not bothering to take them to the shops, not about the DCs' needs.