They covered this perfectly in an episode of Friends.
[Rachel is nervous about leaving Emma alone in the apartment for a few minutes]
Rachel: What if she jumped out the basinet?
Ross: Can't hold her own head up, but yeah, jumped.
Rachel: Oh my God, I left the water running.
Ross: Rach, you did not leave the water running. Please, just pull yourself together, okay?
Rachel: Ah, did I leave the stove on?
Ross: You haven't cooked since 1996.
Rachel: Is the window open? Because if there's a window open, a bird could fly in there.
Ross: Oh my god, you know what, yeah, I think you're right. I think... listen, listen.
Rachel: Huh?
Ross: A pigeon, a pigeon. No, no wait, no, no, an eagle flew in. Landed on the stove and caught fire. The baby, seeing this, jumps across the apartment to the mighty bird's aid. The eagle, however, misconstrues it as an act of aggression and grabs the baby in its talons. Meanwhile the faucet fills the apartment with water. Baby and bird still ablaze are locked in a death grip, swirling around in the whirlpool that fills the apartment.
Rachel: Boy, are you gonna be sorry if that's true.
Some children are fine to be left and some aren't. Only you know how much you trust your children and how confident and independent you have taught them to be. I sense some trust issues with your DCs and your DH. No-one can come to terms with them until you do. You have to work out what is acceptable to you and decide if this is rational. Then you need to calmly discuss this with your DH and a conversation with your DCs would also not go amiss...... Soon, your DCs will be 11 and off to big school where they quite often start to take the bus to school (I did when I was 11) on their own. Then they are off to college and then off to Uni or out to work. I doubt very much that you will be holding their hand when they are 30, so you have to decide in your own mind, how you will get from now to when they are 17.