No child is all that predictable from day to day - though some children obviously have a lot more to deal with than others. So, go out whenever you please, but when they kick off, then yes, take them home. At that point neither they, you or anyone else are benefiting from their presence.
I found this very difficult to do - I was a single parent (still am), job, caring responsibilities, the works, and if I had to walk away from a trolley of food, I didn't know when I would be able to go back into the shop. But I did it, because it was the right thing for DS, for me, and for everyone else around us at that point.
There is, amazingly, a middle ground somewhere in between "lock your children in the attic" and "do what you like, however inappropriate, with resources that other people also use".