I kinda think you are if the kids don't want to go.
Would you like to be dragged along to go see someone you don't want to see, in an uncomfortable situation you don't want to be in, with no choice to say no, and no choice about what happens in your life otherwise either? Even the things that are "for you" are all decided on and arranged by someone else... And now that person is using that as an excuse to force you along on an uncomfortable day. As if you owe them for what THEY have chosen.
Seriously, would you like it? Of course you wouldn't like it. You didn't ask for or choose any of it, and now you're being treated as unreasonable for not liking it on top (and for "not being good"!) .
Of course all this is unfair to the kids, and of course its selfish - you had them, parenting is literally an obligation and "I've parented them all summer!" is no excuse for anything. Yes, that's what you're supposed to do, and that's an obligated you accepted when you had kids. You are owed no rewards for it.
But in the absence of a father or relative (I assume) to take care of the kids doing something more comfortable for them, and you being so determined to see your friend, I suppose there is no alternative. Off merrily you go, enjoy.
But call it what it is, at least. I really don't understand parents who treat their children as lesser human beings whose feelings and opinions don't matter just as much as their parents. Not directed necessarily at you OP - you care enough to be posting about it. This is more about the replies you've gotten. At least you give enough of a shit to question yourself and think about your kids feelings... Some of the people replying blatently wouldn't.