You keep justifying it. Which is a shame, because next time, and next time, anybody say something to your kids, you will blow off, swear and shout abuse, and this way teaching them the same.
You ruined an old mans outing.
He could have been from a care home. Maybe it was his birthday. Maybe it was the only time the carer should take him.
I think you need to give advance notices any time you and your family go on outings, so that people can avoid.
Put an advert in the paper:
"We are a really foul-mouthed family of 4, we intend to go to Butlins on the 24th of July, if you cannot handle shouting and swearing and being verbally abused, may we kindly suggest you choose another time."
Or in language more appropriate for your family. 