Totally wouldn't blame you for going home and I don't think it's your fault that they are bickering and playing up - normal for their age and a holiday/different routine always makes behaviour worse IME. Nightmare when you're on your own and outnumbered as well.
But also - natural consequences, call their bluff and be matter-of-fact. And for the 6-year-old bullying his brother, the consequence for that should be treats and priorities for the brother.
Shoe/PJ fuss. OK don't wear shoes then/keep your pjs on, I don't care. Chuck shoes/clothes in a bag and say they can have them if they ask nicely, and set off.
Refusing to get dressed, leave the flat etc. "Oh dear, I guess we won't be doing <insert whatever part of the day will appeal to them, ice creams, show, amazing climbing frame etc> It will be pretty boring sitting here all day though. Just say if you change your minds." No TV, screens, snacks etc, just sit there. They'll be bored stupid in about 5 minutes,
Seat fuss "OK I'm sitting here to stop you two arguing" (take middle seat). Ignore fuss and say "well you wouldn't make your minds up, behave yourself next time"
ANY teasing/bullying "OK DS6 you have been told over and over and you are being mean again. That means DS4 gets a treat/to decide what we do this afternoon/to pick the seat he likes/etc and every time you bully him he will get another treat or choice." I have found this stops it in its tracks - remember to then praise for treating his brother better.
Some great advice I read on here long ago was to play the 'bored cop". Avoid getting worked up and stressed. Just chill, point out they are not going to get what they want by dicking around, let them experience natural consequences and allow some time for them to make the right decision.
Then be effusive in your praise "I'm so happy we all managed to get out of the house in the end and had a great day. it was loads of fun doing XYZ with you, what did you like best?" etc.
Not judging you though, nor am I claiming to be parent of the year (far from it) but these things helped me when my DC were these ages.