Op, if it's any consolation I had a 'peacemaker' intervene in an argument I had once and it does make you feel like you were in the wrong.
What happened was, I was travelling home from a salsa class late enough at night approx. 10pm on a bus in London. The bus was fairly empty, and it was a long journey and I wanted to be on the top deck so that I could recognise when I was getting close to home, so I'd know when to get off, as I had never travelled the bus route before.
As you might imagine, the bus was pretty empty apart from one girl at the front of the bus and maybe a couple further back. So I sat also at the front of the top deck.
Next thing a man came up and sat beside me. Bear in mind there were probably 50 empty seats on the bus, and he chose to sit beside me. But I turned on my danger radar and said nothing. Then he started sort of hopping his knees and fidgeting with his fingers on his knees. AND THAT'S WHEN THE ROW STARTED.
So I tolerated it for about 30 seconds, again bear in mind, the time of night, the empty seats on the whole bus and that I was a lone female and he was male.
So I said 'I'm sorry, but you're making me uncomfortable, would you mind moving to another seat'. WELL he tore strips off me about HIS RIGHTS etc. This argument went on for about 30 seconds and I said - please just MOVE - THERE ARE 50 OTHER SEATS ON THE BUS, WHY DID YOU SIT HERE?
Guess what he said?
YOU ARE RACIST. YOU'RE RACIST. THIS IS RACISM.
Now genuinely, and I'm not kidding you that up until that point I hadn't even registered what colour he was, so it certainly wasn't racism. BUT Miss Has-to-stick-her-nose-in in the adjacent front seat goes 'JEEZ - LEAVE HIM ALONE - HE CAN SIT WHERE HE WANTS'.
So yes OP, I totally understand how you can pick a battle, but then get totally undermined by an uninvolved stranger. I felt like some sort of a deviant madwoman for a while, until I stewed over it (days) and realised that no - he had no good reason to sit beside me in an empty bus and I'm still convinced he had no good reason for doing it.