I think he thought Brett threatened him - that's different from thinking he was going to physically hurt him at that time. And I think they glossed over how aggressive Brett was being because they couldn't acknowledge a threat of violence and let him stay in the programme although apparently Selina pushed Charliene and they let her remain so who knows?
Again, maybe it was an editing issue but it didn't sound as though Scott had been particularly arsey when he asked Brett to move the irons or whatever it was that caused Brett to tell him not to talk to him like that.
In my earlier post, I said it was poor that Scott ignored Sam so I completely agree on your manners point.