I have three older kids, no answers, but just a few thoughts-
I feel like you probably overreacted a bit, especially when you took his books from his room. Taking away the phone would be a reasonable consequence for the rude texts. Perhaps just say: "You have to go to bed now, and you are losing your phone for a day because of the rude texts". Then refuse to engage and just keep saying "bed".
The violence and bossiness is more complicated, and I think can only be talked about when everyone is calm; I am not sure that punishments ever help with this sort of issue. It is more a situation where he needs to be separated from his brother until he calms down enough to see how violence makes everyone feel. He also needs to be aware of how serious the consequences can be as he gets older and stronger. I had a serious discussion with my oldest son when he was this age and pointed out that he could easily do something which he would regret for the rest of his life. I kind of shocked him into thinking about how scary and dangerous physical aggression is when it comes from a boy who is as big as a man.
In general, I think you should do what you can to avoid the kind of power struggle situation which developed. I would avoid: "I told him to stop and go to bed or he'd lose console privledges after school today", because teens will see something like this as a challenge and will feel like they are losing face at that point if they back down and say: "I am really sorry Mum; I was rude and unpleasant, and I feel bad. Maybe I was unreasonable because I get cross with my brother's childishness and sometimes I wish I had more privacy." Of course ultimately, you want to get him to the point where he can say something along those lines, but it takes years...
In theory, my goal is to get my kids to tell me how they feel- (perhaps annoyed at a brother, frustrated at the perceived injustice)- and to get them to understand how their behaviour makes me feel. I hope that once they feel really listened-to they can slowly become less unreasonable. Also, in theory, they will feel that I understand their feelings and we are working together towards a common goal of making everyone happy, considerate, and independent. It is impossible to do this in the middle of a quarrel.
Have you read "How to Talk so Kids will listen and Listen so Kids will Talk"? It is one of the few parenting books which I thought was quite helpful. There is a teen version by the same authors which might be even better, but I haven't read it, so not sure.
Sorry I have rambled on a bit and repeated myself, but am too lazy to edit or clarify, maybe because I don't have a clear answer.