I don't think Miles or Bruce will either leap to Rob's defence or castigate him because he's probably completely irrlevant to them as they feature much higher up the pecking order in that family. They probably barely know he's alive and from what they do know of him they think he's weak and overcompensates for his failings - as a child he would have been a really sore loser. I imagine they feel a sort of casual disdain for him when they think of him at all.
I see Rob as having a huge inferiority complex with an extra helping of misogyny that manifests itself in aggressive behaviour to compensate. He targets this behaviour at anyone he feels irrational hatred for: woman and gay people (Helen and Jess; Adam and Ian plus he called Kirsty a dyke because it's the worst insult he can think of).
Ursula has fed the misogyny with her sexist and outdated views and worshipping of the males but there is a very clear pecking order of husband, heir... then spare which directly contributes to the inadequacy Rob feels.
Rob is no doubt very bitter (we used to hear some of this in earlier episodes when taking about his family) and blames everyone else for his misfortunes, even those of his own making (quitting his job at Berrow before the investigation into his conduct) and refuses to acknowledge how he is complicit in them at all. Although he must be aware of them because he takes great care to cover them up (deleting photo at the cricket/forging records at Berrow) and has a real need to be seen as a saviour and someone people look up to and aspire to be like - so he makes himself look better by sabotage to either "prove" his opinions or give unfair advantage and the expense of others (the culvert/Ambridge Organics/new lines at the Bridge Farm shop).
Because of this inferiority complex he compensates by being a bully and has invented himself as the hero to boost his own ego and to be a hero there has to be victims (Jess and Helen plus who knows who else from his murky past). He has these women in his life who he seeks to belittle and control because it makes him feel powerful.