Your partner was 24/25ish when his son was born. By and large, you're all the adult you're going to be by that age. But he let his mummy tell him what to do and how it was going to be. Not a good sign.
for the future, what thefirstMrsDeVere said.
When you tried to discuss it "He brushes me off and says that's enough I'm tired," - well pardon my French, but fuck tired. Tired is the normal state of being a parent. And he'll be a damn sight more tired having his eldest stay with you.
TBH, you sound like the only adult in this whole sorry scenario, and you may well have to treat the others as children. (You sound as if you have the right ideas to me - routine, consistency etc). If this is going to work, you will have to lay down the law to all concerned. You are at least in a strong position with regard to housing - the house is yours and he can have no claim to it by the sound of things. In your shoes, I would have to tell him that if the boy comes to stay with the rest of HIS family, it is your way or no way at all. His parents will have no say in the matter (I would probably start with them not being welcome in my house for the first few months, and only afterwards under strict instructions not to undermine and the understanding that they will be kicked out the second they start). Your partner will follow YOUR lead with regard to discipline etc. He cannot claim that he knows best, he obviously doesn't. Your stepson will have to be socialised to better behaviour, and that's going to be tough, and impossible if you, collectively, are not consistent.
Otherwise I would tell your OH that if he wants his son with him, then they can both find other digs, you have no desire to become his landlady.
Poor boy, he has a miserable life in front of him if he continues as he is. (But do ot be guilt-tripped by this. He is not your responsibility unless you choose it to be so and only on your terms. The adults already in his life have all let him down very badly.)