He's a nasty bully and regardless of whether or not he actually hits you or drags you around, the fact that your first instinct when you slip and fall on the stairs, spilling a drink, is to apologise to him speaks volumes. He's really done a number on you 
While a two parent family may seem like the ideal, your DCs need to be protected from this, they cannot be around someone who has so little respect for their mum or for them. He has told you that he lumps you all in together as people who don't respect HIS house. If he hasn't already, he will be turning his anger and aggression on to them unless you make him leave.
If he's serious about sorting himself out he can only do that when he has lost the comfort and security of your relationship. While he feels that you tolerate his behaviour he has no incentive to change.
Financially and logistically it may seem impossible but thousands of women can attest to the fact that it isn't impossible. They've done it and built new lives for themselves under really tricky circumstances.
You deserve better and so do your DCs. Please send a strong clear message to your H that you will not accept him as he is. Empty promises that he's changed mean nothing.
Until he starts to see you as a human worthy of love and respect he will continue to belittle and intimidate you. Generally with men like this, the pushes and shoves become harder and the things thrown become bigger and before you know it you're having to lie to medical professionals about you got your injuries. DV escalates so even if you don't think its bad enough to leave NOW, it will be.
He needs to find ways to deal with his anger and he needs some therapy to help him to empathise and have respect for people other than himself.
Avoiding therapy because he doesn't want anyone to know is cowardly and pointless. They'll all know when the police car/ambulance/hearse is sitting outside your house.