OP-
Your husband is one cunting bellend of a dick.
I struggled to think of words so I put all the bad ones that came to mind into one sentence. Now Im going to refer to him as the CBoD.
He is nothing but a power hungry manipulative shit. He is doing his best to set limits for you, try to get you to obey him and break you down. I would put money on it that if you had adhered to any of his little 'curfews' he would have been 'nice' to you. Because you behaved.
You say you dont fear for your physical safety but you describe being pushed and violent acts happening in your close proximity- I am not for one second blaming you, but you are normalising events that are not normal or okay at all. Ever. DV doesnt start out on a first date- no one will ever go out with a person who says 'goodnight' then slaps them round the face... he's slowly pushing the boundaries and Im willing to bet youve rationalised it by saying 'oh, well he did X, but that was just temper, at least he didnt do Y'. Not acceptable.
The sexual stuff to me just smacks of ownership. Not love, or anything close. You are mine and I can do whatever I want whenever I want.
Do not beat yourself up for lashing out at him and do NOT use that to mitigate his behaviour. We all have our limits and where you have reached yours he is again trying to use it to manipulate you. He has the nerve to call you violent and aggressive? Normally Id say to tell him where to shove it but you seem to be in such a volatile situation.
Record everything. Photograph everything. Email every single thing to your solicitor and if he makes you feel scared, intimidated or under threat- call the police.
He might be a big man in the house where he is the bully, but he wont be in front of the police. Bullies never are- Id again put money on a sycophantic little 'poor me' story. Call your GP and make an appointment for you and the kids so there is a record externally of the police/court that this is affecting you and your children.
You need this CBoD away from you and your children as soon as is possible.
Good luck x