He is the abuser not you but he likes it when you get so upset and do something wrong, anything wrong (because of his abuse) so that he can point out and say, "it's you, not me".
My ex would hit me whilst I was in bed, then pull the covers off whilst I tried to hide, shake the bed, poke me, if I covered my ears at his shouting he would pull my hands off my ears. He would punch the pillow next to my head whilst telling me he had such self control as look he didn't punch me.
This could go on for hours whilst he lectured and shouted at me about whatever I had done wrong. A few times I had jumped up and pushed him out of the bedroom and slammed the door telling him to leave me alone. One time I slapped him as I was pushing him out the door, as he had grabbed and twisted my arm. I was then the abuser. He was so happy and would always bring it up. This was after years of his escalating behaviour.
One time after he had gone on at me for a long time as I lay in bed and I got up and started shouting back at him, I was shouting at the top of my voice and crying, "please leave me alone, I'm scared of you".
He secretly recorded that and later told me how he played it to himself, to prove how crazy I was. When he found someone else (I took the kids to my mum's after that event of I'm scared of you, as I realised how bad it was and later on that week he tried to strangle me), I was later told he played it to his family to prove how bad I was. I had a thread on here at this time.
You are not the abuser, he is. Never forget that, just get rid of him, go low contact or no contact if you can. If you have children go extremely low contact and parallel parent. Put everything in writing only, emails and texts, as every word you say will be twisted and used against you.