OP, part of the issue here is that there are times when you are a willing partner and when sex works for both of you. This is what causes the confusion. It's hard to see that sex is something which is used against you when at other times this is something which you mutually agree on iyswim.
I was married to a man like this. I was a virgin when we got together and he was considerate in waiting until I was ready to have sex. He would have said no at any point during sex, and I have always given him credit for this.
But not long after we'd started sleeping together he left a message on my voicemail saying that as I was coming down for his birthday perhaps I should keep taking my pill so he could enjoy sex with me as a birthday treat. He knew to the day when my period was due, and one weekend he suggested there was no point me going to see him as I would be on. We had a massive row about that one.
Whenever he went away coming back there would be an expectation of sex. He would grope me constantly. Would make references to wanting to give/receive oral, neither of which I was a fan of. Very occasionally I would let him go down on me, but if I ever said no he would become annoyed and say that if only I would let him do it I know how much I would enjoy it.
He would refuse to kiss or hug me during my period, and one day when I asked him he said there was no point as he knew there was nothing else on offer. The day before my six week checkup after I'd given birth he took me to bed and made me have sex because "you need to know that it's possible." I'd had an episiotomy and it was horrid. I cried during and afterwards, even though he was gentle I felt horribly used and as if I had to perform.
Added to all of that he systematically isolated mE from friends and family. Made it impossible for me to go back to work after I had DC, moved us hundreds of miles away from any support network.
This all went on over a period of about twelve years. Thing is, on the surface we had a decent relationship. He enabled me to be a SAHM, we had joint finance even though I wasn't earning, but we had no mutual friends, they were all his, and he made sure he told me regularly how sad he was that I didn't have friends. 
.
Unfortunately the end of the relationship was not my finest hour. I met someone who made me realise that relationships are not supposed to be like this and I had a brief affair. It is something I regret bitterly. But although the affair ended I realised that I had to leave, and I did.
Obviously the reasons why the marriage ended are all my fault according to him and everyone else who knows him., and even my own family, because apparently he didn't hit me so therefore he wasn't abusive.
But it was only as time has gone on that I've realised the true extent of the abuse he put me through and even this thread has triggered memories for me.
It's very easy for people to ask on the surface why you are still with him. When you're there the bad bits are horrible, but they are also mixed with the good, and there will be good bits, otherwise it's not a marriage.
You need to communicate with him. He needs to be made to see what he's doing is unacceptable. I did have these conversations with my H more than once, and things would change for a while, but they always went backwards again.
But the starting point is a discussion where you absolutely lay on the line how he makes you feel. If he doesn't take that on board then it's time to consider the future.