OP, I'm so sorry for your experience. I know only too well how mentally confusing and draining it is when someone who is supposed to love you does something that shakes your entire view of them.
I had previous partners who used to do similar things to yours (when I was asleep or medicated or drunk), despite knowing I didn't like it after the fact. Most would try and tell me I was making a big deal out of it. One of them, like yours, would mutter something along the lines of 'sorry', but then do it again.
It got to the point where I really felt like I was the abnormal one, and so tolerated the behaviour even though it did make me feel dirty and like a piece of meat. It wore down my own belief in my right to refuse until it was almost nonexistant.
It was only until I met my present husband that I realised just what a healthy, respectful, and consensual sex life could be like.
He would start stroking me gently while I was asleep (just on the arm or down my side). I would wake up, not want to have sex, but not say or do anything to refuse (because I felt this would be making a fuss!)
To my surprise, he would stroke me gently for a few minutes, and then when he saw I wasn't responding... he would stop and go to sleep!
It took a couple more times of this happening for me to realise he was waiting for my consensual response before proceeding.
His approach has never changed. To this day, if I respond and begin touching/kissing him back, we proceed. If I say no, he stops. If I lie there and do nothing/continue sleeping, he stops.
It is only in the light of this relationship that I can see how awful (and sexually objectified) I actually felt in my previous relationships. I feel for you because I know how confused I would have felt back then if someone had tried to open my eyes about how I was being treated.
A loving partner will care about how you feel about sex in any circumstances, not just about getting what they want from you.
Sorry for the length! But your post really struck a chord in my heart. You shouldn't have to feel the way you do.