In the above 'situation' basically I was upset and he used sex to try and make me feel better.
You didn't want sex then, so that is rape OP. 
He also nearly cheated on you, or maybe he did. I'm sure he will at some point. 
He's scary and sleazy.
It's not completely abnormal though is it? Is it not quite a typical man thing?
Being rapey and sleazy? It's fairly common but we don't have to be involved with these men.
The 'drama' was caused by me in the first place
No, he had a choice in how he acted, he didn't have to respond with rape.
Do I really just 'give up' on my relationship because of this?
Rape, and nearly cheating on you, or probably actually cheating on you? It's your choice but I wouldn't really say this is relationship 'goals.' You could find a decent guy who doesn't do this.
He's not physically abusive
Oh, but he is.
I certainly don't feel controlled by him in anyway
He's shagged you when you didn't want it, and at other times in ways you didn't enjoy and he knew you wouldn't enjoy, that's why he apologized afterwards as he knew he was doing stuff sexually that was unpleasant for you.
He controls you sexually a) when he forced you into sex you didn't want and b) The calling another woman over for sex is by implication telling you that if you don't give it he'll get it somewhere else (in fact, he probably will cheat on you whether you have sex with him/he rapes you or not.)
But I'm worried its going to be a problem if his main way of communicating is through sex
He's not communicating anything with the weird sex incidents you mention except he is a sexual sadist who gets off on hurting you or shagging you when you don't want it.