That is early physical abuse. Actually it is not even that early an indicator like a small push or pinch. He is abusive and I would leave ASAP.
"The bit I find hard is that I haven't had a lot of love in my life, and he really seems to love me. Well, he says he does. Has talked about marriage and everything. When we're good, we're great but it's times like this I realise I've been an idiot and I'm fooling myself. I'm hanging on for the love and putting up with all the shit that comes with it. He expects all the affection to come from me, and he is always right, he can't stand me sticking up for myself when he accusses me of something, says I'm being stubborn and I can't accept I'm wrong. In fact, the more I sit here, the more I realise I probably didn't need to ask. It does make it easier though, because I've now got back up on how I'm feeling, where as before I would always doubt myself and let things go in an attempt to 'make it work'."
All classic signs of abuse, especially his seeming to love you. Sometimes abusers will pick on someone they know is vulnerable emotionally and groom them by seemingly filling that emotional void for them. That way they latch on to you later. Then when very serious abuse begins, he will start telling you that you are nothing without him and that other men will not love you or treat you well. That way he has groomed you into feeling you need his seeming love and attention and will feel frightened to live without it. According to the fact that you are "hanging on for the love", I would say you are already reaching that point.
I'm so sorry you are going through this, I don't say this often as it is very personal for me, but I was with someone who did this to me and when I look back on it I shudder to think I f I had stayed. We almost got married! It took the interference of others to get me away from it and I hope you will heed what people say here.
Please do what is right for you and take care of yourself, you are worth more than that.