Haven't read through the replies so sorry if I repeat.
The fact that you are questioning it and not asking for a solution says that you want out. Everyone has good and bad but you can't hold out for the rare good times. There will be someone else who will give you lots and lots of good times.
If you stay you are just giving him permission to continue treating you this way. It will not get better unless you do something about it!
Break it off. Do things that make you happy. See friends, go out, pamper yourself. Cut contact with him, just for a month. Ignore the calls, ignore the texts. If he contacts you a lot just simply say that you are taking some time for yourself and you will contact him when you are ready. Make it so you rely on yourself to feel good not him. If after that you want to sort it out then try. At least then you will know you are with him because you want him not because you need him. He won't meet someone else in that time, if he does you still have the upper hand because you are the one he has history with and you don't just turn feelings off. If he loves you and wants to make it work he will be there after the month wanting to change. If he's not then he wasn't the one and you have lost nothing. You don't want to waste any more time on him if he isn't the one, don't put up with it for another year just to look back and say why did I not end it sooner.
I put up with a marriage that was awful. My husband was controlling, he put me down and made my life miserable. We had great times, because he loved me and he really did but he was so scared of losing me that he put me down so I thought that no one else would want me. The relationship was about him not about us. He kept me to himself so I couldn't meet anyone else. I stopped loving him because of it, I hated him. I was putting up with it because I thought 'I'm married I should make it work', to prove a point to other people who said it wouldnt work, because of the house, because we had a child, because i thought he'd make my life hell if I left. Now I never did have the strength to end it, he accused me of cheating and left me, I didn't fight it, I packed my bags and I went and for 6months he begged for me back, for 6months he wanted to sort it, I didn't. I look back on it and think I should have left a long time before. I wish that I hadn't wasted that much time on someone who made me feel that way