Oh my goodness darling, this madness needs to stop right now.
This situation really isn't your fault, per se. I'm afraid this man deliberately targeted you a couple of years ago, and actively selected you as someone who would (most likely) put up with all his arrogant crap.
Imagine him as a hungry lion watching a herd of antelope. Then his eyes lit up when he spotted one of the antelope was limping...well, you were that limping antelope I'm afraid.
He actually wanted someone who was a bit vulnerable, too open to suggestion and quite easily manipulated. Why? Well I could answer that, but it would be lengthy and actually it doesn't matter. Because even if I could make you understand why he targeted someone like you, you will still never, ever be able to change him.
People do not change. Let me just repeat that. People. Do. Not. Change. So you can forget about changing him, or him changing himself. It will never happen.
I do understand that he has got you so confused that you can no longer trust yourself to know black from white. And, of course, that is exactly how he likes you. Uncertain, unsure, hesitant and apologetic. Do you find yourself apologising to him much of the time? I'm guessing that you do?
In that case please trust the experienced judgement of me (and all the other experienced posters on this thread) when we tell you to disengage from this man. Disengage and run as fast as you can. He is bad news and bad for you in every single way.
You say that if you ignore him then he will 'punish you' by ignoring you for days and days.
Can you not see that he is already ignoring you and has been for 2 years now. You say he doesn't listen to you. He doesn't care if you're feeling down. He ignores your sexual needs. He doesn't see you for a week, then only calls round after having been to the gym.
He. Is. Already. Ignoring. You. He always has..and he always will.
So what if he doesn't come round? What is it you will miss exactly? His spiteful sarcasm? His constant belittling of you? His crap and selfish sexual performance? His blatant disdain for any concerns or worries you have?
And you would miss all of that? Really?
You need to wipe him off your shoe like the piece of nasty dog shit that he is and sashay on down the road.
Despite having known this arse for 2 years you haven't had a relationship with him. Leastways, not a healthy, happy and loving relationship. What you have had is something twisted and wretched and ultimately damning and destructive.
Walk away now, right now. No excuses. No ifs or buts.
If you don't walk away now this man won't stop until you've bought a house with him. He will then most likely insist you have a baby. He won't marry you, of course. He'll use marriage as another carrot on a stick to beat you with. Always making you jump through higher and harder hoops to prove you are worthy of him marrying you.
You will never jump high enough though. He doesn't want you to. He just wants you endlessly and pathetically jumping, jumping, jumping.
This man won't stop until you have enabled him to buy a house and you have a child/children. And if you don't walk away now, you will be back posting on here in a few years time saying that he has started screaming at your children, and last night he threw one of them across the room...
Because this won't end with you. He will want to demean and abuse any children you have too. Because he is damaged and flawed and very dangerous to be around.
You cannot make this right. No one could. You need to protect yourself from him and walk away. Please, please just walk away.