You shouldn't allow anyone to speak to you like that.
He's made his choice to leave and you should end it permanently, no more contact, that's it.
Have you actually asked him if he wants the relationship to end?
It sounds like a selfish fucker's tactic to get his own way, you put bounderies in place, deny him sex and he leaves you, you then pine for him and he enjoys you begging him for his affection and company.
Pretty common tactic for abusers to keep you putting up with this shit.
The only way it ends is when you end contact totally, there is no winning or making him realise he hurts you, he knows, it's intentional, to get you back in the box and for your opinions and wants to never be met.
Keep away from him, he ain't nice.