This was me three years ago, minus the children and the shared home. He started being indifferent and playing mind games quite early on, but like you, I excused it. Multiple repeated instances of nastiness interspersed with apparent generosity, compliments and kindness. Long distance so in some ways I kept thinking, fresh start now, I'll go back and he promised to change and was sorry and said all the right things. I had known him a long time... so I thought, well, we have shared history, we can work things out. Lost count of how many time I walked, ended it and then went back. Two or three days into being back with him, he'd make a snarky comment, openly admire another woman, craning his neck and ogling and then make a snide "joke" about my appearance, something I couldn't change like my skin, my bone structure etc. We'd be having a nice time relaxing at home and he'd announce he was bored out of nowhere. Then have another verbal jibe at me that I didn't do the dishes, despite insisting I did not need to wash up and he would do it. I could never win. I was in a power play, a game for him to keep me in my place, insecure about myself and constantly forgiving him.
When I called him out on what he had said or done, he would either deny it happened or shift the blame onto me, claiming I had said or done something obnoxious myself.... completely untrue. I could be sitting in silence and he would literally come out with some observation about how I didn't look good enough, then deny it, say I was too sensitive, I always misunderstood him.
I never would have named it abuse until I started to google for explanations about why someone would behave like this. Little did I know I was trapped in a version of the abuse cycle, honeymoon period (albeit very brief), followed by tension build, verbal aggression then "apologies". What was confusing was the really "good" times, which could be great. Laughter and getting drunk and great sex... however, now I'm several years out, I look back and understand that the good times weren't actually real if he never really loved or truly ever even liked me. He had had two failed marriages and the second one had a restraining order on him.
So I should have known, right? So many red flags. But when you're in the middle of it and you have "bonded" with this person, you can't see it. It is a trauma bond, have a look online for information about abusive relationships and I bet you will recognise yourself and the situation.
I wasted time but it was a huge lesson in boundaries, self-love, respect and that there really are some horrible people out there who only want to use and put others down in order to feel good about themselves. Please don't waste the rest of your life and expose your poor children to this. The impact on your mental health will be worse than you realise and for young developing brains, even worse. There is a known medical phenomena, dysregulation of the HPA axis, check it out:
narcissistfamilyfiles.com/2017/03/02/adult-children-of-narcissists-face-trauma-induced-health-risks/
There's a ton of information out there about abusive relationships. I'd urge you to open your eyes to what has been done to you and is continuing to be done to you and your children.
It has taken me a good few years to get my self-esteem back on track and I'm ready to date again. In no rush, been single for over three years now, but I will never tolerate any red flags again. This man you are with, he is telling you he hates you. Believe him and start planning your future without him. In ten years time, would you want to be with this person, in the same place, hearing the same manipulation? Never receiving caring, consistent love or respect?
You are wasting your love. Turn those feelings on yourself and love yourself and take the rose coloured spectacles off.