op - please make sure you protect yourself in every way possible. Get an emergency kit together for you and the dc's, so that you can leave at a moment's notice if you feel threatened. Also, work out a plan with someone you trust - a friend, your mum or another close relative, so that if the worst comes to the worst and you need to escape quickly, you have somewhere to go for an immediate refuge.
Keep a journal of everything that's happened and continues to happen - any controlling behaviour, any odd emails, texts and letters he's sent you which might allude to what's happened or apologising or even making excuses for it.
But either way, it's critical to keep the journal and the emergency kit somewhere he won't find them.
This happened to someone close to me. It'd been going on for years before she mentioned it to me, saying that other people in the past had been incredulous, dismissive or minimised it - things along the lines of "it couldn't have been that bad if you're still with him".
Anyway, the moment she had an outlet and was able to talk it through and realise that, despite his best attempts to gaslight, manipulate and control her, what he wasn't doing wasn't acceptable. She'd told him numerous times that she didn't consent to this but the moment she started getting external validation, he started escalating the controlling behaviour and getting worse.
Then it came to a head, the police were called and he was arrested. In court, his solicitor did his best to paint both her and I as bitter, manipulative women who planned together to get an innocent man convicted and jailed because she'd benefit more than if they divorced.
Luckily, our texts discussing it and checking in with each other, along with the journal she kept and his attempts at gaslighting and threatening her were all consistent and allowed them to identify 5 specific occasions to charge him with rape. (there were many more but they need to be able to prosecute on proof of specific instances).
He's now in jail and she's rebuilding her life. The Freedom Programme was also something she found useful.
I don't think people on here are trying to minimise it, if I'm honest - I think it's more that they don't understand what it's like. God knows it's something I wouldn't have been able to truly understand before this. But, regardless, his behaviour is abhorrent, grotesque and illegal. Do NOT let him (or anyone else) convince you that it's normal, acceptable or pretend it isn't happening and you're paranoid.
But please remember that his current behaviour isn't the whole picture and if he feels he's losing control over what he clearly considers to be his property, he'll escalate to stop it getting out and to keep you quiet.
Please, please, please - get everything ready so that you can take your kids and leave at a moment's notice. Yours and their safety is paramount - do everything you can to protect yourself and trust your gut - if you feel scared, then you need to leave.
Sorry for the essay OP but, having seen it almost destroy an incredibly strong, intelligent and wonderful woman, my heart's breaking for you and the situation you're in. 
You'll deal with it in the way that works for you and once you do, you will be fine. And in the future, when he's rotting away somewhere wondering what went wrong and blaming everyone else but himself, you'll be the strong one that he couldn't break, despite what he's putting you through now.