Run a mile!!
"If he doesn't hear from me for a few hours he has been anxious and he said his mind runs away....apparently his wife used to cheat."
Never, ever stay with someone who thinks like this. This is absolutely the start of controlling behaviour and it only gets worse. He wants you to shrink your world so he is the centre of your world and you have no other friends or interests away from him, and is he prepared to emotionally blackmail you now to start this process. He will chip away at you, but by bit. ONce he has his claws into you, he will eventually bully, shame and intimidate you to get what he wants.
Also this "Once he was upset because he bought me some new dungarees and he was upset my ex saw me in them before him because I dropped off the kids." is fucking ridiculous and he should be ashamed of himself for saying it. Don't brush it under the carpet, surely you would never dream of saying something like this to someone,
Men like this will drive your friends away as you're so ashamed of his behaviour, or because he just makes is such hard work to see them as you then have to deal with his sulking and nonsense.
Men like this stop you being able to enjoy the simple things in life as you'll be contantly aware that he'll be unhappy that you're not in contact with him, or dealing with constant texts accusing you of cheating because you have dared to meet a friend in the cafe for a coffee.
What he's doing at the moment is compliance, testing, seeing what you'll put up with. He'll keep on pushing and pushing.
There is no winning, there is no level of understanding that will help. He is a toxic abusive prick. Please save yourself a world of hearbreak and run.
Have you done a Clare's Law request with the police? I would strongly suggest doing this, and at the same time making them aware that you are leaving a man who you know has had to do a course for DV before.