Yes, stop talking to him.
He's not the 'him' you know, or knew, any more - or at least at the moment. He's behaving like this as he's completely switched off to you right now - you don't matter. He's engulfed by it. He won't be for long, by the way - reality will bite - and he'll come to and it's probably then that the more 'normal' reactions you might have expected will start. The real sorrys, the horror etc. Hopefully, by then you'll be well past this point and will simply look at him, no words to be said, and shut the door in his face.
You know what you need to know. The far FAR more important issues now are those to do with protecting you and your children. Make copies of all bank account details, his salary, pensions etc. GET HIS NATIONAL INSURANCE NUMBER! Joint account - lock it down, take everything out if possible and move it to yours. Etc.
You won't get through this - but nor should you want to. He's done you a favour, really, in getting to this point while your children are so small, and you can start again. He's not a keeper and never as - weak, deluded, insincere, flaky, cruel, selfish. This is what he is when the chips are down. No decent woman would want this for a husband. It's awful and painful but at least you know now. In ten years, you'll be looking back at this and thinking THANK GOD!
Get all the financial stuff absolutely sorted, then tell him he's out, and you'll be talking to a solicitor about staying in the house, getting onto CSA, etc. Any nonsense with him saying he's not leaving - tell him if he doesn't you'll be sending a few enlightening emails to a few people. And tell him he also wants to think about how a dated picture proving that he was viewing and sending porn when he was in sole charge of the children might look when it comes to discussing access arrangements, if he wants to start throwing his weight around there. I'd also be making a puzzled comment along the lines of - how strange that it was only when she showed that she was actually sniffing round half the office for a shag partner that he got mega interested! Most men would have felt a bit used and taken for a ride at that point, but he realised how much he liked her and got jealous of her because she showed that she was up for it with the first one in the queue?? Err, ok!
But above all, detach. He's a worm. You simply need to get rid of the infestation asap.