So sorry that you are going through this.
Can I make one suggestion - no matter what you think you might do or want in the future - make him leave now. Even if it ends up only being for a week or two. Tell him you don't know what you want, you don't know if you can forgive him, and you need time to think. He NEEDS to sweat and panic. Or there's a huge danger that after the first few weeks of upset, Mr. Cocky will look back and think 'well that wasn't so bad - she was ok about it really - she clearly is prepared to stay with me if I cheat - well hurrah!' And although he'll be telling himself he'll never do anything like that again, the fear of retribution and consequences will have gone.
The 'resolution' of this looks to have been dangerously easy for him. You are crushed, but from the outside, it looks very much as if despite some shouting and crying, he's basically got away with it. He might have to keep his head down for a while, but phew - life basically goes on as normal.
This is not good, especially as it looks as if he's nowhere near being the remorseful creature he claims to be. Look at the facts. HE ONLY TOLD YOU BECAUSE HER HUSBAND FOUND OUT! If he hadn't, and OW hadn't suddenly had a fright at, no doubt, being chucked out on her own ear, THEY'D STILL BE FUCKING. If he'd found out and OW had decided she'd rather have your H, he may well have come home that day and told you he was leaving. You'll never know. But don't kid yourself - this little scumbag hasn't proven anything except what a shit he is, and there's a good chance he's only there because his new little love plans have fallen through.
Given that, you're taking a HUGE risk here. Now, I'm not saying that you shouldn't take it - there are always good reasons for giving something another go. But DON'T kid yourself. If he had told you for no other reason than guilt, if he had truly ended it - you would have more reason to think that this could be rebuilt. But that's NOT the case. He got busted. Very different. So be very very careful about making it anything other than very hard for him.
Oh, and he's doubtless lying about lots of the stuff he's telling you - the saying he told her he wouldn't leave you, for example - let's face it, this little worm wouldn't have said anything of the kind to his mistress while they were lying there entwined - come on, really? It's far more likely that they had lovey-dovey conversations about their new life together before it all blew up and he got bounced back to you.
I hope this doesn't sound too harsh, but this man sounds like a complete stinker. Don't forget or downplay the reality of the course of events here. He hasn't proved that he wanted to end it, that he wanted to stay with you, or ANYTHING. All you know is that he had the choice of you or nothing. I'd be very very uncomfortable with that.