There is no option here except to get rid of this man.
You don't 'plead'. And you offer no further ultimatum. The only way you are going to feel happy and right within yourself is if you are now true to yourself.
You don't want to be with a lying scrote - so you decide not to be with a lying scrote. You don't give the scrote the power - you take the power and say 'you've shown that you aren't up to it. Goodbye.'
Think about this. What he is doing is demonstrating that having her - in whatever capacity - is more important than your feelings, your relationship, him and you. Just the same as if he was still having the affair. He's talked the talk when found out, so he doesn't lose his comfy life. But he cares no more for you now than he did when he was shagging her, and is still lying to you and making a mockery of any supposed 'marriage' between you.
That tells you all you need to know. You will not get what you want from this man. He's third-rate goods, if that. Can't commit, can't stay faithful, can't be honest, can't value his marriage. Can't stop lying.
He's setting himself up for the wobbly, uncertain, trashy, third-rate emotional life he's currently offering to you. Let him go have that as an excuse for happiness, but for god's sake don't join him down that lonely, unsatisfactory path when you've got about 100x more spirit, honesty and love in you. You don't need a man like this - nobody does.
He'll find that out. Just make sure that by the time you're hearing the third sob story about how the latest love of his life has thrown him out after she found out about that meaningless series of texts between him and the woman in the pub, that you're sitting pretty (and divorced, and secure and happy) and not the mug staring at his lying face across the breakfast table.