He’s not begging or pleading because at the moment he still doesn’t want to choose. He’s sad he might be losing you but not enough to make a change. Whilst that is upsetting, at the same time, if he was claiming to have seen the error of his ways it would be difficult to stay strong and you would likely take him back. But the trust is gone and that is very hard to rebuild. He would need to do the majority of work and to be frank it doesn’t sound like he has the mental capacity to do it.
What he gets from you (and now your children) is emotional support. Whilst he’s out there being a white knight to this younger woman with health problems his MH problems are being supported by his family. What happens when you have checked out and moved on. Does this younger OW who obviously needs a man to look after her want to take on the responsibility of dealing with his issues too? Will they be solved by having another family and additional financial responsibilities?
It’s sad that he can’t snap out of it but it’s also extremely unattractive and frankly pathetic. He may never get to the begging stage because by the time he realises what he has lost he will know there is no way back.
You are amazingly strong. Visualise your future without him. All the things you would like to do as a family of three. Plan holidays with your friends. Imagine a future where you can experience the first butterflies of love again. You have so much to look forward to regardless of whether you are in a romantic relationship with him. Keep your head up high, your boundaries clear and nurture the relationships of those people that support and protect you.