So, in the hypothesis you outline, it would be the kindest thing to say that to the partner you are leaving. Adding, perhaps a sincere apology for not raising any doubts or misgivings about the relationship in an open way when they first started having feelings for someone else and started thinking that perhaps the current relationship wasn't right after all.
Dealing only with that example and not other types of affairs where there's been longstanding and openly discussed unhappiness, or affairs where people think that fancying someone else must mean the current relationship is terrible even if they'd been truthfully happy beforehand, being honest with someone you've once loved and whom your children still do love, gives them some choices.
In your example, the healthiest response from the person left behind would be: "Okay, this isn't my fault. My partner didn't raise any unhappiness with me before he chose to have an affair and I can see that an intense, dangerous and exciting new relationship will always show an established, safe, comfortable one in a negative light. There's nothing I could have done to stop that and although I wish that he'd shared his doubts with me before getting too involved in the affair and I feel very angry about the deceit while it lasted, this was outside of my control. I won't beat myself up about doing anything wrong and if I have a relationship again, I'll talk to a new partner about how important it is for him to share any doubts or wobbles about the relationship as soon as he has them."
Contrast that with the alternative of leaving a marriage with children, refusing to attempt (or engage with) counselling, claiming longstanding (but previously unexpressed) unhappiness and insisting no-one else is involved.
If there's one person on the receiving end of that little speech who doesn't constantly torture themselves with an inner dialogue of 'what did I do wrong? will he change his mind? Is he ill? Why has he changed so much? Why won't he try to work on things? How can he not want to wake up with his children every day?' then I've yet to meet them, or encounter them on Mumsnet.
It's cruel and indefensible to lie and it's never done for altruistic motives. Pretending it is, is just another layer of lies.