Just wanted to say that I've read lots of posts recently where people have said they wished their parents had split earlier. They had either suffered from living in an unhappy environment and/or felt their childhood had been a lie to some extent when they realised that their parents had only stayed because of them.
Five years is a long time to try and see if you can live together after an affair. You're dreaming that he's untrustworthy because he's shown you that he is - both the affair and the massage parlours.
I have similar aged children and am struggling with trust/support issues (not affair related). Mine's only been a few months, but if my partner said to me what your husband has, that would be a game changer. Perhaps your answer is that no, apparently you can't be civil enough. Therefore you will take his advice and fuck off, and in true mumsnet style this will be to the other side of fuck off where you'll fuck off some more !!
Just because you agreed to stay (although you say you didn't have the balls to leave) you're not tied to that decision. You hoped you could put it behind you, you had counselling (he went to massage parlours!), but it's not working for you. You not being able to trust/move on, however you see it, is not the problem. He's the problem - he had the affair, he changed everything.
If you didn't have children with him would you stay ? I know that's the world's biggest "if", but the answer to that question is the key. I know my heart breaks a little bit more each time I ask myself that.