Can't believe you didn't notice? How in God's name was it your responsibility to guess he was having an affair? I'm glad you recognise that for the BS it is.
I was saying on another thread recently, that because of the enormous trust default in a loving relationship, an affair is normally the last thing anyone considers, especially if someone has never been through the pain of infidelity. And in your case, that trust default extended to your "friend".
If you had have suspected anything, or queried any change in behaviour, have absolutely no doubt that until he was committed to leave, he would have lied through his teeth to you and fobbed you off with work stresses or depression as being the cause.
All he is trying to do is to mimimise the guilt by pretending that this wasn't as great a shock to you, because making you feel stupid for not tumbling this makes him feel better. I'm so glad you're not falling for it. You have done nothing wrong whatsoever. If you'd been a jealous, questioning person, he would have blamed you for that too. Take huge comfort that you have behaved like a normal, sane rational woman who trusted. Never let anyone make you feel ashamed of that.
I'm glad you've been able to say your piece to the OW. I hope you made her squirm. Whatever script she's following, there is just no getting away from the fact that she has betrayed you in a horrible way. No amount of rationalising can ever convince her or others that she has behaved with decency.
I really hope you get some answers to the myriad of questions you must have. Have you given any thought yet to what I was saying downthread about trying to reconstruct the last two years so that it now makes sense?