Sweetheart, faff, I'm so so sorry this is so painful and hard. I know it must be breaking your heart and I know it's hard to hear. but he won't have been unenthusiastic for the last three years about your relationship because of guilt of an ended affair. That doesn't make sense. People just aren't wired that way. If an affair has ended, particularly badly, people tend to either (rarely if there are kids involved) leave having realised they cheated for a reason or, much more commonly, throw themselves back into the original relationship almost grateful for its existence after an infidelity which has proven to them the grass isn't greener.
I know this is hard to hear but the fact is by far the most likely explanation is that this affair has been on and off all this time, beginning around the time that he claims it ended and that recently someone has been pressuring or threatening him to tell you. Either the OW or the mutual friend you mention who may be unwilling to conceal what she knows.
Whatever, the fact is you were just as committed to him three years ago as today, so does it really matter, in terms of hurt and morality when he did what he did. Someone who has definitively ended an affair years ago does not admit to it years later out of random 'guilt' and then show no real motivation or passion for repairing the damage. That's just not psychologically believable. It's not how people are.
You know this young person who appeared to be 'in love with him' probably had reason to have confused emotion, right? It is very often the case that an adulterer explains away bad behaviour with this explanation. (I am not saying it NEVER happens that random people get obvious crushes, but it's a vast vast minority in cases like you're describing)
The fact that he has lied to you in the past does not surprise me. Nor does you having suspicions which he ridiculed. This is par for the course. I am afraid that to repair this, he would have to be really really brave and honest about his character and past. I am not saying he doesn't or didn't love you, nor that any of your relationship up to this point is false. But From everything you describe, he is someone who has massive problems with honestly owning up to his flaws and misdeeds and if that is the case, he cannot address and repair them with you until he is honest about this. I promise I am not trying to paint him as a villain, this is very human and many people have infidelitous behaviours in their past, for many reasons. But you are the person we are caring about and worried about here and at some point when you feel strong enough, you may need to let him know that he can no longer treat you like this and he is going to have to show some completely different behaviour, around honestly, vulnerability and courage, the opposite of what he has done so far, in order to keep you and to save himself.
Why do you doubt you could discover anything else? Are you not intimately involved in his social and professional circle? If not, why not? That, again, is par for the course if someone wants to keep secrets. You are well, well within your rights to speak to any of the women you mention here, and though many would talk about 'dignity' and pride' in relation to this, which I can totally understand, and i know that this is too early for you, but I would say it sounds to me like you are well out in the cold on this, with no help or protection but a husband who I'm pretty sure is still lying to you possibly about a number of things. If you want to reach out, you can. There is already an almighty mess. I'll bet at least one of them is waiting for your call. Or threatening to make it themselves.
I am so so sorry he is treating you this way. I would suggest speaking to some people, snooping, or challenging him more directly in counselling. What is your relationship like generally? Do you feel comfortable challenging him when you think he is being dishonest with you? I get the feeling from what you're saying that he is somewhat central in his idea of this relationship and unused to having to explain himself.