" I do hope though that you all stay happily married and never find your self in a situation like this."
Nice. 
FWIW, I was in a shitty abusive relationship for over 10 years, and now been alone for almost a YEAR. I have been OUT 4 times this year, 3 of those for work functions.
I had a MM come on to me recently, FULL TILT. I have not had ANY attention from ANYONE in YEARS. It was so tempting to give in, but I didn't. I managed to resist. He went off in a huff in the end.
I didn't know his life, his wife etc.
You know his wife, she accused you of having an affair with him to your face, which you wrongly denied. THAT should have been your wake up call. It wasn't. You STILL fucked him.
You have NO chance of being happily ANYTHING with morals like that love.
So you sit there and get all snarky with us for judging. I have been in your shoes, and I knew not to do it. So I DO have the right to say what I think. If you were my friend I'd have said it all to your face too. If I had have lost your friendship over it, that would have been fine too. I feel THAT strongly about it.
You doubtless DO feel bad. You deserve to. You know that too.
Now pack it in, find some pride and dignity and move on. We all make mistakes. This was yours. To carry on in this grim and sordid behaviour will destroy what is left of you.