In an ideal world I would not have wanted to go back to work at all and never seen him again or at least not until my feelings of attraction to him were completely gone.
Seeing him in the office Friday made me feel that to make a brief ice breaker contact with him, apologise, and then leave it there, would be constructive and bring closure.
I didn’t respond to his message. I know him well enough to believe he meant it when he said “no stress”.
Neither of us has cheated on our spouses and the alarm bells rang for me to motivate me to improve things at home and work on my own faults.
I repeated to myself many times to think about how would my DH feel about this or that and be less self absorbed and more empathetic, it doesn’t come naturally to me unfortunately 
I spent some time watching marriage improvement videos, cheesy as it sounds, I felt the best I had about my DH in a long time, we kissed and hugged each other on and off all day.
I really want us to work out and I will make a big effort to make that happen.
As for this man, he is purely a fantasy. He was meeting a few needs that my DH was not meeting (he made me feel desired, he was attractive to look at, plus we sometimes had interesting (but not intimate) conversations) so I need to get back to making sure my DH meets those needs and absolutely minimise contact with him.
It hasn’t been a pleasant experience but a learning one. I do think that if I hadn’t brought it to a head I wouldn’t have learned what I have or be on a path to “marital improvement” now, so it’s not been that bad in the end.
Sorry for blabbing on 