I'm sorry OP, but everything he's done here is straight out of the affair playbook.
Man not entirely unhappy in relationship, but a bit bored.
Attractive female colleague starts at work
Flirting and banter at first, man convinces himself it's all harmless because nothing physical.
Starts confiding in the OW. Spins the OW all the "my wife doesn't understand me/we don't have sex/I'm lonely/I wish I had someone like you/I married the wrong person" lines. This is how these men often groom their affair partners into believing it won't be just grubby sex in the stationery cupboard, it's something real and the OW is special. These lines also have the purpose of minimising the wrongness of the affair by turning it into some kind of star crossed lovers bullshit.
Once it turns physical, there's a honeymoon period where it's all fun and a bit thrilling and illicit and very intense. Men often do a lot of "future faking" during this period, leading the OW to believe that he is really in love with her and will leave his wife for her. When all the while he knows he has no intention of it. Women are more likely to have "exit affairs" but most men who have affairs are cake eaters. They want the security of the married life and family and the excitement and sex with the bit on the side. But they are rarely honest about it, and this is what gets them in hot water.
A lot of affairs naturally fizzle out when the honeymoon period ends, because the reality sets in of what it would actually mean to leave respective spouses, divorce, shared custody of kids, etc. By then, the shine has gone off the affair partner too and the grass isn't necessarily looking much greener for one or both parties, and the affair can't go anywhere. Affairs that naturally fizzle out and both parties agree to part ways when it's run its course often go undiscovered, especially where both have gone in with the understanding that nobody was leaving their marriages from the get go. However, if someone has future faked, it's not that easy to get out. And IME it's usually men who get caught out by this.
The problem with men who hook their OW in with emotional bait, declarations of love and future faking is that they often can't extricate themselves easily if the OW has genuinely fallen for them. An OW who believed in the love story she was spun and then realises she's just blown up her life for a man who all along just wanted a bit of grubby sex in the stationery cupboard to liven up his dull married life is going to feel duped and lied to, and is much more likely to blow up his life too.
Meanwhile, these men have the AUDACITY to then crawl back to their wives and groom their wives to feel sorry for them and blame the OW for everything so they can slot right back into family life like nothing happened and they get away with it.
I can absolutely assure you that no line that comes from a man who behaves like this has any real substance to it. They will of course believe their own bullshit at the time, but only because what is coming out of their mouths serves their own interests. He wanted to hook the OW in, so he said what she wanted to hear to get his end away. He now wants to hook you back in, so he's saying what you want to hear, and will get him what he wants. Until the next time an attractive female colleague crosses his path at work, obviously.
He's played you both. Don't let him do it.