Bones, you can only know whether you two could start again.
But you asked what do we think, and for what it's worth, I think he's wavering, because he realises now exactly what he's lost, coupled with finding out the grass isn't greener.
My ex sent me a long email three months after he left, saying he was in a bad place, didn't know what he wanted. He was dropping strong hints that he had regrets. If I'd clicked my fingers, he'd have been back in a heartbeat.
I considered it, even though he was by then living with OW, I gave it serious thought. Twenty plus years together, three kids....
But I ignored his email and pressed on with the divorce because dear Bones the damage was done. The trust had gone. And if I had him back I knew their would always be that niggle, that he was back because things weren't great with her, rather than he couldn't live without me.
I think you are different, I think you want to try again. That's fine. But don't minimise the OW, I mean I think it's unlikely that there wasn't a kiss (at least). I'm a bit baffled as to why push ahead with a divorce and go through the stress and expense of it if you think you may later reconcile?
Perhaps tell him you'll put it on hold for three months. See him to talk things through, tell him your concerns, set your boundaries and both of you keep an open mind. Joint counselling if you think you may give it another shot.
Most of all, be brutally honest with yourself. Do you still love him? Can you trust him again? Can you cope with finding out later down the line that it was more than an EA? I think it probably was.
Good luck