I doubt that it was a 'total chance' that he just happened to walk you back to your car after the others had gone 
he shouldn't have kissed me, but he doesn't make a habit of it, I know
Unless you take drastic action to stop it, you know damn well that it will become a habit. There'll be another kiss, and another, and a quick fumble, and a longer fumble, and the inevitable will happen - and all because you're 'bored'.
You know him so well from your once weekly meetings, do you? Do you know him well enough to know that if you leave your dh to shack up with him, you'll soon be bored again?
DH and I make a big effort, but can't ever seem to make it as exciting as falling in love again
You've already fallen in love with your dh, you've done the butterfiles in the stomach, the will he/won't he phone, the wonderment that this paragon has chosen li'l ol' you; now start falling in love with him all over again in the knowledge that he's proved his love for you, that he's (maybe) the father of your dc. that he's your solid, reliable, rock of ages - and wonder at the fact that he chose li'l ol' you to be those things to him.
Like lots of things in life, a good marriage is only as exciting as you make it. Next time you're feeling bored take up knitting, or take your dh to bed and blow his socks off.
And can the once weeklies unless you can trust yourself to make your own way back to your car without an escort.