Sadly, I've been here a few times. It's amazing how bold some men will be!
You deal with it at the time by turning it immediately on to them with a, "What the fuck are you doing?"
Make it clear from the very start that their advances are completely unwelcome.
When he was to due to get out he turned to me for a kiss and asked several times before accepting my no.
Men will generally not push if it's an assertive no because then you might be 'the sort of woman' who will tell his wife/make it publiclyknown.
If you are a bit flattered and flirty back (which, from your op it sounds like you are with all the talk of mixed feelings and how, if you were both single you can see something happening), you are essentially colluding with him - I'm saying no, but I might be able to he persuaded...
Don't he flattered if a married man tries to kiss you. Be offended. He's looked at you and judged you to have poor boundaries, poor morals, a lack of self respect.