I've always found the Strictly curse a good way to explain this.
Noone goes on Strictly Come Dancing with the expectation that they're going to shag their dance partner.
But there you are, thrust together with someone you've never met before for weeks on end. You have to work closely with this person.
A couple of weeks in, you start to feel some chemistry building. It worries you a bit but it's fine, you're probably getting kicked out next week anyway. You've not seen your spouse in weeks, they're they other end of the country looking after the kids, you're probably just craving some intimacy and you've been working so closely with this person.
A few more weeks, youve definitely got a crush. But what can you do? You can't just drop out suddenly, spouse will know something's up, and you can't talk to them about it, they'll get so upset.
A few days later a lift goes wrong in rehearsals, and you collapse in a heap on the floor. Laughing, relieved that no-ones hurt, suddenly there's this moment of tension, looking into each others eyes before you take control of yourself and stand up.
The next week you're in the dance off, but if you get through this you're through to Blackpool. The judges pick you, you're both elated, so proud of each other. After the show you find yourself together in a quiet moment, and out of nowhere your partner kisses you. 10 -15 seconds pass and then you pull away. What the fuck are you doing! How on earth are you meant to tell your spouse about this?
You avoid your dance partner as much as possible for the next week, and your spouse comes to the taping of Blackpool. You spend the night together in a hotel, but you're feeling guilty, and things are fractious. You end up arguing.
Things get back to normal with your dance partner over the next week, you don't talk about the kiss and fall back into an easy rhythm. There's a night out with the remaining contestants and dancers. You both get a bit tipsy. You end up in a corner having a deep heart to heart with your partner, you've never felt so close to them. You've know you shouldn't be talking like this, but you've already kissed them, how can it get worse? You leave separately but at 2am find yourself stood outside your partner's room.....
Obviously Strictly is an extreme case, it's a wonder anyone gets out of it with their marriage intact. But it works as an example for that reason. Most affairs have far fewer triggers, but take a lot longer to happen, they're a much slower boil (I'm exclusively talking about affairs where the instigators aren't looking for one here, obviously there are plenty of people out there desperately trying to find someone to cheat on their spouse with.)