I'm surprised that so few women seem to understand there are a lot of men who treat sex as entertainment, stress relief, even a hobby. Its only relevance to the home relationship is that they know they are cheating, but probably feel that as an unreasonable condition they try to uphold for the wife's sake. If they fail a few times and she doesn't find out, no harm done (in their eyes).
Imagine you drive the ideal car for your lifestyle. It's comfortable, works well, looks good; you love it and are proud of it. Somebody offers to lend you a top-end performance car for the day! Naturally you bunk off work, climb into your Bugatti and spend the next several hours having the time of your life with this sleek engine of unprecedented power.
This is not a metaphor, because people don't swear fidelity to BMW or Volvo, but it's got some legs: Most drivers recognise the Bugatti would be a shit choice as a regular car and hand it back with a big smile on their face. Some, though, find this experience so much more rewarding that they abandon their Range Rover and all the needs (children, shopping) it satisfies. They keep renting performance cars until, one day, they buy one and depart forever, leaving the family car and family behind.
It's not the family car's fault. It's just that he couldn't have a BMW and a performance sports car, so he did the next best thing.
Plenty of women like "driving Bugattis" too, but the potential risks and consequences are greater for us so we tend to weigh things up more carefully.