Historicallyinaccurate
I'm not addressing the question of whether it's OK to cheat. I'm happy to state that I think it's not. However, I think we can agree that as it's bad, it's worth understanding all its causes. If this was a discussion on how to reduce smoking, I expect that simply ending the discussion with "it's wrong to smoke" would be considered inadequate by most people.
The question I'm addressing is the question posed by the OP: why men go off sex. My reply is that medical reasons aside, they don't: while individual cases may vary, it's truer to say that over time they may go off sex with an individual person if that person is not particularly interested in sex. You say that all I'm doing is merely underlining the difference between male/female thinking on sex. Actually I'm pointing out the biological differences, and there's absolutely nothing mere about that: in fact it's critically important.
I mentioned Steven Pinker earlier. He mentions a couple of experiments. One involved an ego depletion task (as I understand it, an ego depletion task involves doing something dull, which tends to reduce a person's self control). It basically measured a person's likelihood to stray. Before the task, men and women were equal. After the task, men were much more likely to stray (another experiment cited by him showed that people in long-term relationships were less likely to get it on after ego depletion than those in new relationships but none of us should be surprised by that). There is plenty of reason to believe (for evolutionary reasons) that men are naturally less monogamous than women. In the West, the traditional solution has been to restrict it to committed relationships, with a certain amount of informal polygamy. In other cultures, the solution (at least for powerful men) has been open polygamy (by contrast, compare how uncommon polyandry is).
The rather depressing conclusion is that for men, monogamy becomes naturally boring after a while and results in a greater temptation to stray than women, unless the couple make a specific effort to mix things up and keep it interesting.
Here is another interesting article, this time from the Guardian. Heterosexual men were most likely to orgasm during sex: heterosexual women the least. No doubt the instinctive conclusion a Guardian reader might draw is that it proves once again that men are such selfish, nasty creatures, probably made from slugs and snails and puppy dog's tails etc. Actually, to me it suggests that heterosexual women (or at least those who identify as such) are just less into it.
Of course there is an alternative to straying, and that is simply topu t up with it. At the extreme end, it involves the complete suppression of one's sexual appetite, and the history of monastic communities round the world show that this can be possible, although they also show that natural desires can't be entirely quenched.
Youzanne
I appreciate that for you the above might seem rather depressing. However, from what you say it sounds like you are anything but boring sexually and your husband is probably telling the truth when he says his libido has been reduced by fatigue and stress: they are probably the two most important causes. I can speak from my own experience that it is a real killer. Please don't be discouraged. The other big killer is embarassment. I'm sure your DH would like sex more often and should realise that getting embarassment out of the way will enable this. I think counselling (perhaps a sex therapist?) is a good idea and well worth the money. Good luck.