coccyx can I ask something?
Why didn't he invent an excuse at work and come home immediately? Have a think about that too, because if he would have come home for some other emergency, then he is not taking this seriously enough - and your pain, shock and confusion have all been made doubly worse because you've been unable to confront him face to face. The cynic in me also knows that this delay has given him ample opportunity to wipe his phone/laptop of any incriminating evidence and if he has finished with the woman, you have now lost the opportunity to hear what he said.
I completely agree with a poster above, that it seems that you are blaming yourself alone for your poor sex life, but please challenge that most vigorously. It doesn't sound as though he made any effort either and it would be interesting to know when that effort stopped. Manipulative people of both sexes will often want you to believe that they want sex all the time, but closer scrutiny tells a different story. If the supposedly higher-sexed one of the pair keeps asking for sex at inappropriate times (i.e. when you're cooking, about to go out, the DCs are in the next room) or their seduction technique amounts to asking for sex after a day without any kind gestures or affection, in reality this person knows he is going to be rejected, but wants you to take the blame for that. The truth is, that person didn't want sex at all, but won't take the responsibility for it.
I also wonder how labour is shared when he is at home? Four kids is an enormous workload, so the man who fails to pull his weight is also sabotaging his own sex life.
If he's been meeting the same woman for a long time, there might well be feelings involved, but he will be keen to deny them and present the affair as "just sex". What he might not bargain for in this lie however, is that in actuality that might make you feel worse in the long run, because it doesn't fit with the personality you thought he had.
I also think, if you have got any hope of saving this marriage, his long absences away from home have got to stop. Given that he seems to be prioritising his job above his marriage right now, I'd call him on this vow that he "will do anything". Will he change jobs and will you insist that he does?