If I was the mother, I'd be fucking furious with him (if I had any energy to do anything other than grieve). The kids shouldn't have been there in the first place - what kind of creep takes his babies along on a date with the 'mistress'? And if he had taken them out on his own, he might have been paying a bit more attention to them, not the other woman.
The man is guilty of severe negligence, at best, IMO. Saying 'the same thing could have happened to a different man or woman in different circumstances' doesn't make what this man did right or excusable.