You reasons for leaving a spouse or having an affair sound like the talk my father gave me when my (violent, drunk, drug taking, adulaterous) husband and I broke up.
Apparently if a woman leaves her husband she is a slag, if a man leavs his wife it os because she is not being a good wife, getting fat, not shagging her husband as regularly as he likes it, not keeping the house as he wishes and not bringing their children up as he wishes! Sad, archaic, sexist thinking IMO.
That says a lot about my father as he is living in a very sad relationship - he hates his wife (my stepmother), calls her fat, says she is dirty (she is - I was thier cleaner for a while and had to do their washing - first unsticking her used sanitary towels from her knickers that were in the laundry basket!), she dotes on my half brothers but in a way that is unhealthy - letting them do exactly as they want - smoking pot, getting drunk and fighting - anyway I could go on and on.
I am now happy, in a wonderful marraige with a man who talks to me about anything that is wrong in our relationship and who I don't lie to because that is the downfall of any relationship - what you are doing by having a relationship with a man like this is allowing for his dispicable behaviour - the lying, cheating and the problems that will be a skeleton in cupboards for years to come. Your kids parentage will come out oneday and it will be your families dirty little secret that will take years to get over with therapy and family division thrown into the mix.
And your husband lost his temper once, by no means acceptble to hit anyone but in your post you painted him as someone who was a relly bad person when in my mind the true father of your children is far more dispicable - he isn't even man enough to be a father to the children you have had together - he is happy to leave that to someone who is apparently violent! Can you really not see what kind of man he is?
And I don't really think you believe that his wife has driven him to seek a mistress - I don't see you coming out on posts when women on here find thier husbands have been unfaithful blaming them, saying it is their fault for not being a good wife.
You arguements for cheating are flawed because when all is said and done cheating is just lying - it is building something on a lie, so the relationship with the mistress can never be hapy and fulfilled on both sides - nothing that is built on a lie can ever be.