I absolutely do not condone it, but I think a lot of people forget that cheating (depending on the extent and the definition by the people involved) is not always a deal breaker for everyone.
OP, as upset, disgusted and hurt as you understandably are right now, I think you need to take a step back and really think about yourself, how you truly feel about this situation, what the future looks like for you and most importantly your children.
A lot of people here, myself included are angry for you and naturally want you nowhere near a man you trusted completely who has betrayed you and put your health, family and peace at risk. But at the end of the day we are strangers on the internet. We are not the ones who will help you financially, put a roof over your head or raise your children with you.
I honestly think you should speak to one or two people in your real life who genuinely love you and whose judgement you trust to be solid, honest and unbiased. Then really think about what you want and how you want the rest of your life to look.
One thing I will say though is whatever you decide, do not feel tempted to help your husband clean up the mess he created at work or emotionally carry him through the consequences of his actions. He needs to deal with that himself.
If you do decide to stay, make him work for it. Make him prove himself properly, meet your conditions, rebuild your trust and show through actions that he deserves another chance.
I know you mentioned divorce already, but if that ends up being your decision, let it be because it is genuinely what you want and not simply because strangers online are telling you leaving is the morally right thing to do. Please know that no matter how difficult it feels right now, you and your children will be okay.
I wish you the very best of luck, and will be sticking around to see how everything pans out.