Leave him now.
Sexting is cheating.
This exact thing happened to me, with a few differences:
I found all my partners texts, dirty texts, FB messages, dick pics to various different women, arranging to meet etc. I found all the texts 4 weeks before our wedding. One of the earliest messages I found dated back to when our daughter was 2 weeks old.
I also have 2 older children. I wanted to keep 'the family unit' together.
The marriage lasted 20 months because of the same feelings you have - betrayal, lack of trust, humiliation (and because he carried on being a massive twat).
That was over 4 years ago. The separation was hell.
It won't be easy for you to split but you WILL get over it.
Chucking my ex out was the best thing I ever did!
We get on now, my only regret now is for putting up with it for the length of time I did.
Good luck with whatever you do, but I think deep down you will know what the right thing to do is.