I'm so, so sorry you are going through this, I'm glad you've got friends there today supporting you.
You did a very difficult thing in forgiving him for the first affair and agreeing to work through it. It takes a lot of tears and strength to do something like that, and he has basically shit all over it and thrown it back in your face. I'm so sorry.
I'm not always automatically 'LTB' as I think sometimes humans make mistakes and some things can be worked through if there is genuine remorse and trust can be rebuilt. However in this case I'm saying end it. He's already seen once already how much hurt it caused you, and he should be on his knees with gratitude that you even gave him another chance. Instead he's gone and done it again. You will never trust him again now. I'm guessing that you hadn't really managed to trust him again after his first affair, given that you felt the need to check his phone (no judgement here, I know how it feels).
From my own experience, I would say the most important thing now is to make him leave (you can call it temporary for now) so you can think things over and try to get a clear head. This is the mistake I made after my husband did something awful to break my trust. I threw him out but he was begging forgiveness and I let him back in less than a week. It's the biggest regret I have, not giving myself more time, because 6 months down the line I'm still not sure I did the right thing.
Right now you will be devastated and terrified of the thought of breaking your family up. This was me, and because (as above) I didn't give myself enough time on my own to think things through and get past that initial feeling of fear and devastation, I gave in and let him back. Right now, you are maybe even tempted to give him yet another chance - and I don't judge you at all because I know how it feels. However I promise you this, you will NEVER trust him again now. It will never feel the same. He'll promise you anything and everything, he'll probably offer to go to counselling, constant access to his phone, but nothing will ever be enough. I wish I'd been brave enough at the time to end it because now I'm just living a daily nightmare so please dont be me. As someone said above - it really won't ruin your kids lives, I see so many posts on here from women who have left and are so much happier now that they're not living with that constant uncertainty of not trusting someone.
for you