I'm really sorry to hear you are going through something so painful. Everything you have said above, I can absolutely relate. Those were the exact thoughts I had when I first found out about my DH. Apologies this response is really long. :) I hope what I'm about to say can help you if you want to see a positive example in staying and persevering but because I don't know you or your husband personally my experience may not be right for you.
I always said that I would beat the crap out of my DH, divorce him immediately and make sure he pays for it. I would also judge other women who stayed with their cheating husband/BF. All this was before it happened to me. Easy to talk tough but tougher to carry through. It was super hard staying with him because he was a constant reminder that he could do this to you and your relationship. I felt the same as you when I saw the incriminating text, I immediately lost my best friend, my life and me.
I chose to stay at first because I loved him and kind of in denial but eventually it was because he realised he made a mistake and we both wasn't ready to give up on us. I gave him a cat door to crawl through for him to make it up to me. For you to heal, he has to make most of the effort and be willing to. He hurt you and it is his responsibility to make things right.
My DH's actions after was my indicator of his commitment to making it right. Whatever he said to me I didn't believe him because he lied to me for months so why should I believe him now. When he stopped making excuses, told me where he was going in advance, who he was with, answered any and every question I had for him even if I had asked him before no matter how painful it was for him to admit, he was patient with me when I cried or was angry, he listened without interruption and when he said he will do something (even if it was something small like washing the dishes) he will do it without question, excuses or hesitation. These are the things he needs to do to help you you heal. It wasn't overnight and it was a painful journey. We argued a lot but overtime he realised he needed show me more than words and when you see him making extra effort that is when you know he understands the damage he caused you and he want to make it right. Over time the arguments became less, he became more understanding to my emotions, he was honest with me about his emotions, he stopped blaming me and took 100% responsibility of his choices.
It's been your first two weeks since you found out and I think it's normal to have your kinds of fights in the beginning. The dynamic of your relationship has shifted and it feels like you're arguing with a completely different person you were once married to and you too became a different person. Most of my arguments made no sense. We had so many fights to the point where I couldn't bare the sight of him and he was so frustrated with me constantly crying everyday, at work, at home, having a shower, in public. Just absolutely everywhere and I couldn't stop it. I frequently had thoughts that's it, I had enough of this BS.
I had no idea who I was, what this relationship was, who he was. All was very confusing. This is going to sound selfish but you need find yourself again and you will also need to be ready to heal before you are up for forgive him. You don't have to forgive him too quickly either, take your time, know what you are forgiving him for (like lying, breaking promises). Forgiving is a choice and is not for him to feel better but for you to move forward and let go. That is why it is important he needs to work on building trust so you can focus on putting yourself back together. You can't do both. You can't fix what he has broken. Just like a broken robot can't fix itself (not that you're a robot but you get the idea :) ). He has been selfish for long enough. His cheating actions was for himself, his ego, his insecurities and attention all for himself. Nothing to do with you. It is time he gave you back the attention you need plus interest.
I highly recommend investing a few sessions with a good councilor and possibly one who has dealt with PTSD because you will have trigger memories which will create havoc on your emotions and sometimes in a very inconvenient place (like in public at dinner). After the first few sessions we really saw improvements in the way we dealt with our emotions and how he dealt with helping me heal.
There are lots of blogs and vids out there directly from couples recovering from affairs. Here are some I thought was helpful but at the same time try not to keep reading for too long. I stopped reading these for about four months to help understand why he did this and how to feel better but once the why questions have stopped, I didn't need it anymore otherwise you run into stewing territory. There are also good posts for the cheater from an ex-cheater as well (less condemning I think). If either of you have any questions, you can email them and they are really helpful.
www.youtube.com/channel/UCeneINV1yWUhjo1xCjz099Q
beyondaffairs.com/
www.emotionalaffair.org/
The other woman is a separate issue. My DH also had an affair with someone at work and she knew he was married (I know this because I've seen their texts). I found it really hard to ignore her because I firmly believe she is at fault for her part. Ignore those people that think the mistress/manstress shouldn't be slut shamed but in my opinion, if they knew the other was taken they are also at fault, no question. Anyway, your recovery and the recovery of your relationship has nothing to do with her. She was convenient and easy but not the symptom because he was the only symptom. The less she is the focus of recovery the less significant she becomes in both your lives and that is who she is... insignificant. (no I'm not bitter at all...)
Part of his commitment to recovery he has to be the one to distance himself. He deletes her phone number and email addresses, all texts, pictures, social media (Linkedin and Facebook in particular), personal emails and he should show you that it is all gone. There should be nothing to hide now. He has to understand if he is keeping any secrets that doesn't help you in any way. He can no longer use the I'm trying to protect you that's why I didn't tell you as an excuse because it will now be suspicious.
When there is a difficult decision I give my DH the obvious options to chose from and tell him why one makes you happy and why the other doesn't. You too have to be as honest as you can to him about your feelings. Even if you're not sure what exactly what your feelings are just say that. I do find that giving him clear choices helps him understand where you stand and he doesn't need to guess. Sometimes be prepared that he won't make the choice you want and may change last minute because although you may think it is a simple choice (like duh choose the one that makes my wife happy) like not going to a party when she will be there, you will need to remember he is also going through a breakup (no sympathy there but that's the painful truth) and he is going through a thought process weighing up multiple decisions at once. eg if I go, my wife will be upset, but if I don't go people will ask why I'm not there and I'll have to make up an excuse, but if I go, she will be there and I'm too ashamed of what I have done, but if I don't go, people may find out what happened and gossip, but if I go what do I do when I see her and if I speak to her do I tell my wife, and if I don't go my wife will be relieved, etc. I tried not to pick a fight with him and let him choose because you have given him exactly how you feel about each option and if he chooses the one that makes you unhappy because of his ego and if (remember if) something happens he has no reason to blame anyone else but himself. But ultimately, he will be happy he made the right decision himself and not forced because if he was forced to do something it won't feel right to him no matter if it was the right one or not (if that makes sense).
At the end of the day, it is a tough choice to stay or to leave particularly when you will be feeling so many emotions and decisions at the same time. There will be a clarity moment that the decision you make will give you the most peace you have felt in a long time. As I write this, although it is still a little painful and I have very few bad moments, I believe I have made the right choice to stay and I don't feel like a mug for staying. I am really thankful for that because TBH we are in a much better place than before the affair. We are both more attuned emotionally, we are more open and honest with each other, we make special effort to take time out for each other and we say what we mean.
I hope this helps a little. Please take care of yourself. This is a really tough time and can be very confusing at times but there is a light at the end of the tunnel and it can be a long tunnel no matter what you choose.