He is begging you to give him a chance to to try again...but to try what? He's been lying to you and cheating on you, for years. He stopped trying in your marriage years ago. Why didn't he choose to do the right thing then? Why now? The difference is he's been caught and he doesn't want his reputation to be damaged. It's funny how he has morals when it comes to divorce, but none when it comes to lying to and cheating on his wife. His morals only seem to extend to believeing in things that benefit him and make him look good.
You say you don't believe in divorce.... what does that mean? Does it mean that no matter how badly someone treats you, you should stay with them because you married them? Do you think that he can treat you like rubbish, lie and cheat but you still owe him your loyalty and forgiveness?
He's been with this woman for years. I doubt he's going to give her up now, or for long. Even if he did, you'd never truly know if he had, because you now know you can't trust him. He might just find another woman to cheat with.
So, you could choose to forgive him, yes. And you could choose to cover up his cheating and pretend to everyone that everything's just fine. And he continues to conduct his affair. He has everything he wants - no divorce, no shame, no pressure from his wife, and he gets to still conduct his love life outside the home.
What do you get? Not a loving marriage. Having to lie to family and friends. Not single so not free to find someone else yourself.
My words sound harsh, but I feel for you. I think you deserve more than a husband who treats you like this and then expects your loyalty and protection, for the sake of his reputation.
You don't have to make any fast decisions. You don't have to make any promises to him. You can make a decision now and change your mind in a few weeks or months or years. Don't be rushed. Don't be pressured in to a dishonest life or one that makes you unhappy. Know your worth. Protect yourself. Your husband has been putting himself first for years.
I wish you all the best.