This is a bit waffly as it’s early!
Op remember you can leave him for any reason. His anger at you asking him to reassure you is even a reason. I think if someone gets defensive they need to explore why. It may be because he hates what he did to his wife before. It may be because he hates being accused. It may because he’s getting laid every Friday in his office. It may be because he feels he deserves 100% trust from you. Who knows.
The fact you are now older and wiser and have recognised lying to his ex wife for a year is poor behaviour and you no longer want your life tied up with a poor quality individual. Is a reason to not be together.
Sometimes when you really think about a persons past behaviour you see them in a different light and then you can’t unsee it. As we mature I think things we once saw as acceptable become deal breakers.
You changing your view on his previous behaviour is okay. It’s reason to leave if you want.
From his perspective this is why cheating isn’t a great idea. If he is now an honest citizen his cheating has cast doubts. However as I said upthread if he never did the work to discover how he could choose to emotionally abuse his wife for over a year then he’s not someone I would want to spend my one precious life with. I don’t think he ever did the work.
‘Why wouldn’t you cheat?’ To me ‘because I love you’ isn’t a great answer. So what about the the days you don’t love me (hell I am really annoying at times, I really suck at certain things)? Because in a long marriage I am going to face many a challenge. I don't want a fair weather spouse. Being honest is being a bare minimum spouse in my opinion.
I don’t cheat for me. Not my husband (he’s annoying at times). I chose to commit, I chose my vows (I wrote them myself in fact 😁), I stood up willingly in front of my family and said them. My words meant something to me. If my word is meaningless then who am I? Loyalty, honesty and integrity are important to ME. My husband is my collateral damage. But I’m doing it all for me. I matter to me. I look at myself in the mirror and liking what I see is important to me. My husband is the collateral damage to my choices.
He may be fun, sexy, intelligent. You may love him. But love is not enough. Core values matter.
His ex wife will have been told, you are better off without him, he’ll never change, etc etc. She may have blamed herself (unmet needs), her self esteem may have been damaged, she may have thought he was happy and having fun while she was destroyed. She is likely to have thought his new life will be great with his affair partner, that the AP will change him etc while she picked up the pieces.
Yet here is is, has he changed, did he do the work or was she better off without him?
Op if he’s a sneaky cheater he will have learnt his lesson from before. If I wanted to be a sneaky cheater I could be very easily (leave NO written trace - you can deny then). If I was a sneaky cheater and the AP was a colleague do it at work, and make sure they have just as much to lose. Hell don’t date a single lady - too risky as he found out. School boy error there. He trusted his biggest secret with a woman who he apparently didn’t know at all - she told his wife - another school boy error! Sometimes cheaters are shocked ‘I never thought the ow would do that’ - the irony. They are often upset at being lied to or deceived. An example of the odd values and twisted logic of the common garden cheat.
I won’t cheat on my husband but it’s not because I’m not capable, I just choose not to.
Read how to help my spouse heal from my affair. Maybe don’t let him see. Also not just friends by Shirley glass. And also cheating in a nutshell.
Read Gottmans seven principles of making marriage work. And his 8 dates book. Read them - you may want to share the last one with him as a talking point. Mark Manson is good and has YouTube videos (I get some find him a bit much but I like him). Consciously decide what marriage you want.
Then decide if you are happy with your marriage. I’m not happy and i can be happier elsewhere is okay. As is I’m not prepared to be with you anymore I’m not comfortable with your values. This is what I see as ‘growing apart’.