...he was so upset with the prospect of us never having children this girl caught him on an off day at work and it just progressed from there.
So his cheating is the OW’s and the infertility’s fault? Spare me. He is responsible for choosing infidelity for 8 long months. Infertility happened to both of you, yet you have not abused his trust and twisted the knife. He had a range of ethical options to use to deal with his issues. Making a fool of you and leading a double life with OW are not among them. I think he just fancied OW and went for it.
Regarding the restoration of trust, did he confess on his own or did you discover the affair? Did you see their messages? Moving forward will be impossible until you know exactly what you are forgiving. I wouldn’t even consider staying unless he admitted what he told OW about you, your marriage, and his feelings for her. His hesitation when you asked if he loved her spoke volumes.
It sounds like you are a challenge to him now, hence his sudden regard for the marriage, plus he desperately wants to keep everyone from learning what a bastard he is.
In truth, he is a very bad bet. His character flaws are still present. He is selfish and has weak boundaries. He feels entitled to pursue illicit thrills/intimacy...and will likely cheat again. You know he is capable of massive deception and manipulation.
Are you willing to spend the rest of your life feeling perpetually unsettled?