I'm so sorry you've experienced 9 miscarriages. That must be beyond distressing.
It's never an easy decision to just leave a marriage, but I wonder what message he receives in his head, when you forgive him.
He's sorry that he was caught and that he might loose you, not that he did it in the first place.
While he's in the begging stage, I suggest you direct him to a helpful place called the Www.survivinginfidelity.com
He can post the situation, without people hurling insults at him and he can get support from other wayward and betrayed spouses, but he has to start by recognising this is a betrayaland he needs to believe that you are not prepared to tolerate this.
Consequences are very powerful. What consequences did he face after his last affair?
If the answer is none, then that's probably why you're here now. If he faced any, then they clearly weren't enough to deter him.
He'll be going LC (low contact) with her till it all blows over.
Think very carefully if you really want to have another child with him. It will only tie you together for longer.
Can you ask him to move out for a couple of weeks to give you space to think. He needs to feel that you could leave the even though you won't be doing so.
Once again, I'm sorry about all your miscarriages, it must take such a toll on you mentally, emotionally and physically.