OP if you were prepared to leave for this other man then you are capable of leaving.
The reason why you won’t leave now is clearly because you are afraid to be on your own. Be honest with yourself.
I had an affair several years ago. I was in an emotionally abusive marriage and had several times brought up the possibility of leaving but never had.
Then I ended up having an affair and I realised that having had the affair, and being discovered, meant that I actually did have what it took to leave and now I had a reason to leave where people would accept that rather than if I’d just left because I was unhappy. Iyswim.
The affair ended before we split, but by then things were in motion so I just carried on and we separated. My DH would have taken me back no question but I was already out by then.
As for the OM, he got together with someone else but kept telling me how he was just waiting for me to be free. But I didn’t want to pursue that any more because even though I had had an affair I didn’t want to be someone’s reason for ending a relationship. Besides which, I was free and realised that we probably would never have worked anyway.
I got together with my current partner, and OM acted as if I’d betrayed him, even though he was with someone else by then.
I haven’t spoken to him now for eight years. I didn’t have to block, I just stopped speaking to him and he concentrated on his gf who he stayed with for five years and treated like crap anyway.
You were going to leave for OM. Just because OM isn’t in the picture any more doesn’t mean you now have reason to stay.