OP,
How is your life? Are there things that you can do to make yourself happier, so you are not looking over the fence at the OW.
It might look like they are having a great time, but I'm sure there are stresses in their lives too. If he is older and has a young child, that is not going to be fun. He's also lost his younger children. If they are in their 20's and making their own way now, they will soon dump him. They won't want to go over and see him if she is there. I wouldn't go near him. I would blame him, and hold him in contempt. He won't get to have a relationship with his own DGC.
If you made a list of what you have lost, versus what you have gained, I think you would be surprised.
I love my DH, and we have been through some very tough times. I also know that I can't take another one of his episodes. He hasn't had affairs, as far as I can see, but every decade goes though a "don't want to be married".
I know that I would be very upset if we broke up, but I also know there would be upsides too. For example, we only get to do what he wants, and I have a long list of things I want to do. I end up eating what he wants, he's fussy, and he is high maintenance. I wouldn't have to do his washing and ironing and cleaning up after him as he works long hours.
In some ways I would be free. I might even meet someone who thinks about me, rather than his own needs.
You need to look at what you are doing, and see how you can enhance your own life. You have offloaded a dickhead. They are both horrible people who your kids don't like. Let them fester in their own stench.