I'm sorry, think you're being mucked about too much.
I was with a married man for 4 years and it was awful, probably much worse for his poor wife and he was on and off with me all that time.
I really thought he loved me, but he never did choose me, even after he left her...he went off with another woman instead.
I know exactly how hard it is to leave someone if you adore them - every breath I took, every thought I had was for that man I loved - so I won't tell you to leave him. But please know he's treating you and her very badly.
Think of it like this, you're thinking only of him, but he isn't considering your feelings or hers at all. That might just make you angry and embarrassed with yourself enough to leave...atleast until he makes a choice between you and her.
I have a feeling he would still be drawn to her even if he left you - you would never know if he was still seeing her, obviously he's capable of deception...my ex was, on a grand scale, and in the end he cheated on me between leaving his wife and getting it on with the other woman.
I deserved it I daresay, but I imagine he is still lying and having affairs, it's a compulsion and I am a much less angry person these days since he left my life.
Sorry bit of a rant...
Good luck