It's a really difficult situation. I went through a similar scenario years ago.
Both committed. Both in failing relationships on our way out. I was getting out of my failing relationship honourably and it looked like he was too, at first. We introduced each other to families and friends.
Then he started to reverse and I became the OW, a position in which I never wanted to find myself in.
Being told to be patient. To wait. That things will change.
I knew the 'ex wife not ex wife' from work and she did have a reputation of being manipulative, highly strung/difficult/mental health problems and I had seen how difficult she could be with my own eyes.
She had real anger issues and nearly lost her job because of these and bullying somebody in her department, who then left . (I knew if this not through him before we got involved as one of my friends was a director where she worked). She really was a bully and a nasty piece of work. It happens. Not every wife is a victim.
However, I did not want to be with somebody who was not able to confront the bully and confront the situation, I did not want to be somebody who was, ultimately, afraid and who was reversing back into that relationship that he had claimed to hate. Somebody, I realised, who had probably contributed to the mental illness of the ex-not-ex wife.
Reason was telling me that, yes, she was bad but he was probably worse and I was probably the worst to get involved with them both. The drama. The lies. I knew that leaving was the right thing to do rationally, emotionally it was a lot harder.
I did not last 5 years but 8 months in total (we knew each other as colleagues/friends for longer than that). The first three months we were openly together, then 5 months when I had become the other woman.
Things had to be more and more secretive and because there were professional connections this was potentially damaging to my career too.
I left the situation after 8 months, in total suddenly. I did not look back.
I left him, left the job, moved on. It did hurt because when emotions haven't yet caught up with the evidence really hurts.
But it was a positive thing to do. I got my self respect back. I got a better job. I slowly healed.
I do not know how much of this resonates with you, but I hope you find a resolution, and a way forward.