Incredibly sad, a story which has played out a million times for many a loyal loving wife.
It doesn't sound like he's got the sense to realise what he's got and the truth is you are dealing with so many what if's.
Does he love her, does he want to be with her, does he want a new life, willingly not just because you are forced into ending your union.
The difficulty is many wives know these infatuations can burn out, and ending a good marriage for a man's temporary madness can seem foohardy, but whatever the course, your marriage as you knew it has already ended, he has destroyed the trust and the safeness of your union.
You don't need to make rushed decisions for him, take your time and assess his behaviour, your head and heart will guide your decisions and only you will know how much you can take before severing the chord on the relationship.
I would say though, do not allow him to turn this around on you, he is the one who has ended your marriage, no woman wants this kind of ultimatum, being forced to share a husband, he should be the one to leave.
Take support from others this friendship of his needs a light shining on it, allow him to feel shame, their friendship needs to be judged.