I've been where you are and it's a bleak place.
I had 3 months separation time to 'be myself' and it's true that it does give you perspective. For me, it made me realise that, despite what had happened (a full-on 5 month affair in which he began to make plans to leave me and make a new life with 'her'), I simply loved him and would be happier together than apart. We had been together for 13 years at the time.
He missed me in my absence; as a result, he asked me to continue our life together. I'm not a complete doormat, so some 'provisos' were made and I agreed that it would become a turned page, to which I would not refer.
We are now 11 years further down the line and have a strong bond. I would say that we both learned some valuable lessons, and are more attentive and supportive of one another as a result.
I chose to try and make the relationship work because I could still see the man I loved (sometimes only a few toes poking out, sometimes a bit more of him).
I believe the affair was sparked by factors beyond physical attraction and, in part, by a mid-life crisis/depressive phase. He felt discontented with life in general and thought the 'solution' lay in breaking free of our marriage.
Even now, I cannot forget what happened, although I have certainly forgiven. There have been difficult times: any cross word takes you straight back to zero-hour and the words uttered in those dark days. However, the pain of that does lessen with time.
No one can know if a relationship will last, regardless of an affair. I think it's best to take it one day at a time, try to be honest with each other and treat the other person as your close friend (giving them the same respect, care and best of yourself that you would hope to receive in return). It benefits no one to rehash past arguments, so it's vital that you feel capable of rising above any resentment.
No one can make your decisions but you: evaluate how much you still love him and if you want to fight to make it work. Be under no illusion that it's an easy path.
If you try and it doesn't work out then you will know you gave it a shot; don't feel that you failed - it was simply a learning experience. Go forth then with a lighter heart, and enjoy the world with new eyes.
It is possible to be happy solo - and much easier in many ways of course - so don't cling to him through fear of the unknown, only through love.
(I'm assuming that there is no domestic abuse in your relationship... if there is, please seek professional advice xxx)