He's married, you knew this when you got together with him, FGS he was still living with his wife, so stop with the "I was so naive, blah, blah, they weren't sleeping together, blah, blah".
Now that your conscience has finally woken up, this is very simple.
Take how upset you feel, multiply it by about 10 million and you're nearly at a fraction of what his wife will probably feel when she finds out, which she will if you carry on.
With this in mind, delete and block his number from your phone. Block him on all social media. Delete his email address and block it. If he tries to contact you again (which he will), do not respond, other than to tell him to fuck off, that you've kept copies of all correspondence and you will send them all to his wife if he tries to contact you again. That should see him off nicely, because, as you know, he has no intention of ending his relationship with his wife.
Then busy yourself with work, friends, the gym, a good book, whatever.