Why on earth would you torment yourself further by meeting him?!
That is utterly, utterly insane. You know damn well that he'll try to talk you into staying, that it won't go well. You are setting yourself up to fail.
Text him now and say that it's over, and you want nothing more to do with him. Remember how much he's changed you while you send it, and don't let yourself chicken out. Then block his number, and delete it. Give yourself no way of contacting him, and stop all the ways he has of contacting you.
You can't be friends. You aren't great friends. You started an illicit relationship and threw away any chance of friendship when you did: this isn't a star-crossed lovers situation, you have to cut him off completely. I'm not sure that a man who is trying to manipulate him into doing exactly what he wants, despite what it means for you, and who 'struck' while you were low and vulnerable would be a great friend anyway.
You have to 100% walk away, and cut off all contact. No goodbye meetings, no "I miss you" texts, no quick catch-ups, no rants about how if things were different you'd be together. You can't do this by half-measures.