I'm glad you've come to a point in your life where you've realised that you don't want to live like this any more.
Please, please make sure that when you tell him, and subsequently when you're discussing your decision to leave, that you do not place any of the blame at your feet.
That doesn't mean you haven't done anything wrong, every human being makes mistakes. But it means that you won't be giving him the fuel to guilt you with.
If you say something like 'sorry I asked you to stay before, I should have ended it earlier', he will take that and run it into the ground, telling you that he would have been better off before and you gave him false hope, wasted his time, blah blah blah. You were entitled to come to your own conclusions in your own time.
If you say 'I'm sorry I didn't get back to a place where we could be intimate after your affairs, I'm sorry I struggled to forgive you', well again, suddenly it's your fault and he is blameless.
You can process 'what you did wrong' after he leaves, and if for whatever reason you think you owe him an apology, you can tell him in two years after you've healed and moved on.
And then of course, by not apologising during the breakup, he will probably say you're 'cold', 'self righteous', 'never think you're wrong about anything'. Again, blah blah blah. You're not. You are coming from a place of strength in order to make this as quick and as painless as possible, and you don't need to apologise for that. If anything he'll thank you for it later. Don't attempt to justify why you're being so decisive even if he begins to tell you that you're being selfish. Tell him what needs to be said, tell him what needs to be done, tell him when you expect him to be gone by. Don't play any games he invites you to, because they will inevitably suck you right back in and you'll find yourself back here in a year or two asking for more guidance. (Which there will always be, so don't worry if you don't succeed first time round. You're human and people here care enough to see that).
Very sorry for the essay. I really hope that you can feel proud of yourself for how far you've come and how you're handling things. You deserve happiness now x