When you think of how you feel, how to end it, how he feels etc. try thinking a different way.
It is like you are an alcoholic and he is vodka.
The vodka was fun at first, getting silly then having a bit of a hangover was OK. Unfortunately you turned into a habitual user. An alcoholic. You are addicted. You think of how to get your next drink all the time.
Recently you woke up, realised what you have sleep walked into and have decided to kick the drink.
Do you worry about all the poor pub landlords and how they'll feel? Do you worry about how the distillery will cope? Would you recommend keeping visiting the pubs but planning to drink only water? Would you bring vodka into your house?
No of course not. You would go teetotal and avoid like the plague alcohol, anywhere with alcohol and anyone who is your drinking buddy.
Easy.
Would you expect it to be easy? Would you recommend waiting until you think it will be easy? Would you keep drinking because some days you expect it to be hard? Would you keep drinking because your drinking buddies want you to get hammered with them? Keep drinking because if you stop they'll say you are no fun any more, because they'll say you are over reacting you don't need to drink less or even come to the pub and just have an orange juice it'll be fine.
No you wouldn't. You'd expect it to be rough. You'd plan a support network. You'd do courses and listen to podcasts etc to keep you going one day at a time.
Ending it with a narcissist is the same. Junkie getting clean rules apply. You can do it.