The fact that the OP wanted more from this man speaks volumes about the fact that this wasn't actually just a cheap shag for her. The fact that she ended it because he didn't want more should be a wake-up call to her that her marriage is in trouble, and if she didn't realise that before embarking on an affair, she needs to do some serious thinking as to what has gone so wrong in her marriage that has led her in the direction of attention from someone else.
OP, you've ended the affair, but this isn't over. You know deep down that if this man had agreed to more you would still be involved with him, and it would only be a matter of time before you slept together.
You say that he's been honest with you and is being nice to you. Listen to what he's said to you. He's saying that monogamy isn't for him. that sex and fun outside of his marriage is perfectly acceptable to him, so much so that he's prepared to be up-front about it. You don't want to be with him because of this, but the reasons why you turned to him in the first place still remain. You loved being texted first thing in the morning, called beautiful, all the other attention he gave you. But you know that this is just the beginnings of relationship stuff. It's the stuff which draws us to people in the first place, but no amount of good morning texts will pay the bills, be a husband, be a father to your children, be all the things which a well-rounded, solid relationship should have.
You know that it's not just being called beautiful that's missing from your marriage or you could have a discussion and bring that back. So think about what it is you want from a marriage, and what in your marriage doesn't give you that, and then work on those things.
As for the OM, delete his number. He's gone. He's not interested, he doesn't love you, he probably texts ten other women the same thing every day. And then start to divert the attention you've been giving to your OM to your marriage, either to work on the things that are wrong in the marriage, and for that you will need communication with your DH, or to end the marriage and seek the happiness you are looking for somewhere else.
It's ok to end an unhappy marriage, even if it's to a good man. Relationships end, that's life. Equally it's ok to realise after being stupid enough to be drawn to someone else that the marriage is in fact what you want, and to work on that to put it right.
But you need to do one or the other or in a year's time you will be back where you are now.