Honestly, OP, I would let yourself off the 'moral' hook entirely here. You don't want your DH to die, or to divorce him, this is a completely harmless fantasy scenario. (Have a look at a recent thread on fantasy lives on here for comparison -- many, many people have involved, vivid fantasy lives that they spend hours a day dwelling on, and which involve scenarios which involve disinventing their children and marriages.)
My therapist said once that she sees this kind of fantasy (involving another person) very frequently in clients. She would tell you that this fantasy is doing some kind of 'job' for you. It's useful to you, or it has been useful to you, consciously or unconsciously, and it may be helpful to you as a way of coming to terms with it to try to think about what role the fantasy is playing in your life, what its 'job' is. Why him etc.
I suspect, for instance, that the fact that you hardly know him, that he's a comparatively distant and unavailable figure, is key here. You've chosen someone at a safe distance. Not someone who is going to proposition you and present you with the real possibility of breaking up your life as it is. There's a kind of wisdom to your choice.
(For instance, I now understand exactly what role my own 'crush/fantasy' played in my life several years ago, though it's easy with hindsight. That I chose someone who was a little bit available, but not too available, and that I cast him to play a particular role in my head, when he was really quite different. Also, something you might find helpful, when he seemed to start 'seeing' me, I was actually horrified. I remember seeing his name come up on my phone a few times in quick succession, being horrorstruck, and not picking up. I think unconsciously I knew I wanted to keep him at a safe distance.)