Skidoodle, while you may be right, I feel that's a bit harsh. The OP has tried to do the right thing here, and hasn't actually been unfaithful (although I agree there's a philosophical argument here about emotional infidelity - but aren't we all guilty of that to a greater or lesser degree when in a relationship?).
ChgdnoM, you need to make a decision. I'm afraid that neither decision is not going to easy. You won't make a decision, know it's the right one, and somehow everything will fall in place. It's not going to be that simple, unfortunately. But you need to decide what you value more, and then follow through that decision wholeheartedly, without questioning yourself if you've done the right thing.
In order to survive, marriages do not always require the heady type of love all the films and books tell us about. In fact, this type of love usually diminishes over time anyway, even in the most passionate of relationships. Hopefuly, it is replaced with something more lasting and intimate. If you think you have this with your husband, maybe staying with him is the right thing for you. What you feel for the OM WILL fade eventually, and what's left may not be worth sacrificing your family for. If it helps, I read somewhere that the cascade of hormonal reactions triggered when we fall in love, usually wears off after about 6 months, so maybe you should reassess then.
OTOH, don't be self-indulgent and use hurting your husband or your children as an excuse to stay. Being a martyr like this will only cause everybody pain, yourself included, if staying is not the right option. Your husband deserves the chance to be fully loved by someone, and kids are very perceptive and will pick up on your unhappiness.
It is possible to love two people at the same time, though in different ways, I think. If you love your husband, though, in your shoes I would stay, regardless of the attraction to the other man. Marriage vows do not mean you stop finding others attractive, just that you do not act on it. This is your first real test. FWIW, I think you are trying hard to do the right thing, rather than enter recklessly into an affair, and that's good. Your feelings for OM will subside and you can get past this.
Hope you do the right thing and that it gets easier.