The sad fact of the matter is, your relationship with your husband is doomed.
Either you come clean about what has been going on these past two years, and inevitably break up.
Or you cut contact with the OM, and focus on your marriage. And you just know what's going to happen then, right? You end up resenting the absolute hell out of your husband, who in his ignorance, has no idea what you've 'given up' for him.
So you're even worse off thwherefore you went into this affair, hating and resenting him even more. You've made this huge sacrifice of giving up the OM, and your husband doesn't even have the damn grace to appreciate it!
There is no way an already unhappy marriage can survive that.
What isn't doomed is your relationship with yourself. If you stop being so passive in all of this and wrest some control back for yourself, you might actually get a bit of a much-needed boost to your self-esteem.
The other thing which isn't necessarily doomed is your relationship with your children, but in order to ensure you haven't done irreparable damage there, you really now need to step up to the plate and stop letting the men in your life make all the decisions for you.
Again, this is all completely your choice, but it's whether or not you have the strength of character to follow through. It's kind of too important not to, but of course many people before you have failed. There's unfortunately no particular reason you won't fail, too.