Op - I think you need to get out of the drama triangle. You haven't technically had an affair if you haven't had sex with the OM. Sounds like you have developed an intense attraction and attachment which is hardly a hanging offence in 2019, or at least not in this country, thank goodness.
To all those who say her husband she kick her out - he doesn't WANT to, so why should he? Let HIM at least decide FFS. It's not up to the OP or anyone else what her husband decides to do. He is probably hoping that the whole thing will blow over, which these things nearly always do, especially as the OM does not appear to want to leave his wife. Sounds like the Op's partner is reacting in an authentic way, rather than just responding to his ego. He is hurt but hoping it will all go away as he wants the partner he knew before back.
I do feel for you OP, but really this is all too melodramatic. People who are married or attached form attractions for other people ALL THE TIME. It happens. It is a fact of life. It happened hundreds of years ago and it will happen in hundreds of years time. There mere fact of having a ring on your finger does not make you oblivious to the attractions of other people, despite what Disney would have us believe.
I don't think you are behaving badly, really. You formed an attachment which you are now miserable about. You've been honest with your partner which is a lot more than a lot of people (men) would have done. OM is doing the decent thing by his family. You are detached from your husband at the moment - as he has naturally noticed - because you are still in the throes of a crush. This is normal, although of course upsetting for your partner.
It will all blow over from the sounds of it. If, in due course, you and/or your partner decide you are no longer happy in your relationship then you can decide what you want to do. Take OM out of the equation as he has already stated what he wants for the time being.
You are being far too hard on yourself and far too melodramatic in my opinion. It might help to have a few counselling sessions either alone or with your partner to thrash things out? For what it's worth I've got several sets of friends who had affairs and then got back together afterwards with an even better relationship so I would keep an open mind on everything.
Also, I would like to add that in my experience and my opinion men react differently to their wives' emotional affairs/attachments/crushes than women do. When I was married I not infrequently had crushes and they always had the effect of making my then husband far more attentive and interested than he had been prior to the crush when he was generally obsessed with work and really not that interested in me at all. My 'crushes' forced him to confront the reality that other men were actually interested in the wife that he was neglecting! (Not saying this is the situation in your case, Op, but the fact it one can never become complacent.)
Just sayin'!