I'm so sorry OP. No one deserves this shit.
But, a slightly different perspective perhaps. You've mentioned a couple of times that your husband is very handsome and charming, and I hardly imagine at the moment that you're blinded by affection...
It doesn't excuse the affair, but most middle-aged men aren't handsome or charming. The small number that I've known who've been genuinely like this (not Brad Pitt gorgeous, but the good looking guy in the room, at the work party etc., the one who has always found flirting easy etc) have all had affairs. One of my friends (divorced) left her husband after an affair of six months. It began (corroborated by others!) when a junior colleague climbed through the handsome man's hotel room window on a work trip away. Most men never get to see how they would behave in this reverse-Rapunzel set-up. A lot more men (by no means most or all, but a lot) would cheat if it was easy for them to do so.
You need to think about the actual relationship you've had, and your own genuine beliefs about fidelity. But I'd also be on your guard for being played by this other woman. How did her husband find out? I can't find the statistics but the chance of a wife finding out about a husband's affair is apparently much higher than vice versa. He's also left her, which she could probably have predicted. Was it an engineered discovery? Or, put it another way, after the first plan of withholding sex for a year hadn't worked (in enticing your husband to leave you), did she consciously or unconsciously go nuclear because this was the only chance of getting him to herself?? Not that this is the remotest consolation to you, but, if he is indeed very handsome and charming, she's probably head over heels in love with him and not thinking rationally about nuking her own life too.
He didn't want to leave you and he didn't want to move in with her. People never like what they don't want any more when it's forced on them. I would kick him out and see what he does. I suspect he will end it with her next week, if he sees the possibility of losing his family life. (Better to have the taste of being deprived of the children, wondering what they think is going on, what they will think of him as adults etc, than to have him knocking about the house.) Also it suggests a degree of aloofness on your part that will unnerve him.
Note that I'm not saying that you ought to have him back or want him back. But I think it's perfectly reasonable if you do. Lots of marriages survive affairs, and sometimes improve.
(If you do work this out, you'll need to have sex more than once a month. If you really don't want to do that, with him, or at all, I would end it now, because this is a story that will happen again.)