I really can't understand other posters saying the OP's DH has done nothing wrong - he has. He's betrayed her, in the worst possible way.
I'm open-minded about swinging - I think if both parties are up for it freely & without pressure then all good for them.
What I cannot understand - and I doubtt that even seasoned swingers would OK this - is he went behind her back.
If you are in a (you believe) heteromongamous relationship, then the ONLY person you discuss planned sexual activities with, is your partner. You do not go behind the person you are intimate with's back, and stage a possible extra marital contact. You just don't.
Even if the DH believed (via pillow talk) that his wife was 'up for it' he should have told her first.
I can't believe that others believe the DH reacted appropriately. The correct approach would have been to immediately all an end to the night, and talk to the OP in private - at home - later. Not allow his dick to become interested in his fantasy coming true.
Even if the event wasn't staged, and the other woman was acting impulsively, the DH should have made excuses and left - then sounded out his wife. By making it clear he wanted to shag the other woman, he's airing their private stuff openly, and putting the OP in the position of party pooper - instead of talking about it alone, then talking to the other couple, if the OP agreed.
Your biggest problem OP, is not that your husband wants to swing, it's that he doesn't respect you, your feelings, or your sexual boundaries.
I wouldn't trust your DH with her other woman now, even if it was just the fantasy, the truth is you know he'd happily shag her.