The more this unravels, the more obvious it is that this was never a harmless friendship after all. Friends do mention their partners and families, rather than never mention their romantic relationships at all. It sounds like you have been using this friendship to prop up your marriage for the last 18 months. Beyond an initial enquiry about whether his wife would mind, (a question no doubt fuelled by your own situation, but not one that most "friends" would ask of one another) both your H and his wife have become faceless and invisible.
Reading between the lines, he has responded today with matched or even more extreme flirting. He was probably waiting for you to make a move so that he could pass the responsibility over to you, for changing the terms of your relationship.
What I am trying to get across to you is that by acceding to your H's unreasonable demands, you made a choice and if you are "rebelling" now, you are making a different choice, but the difference is that you are not being transparent about that new choice. This is why secret friendships are controlling, except the deceived parties have no idea they are being controlled.
Your language is interesting too. You speak of rebelling and you ask us if you "have to lose" your friendship? This is the language of a child rebelling against a controlling parent.
What I am also trying to get across is that you have choices and decisions to make about your marriage, but express them openly and honestly, not covertly. And as soon as this friendship became a scret, you crossed a line. Whether your friend told his wife about it or not is a moot point, but assuming he did, that friendship has now crossed the line for him too.
If you are a good friend to him, you will not want to cause problems in his marriage, because no doubt his wife would be horribly hurt if she knew that the two of you have been flirting. Take responsibility for your own choices and don't intrude on someone else's relationship to prop up yours.