Well, I'm sober OneLieIn (I'm always sober!) so let me be your sober self for tonight :)
The comment is symptomatic of a much more serious problem. You can look at it in a number of different ways; you don't have to decide tonight.
I think option 1 is go nuclear and take his sorry arse to the cleaners. He should be reprimanded for his behaviour and a message should be sent to the other men who condone or ape it that it is not acceptable. Except the main loser in that will be you, because you'll be the whistle-blower and perpetually on the outside thereafter.
Option 2, at the opposite extreme, is to leave. Rome wasn't built in a day and the playing field won't be levelled in one either. Bit by bit, this kind of attitude will be broken down; you get to decide how much of a role you play in achieving that, and how much you hand on to the women who come after you. That's not a defeat, but it's a tactical withdrawal.
Option 3 is to speak to him again about his conduct. It may not actually be intended to be patronising and dickish, but caused by lazy thinking, assumptions and unfamiliarity with the concept of women as people. I think you can make the point that there's a pattern of behaviour that needs sorting out - it isn't your job to have to keep coming and pointing out how excluded his behaviour makes you feel.
Personally, and I'm in in a kamikaze mood about my own career, I'd leave and then take the fucker to town. But that kind of attitude is why I took leave (without authorisation) this morning, to prevent me going nuclear in the office. Can you take a few days out to think it through?