I actually think David Cameron is the one who cops most of the blame for Brexit. If there is an avenue of sexism to analyse here look no further than the last election where a mediocre man (Corbyn), actually loses the election but is somehow heralded and trumpeted as a victor/hero in popular imagination, and the woman who actually won the damn thing is derided, criticised and hated.
Don't forget women have a 100% success rate for leading a party to a win in general elections in this country, and thus far no woman has ever lost one. Also one of the most uniformly liked/respected potential future next leaders of the Conservative party is Ruth Davidson. A woman who has almost single handedly turned around the party's fortunes in Scotland.
I don't doubt there are some misogynists within the Tory party, but no more so than any other party, and certainly not enough to hamstring the political fortunes of women to quite the same degree as in other parties.