If I get it right, the women who attend them aren't anticipating that vibrators will be passed around between men and women. They really might not be the sort of people who,as you suggest, would be happy with this. I'm not sure that there's reason to expect this, unless they're seen as women with loose morals.
Firstly, I'm not sure who would see a woman attending an Ann Summers party as having 'loose morals' unless it was someone who had just got off the train from 1950.
Secondly, even if you did associate the parties with 'loose morals' why would it make your morals any looser to pass vibrators around with men, than it would to pass them around with women? If you are coming from a place that says it's immoral to share anything sexual with someone other than your partner, I really don't see why sharing those things with women would be any 'better' than sharing them with men.