I think it's whether it's used to "other" the person it's said towards. So in my part of the UK we use "Love" a lot. "Alright love", "Here you go love", See you next week Love". It's used by women to women, men to women, women to men. Not so much by men towards men, presumably because despite the enlightened society we now live in, it's still considered a bit gay, and because we have the all encompassing "mate" to use instead. But it's a universal thing, it's just something you put at the end of a sentence, it's not actually meant to indicate anything about the person you're talking to.
However in my last job we had the "girls". We were a software company, so very male dominated, and aside from the accounts team there were very few women.
So the accounts team became "the girls". And maybe if it was just a descriptor, that'd be fine. But any time we had visitors in the office, the directors would get "the girls" to sort the coffees and biscuits. The sales guys are incapable of booking their own hotels, better make the girls become their mothers. Communal fridge so filthy that entire civilisations of bacteria have risen and fallen? Eventually one of "the girls" will crack and clean it.
It's not the name that's misogynistic, it's the expectations that come with the name