I dislike 'ladies' because of the force behind it, which is chivalry, and chivalry is the carrot to which domestic violence is the stick: both say, we may put you on a pedestal as a 'lady ' or knock you to the ground, but either way, you are ours to position. Both chivalry and male violence against women say: obey the patriarchy: you may get a pat on the head! Disobey it and you will be hurt, possibly killed.
Also, it comes from the idea that calling women 'women' is insulting, somehow. Therefore you flatter them that you think they are dead posh, and call them 'ladies.' Speaking from a peerage/etiquette point of view, 'ladies and gentleman' makes no sense. it should be "ladies and lords!" and/or "gentlewomen and gentlemen!"
Bit of trivia for you: In the early eighteenth century the term 'gentlewoman' was often a sarcastic euphemism for 'prostitute' - after all, these women didn't work for a living, and gentlemen don't have to work to live, so therefore.... (read the start of 'Moll Flanders', and her childhood, for a description of her getting this wrong as a child, and everyone laughing at her wanting to be a 'gentlewoman' when she grows up). By the time Austen wrote 'Pride and Prejudice', around 1806 (so 100 years later) in the scene where Lizzie defies Lady Catherine de Burgh in the garden, and states she's as good as Darcy, she says "he is a gentleman. I am a gentleman's daughter." She doesn't call herself a gentlewoman because it would be an insulting thing to say about herself. By this point, no one says 'gentlewoman' and women of middle-class men (say, doctor's wives or lawyer's wives) who are not aristocratic but nonetheless do not have to work with their hands - are elevated up a social title, and called 'ladies' even though they aren't "lady so-and-so."
The idea that 'women' - a gender descriptor, ffs! - might be construed as insulting (because who'd want to be a woman, after all?) is pretty horrific.
And yet I use it. "Say thank-you to the nice lady, DS" ... I use it because I'm socialised to think "Say thank you to the nice woman, DS" sounds somehow rude.