I know it's petty but it really grates on me. Went to see my 32 year old brother today and he was saying how he went out the other week and a group of girls got talking to him and now he things he may be starting a relationship with one of them. From the phrase girls I thought she must be early 20s or even younger but he showed me a picture and she was quite obviously his age. So it's confusing for one.
It also feels like an attempt to almost patronize or even infantilize women, by using a word meant to refer to children when describing them. And in my experience, the ones who like to refer to fully grown women as girls tend to be the same ones who objectify women and their body parts (I hear men saying "Did you see the on that girl?" a lot more than "Did you see the on that woman?) for example. Because referring to her as a woman would be admitting she has full autonomy over her life and body. We don't really see it done in reverse, can't say me or my friends have ever referred to a male as a boy, except when we were in high school.
I accept most of the time it is innocent but I don't like the connotations it has.