I think for me, it's something that's quite hard to put your finger on but, broadly speaking - a lack of cohesion or a lack of confidence.
For an outfit to be "cohesive" it needs to look intentional. That doesn't necessarily mean everything is "matchy" but just that, there's been some thought given to how everything works together as an "outfit".
An item of clothing that might look "frumpy" on its own could look really cool if styled as part of a "look" that considers colours, proportions, accessories, shoes, bags, coats, fabrics.
For confidence - I honestly feel if you "own" your look and style and wear your clothes with confidence, it's quite hard to look frumpy! I would tend to see a middle-aged woman who dresses head-to-toe in microtrends, tries to emulate the style of Gen Zers and doesn't have a sense of identity and self-confidence in their personal style as quite insecure and "frumpy".
I think women who "own" their looks - wearing classic pieces with trendy "elements" or ignoring trends and "classics" altogether and embracing bold colours and patterns - are far more "stylish" and less "frumpy".