I think it is very much dependent on your build. People's perceptions will also be skewed by your "normal" and what's "normal" around them.
Age is a factor. At 5'2, I'm about the height of a lot of 12yr olds. I'm a pear shape, so my top half is small framed and people percieve me as being small and healthy (I am!). I would hope that I weight a good bit more than the average 12 year old because I'm a mature adult with the muscle to run half marathons and the fat reserves to bear children and breastfeed them. In my teen years activities that 12 year olds are not designed to do yet. In my teenage years I looked healthy with a bmi of 18-19. I ate well, exercised, had long glossy hair and strong nails, just a naturally slight build. Through my 20s and 30s my BMI has increased and I've settled at a healthy 22-23.
If I spent a number of years with a spare stone or two on and that was my normal, people might comment that going down to my current size with small bust and hint of ribs/ clavicals might be too far, such is the difficulty of people's perceptions.
BMI does tend to work for most and it is a broad range for good reason.
If you have to put in a lot of effort to maintain a lower BMI, that can be a bad sign, especially if it dominates your lifestyle and priorities and interfers with "normal" activities.
If your health and comfort are suffering, that is a bad sign.
Gentle weightloss is easier and healthier to maintain than rapid losses. People also have more time to adapt their perceptions of you.