If I go out in a pair of jeans, muddy boots, a fleece and no make-up I don't feel great about myself and there is unlikely to be a spring in my step.
This sentence really clarified things I've been wanting to say on this thread for several days. I am the EXACT opposite of this, and it's really helpful to be able to explain it using that contrast. I feel my most happy, bouncy and confident in exactly this attire, because I feel most myself. And it normally means I'm doing something that makes me really happy.
I look at myself in the mirror every day, with no clothes or makeup on. I know what I look like (face ok, body pretty happy with). What I wear doesn't really affect that. I buy things primarily based on practicality,(warm/cool, comfy functional, versatile, good quality, timeless) and, if I have a choice of similar things will automatically choose what I think is the most flattering cut and colour and nicest fabric. I make an effort to look suitably put together for any given occasion. It wouldn't damage my confidence exactly, but it's not great feeling massively out of place if its easy not to. That's where my concern ends and doing so certainly doesn't actively enhance my self esteem.
I'm sure there are thousands of people who see me every day who think I don't look attractive. Others might. Some might think otherwise if I was wearing something different. Others wouldn't. I'll never know one way or another anyway, and I doubt I'd do anything different even if I did. As for any other judgment beyond attractive / unattractive, that surely boils down entirely to what you're wearing (boring/individual, dated/fashionable etc). And my perceived ability or lack thereof to choose clothes according to the personal taste of some random person certainly doesn't affect my self esteem.