Well on the plus side OP, you will look groomed if you are accompanied by him even on your most ungroomed, 'ugly days'.
You cannot stay in this relationship if you are going to do what some posters further up the thread are suggesting, ie. taking his clothes, burning them, marching him off to buy news ones... those suggestions make me shudder. So controlling and the only men that would be with those women aren't ones that anybody would want to be wth. 'Local weirdo' 
So, if this is actually a 'DP', ie. somebody you care about, then the only thing to do is have a really frank conversation about the way he dresses and tell him how it upsets you - but that it's about you, not him. Because he's an adult and free to dress how he wants to.
My husband wears combat stuff, all in black, with large pockets on his trousers that he fills with random crap and which knock stuff off the low tables all the time. He picks it up, I raise an eyebrow and that's it. I don't sweat the small stuff because now, if we go out somewhere, he asks me what I want him to wear - and I tell him. And he does.
Would I prefer it that he wears the 'going out' stuff all the time? Yes, because I don't 'do' casual myself ever, but that's my choice - and combats are his.
Decide on him - then tackle the clothes issue - or move on.