I know what you mean OP: there's a sort of Puritanism about some of these clothes -- sort of anti-sex, anti-glamour.
I grew up in an incredibly middle class town and there was a weird competitiveness among people of my parents' generation -- affluent, if not wealthy people who could have afforded to dress much much better, to look as dowdy and asexual as possible.
I think its partly that labels are thought to be a bit common and certainly if you're spending a lot of your disposable income on them there's a moral judgement.
Also subconsciously I think these people want to send a message that they are too clever to think in great depth about what they are wearing.
I also think its a hangover from the Victorian era with its sex phobia. There's some thought process that goes: showing leg or cleavage makes you look poor. And anti-feminist.
It's all bollocks but I've lived among and around these people long enough to have some insight into the way their minds work.
I'm not necessarily a huge fan of knock of Louis Vuitton either, not really my thing. But I do think there's no shame in wanting to look attractive and well-groomed and its very odd that people want to go out of their way to avoid these things.