Because people think you're posh therefore rich, therefore wrapped in cotton wool by your family, therefore an ignorant snob, therefore judgemental, therefore unable to contribute anything of value or funny or interesting.
I live in the north but grew up in the south and I've lost count of the amount of times I've been treated like I'm a "soft southerner" No one knows (because they haven't asked) that I grew up on a council estate, had a heroin addict step parent, my mum left us when I was 1, I failed all my GCSEs and have had to work really hard to get qualified in a reasonably good job and lead a stable happy life.
It's just assumed I'm posh so I need to shut up.