Stereotype - your opinion please?

[OldandJaded]

I'd like peoples views on horse owners/riders please, and why you hold those views.
I have noticed that as both I tend to be stereotyped as posh, like I think I'm better than I am, rich and a snob.
It couldn't be further from the truth, I love horses and always have and I work hard to afford them (large horse with small pony for company) I don't do anything else, don't have holidays, don't go out for nights out as many of my friends do (unless its a special occasion) and I budget to afford owning what I consider the most amazing and graceful friend I could ever wish for.
I'm genuinely interested to know why because I choose to spend my money and time on an animal instead of nights out, new clothes and shoes, handbags or other sports, that I'm automatically considered 'posh' - is this a general stereotype out there or have I just been unlucky?
An incident this morning has got me thinking (along the lines of child wants to pet horse in street as I'm riding, I say no as traffic coming both directions and I just want to keep stuff moving and not cause a hold up - got called a posh and asked who I thought I was)
Myself and a few of my friends have noticed this attitude towards us - can you give me your honest opinion on what you think please?