"And that attitude towards horse owners and riders I'm afraid is the root of the problem. It's the fucking ridiculous stereotype that to own or ride a horse you're a stuck up snob, and somehow therefore deserve to have your life put at risk because someone has a skewed view of owning an animal."
Indeed. Most people I know that have a horse(s) have hardly any money left after horsing costs are catered for :D
If you're using a public road you have a responsibility to share it considerately with other road users, regardless of what mode of transport it is - pedestrians, cyclists, horses, lorries, tractors. Everything and everyone.
If you're think you're more entitled to be on the road than any of the above (and any I missed) then you're a grade A bellend.
That goes for inconsiderate (dangerous) drivers, self-rightous lycra warriors, white van men, audi/bmw/range rover drivers, boy racers in shit corsas and whatever other stereotypes you care to think of.
I use the roads in various ways:
Drive a car, ride a bike, occasionally brave riding a horse.
I treat other road users how I'd like to be treated - so for horses I slow down, pass wide and slow, wave back at the acknowledgement I (usually) get, swear at the ones I don't, quietly after I've passed 
If I'm on a horse, I've been lucky - never had someone fly past me - roads aren't wide enough, though I did get one farmer (probably a contractor) with his hedge murdering machine refuse to turn it off when he could clearly see my horse was not a fan of the racket it was making.
Most of the time any problems with horses being startled are sodding pheasants flapping out of hedges just in front. Who knew horses could leap about 6' in the air from a standing start?
On my bike, I don't ride in the gutter, but I don't sit in the middle of the lane like some self-rightous prick either. I will avoid potholes (of which there are bazillions) because my bike doesn't have suspension and my arse doesn't appreciate it, also I don't want to knacker my wheels/tyres. But I won't swerve around like a nobber either. Shoulder checks and signalling are a good thing.
I wave drivers past if they've patiently sat behind me and I can see over the brow of a hill/round a corner that it's safe for them to pass. If they pass "too close" then they might get a sarcastic wave, but there's sod all point being confrontational about it. I'm on a bike, they're in a metal killing machine. I'm not going to win.
I pass cyclists how I like to be passed if I'm on my bike. I thank groups of cyclists for waving me passed, or moving back into single file if it's a narrow road to let me past. I'll get annoyed at groups of riders that don't.
TL;DR - don't be a dick about other people taking up precious space on "your" road.