Why do people say 'it's all downhill from here'?

[R9090]

Surely they mean it's all uphill from here (more difficult)

Or have I misunderstood

Because down is generally worse. It's not about the effort of getting uphill as opposed to downhill bit where you end up. Down as in feeling down, down as in where we traditionally portray Hell to be. Down as at rock bottom.

I always took as the metaphor of having reached the summit (ie a great achievement, peak [geddit?] performance), that there's a resigned recognition that everything beyond that is a bit of an anti-climax, or even carries a sense of failure - eg a football team that wins the World Cup and then subsequently struggles to impress.

'Its all downhill from here' means it gets easier.
its going downhill means its getting worse.