Most people work hard, but a large part of earning very high salaries is luck, which is being in the right place at the right time.
I don't think it's just luck that separates high and low earners (though I absolutely believe luck is involved which a lot of high earning people will not admit but I'm not having that conversation again so no one bother picking me up on it!)
BUT I think the other thing is strategy, when I look at the lower earners in my family and what has set me apart, it isn't hard work as you say, they all work hard (most of them work physically harder than me), but it's the fact I have strategised my career; be that qualifications, training, mentoring, I have regularly changed jobs against the comfort of familiarity to get up the next rung, I've taken on horrifically long commutes in order to get to the next step and made really difficult short term decisions (especially whilst having young children) with a view to it paying off in the long run. I've never coasted, whereas a lot of my family have no interest in that, they stay in their jobs for years, they go to work, do their job, come home and don't think about it again (obviously a very valid lifestyle choice!) for me, when people say "they've worked hard" it's those strategic and challenging decisions they mean rather than the physical daily work.