You sound lovely
Wish you were my neighbour!
Does the money issue bother you, or have you kind of let that go?
I agree with AgentZigzag and Shylepite - he was procrastinating and then started to feel guilty, which he didn't like, so he started avoiding you to swerve the guilt, and then the bad feeling sort of snowballed for him. So you have become a trigger for him feeling crap and cross - although it's all his own fault.
Some people who don't have oodles of emotional intelligence can start unconsciously blaming the person they have wronged/short-changed due to this!
He might even have tailored his thinking to match his feeling of annoyance now, convincing himself you didn't pay him enough for the job or something - justifying his silly behaviour.
Anyway, who knows. If I were you, I would pop round, as others have suggested, and say to him or his DW that you just want to let them both know not to worry about the garden stuff anymore and that they must pop over for some tea and cake soon.
Then leave it and carry on as normal - smiles, hellos if you pass. You can't do anything else. You've behaved impeccably.