If I was arranging flowers it might be lovely "me" time but pruning trees and weeding is like cleaning, a necessary regular chore, not "me" time. It needs to be done.
DH has flounced off blaming me for ruining his day because he has to spend some time with the kids when he wants to do his thing (waxing surfboards as you ask).
I am I being unreasonable?