Although I really dislike rudeness, I think some of the problem might have been circumvented by a bit of forward planning and maybe better communication.
Apologies if you did all of this, Conway, but when dealing with a self centered teen I think I would have taken pains to be clear to him that the trip would take no more than X amount of time, and I would have assured him that he would be back in time for his match. Then I would have honoured that promise about time.
If it was a spur of the moment idea of mine, or something that occurred to me on the day of the event, I would have given him the option of joining in or staying home, with the comment, 'well this will be a girls' only trip this time, but next time you and gran and I can spend some time together', just so that he is on notice that he can't get off the hook (assuming he and gran have a decent relationship). Then for next time, I would try to involve him in deciding what he and gran would like to do together and certainly let him tell you what is a good day for him instead of asking him to drop what he has on when an opportunity comes up.
When you are dealing with teens you can't expect them to just jump when you say it -- that is not to say you should tiptoe around them, but rather that you should expect to engage in a certain amount of give and take, especially if something comes up at short notice and there is something else preoccupying the teenager.
Looking at another angle here -- does DS ever help out his gran in dealing with grandad? Does he mow their lawn or do little odd jobs for them? Is there any way he could be useful to them around their home or garden? This might be more meaningful for all concerned in the long run than tagging along on trips.