Has he always had a relationship with his grandparents that he would say is good?
I'm going to come at this from a similar but different perspective.
My DM has always been upset that I wasn't more engaged or interested in my very elderly grandparents. I'm autistic and had an unusual childhood but I'm not adopted.
My DM often confused her relationship with my elderly grandparents with her relationship with them.
By the time I was a teenager they were extremely infirm and had mentally deteriorated greatly. Id never had the relationship with them that she had, but she obviously remembered them in happier times when they were well and happy. I had no memory of those times.
She obviously loved them so much and had a lifetime of memories with them. From even my earliest memories of them they were confused, unwell and not too sure who I was. I never had a strong "relationship" with them as such.
But she was so very desperate for me to have had that relationship with them (that had never really existed) that she was so upset that I wasn't more devastated by their decline.
But I felt I'd never really had a relationship with them in the way she was imagining, plus being a teenager (and not neurotypical) which meant she was interpreting me as being cold and disengaged. Whereas I felt I wasn't losing anything because I'd never really had it.
Sorry if that comes across as cold, I'm not great at trying to explain what I'm saying. But perhaps he didn't have the relationship with them that you remember or wish? So he's not feeling the same loss and sadness.
I always tried to respect my Dms feelings, but I felt I couldn't pretend to have the feelings that I didn't have.