My older 3 were all around 6ish before they showed genuine wish to stay over at my parents, even though they were familiar with being at her house during the day. We were there a couple of times a week at least. My mum never pushed the issue of sleep overs, and i never felt the 'need' to have the kids out of the house all night when they were little. If (X)DH and i wanted to go out GPs would come to ours and sit. I think they slept at my mums a maximum of 3 times in their life.
With the ILs, (now outlaws) the kids showed 0 interest in being at their house much at all, probably because the ILs showed 0 interest in having them their. ILs youngest sons DCs were the 'golden' grand-children and the other GCs (not just mine) were just background noise, very sadly.
My parents and the ILs were within walking distance. This is years ago - older 3DDs are teens and 20s now.
With my 2nd (and current) DH - his parents are very hands-on with their daughters kids (because she uses them for child-care) with sleepovers every week or so. However their sons DD has only slept round there once in 3 years. It was a bit of an emergency which prompted it and i seem to remember it didn't go well and hasn't happened again.
DH and i have an 11 month old together, who wont be sleeping at my mums ever probably, because my dear dad has passed on and mum is rather frail now. DD4 may stay at her other grans some day, but as they live 2 hours away it will have to be for a good reason, really, and be when she is older and much more familiar with them than she is now. At the mo they only see her once a month and she is very shy with them.
But that's life - every family is different.
That was long! 