I do think it's always been quite common for young parents with very young children to take the children to 'nan and grandads' - not only to 'visit' but also to occupy and entertain them, as young children can be very full on and hard work. Even the ones who are 'good' and quite well behaved.
My cousins (as adults,) and also some of my friends, frequently visited their parents with their children, with the 'they wanted to come to nan and grandads' line. In reality sometimes they brought them to occupy them, and for a bit of a break. Sometimes, my auntie(s) would take them for a walk or my uncle(s) would take them to play footie, or to feed the ducks. My parents occasionally did the same with mine when I visited.
Nothing wrong with that, because (as I said) looking after young children is hard work and demanding, and it was common for families to help each other if one of them was struggling. And my extended family enjoyed having the much younger family members around.
I do think it's sad indictment of society, that some grandparents regard their adult children and grandchildren visiting 2 or 3 times a week for a couple of hours as a nuisance and an 'intrusion.' My aunts and uncles (and my own parents) just LOVED seeing their grandchildren, and would play with them for hours sometimes.
These 'grandparents' should be careful with their rudeness towards their adult children, and giving them the cold shoulder (and sarcastic demeaning remarks) when they come, because they may get what they wish for. One day their adult children may think 'well they don't seem too arsed about me OR their grandchildren, so why am I bothering to stay around here?' And they may leave the area.
Growing up in a friendly, family-orientated community, where kids played together all the time, cousins often visited, (and went out to play with me and my brother and our friends,) and grandchildren frequently went to their grandparents homes (and were welcomed,) I can't get my head round the 'maybe they don't WANT you and your kids around 2 or 3 times a week' attitude.
I mean, FFS, it's not like the OP is stuck there 4-5 hours a day every day!
Some people are becoming curmudgeonly, even with their own children! No wonder so many older people are so lonely now if this thread is anything to go by. Probably because many of them made their adult children and grandchildren so unwelcome over the years.