This part stood out to me like a sore thumb: 'she said 'she doesn't know what else she has to do'.'
Why would she say this, unless this was a repeat occurrence, OP? Be honest now, have you crossed their parenting boundaries before (besides keeping them out past mealtimes)? Because that sounds like the sentence of someone at the end of their tether and who has tried to be firm, and it's falling on deaf ears.
There is nothing more infuriating that a person coming into your home and repeatedly crossing your boundaries and rules in your home. Moreover, when pulled up about it, arguing the case and saying essentially along the lines of 'Can I not just do it my way, because, grandma'. Worse still when it involves someone's children - people are fierce when it comes to their kids.
Perhaps if you respected their boundaries more, they'd eventually come to give you more benefit of the doubt. Just an idea.
Some people are overly controlling of their kids. Not much you can do than adhere to it, if you want a relationship with them, really. Don't worry about disagreeing with mum so much, they're her kids, not yours. You had your time.
It's not your place to in any way argue with their rules around their kids. That's what would get me. I'm not about to overly and repeatedly, explain myself to someone. Couldn't be doing with the grief tbh.
Enjoy your time with your grandkids and stop pushing boundaries because you feel you know better, let them crack on with it.