Gnome: re garden accidents. I would let my children play - of course I have, many times - in other people's gardens. It would not cross my mind to assume they were dangerous places: because I trust the parents to have that normal level of responsibility wrt to chemicals and tools, the same as I do.
I would never ask if they are supervised while in the garden because of those risks: I would assume the risks would be absent. I think this thread demonstrates that in one way, if the crowd earlier are normal/typical, there is a high degree of recognition that gardens need to be safe when children are playing. Since the OP has this regular agreement with the neighbours it's normal to make the same assumption.
If they had a pool however I would always ask about the supervision. Pools are not normal in gardens and again, as demonstrated by this thread, because of or perhaps despite that, there is a very high degree of complacency and irresponsibility about pool safety.
I think it is strange to be so dismissive about pool safety and to have as your chief concern general garden safety or even manners.
I'm not saying you are, you have a view on both, although it favours general garden safety, not at the derision of pool safety.
My primary concern, the one I would ask about and ensure was well covered, is pool safety. I have asked about pool safety loads of times but never general garden safety. That is a given, considering I know the parents.