Yes, demented, that's a good point and I don't think you have to have children with special problems to feel like this. On some occasions, when my sons are having a major wobbly, I have had to detach myself from the situation and ignore them, simply to keep sane.
I have been thinking about this discussion a lot. I don't think it is fair to judge a parent on a fleeting moment (unless the child is subjected to violence or physical risk), but, I agree with others here who say that some children out there are having a consistently awful time at the hands of their parents, and yes, we should probably intervene more.
I can look back at my son's friends and acquaintances over the last five years and think of at least three or four children who seemed neglected or worse. In cases were I intervened, it was to offer a meal, a few hours invitation home, a visit to the park, ie temporary measures, a sticking plaster. In only one instance did I think of tackling the mother or the authorities, and that was only because I knew the mother closely and so could be sure things weren't right.
IME it's much easier to intervene at child level than at parent level. These children were school age, and I hoped that the school would pick up on their problems. Is that a naive hope? Failing that, I hoped a relative or friend would talk to the parent.
The only thing I felt I could do was to offer the child friendship and keep those invitations coming. Anyone can judge a parent, it's what you do afterwards that's the hard part.