It makes me angry when it makes you angry that other people choose, eg, animal charities over human, overseas charities over homebased (or vice versa), children over old people etc. The fundamental point is that it is just that: charity. Nobody "has" to give a brass farthing to any one of them, but they all (arguably) do a vital job which otherwise wouldn't get done. Furthermore we all have our own priorities. It is not for one person to dictate to another what should be considered more worthy (which I think was, at least in part, SGB's original point).
So, say you see me popping a spare quid into the RSPCA box (which actually I wouldn't, before Valhalla looks at me funny!). You might feel like nudging me and hissing "psst! There are children starving in Africa, you know!" Well, there's a fair point in there; but for a start, you don't know I didn't bung two quid into the Oxfam box in the previous shop. And to carry on with, some people think charity begins at home. You might think I should show species loyalty; I might think that animals are suffering because of what human beings do to them, and I for one would like to redress the balance. Supposing I really couldn't care less about people suffering in other countries (as unfortunately many don't), the alternative is not that I bung the money into a starving children box; it is that I keep all my money to myself,.
You might see me bunging a quid into the NSPCC box, and again the nudge and whisper "The NSPCC have plenty of money, what about supporting a less fashionable charity?" To which I could reply "As long as the NSPCC have a job to do, they need all the money they can get". Or "I can't be naffed to research unfashionable charities, but here's a box and I feel like putting my spare change into it, what's your problem?" Again the alternative is not that the local burns unit, a lonely old person, a mentally disturbed teen or a flooded African village get the money instead, but that the money stays in my purse and probably ends up contributing to the annual profits of Messrs Rowntree and to another inch on my already ample hips.
I could put this well-travelled coin into a missionary box because I believed strongly that bringing the joy of Jesus's love to those living in sad ignorance is a truly charitable act; that saving an immortal soul from damnation is more important than saving their bodies from earthly discomfort; or even (in the teeth of the evidence!) that spreading the Gospel contributes to world peace. Should I then have a right to insist you do the same? It would be so clear to me that if you teach children about the love of God they will not grow up to shoot their neighbours, I just couldn't understand why you would want to focus narrowly on one parochial, superficial issue like adults with facial scarring. On the other hand you believe that Churches peddle pernicious nonsense, and that the joy you can personally bring to one young woman by restoring her disfigured face is what charity is all about.
Could go on, of course (and on, and on...), but the point is, you have to give to what you believe in. If you have a cause you support passionately, it is up to you not only to give it money, but to spend time and effort raising its profile. People who give to the "wrong" charity aren't depriving yours, they're giving where they see a need. It's up to you to persuade them your charity's need is greater!