Just say a polite no.
Unless the collector starts badgering you don't treat them to your reasons. Especially if they include: "She's taking voluntary redundancy and her husband is rich and she's just come back from maternity leave anyway and do you know where she lives? I wish I could afford a house like that..."
I got really fed up with that when I was told by my boss to go round collecting for a colleague who was all of the above and was embarrassed when she found out he'd ordered a collection for her. She's a nice woman, much nicer than the people who whined on at me.