Well, it depends. If you're some super wealthy Londonite with a nanny and six cars and a wine cellar, you probably feel guilty about lots of things. If you live on tins of beans then yes, you should feel guilty.
If your DD is a 3 month old, it's a bit different to a 22 year old.
If it's filled with crappy candles, it's a bit lame.
If you just bought it to put on Instagram with the hashtag #lovemykids or #makingmemories, you should feel very bad.
In the middle... meh, honestly I've found that children simply don't appreciate expensive things. In their eyes everything costs the same amount, "About a pound probably", and I find their reactions to expensive things disheartening. They don't know they're expensive, or what expensive really means, so it's not really their fault. In the end buying expensive things does feel like a waste of time so I don't now, saving the money for the future when they will desire and appreciate expensive things more.