Agree that generally, giving a gift back is rude.
My parents talked for a long time about getting a steamer for cooking veg. One Christmas I bought them one. Within a week they had brought it back to me saying 'We don't think we'll use this after all'. I was a bit put out, since they're quite difficult to buy for and I had been paying attention to their conversation and had picked up that this was a planned purchase for them.
Another year, I bought my mother a foot spa as she spent a lot of time with her feet in the washing up bowl. Again, within a couple of weeks it was back with me as she 'didnt need it'.
I've learned my lesson from this and they get vouchers for Christmas now.
But the crowning glory was the day she gave me back all the kids photos. You know, you get the school pictures and you give one to each set of grandparents? She handed me back a pile of them, still framed, with the words 'I don't need these, I know what they look like'. [shocked]
Two years later when we had professional pictures done she was might narked not to be given a copy. 
All that said, however, I'd rather take my cake home and scoff it myself and not share at all oh no than think of it going stale and wasted in someone else's house...