OK, let's not get diverted to aleph-null and 0.9 recurring yet. 
So, I think, @goodwinter, you agree in what we shall call Scenario 1 (chest starts full with 1, 2, 3, ... and B takes out a coin a day in the order 1, 2, 3...) that the chest should "end up" empty. (I've put it in inverted commas because there is no end).
Now imagine we study the set-up in a bit more detail. We've told B that the chest is infinitely full, and the way it works is it has a little drawer at the front, and when she opens it she can see the next few coins in the sequence and can take the one she wants. She can't see into the back of the chest, but that's fine, she's an angel and believes us that they are all there.
But what if in practice A is a devil and is being economical with the truth. In fact, the chest is empty, but every night A sneaks along with his infinite bag of coins and puts the next ten coins into the chest. Why does this make a difference - you could just think of the bag as extra storage for the chest? From B's point of view (and ours) every coin that exists appears in the chest and gets removed eventually.
The process may be infinite but each number involved is finite and that means whatever number you care to name will eventually be removed from the chest.