On the day I give blood, I always make sure I go to the gym before rather than after. My blood seems to come out faster! I'm always a bit touch and go on the iron test (have failed it a few times)- so I take a multi-vit with iron the night before and it seems to help.
I go back to the gym the following day (maybe 24hrs later) and suffer no ill effects.
I read somewhere that it takes 24 hours for the blood volume to be replaced in your body.