If you knew someone who had been signed off work with (work) stress, caring responsibilities then bereavement of a very close relative, would you think it was wrong to see them in the gym, shops or pub whilst they were off?