Depression is an deep, dark and agonising thing to suffer from.
Your DH's mental state is probably so debilitating and all-consuming that he simply can't image a time when he'll be able to deal better with day to day life, let alone looking after a puppy/dog.
The realisation that he has taken on a long-term commitment has probably just hit him, which may well be freaking him out. The change in routine will be very hard for him as well. He also can't imagine how he could every possibly feel better, which is why he can't get to the point of acknowledging that anti-depressants (or similar) may help him. It took me 4 years to acknowledge that I needed help.
Don't let other people push you into a decision - you have to feel you're doing what's right for you, although I know it's incredibly hard. It may well be that it's just not the right time for both of you to take this on.
It's very, very hard but you can't blame him for feeling this way. It's awful if you've bonded with the puppy, but susiedaisy is absolutely spot-on in her description - it was exactly the same for me.
You should talk to the breeder, or to some local/national rescues about rehoming. Some may not be sympathetic but you will find some who will understand where you're coming from and will help.
I think you may have to decide to return the puppy, it will be very difficult but it may be better than feeling guilty and torn between you, your husband and the puppy. But try not to let any guilty feelings get too much for you, just try to acknowledge that it wasn't the right time.