YANBU. That's actually really bad, that he has only looked after her for 15 mins I know things are harder when she is bf, but there should still be an hour or two at some point during the day where he can take her out, or you can go out.
What shewhowins says. Other than the fact (and I am just assuming here, so please don't be offended if I am wrong) that he is working and bringing the larger salary while you are on maternity leave, is there anything you are gaining by him being around? You sounds as though you are pretty much doing everything yourself.
Is there a limit in your head? If talks won't change his involvement, and you're not currently in a position to leave him with the baby for a day - will you just put up with this, or would there eventually be a dealbreaker?
I don't think many people are natural parents. We all have to try, I'll bet you didn't wake up pregnant and know how to be a mother. You either read, asked, or, like most, learnt on the job. It's hard. His excuse is flimsy, and the shopping trip appears to be a red herring for the other deeper problems in your relationship.
I think that if he has brow beaten you to a point where you can't talk to him for fear of being shouted down (I know you shout too, but it sounds like he starts to shout, and you get drawn in) and he has spoken many words but taken no action then somethings got to give. Have you got any childcare? Even for an hour where you can attempt to talk to your dh without the baby, a bit more of a neutral setting? It's very early days in terms of parenthood, and new born babies can put stress on a relationship like nothing else. If you can perhaps try for marriage counselling. Perhaps ask for this thread to be moved to relationships, you obviously need some support, and aibu can sometimes draw some negative people. Good luck.