Dh works full time and we have 3 dc's, the youngest of whom is 4. I have mostly been a SAHM but have also worked part time at times. Because of the nature of dh's job (involves travel and unpredictable hours) I can't go out to work. He has always acknowledged this and has always said he appreciates that he doesn't have to worry about who's going to pick up the kids, what happens when they are ill, etc etc.
In the last year I have been studying something which would mean ultimately that I could work from home. I use the 15 hours of nursery care to study, and we agreed that I could have Sunday daytimes as well, i.e. dh would look after the kids while I worked.
The problem is that week in week out these Sundays are a nightmare. Dh gets in a huff right from breakfast on Sunday morning and complains about every little thing, so that by the time I go up to start work everyone is already in a bad mood. I am trying to work and I can hear arguing and crying and all sorts. He complains if there isn't enough food in for lunch (shop is round the corner) and behaves generally as if the whole thing is just a drag. Final straw came today when he burst in, having taken them to the park and ds2 had lost a rucksack with dh's phone and wallet in. He was almost incoherent with panic and dumped the kids and rushed out again to look for it. I then had to deal with two hysterically crying children, ds2 weeping and saying "it's all my fault" and ds3 was crying because he had been scared by dh's handling of it. 
It was awful.
What do I do? I can't face another Sunday like this and it's not fair on the children. It's not that he won't look after them, it's just that when he does, it ends up like this.