I feel really sorry for you. Money matters have a way of eating at the heart of even a really strong relationship. I have cried so many sleepless nights over finances, more than some other seemingly stressful things, you literally can't sleep as all you do is lie there calculating just how much you haven't got.
The thing is: you are both blaming each other and angry, but the problem is money. If he only just finished the job, and you had no hand in it, he may be right that you starting to harass the person to pay a few days later may be too much. But I totally understand the desperation which would drive you to want to do this.
OP, I just hope your luck changes, the period in which we were in serious financial difficulties was the worst of our marriage. But we have staggered through it.
Is there anything you can do about the mortgate, get a payment holiday? I might have a serious chat with the mortgate company. I think it is the unknown that is stressful, in some ways, hitting rock bottom (we lost pretty much everything) was actually better than trying to keep it all juggling in the air!
It's very easy for people to say: he's not a great guy in a financial crisis or he shouldn't snap when you go on at him, of course not. But, the reality is that we are in the middle of a horrendous depression, you can't get work, and this does affect how people act, even though of course it shouldn't. The pressure on him is probably fairly terrible, he made a judgement call to set up his own business, probably at a good time when he saw a strong future, and it hasn't worked out. Thousands of people have lost their businesses and their homes and it isn't all because they are crap with money.
Keep on in there, it will get better at some stage. Try to get back that feeling of being a 'team' fighting against the world, rather than fighting each other.