I sort of understand your dh. I inherited enough money about 5 years ago to pay off half our mortgage.
Rather than actually pay off the mortgage, we changed our mortgage to an offset mortgage which means that we benefit as a couple through lower repayments, but I retain the actual money in my account.
Somehow, that money is special to me. It was given to me by my grandparents and I still think of it as "my" money.
DH is fine with this and knows that if I die, it will go directly in trust for dd, and not form part of my estate which goes directly to him.
About 2 years ago, DH recieved an equivalent amount when an investment paid off. Again, he retained the actual money in his account, but our repayments lowered again as it was counted against our offset mortgage.
I think that if your dh was refusing to use the money and it was causing you financial hardship then he would be being unreasonable. However, I think that you are actually the one being unreasonable, expecting him to pay off the mortgage with his inheritance money, just because you don't like your job.