What would happen if you both agree not to have any credit cards or loans? Your joint income, as you say, is pretty high so you should be able to live comfortably within it.
By not having access to credit, you cannot run up 'secret' debts that your dh has to pay off. That should help break the cycle of distrust on his part and resentment on your part. Likewise, the deal is he cannot use credit cards to fund his hobbies.
In return for cutting up your credit cards, ask for more cash each month - the equivalent of what you'd spend with credit cards. Also ask to have access to his bank statements - and he gets full access to yours - so nothing can be hidden.
I feel sorry for you as you seem so cut off from the financial planning for your family. That must make you feel really insecure and feed your resentment. I really think your husband must should talk to you about plans for this £80,000 he (you) have acquired. I think it's really unfair if he keeps you in the dark. WWW is right, all your money is family money so you and your dh should have equal say in how it is spent and saved. It is about much more than you getting direct access to all the cash, which IMO is a bit of a red herring.
I can sort of understand your dh being cross with you for running up debts outside your monthly agreement. It might not be a lot in the grand scheme of things, but it is the surprise element. For you, running up a £2,000 debt is a gradual process, so you get used to the idea. For your dh, no matter how much he earns, it must be a shock to suddenly find there's an extra £2,000 dent in the family's monthly income.
On a much smaller scale, when I was supporting my dh, if I knew he had plenty to cover his needs and he then announced he owed 'X' amount of money to a friend, I used to be livid! Quite unreasonably so, sometimes but it was an emotional reaction. It made me think he really could not be trusted with money. But them I was spending less on myself than he was, so a different scenario I suppose. However, the rows we had then have put us both off having any personal credit cards or separate loans. I shudder to think how we'd cope if we could both run up 'secret' debts as neither of us are brilliant with money.
Anyway, getting back to you, if your dh refuses to up your monthy limit in return for you (and him) not using credit cards, I'd cut the amount you spend on your family food bill. £600.00 is quite high. If your husband complains about a lack of his favourite stuff in the fridge, tell him you are economising so you can live within your spending limit.