So sorry to hear you are going through the last stages. My DH died in February and it's been a learning curve.
Joint account(s) should not be frozen but when you do notify them, they will remove your DHs name. Same like the deed to any joint property, the Land Registry simply removes his name and you become sole owner. If your property is not joint tenancy, work hard now to make that happen.
For starters, the banks won't know he's died until you tell them. So the bank accounts will remain active until you contact the bank. The banks are not included in the 'tell us once' service (not much is actually and you'll still need to contact the council, etc) and need to be told individually. The banks have all been very kind but the employees are strangely unaware of how to help beyond freezing an account which they seem to think will be a real comfort but is not really here nor there.
If he is able, ask/help your husband now to make a list of all his accounts and login details for them (not just bank accounts but utilities and apple ID and devices like laptops and anything else that needs a password) so that you can deal with most of this online. My DH did this well before he went into hospice without telling me and then when things were truly dire and he couldn't quite remember his passwords, it was really useful.
I didn't tell the banks immediately and being able to log in to his bank accounts was helpful so that I could see the scope of his committments and estate. Also, those accounts continued to pay utilities until I could transfer them into my name.
If your DH had ISAs, and you are his heir, you can keep these as ISAs without losing the tax advantage. When you contact the banks, ask how to do this, since I have found them all to be strangely unaware of how to help. There are some sensible things to do but you will have to guide them through the process.