So sorry to hear this, OP. I expect by now you've registered the death, or are gearing up to do it. We found when my Dad died that getting that done was a big step. We were able to do it over the phone, which I hadn't expected, and it made things a lot easier as I had brought all the paperwork down to the living room and my Mum and I could refer to that as needed during what turned out to be a longish call. We used the Tell Us Once service and it was very helpful. DWP, DVLA, HMRC, Passport Office, NHS, council tax department, library, emergency alarm bracelet service, electoral registration - all notified as a result of that one call.
We heard very quickly from the DWP and HMRC. We got a small repayment of Dad's PAYE (could have gone the other way, of course). Mum's council tax dropped because of the one person discount, which might be relevant for you.
I looked through their paperwork to identify all the regular payments and set to work to contact them in order of urgency. I also went through Dad's emails to see if there was anyone or any organisations there that needed to be notified. It took quite a while for post and emails to stop coming, which is unfortunately just how it is. The odd one needed a reminder.
We started with the bank, and their bereavement team were really good. The joint account was changed immediately to Mum's name only. One thing we hadn't thought of was that their credit card, which they had thought of as joint, was in fact issued to Dad and Mum had a second card. On his death the card was cancelled. However, we did an online application for a new card and she had that in under a week, so it wasn't too much of a problem.
It was reasonably easy to change the BT account and energy account to Mum's sole name too.
They owned their own home so had buildings and contents insurance. I notified the company and they charged an admin fee to change the policy over to Mum's sole name. I thought was poor form in the circumstances, but we had no choice but to pay. We didn't renew with them, though!
It is a lot. However, it will get done. Don't overdo it. Not everything needs to be done at once. 