Some odd responses on here Op. I dont think you sound controlling, just sensible.
We have substantial savings, partly via windfalls but also because we still shop in Aldi, Asda & poundshops despite having a decent disposable income. I do occasionally buy in a local Waitrose, but its generally for something specific or if I am close by and just need a few bits. But then both DH and I have a similar attitude to you to money, we would see a full Waitrose shop as wasting money.
We had some windfalls in 2016, the majority being 'mine' (inheritance and a work share windfall) - all has gone into joint savings, being drip fed equally into ISAs, some used for a newer car for me, some for a big family holiday (and 5k is nothing - ours was more, with a 3rd adult and a teen). The rest is sat as a nest egg for possible future projects eg maybe a campervan, or for kids, and generally to ensure day to day life is easy.
I do not see the money as mine in any way. It is family money. If DH wants to buy something large with it, no problem, although I would expect us to discuss it. Similar with me. I did spend more on my car than originally planned, but I wanted a certain spec, and we agreed that spec, and physically bought it together.
If DH eventually gets an inheritence (no guarantees we know) and decides to secretly squirrel a lot away in his own name, I would be well pissed off, and would be questioning our 20+ yr relationship. There's nothing wrong with treating yourself, as long as the money also works for the family.
We too have a child at uni and a younger one, I am guessing you may eventually have 3 go to uni, which in itself is expensive!
Why not try and have a sit down with him, on the pretext of whether you should pay some of the mortgage off? Would he discuss that? We have been mortgage free for a while and it is a lovely feeling.