So my answer probably wouldn't work for you, but in the same mindset, aiming to work less/retire etc, I'd look to save a lot of that money, especially as it appears to be entirely discretionary - you say that you pay for groceries and bits from B&M on DHs credit card, so there's probably more spending there that you aren't counting, on top of what you are spending.
The sort of things I would think about, when deciding what to do - I'm not asking you to tell me the answers to these questions, but more, think about it yourself/talk to DH about:
Is your house filled with stuff and does this bring you joy? Also look at what your DH is spending so you get the bigger picture - what's going on the credit card each month - most allow you to download and categorise a year's worth of transactions, so you can see what you're spending where, although it's not 100% accurate, eg if you buy lots of clothes, books, alcohol, make up, household goods at the supermarket, it will categorise all these as groceries I believe.
Do you have good pensions and what's your mortgage rate?
Do you expect your DC to go to university? Do you want to help them with house deposits?
How secure are your jobs and do you like doing them? Are you on track to retire when you want to? What assets do you have - house, savings and investments?
It's probably also worth looking at what your DH is spending, if you say he doesn't save.
Obviously there's no need to live on an austerity budget, but what you don't want is to find that you can't afford to retire, you can't help your DC, one of you becomes ill or loses their job and you've nothing to fall back on, because you haven't saved enough. Also, if your mortgage rate is above about 1.5 -2%, it's probably worth overpaying that, rather than saving elsewhere, as that's a guaranteed return.
I'd probably be aiming (or overpay the mortgage, but watch for penalties) to save £1500 to £2000 per month and then spend the rest without worrying about it so much, but be mindful of buying stuff you don't need for the sake of it.