Hello - we moved from London to Lewes a few years ago. While I'd loved living in London, I didn't miss it at all when I arrived here - partly because I realised I am at heart a real outdoor type and love cycling/walking/running in the hills round here, and also the town is so beautiful. Also, Lewes is packed full of really interesting, friendly and lively people and our social life actually improved compared to London. Admittedly the formal nightlife (was) a bit thin, but there was, pre-lockdown, loads of music, dj nights, poetry readings, comedy that sort of thing, all of pretty good quality; ok nothing on the Brighton scale either but certainly enough to keep a parent amused. We've also been to some fairly hedonistic parties too, but I'll keep that quiet. It was, admittedly, a shock how quiet the HIgh STreet was after say 5.30pm, mind.
Also having Brighton 15-20 mins/£20-£25 taxi ride away is pretty good given the shops, restaurants etc. I'm not a big fan of Hove (except the bits that are basically Brighton eg round Seven DIals or Brunswick Town) as I seems a bit blingy/land rover in bits and feels a bit Clapham-on-Sea but maybe I just don't know it. Mind you, Lewes has a fair share of trustafarian types which are just as annoying; people who have no discernable source of income & work intemittenly as "artists" but seem to have big houses and nice lifestyles all the same. There is, as others have said, a very Guardian-reading vibe which may annoy some too.
I don't get the point about having to drive to supermarkets - the Waitrose is basically slap in the middle of the shopping area and the Tesco & Aldi 5 minutes walk to the north of there?
On another note - Lewes is not quite as middle class as it first appears. There is a big, mostly white, working class living on estates such as Landport and Abergavenny, which have patches of severe deprivation. If you go to some of the primary schools it will be immediately apparent this is not as middle class as the stereotype suggests! I would add on the other hand that it is becoming more diverse but nothing compared to London obviously. Brighton & Hove is also one of the least racially diverse cities in the UK.
One other thing - after a few years here, it's great to find that when you walk around you know a lot of people out and about. Some might find that claustrophic after the anonymity of the capital.