It depends on what you mean by 'less nice' and what your priorities are. We lived in a very posh, chocolate-box bit of outer London where we had a tiny house with only one full-sized bedroom and no garden.
We moved to somewhere three times larger and near to DH's work, close to a park, and with a gym and shops within five minutes walk.
The area we now live is much more socially and economically mixed, and I prefer not living in a social ghetto. Reported crime rates are higher, but I come originally from one of the poorest cities in the UK, so don't notice in practice.
We save a lot of money because everything (childcare, shops, building work) is cheaper - they had loads of gymwear shops in my old suburb, but I used to have to cycle or take a train to buy sensibly-priced running gear.
I wouldn't move to a less nice area if it was a dump (e.g. regular break-ins and stabbings in daylight, suitcase stolen if you leave it outside for few seconds), if it had fewer amenities or a worse commute, or required a huge lifestyle change. We looked at another area, further out of London but, although nice, it would have meant using a car for everything.