We're doing similar. We've sold our 4bed/3 bathroom and getting a 3 bedroom in a city. Lots of reasons to move there including better schools, family, connections, etc. We are halving our mortgage to give us more options re working/childcare. Currently we're in rented so have decluttered a lot but will need a second go at it. You just accumulate so much stuff as a family- toys, books, papers, camping stuff etc.
I've actually found it liberating to just look at stuff and say "I don't have the luxury of keeping this any more" and I don't need to still have every towel I've ever owned, or wedding gifts never used, or tatty furniture that won't fit in the new place, or enough DIY stuff to open a shop or a massive craft stash. All the "just in case" stuff is going.
I'm looking forward to having less stuff to clean, store, lose things in and be such an obligation. The previous house really I needed a cleaner to keep on top of it. The new one is fresh, light and cheerful and a perfectly good 1950s family house like many of our parents generation grew up in.
Our mitigations are - first job will be to get the loft totally boarded out with a decent ladder and shelving. There is a garage and a conservatory. Also we are getting a wooden garden office put in for a bolt-hole/guest hut/home office. I think we will put in a downstairs loo but we are resolutely resisting an extension.
Every inch of space will have to work hard and have good storage. But I am looking forward to it. Life is about people, not stuff. In the end, if it's a failure we can just move to a larger house in a slightly less desirable part of the city.