This is something I have been thinking about recently. Ds1 has graduated and left home, ds2 graduates this (though he may go on to do a PGDE), and ds3 graduates in 2020, so just over three years from now, we will have a properly empty nest, and dh wants us to think about downsizing.
I am very ambivalent about it. Whilst we'd be able to manage, with three or even two bedrooms, we have a lot of stuff that needs downstairs space - I have a room that is basically a library, and even with three walls of books, I still have to have regular culls, so they fit on the available shelving. I love my books, and cannot get on with e-readers, and I don't see that changing as I get older.
I knit, and need a fair amount of storage space for yarn, and I also have other hobbies that require storage space for the stuff involved - art materials, fabric, sewing machine etc. We also have a piano - we inherited it from my mother in law, and so it has sentimental value for us, as well as being useful for me, because I sing in a choir and sometimes need to note-bash new pieces.
Then there is stuff which fit nicely in our current house, and that I enjoy having around me - an oak dresser with pretty plates on it. Yes, I could store the plates in a kitchen cupboard, but I love the look of the dresser and plates.
I look at the things we have accumulated over the years, and the thought of having to get rid of enough of the things I love, that what remained would fit into a much smaller house, daunts me. I suspect many older people are in a similar position - my mum moved from a reasonably sized three bed bungalow to a tiny two bed retirement bungalow, and even though she got rid of a huge amount of stuff, her new place is still horribly over-full and cluttered, and this has contributed to her feeling like she's not at home there, and she is finding that very hard, and depressing.