I really empathise with this.
I live in a lovely 4 bedroom house, with DH and our 2 children, so have a spare bedroom, mortgage paid off, loads of money to go on holidays, which I love, and much prefer spending my money on this than more house. Our house is in a lovely location, with an area of woodland behind us that will never be built on. We have a garage, big garden, space for 5/6 cars.
BUT, I just keep comparing myself to other people all the time. My brother-in-law who lives in a 7 bedroom house, friends of ours who live in a 5-bedroom house, my sister who lives in a 4-bedroom house and she hasn't done a single day's work in her life, whereas I worked hard all through school, and have worked full-time for most of my adult life. Why does she get the same as me? Even though it's not really the same as me, because none of these houses have the big garden or extra parking, my sister's doesn't even have off-road parking, never mind a garage.
I'm obsessing over number of bedrooms, basically. I'm a perfectly intelligent person and don't feel like this about any other material things. I drive a 10 year old car and don't give a shit what anyone else thinks about that. My DH is perfectly happy where we are and really doesn't want to move, so we won't, and deep down I don't really want to. I keep looking at houses in our area on right move, and they are all new builds, and yes, they have an extra bedroom, but overall there is less space than our house - less parking, smaller garden, overlooked behind - it would be utter madness to spend probably another £200000 buying one of those new builds!
I just wish I could feel really happy with what we have - I used to, but I now feel like everyone else has caught up with us, and I should be living somewhere bigger and better than them. It's so pathetic.