My advice would be to:
Only look while you are 'fresh' by which I mean don't try out beds all day long in one mammoth searching session. When you are tired at the end of a long day a row of wooden packing cases will seem comfortable in comparison once you take the weight off your feet!
Wear loose comfortable trousers and top - to replicate pyjamas as far as possible. Sadly, not much you can do to replicate your birthday suit if you sleep in the nude! You won't be able to properly assess comfort otherwise if all you can feel is your clothing constricting you. If wearing a coat take it off. Same for shoes. Afterall, would you get into bed at home with a coat and shoes on?
Don't test beds by sitting on the edge of the mattress. That tells you nothing useful.
Lie on the bed and relax. Lie in your preferred sleep position and move around as well. Spend enough time lying on the bed to really get a sense of the support.
What you are looking for is a bed with enough 'give' to support
your shoulders and hips and also provide enough support for your spine. You should be able to, without having to use force slip your fingers/ hand under you. If you could run your hand easily between the mattress and your body it is too firm. You don't want voids. When the level of support is wrong your body will be in tension because your muscles are having to work to keep you comfortable and you definitely won't wake feeling refreshed.
Don't think/worry about other people seeing you - they're not buying your bed nor sleeping on it!
The more you do this the easier it is to relax. I never care if I look odd - I'm the one paying the money so I won't be the idiot who sits on the edge and pronounces "I'll buy this one - it seems fine!" Saw and heard that when I was looking for my last bed. If you are self conscious try out beds during the week if you can.
If you are of very slight build it is likely a softer mattress will provide sufficient support. If you are heavier then a firmer mattress might be needed. Also if you have a partner and they are a different build to you, you might need to consider having a dual mattress with soft/medium sides to meet your individual requirements.
Obviously I am talking about sprung mattresses here.
If you have a sprung/sprung edge divan base for your current bed, then that will also have become less efficient in my opinion. I would get a new base and mattress together. Otherwise it is a bit like laying new carpet over existing underlay. The underlay doesn't support the carpet and it stretches and you potentially void any grounds for complaint regarding non-performance.
If you have a solid/non-sprung base and it is sound then you probably can get away with re-using it. If you have or buy a firm/solid base that will make whatever mattress type you put on it 'firmer' to the feel.
The more springs a mattress had the better the support. And pocketed springs are the best as they allow each spring to take the load without unnecessarily reducing the support provided by adjacent springs. And they lessen the effect of any movement from a partner.
Also consider the layers and materials used to construct the pad over the springs. Some come with a summer and a winter side.
And once you've found your ideal mattress don't forget to follow the care instructions for turning and stopping and tailing it. These add to the life of the mattress and your comfort.
Finally, buy the best bed you can afford. There's a lot of hype about changing your mattress every eight years - I am of the mind that you change your mattress when it is no longer giving you a good night's sleep.