Theres so many degrees of "poshness", its all relative. A lot of people who aren't remotely posh may try to look down on others for not fitting into their "set".
My family are relatively posh, though in a rural, landed gentry way, as in a lot of them didn't have to work for a living and lived in Georgian country houses. I grew up in one for part of the time so my childhood was spent teaching my pony to lead down an ancient set of polished stone steps around the back of the walled garden to his field and evading everyone who told me (rightly) that it was dangerous because they were so slippy (but it was much quicker). I was always slightly orange tinged in winter, before fake tan came along, because our water came out of the taps that colour when rainfall was high.
Anyway...I treat everyone the same and am disappointed when they are not. I don't consider myself posh and deliberately don't cultivate a posh accent, although when drunk or annoyed it can become very posh. My mother was the same and used to look down upon people who were either too "brash" (new money/can't stop talking about it) or just ill mannered. She had an inbuilt hatred of housing estates (even expensive new ones), monobloc driveways and men who drink regularly in pubs. She also hated Edinburgh's New Town for some reason, and referred to the rears of the tenements as "looking like falling-down slums".
I know a lot of "posh" people. Some of the older aristocratic Scottish men at the top of the landed money pile are seriously unpleasant with little good to be said for them, especially those around East Lothian/the outskirts of Edinburgh. The ones from Perthshire always seem to be nice. A lot of Eton and Harrow educated men are very decent. The worst I find are these late forties plus women who have never worked but have somehow married into money and then wasted years building up their self image to something far more important than it actually is.
But my family are also very clever and well educated, and thats a whole other level of posh. I moved to the Borders for a while and I remember being out with a local hunt and being asked by one of the snooty wannabee posh girls there, as I was hinting that one of my horses was for sale, whether I was a horse dealer. "Good heavens, no" I said. "I'm a lawyer. What is it you do?"
As for men, I'm single and I can sniff out a whiff of what I can only describe as "roughness" all too easily. Its something to do with the ability to smirk, or rough skin, or a pointy nose, or something! Or just the general attitude. I was speaking to a man online dating the other day and it was all going well - his skin wasn't terribly rough and his feature were relatively symmetrically arranged - until we discussed hobbies. He told me his included "like going to the pub for a couple of pints, don't usually drink much in the house" and that was enough to put me off. I previously dated a man who was unemployed for a couple of years, so its not anything to do with money. Its much more intrinsic.
But in answer to your question OP, I'd say the real deciding feature is whether you have the confidence to stand up to anyone who is treating you wrongly and put them in their place politely yet icily. Or whether you have the confidence to be slightly eccentric and comfortable enough in your own skin to go your own way - this post is an example of that.