no, that's "pardon me"
And obviously quite a lot of people care 
I grew up saying what (lmc with wc roots and relatives) and began to be told by teachers to say pardon at almost exactly the same time I was becoming aware both of class distinctions in language and of the advantages that being thought higher-class might bring. (And read Jilly Cooper's seminal work a year or so later.)
The resulting confusion means I now say "sorry?"
Still cling on to lounge, settee, loo, napkin or serviette depending on context and company (yes, you absolutely can use both [eyeroll]), scent, mirror, mantelpiece, glasses, false teeth, jack, die (? I'm looking at the wikipedia page right now), jam, pudding, teacher. Take issue with their putting lounge against dining-room (two completely different things) and agree with pp that cemetery and graveyard are different too. While obviously dinner can be at either lunchtime or teatime.