Lies, damn lies, misrepresentation, misinterpretation and STATISTICS

(88 Posts)

A thread for anyone who is infuriated by and/or enjoys reading about this type of stuff. I love it, but annoyingly at the moment I can't think of any examples, other than dull work-related ones. For example, if you're looking at something as a proportion of the whole, then you can't consider one thing in isolation. For example, as Trills mentioned on another thread, let's assume heart disease is the biggest killer of adults (which I believe it is). Let's assume 10% of deaths to adults in 1950 were of heart disease, compared to 42% now. Shocking rise? Probably not.

Plus I will attempt to explain the Monty Hall (think that's the name) problem to anyone who is interested and who doesn't already know it.

the deaths to heart disease percentages were made up btw (a "damn lie") grin

I can't be the only one who enjoys this stuff
I estimate 13.2% of the rest of you do

chocoluvva Wed 29-May-13 23:08:21

Ooh yes please to the 'Monty Hall problem'.

(I always shout at the tv when stats for the effectiveness of grooming products are given - eg, 87% of women agreed that their skin was smoother after using 'Magocreme' for two weeks. "Smoother than what?" I shout, "Smoother than when you put nothing at all on your skin - therefore 13% of women find that Magocreme either makes no difference or makes their skin rougher".

And how can anyone's hair be 3 times shinier?

Also, "up to 100% x/y/whatever" means nothing does it? The difference will be between nothing and twice what it was before. Pah. Just Pah.

As for heart disease I'm pretty sure that if you add up the people who reportedly die of that and those that die of smoking, overeating and of not exercising then it comes to many more than the total number who died. So unless some people died twice they really mean "contributed to"

"8 out of 10 owners said their cat prefered it" prefered it to what?

oh yes very good point about the effects being compared to doing nothing.
I heard on the radio today that you're more likely to die in hospital if you have surgery near the end of the week (I think, didn't really tune in till half way through). Flippantly, near the end, he said "and this figure is up to 86% on a Friday". I assume that is 86% more likely to die, rather than an 86% chance of death when you go in to have that ingrowing toenail removed!

I assume you're aware of the problem? Gameshow - three doors. Behind one door a flash new car. Behind other two doors, goats (and not even the posh Oxfam ones!). Pick a door, 1, 2 or 3. So you have a 1 in 3 chance of picking the car. A 2 in 3 chance of picking a goat. You pick door 1. OK...now I, the omniscient gamesmaster am going to open a door - here we go, door 2. A goat. You now get the option to change your selection (so from door 1 to door 3, as this is the only one left). Should you swap?

And the answer is yes, as by swapping you increase your odds by 100% wink

Not sure what you mean, Back. Deaths will be counted as those which have one of the "heart disease" codes in the cause field. The other deaths (obesity, smoking etc) will have whatever caused their death. So in the case of smoking, it may have been lung cancer.
Ah yes, I think I see. The way it works with alcohol admissions is that they assign a rough % of admissions for (eg heart disease) that will have been caused by alcohol. So it may be that research has shown that 40% of heart disease admissions are "caused by" alcohol. I suspect there will be something similar for obesity, smokng etc, and you're right, they will then be double counted in the separate reports (although they're actually reporting different things). Interesting, and something I should know.

"8 out of 10 owners said their cat prefered it" prefered it to what?
Preferred it to a kicking round the sofa and some dog food grin

TheDoctrineOfSnatch Wed 29-May-13 23:39:00

grin

The issue with the heart disease is that it kills older people - people who, in the past may have died of other causes, much earlier, such as infection or industrial accidents. It's not enough to know the proportion of people dying from X has gone up unless you know whether the rate is also going up and what the rest comprises, and what is happening with those.

That reminds me, when I used to ask what I was getting for Christmas I used to be told "a walk round the table" confused or " a sack of cinders". I didn't even know what a cinder was! (still not really sure)

Another one - I believe the numbers of children with complex disabilities is rising. This is because premature infants are being kept alive at younger ages because of improvements in healthcare and technology. While it doesn't seem like it at first glance this is a good thing - these are children who would have previously died.

87% of 74 women sampled would recommend it to a friend.

So 10 (9.something to slightly less than exact) women would tell their friends not to buy it, it's shit. Out of 74. confused

That reminds me, I was recently asked to complete a survey (read out, my answers spoken) in the doorway of my local co-op, within earshot of the tills (it wasn't busy). Luckily I had nothing but genuinely nice things to say about them but I can imagine if I'd wanted to rate them badly I'd have been a bit nervous next time I popped in for a pint of milk!

And I bet the options to select were:

Definitely
Probably
Maybe
Definitely not absolutely not, I'd rather stick pins in my eyes

With the top 3 counting as a "yes" - sneaky!

badblueeyeliner Wed 29-May-13 23:54:43

The skin cream and shampoo ads do my head in! How can they justify the claims based on so few people, who were probably paid to take part in the research?! 'Our advertising and marketing budget for this year is 10 million quid - let's survey 70 women!' hmm

Anyone know what would be a fair number of people to test on?

Great thread!

ClaireDeTamble Thu 30-May-13 08:40:22

Oooh, this bugs me too - especially the shampoo ads. My biggest annoyance though is people's lack of understanding of risk which the media usually play up no end e.g "Eating bacon increases the chance of dying from a horrible gruesome death by 25%", which of course means absolutely nothing without the baseline and even then doesn't really tell you anything unless you know your own personal risk factors.

There was a great article on the BBC website a couple of years ago which explained it really well - I'll see if I can find it.

SanityClause Thu 30-May-13 08:47:21

Well, the reason they don't do better research into cosmetics and hair products is because if they were actually to prove some benefit about them, the product may have to be classified as a drug, and they wouldn't be able to sell it in the same way.

DD1, who is 14, often tells me that 84% of statistics are made up. wink

People in general dont understand risk. You see it all the time, especially in the discussions about breastfeeding and formula. Formila feeding increases the risks of gastroenteritis. The fact your formula fed baby didnt get it doesnt negate that statement. Ditto if your breastfed baby did get it. (Not turning this into a breast/formuka debate btw!). Also dismissing research with "but I bet those mums were the middle class ones". Proper research should control for these factors but people always assume that if a randomised controlled trial hasnt been carried out, then the results are crap. Yet somehow no one doubts the ljnk between smoking and lung cancer. I wonder how they got ethics approval for that rct. hmm

Is that reallh the case sanity? If a cosmetic product has an effect it gets reclassified? How interesting.

ClaireDeTamble Thu 30-May-13 08:51:44

news.bbc.co.uk/1/hi/magazine/7937382.stm

A lot of his articles were very good and are available in the drop down list at the side.

Ooh claire thanks fo that. I should be working...must savethem to read tomorrow

TheDoctrineOfSnatch Thu 30-May-13 09:04:31

Stealth, can I ask for this to be moved to Science and Nature Club? Statistics being 79% more important in reaching scientific conclusions than anything else, after all grin.

I don't know, shall we survey a sample of posters on the thread and see what they think? We'd need to weight it to give more weight to more recent posters.
(fine by me wink)

Indith Thu 30-May-13 09:11:03

It is amazing the amount of "proper" scientific research that gets written when the researchers don't appear to understand statistics and risk. And then you get the books that are absolutely based on actual scientific evidence and not at all biased towards the opinion of the author.

Take 3 in a bed for example which makes my blood boil every time the author merrily says something along the lines of "30% of SIDS cases happen in co-sleeping babies which means that 70% happen in cots so actually MORE babies die in cots so it is more dangerous." NO YOU MORON! IT MEANS NOTHING UNLESS YOU ARE LOOKING AT IT AS A PERCENTAGE OF THE NUMBER OF BABIES WHO USUALLY CO SLEPT OR WHO USUALLY SLEPT IN A COT! And that i before you even get around to bf vs ff, usual place of sleep vs place of sleep whenthe death occured and all the rest of the variables. The Politics of breastfeeding is guilty of similar offences too.

TheDoctrineOfSnatch Thu 30-May-13 09:15:01

Exactly, Indith, exactly brew

Indith Thu 30-May-13 09:15:26

Oh and understanding the difference between a factor and a cause. Walking into the road without looking was not the cause of my death, it was a factor. The cause was the car that hit me.

StrawColoured Thu 30-May-13 09:15:33

Fascinating thread!

Advertising with statistics is so persuasive though. There's some body lotion or something that claims to "lock in ten times more moisture" - but doesn't explain ten times more than what. Smearing lard on your skin would "lock in" about a hundred times more moisture anyway. And since when is "moisture" so good for the skin? Humans aren't dolphins, they won't die if their skin is dry.

It's all a load of boswellox

TheDoctrineOfSnatch Thu 30-May-13 09:20:05

Is a cinder what's left over when you burn coal etc, but it's a bigger piece than ash?

Hence "cinders and ashes" from Thomas the Tank Engine.

TheDoctrineOfSnatch Thu 30-May-13 09:23:41

There was a long thread about cycling recently where one poster believed that, since the absolute number of cyclists killed in the UK and the Netherlands was the same, UK drivers must be more tolerant of cyclists as in the Netherlands people were more often on cycle paths than the road.

Factor in the regular cycling population in each country? Apparently that was irrelevant....

bigkidsdidit Thu 30-May-13 09:24:40

Reading with interest

I'm a scientist but my stats knowledge is shameful (in my defence i'm a biologist grin )

I don't understand the game show, can you explain?

I do think when discussing death / illness, newspapers should use actual numbers as well as incresaed risk. I was listening to Today today and there was a piece on long term use of painkillers increasing death rates. They did actually use real numbers and said the risk of death increased from (I think) 2 oer 1000 to 3 per 1000. If you have arthritis and were told risk of death increases 50% you might panic, but 2 to 3 per 1000 would probably be worth it to be pain free.

Indith Thu 30-May-13 09:27:14

Yup. All a pile of shite.

I find it sad that I did a presentation on infant sleep including a nice lengthy section on bed sharing on the day that the newspapers published that research done that said co sleeping was incredibly dangerous. My presentation linked all the research done by Helen Ball and mentioned the most recent suggestion that once other risk factors are removed co sleeping is possibly safer than cot sleeping. The research that anyone can access through the links on ISIS is all very well conducted with groups selected and matched and questioned specifically for the purpose of that study. Then we have this research which makes it into the papers and scares everyone silly nad could potentially have a huge impact on women and their babies because nobody wants to do something that will make their babies a huge 5 times more likely to die. Yet this research was done as a retrospective looking at other studies and using their information so there is no control over methodology and they are relying on information gathered while researchers were looking at other things. But this is what the papers have put out there as gospel.

If you stick with your original door, your chance of getting it right is the same as it was originally - one in three. If you swap, your chance of getting it right is the same as it was before he opened the door - two In three. It seems as though it should be 50/50 once a door is open but the key is that the gamesmaster knew what was behind each dpoor

Indith Thu 30-May-13 09:31:09

YEs bigkids, real numbers are needed. If we don't know the baseline then anything like an increase of 67% or a rise by a factor of 5 is meaningless.

TheDoctrineOfSnatch Thu 30-May-13 09:35:06

Contemplates drawing a probability decision tree in ASCII, decides not to...

At the time of your first choice, you had a one in three chance of being right.

Once one choice is eliminated, by changing your choice you have a two in three chance of being right.

It's a bit mind-bending. But it works because the door opener knows the answer. So if you picked the car door, the door opener can open either of the other doors, but if you picked a goat door, the opener has to open the other goat door. As you were more likely to pick a goat door first time, it's more likely that the unopened and unpicked door is the car door.

en.m.wikipedia.org/wiki/Monty_Hall_problem

daftdame Thu 30-May-13 09:35:16

the big thing about o-cleeping is that they lump in people who fell asleep with their babies on a sofa (which I've done but far from ideal). If they separated out intent then I bet the findings would be very different.

ooh thatnks daftdame that's one I haven't read.
Just finished Naked Statistics, that was good.

bigkidsdidit Thu 30-May-13 09:36:58

I get it! thanks smile

Snatch, I have tried to describe it using a deck of cards, where the idea is to pick the ace of spades. I think it's a lot clearer when the gamesmaster turns over 50 cards.

Indith Thu 30-May-13 09:45:11

not on all studies they don't, tis a bit of a myth I think that they all lump them in together. But usual place of sleep makes a difference. For example a child who normally sleeps in a cot but who comes into bed with parents one night for some reason is at a greater risk thana child who bed shares all the time.

Anyway, I suppose I should start heading towards class. I had ideas about being productive this morning but apparently not. Maybe they need to ban MN from the uni laptops!

Soery good point. I meant on the ones id seen there was never any mention of where the child started the night or, as you say, where they usually slept.
Have a good day!

Just read the sausage one. How can someone "invent jogging"? Thats liks saying someone invented breathing or sneezing.

Just bumping

:'(

Witt Thu 30-May-13 11:16:38

Please can you explain the Monty Hall problem to my MIL. My DH and I have tried several times and we hit the same problem - she wants to win a goat!

:-)

TheDoctrineOfSnatch Thu 30-May-13 11:23:20

Well, that's ok - she can maximise her chances of winning a goat by not swapping her guess!

Or win a car, sell it and buy as many goats as she wants

FREE THE GOAT 2

Witt Thu 30-May-13 11:58:04

We try to suggest she wins the car then sells it for goats but she isn't happy with that suggestion.

We also once tried to explain the infinite hotel problem where an infinite number of guests arrive wanting rooms so you ask everyone to move to the room that is double the number of their current room eg 1->2, 2->4, 3->6 etc. Oh the regret in starting that conversation.

Well thsn there is no reasoning with her

WorrySighWorrySigh Thu 30-May-13 13:09:03

aaahhhh! <contented sigh>

I am home!

KateSMumsnet (MNHQ) Thu 30-May-13 13:12:05

Our survey suggests that Science and Nature is 78% more thready....so we're going to move this thread there.

piprabbit Thu 30-May-13 13:12:43

Radio 4 have a brilliant programme called More or Less for people fascinated by this sort of thing.

Thank you pip, I've had a quick look and it looks really interesting. I might print some off to read tomorrow (I know...I am a luddite)

chocoluvva Thu 30-May-13 14:11:35

grin at "more thready"

bumping

TheDoctrineOfSnatch Thu 30-May-13 15:40:31

Thanks Kate.

I lurve More or Less!

StrawColoured Thu 30-May-13 16:40:50

I lurve More or Less as well.

There's lots of food-related misleading stats as well. I remember all the claims like "94% fat-free" on so-called diet foods.

IMHO breakfast cereals owe their entire reputation as a "healthy" food to their claims in years past of being fortified with vitamins, and this providing "50% of your RDA of Vitamin C" etc etc.

There's an advert for follow-on milk that compares itself to cows' milk, saying it has 20 times the amount of iron than cows' milk (or something like that). What it doesn't say is that if your child was iron-deficient, they'd do far better eating other iron-rich foods.

bran Thu 30-May-13 18:59:10

Message withdrawn at poster's request.

bran Thu 30-May-13 19:04:29

Message withdrawn at poster's request.

TheDoctrineOfAllan Fri 07-Jun-13 16:22:11

It's like the '8 out of 10 coffee lovers prefer Costa to Starbucks' poster has a definition of coffee lovers somewhere, think it was to do with number of cups per week...

just fancied bumping this...

EndoplasmicReticulum Thu 20-Jun-13 20:47:26
TheDoctrineOfAllan Thu 20-Jun-13 21:22:17

Thanks!

bumping this for any more thoughts or examples

kim147 Fri 28-Jun-13 20:39:04

More or Less is a great programme.

"20 mph zones are dangerous"

Want to guess the maths behind the headline?

TheDoctrineOfAllan Fri 28-Jun-13 20:41:56

Um, because they are near schools or other places where more accidents would happen regardless of speed limit?

kim147 Fri 28-Jun-13 21:01:27

The number of injuries went up in 20mph zones year by year because.....they built a lot more 20mph zones.

Now if you looked at the injuries per mile, that would be a very different story.

The number of deaths went up by 70% (!!!) (from 4 to 7)

lol thanks for keeping this going!

EndoplasmicReticulum Fri 28-Jun-13 22:47:17

I saw a lecture by "Professor Risk" the other day.

understandinguncertainty.org/

I liked the way he looked at things. Especially the idea that if you go for a half-hour run it extends your life by.....half an hour. Great if you like running.

Aaaaaaaaaaaaaaaaaaaaaaaaaargh! Is it really that hard

"reduced risk" does not = "no risk"
"heightened risk" does not = "certainty"

these are all common words, we use them in the English language all the time. So why do so many people struggle so badly with this concept?

Trills Mon 08-Jul-13 20:01:43

Hello.

Trills Mon 08-Jul-13 20:02:19

You mentioned me in the OP and didn't PM me to tell me this was here!

I had to find out about it on a lottery thread!

Trills Mon 08-Jul-13 20:04:52

87% of women who were given this product for free said it was "quite nice"

TheDoctrineOfAllan Mon 08-Jul-13 20:36:28

Was it ham, Trills?

rubyanddiamond Mon 08-Jul-13 21:23:38

David Spiegelhalter did an episode of The Life Scientific recently - really interesting if you like this kind of thing:

www.bbc.co.uk/programmes/b02x7h0z

Debsndan Mon 08-Jul-13 23:31:40

What a great thread. My only contribution is when I was having ivf, I got very interested in the individual success rates of each clinic, including one that actually had a detrimental effect on its patients' fertility. I posted about it on a fertility forum and a woman popped up to say I'd got it all wrong and we all had the same chance of getting pregnant as "it either works or it doesn't, so that's 50/50."
I retreated as my brain had started to hurt a lot and clearly there was no point trying to explain...

TheDoctrineOfAllan Tue 09-Jul-13 10:26:09

It should be mandatory for any comment on risk to be balanced by any comments on counter risks.

For example: certain types of SSRIs may increase the risk of foetal heart defects from 2 in 1000 to 4 in 1000 (not exact numbers). However, women of childbearing age who stop taking SSRIs for mild to moderate depression without having alternative treatment arranged suffer an x% increased risk of self harm, worsening depression or whatever.

TheDoctrineOfAllan Tue 09-Jul-13 10:27:55

That was an illustrative post but my point was that changing one risk often means changing another, but only one change gets reported.

Azultrailer Mon 26-Aug-13 01:17:19

I love this thread.

meditrina Tue 27-May-14 09:17:34

Old thread, but I thought that anyone who spots it might also like this BBC story on spurious correlations

BoomBoomsCousin Mon 07-Jul-14 02:48:37

Another good BBC article this time on statistics and risk in medicine.

TheDoctrineOfSnatch Mon 14-Jul-14 22:30:49

Thanks!

TheDoctrineOfSnatch Mon 14-Jul-14 22:37:47

"Perhaps the most notorious example of patients being misled about risk occurred in October 1995, when the UK's Committee on Safety of Medicines warned doctors that a new, third-generation oral contraceptive pill doubled the risk of thrombosis. Thousands of women came off the pill, even though the risk had merely increased from a one-in-7,000 chance of getting the disease to a two-in-7,000 chance. The following year saw an additional 13,000 abortions in the UK."

Now that might not be direct cause and effect but is close to what I was trying to say above - reducing one risk might often increase another.

Trills Mon 14-Jul-14 22:44:32

Definitely ham.

Join the discussion

Join the discussion

Registering is free, easy, and means you can join in the discussion, get discounts, win prizes and lots more.

Register now