Basic science and statistical understanding

[MrsTerryPratchett]

I'm not a scientist. I do have a BSc. but nothing more impressive. I do wince when I hear some assumptions that people make who don't have basic science (and statistical) knowledge. What basic scientific or statistical principles do you wish the general public (and members thereof) knew about?

I'll start. Causation and correlation. A correlation doesn't prove causation. Particularly when that correlation is a correlation of one. "I had the flu vaccine and got the flu really badly" isn't causation.

28 day mortality is a pretty standard way to count outcomes in groups of people. Individual cause of death can be identified by individual records but when tracking population outcomes, 28 day or 60 day mortality makes more sense.

What basic scientific or statistical principles do you wish the general public (and members thereof) knew about?

That there is no such thing as zero risk decisions, and that we can live with a lot of risk fairly comfortably.

Even more basically; mean, median and mode can be wildly different. You can pick the 'average' accordingly based on your own interests.

This

I think what I’ve found most frustrating and worrying is the “little knowledge is a dangerous thing” factor. People who have learned something which sounds a bit clever and perhaps counterintuitive spouting it off when it doesn’t apply.

The best example I can think of wasn’t a Covid thread - it was a thread “Surely it’s not correct that the average income is 30 thousand pounds?”. The OP linked to an article with this fact but didn’t believe it.

Predictably there were a fair number of posters saying “no that figure’s definitely too high/low because I earn 16k/100k and so do all my mates”.

But more depressing were all the people lining up proudly to explain at great length that mean income measures are actually skewed and unrepresentative because one billionaire can outweigh a hundred thousand dinner ladies and therefore the number was bollocks. None of them had bothered to check or click the link and find out that the figure being used was in fact the median so none of their painstaking and patronising explanations were remotely relevant.

It’s like that with Covid - people spouting off interesting stuff they’ve heard which is not applicable to the situation at hand.

“False positives in a mass testing regime can lead to strange outcomes where most positive tests are in fact false”
Yes, sometimes, but not in the circumstances we’re looking at with positivity levels several multiples of the maximum possible false positive rate.

“An apparent increase in cases can in fact just be an artefact of increased testing”
Yes, sometimes (eg when comparing October 2020 with March 2020), but not when you’re seeing an increase in positivity levels.

“The PHE method of recording Covid deaths meant that all deaths after a positive test were being recorded as Covid deaths for ever regardless of actual cause”
Yes that was a fact and a major flaw if they had continued using it which is why they changed it. But no it doesn’t affect the overwhelming bulk of deaths in April and May to any significant extent (because there were so few March/April tests and so little time had elapsed), and nor does it affect ONS death records.

Oh and just in case you were wondering - 30K is the median full time employed salary which is why it might seem a bit high.

That having a PhD, or even a Nobel prize in something other than virology, immunology or epidemiology doesn't make your opinion on covid relevant. That having a PhD or Nobel prize in a relevant field doesn't stop you from being a complete nutjob. The number of covid deniers I've seen using what a Nobel physicist has said to back up their argument is unreal. Then there's the whole "well the inventor of PCR (who had a Nobel prize) said you can't use PCR to detect a virus" thing. Yes, he probably did say that but the man was a complete fruitcake who spent the end of his career trying to get everyone to agree that HIV doesn't cause AIDS.

That "x% of people who have COVID get a false negative test result" is not at all the same as "x% of all negative swab test results are false". I saw someone on here claim that 30% of test results were false negatives - if infection rates were that high we'd have reached herd immunity very quickly! It's an easy thing to misunderstand but at some point you surely also have to apply some common sense.

Not so much statistics as biology, but that when scientists say "we don't know how long immunity to covid lasts" they don't mean you can catch it again two days after getting over the first round. I think most people have a better idea of the hypothesised timescales now, but the media (and the WHO) really fucked up the messaging on that point at the beginning, and caused a lot of people to think that you had no immunity at all immediately after catching it.

"Absence of evidence is not evidence of absence".

Also this.

This thread is both cathartic and educational - thank you to all the excellent and skilled posters!

That there is no such thing as zero risk decisions, and that we can live with a lot of risk fairly comfortably.

And anything with effects has side effects. Just because your aunt's friend's cousin's mate had vaccine damage, that doesn't mean the risk of vaccine damage outweighs the risk of polio, yellow fever or measles. Saying that vaccines are 'safe' is comparative, not absolute. Just like everything else in this life!

@MrsTerryPratchett

Maybe we should start a list of terms that people bandy aroud:

PCR
RNA
Exponential
R0

Any I've missed?

'Source?'... As an entire post. Usually posted by someone getting into a bunfight having stated something to be fact when the very best that could be argued is that their view may have some evidential basis. How I wish that people would understand 1. that science is not some kind of monolith, but instead has many different disciplines; and 2. that just because something is 'science' - as opposed to the arts say - does not mean that there are always, or even often, black and white answers. It riles me because they wouldn't actually want any 'sources', probably wouldn't be able to understand them, and - in the context of Covid especially, being both new and emotive - it wouldn't take them any further because the person picking said sources would of course have picked them as representative of a body of evidence with which they agree, which will not be the body of evidence (or fb post) with which the 'source-demander (!)' agrees. Ah, that was cathartic again!

'Source?'... As an entire post.

I've definitely done that.

I have to be careful not to type evidence or STFU here because it’s standard on another forum.

Tbf I’m also bad at providing the links to stuff here.

😂Ah, sorry! Well all crimes now forgiven for having started such a nerd-tastic thread - and one which hasn't yet descended into emotive science-illiteracy😂

I enjoyed this thread - so yes I am a nerd.

What I hate most is misuse of data and statistics to make a headline or to justify a particular cause. Earlier in the thread someone mentioned the "doubles the chance of getting the lurgy" problem, but here are a couple of annoying examples (not using "real" data).

eg. A month's worth of rain fell in 3 days. It sounds disastrous - get the sandbags out - but on average, if it only rains for 6 days in that month, then it is just heavy rain. A year's worth of rain in one day in the Atacama is just light drizzle.

eg. In London during the first wave, 40% of coronavirus deaths were from BAME groups whereas only 16% of England's population is BAME. Very worrying, until you realise that 40% of London's population is BAME.

Any other nerds wanting to learn a couple of interesting new topics should look up "Simpson's Paradox" and "Bayes' Theroem".

@Hardbackwriter

Oh, also: almost no one really gets logarithmic scales, to the extent that I'm not sure they should be used in anything aimed at anyone other than specialists. Maybe not even then - I heard David Spiegelhalter talking about this on More or Less and he said that there was some evidence that even public health professionals often draw incorrect inferences from graphs using logarithmic scales.

Like the op, I have a BSc in a science so not a scientist but some idea at least . I also work with data and do quite a bit of producing charts and stats to different audiences analysis (though I am definitely not a statistician ). Despite having what I consider to be a better than average understanding of charts and trend data. I don't understand logarithmic scales on charts at all so would agree they should only be used for an expert audience

QueenStromba totally agree with this - frankly even if you do have a PhD in Epidemiology it doesn’t mean you know more about the actual situation on the ground than Chris Whitty.

I have a PhD in Epidemiology. Epidemiology of non-communicable diseases, eg heart disease, diabetes. I covered some infectious diseases stuff in my MSc, but it definitely does not mean I have any insider knowledge on Coronavirus biology, the nuts and bolts of our testing programme, or vaccine production.

The number of people who say “my mate’s a nurse/GP/surgeon/somebody else with no involvement with public health or virology, and they say X” is unreal.

Re: Died of v died with.

Its actually irrelevant.

For example it could be that if someone is in hospital with something else at the time of a peak in cases and they then get covid, they are going to get poorer care simply because the staff are over stretched and on their knees and staff having to have so much PPE has a detrimental effect too. Whether or not they actually died from covid therefore becomes irrelevant but it certainly could well be a contributary indirect factor.

What we should be doing over the next couple of years is the following:

1. Looking at a control group pre-covid and what deaths were like within that group for certain health related conditions like diabetes / or even the population as a whole
2. Then looking at a group who got covid and what happened to them.
3. Having a third group post-covid who were similar to group 2 but didn't get covid.

This isn't something we can do immediately, but something we can do long term. You look at all-cause mortality not just at 'deaths from covid'/'deaths with covid'.

What we will probably see is:

1. A higher life expectacy in group 1
2. A much reduced life expectacy in group 2
3. A life expectacy which is lower than group 1 (but possibly higher than group 2).

Why?

Because we know that covid is almost certainly causing deaths directly and indirectly and this is important to understand and acknowledge. Covid isn't deadly purely because of the disease but also because of the pressure it puts on the health care system elsewhere - all the cancelled regular appointments for other conditions and general monitoring of underlying health.

There isn't a lot of point in being classed as having 'survived the pandemic' because you've not caught covid, but you die during it of unrelated causes either during the pandemic or shortly after it, if had you been 5 years older, you could have expected 20 years more of life expectancy.

After we have a vaccine and the healthcare system is back to functioning as normal and we have caught up with a backlog of treating other health conditions THEN we could look at how deadly covid is for unvaccined people in a new light. THEN we can look at the case fatality rate of covid itself in an unbiased way. Once you have separated out the outside influences.

I've seen an awful lot of people already trying to look at the case fatality rate on MN and its massively premature and which frankly utterly pointless at this stage.

A pandemic is unique because its NOT a normal set of circumstances. Its significant not just because its a new disease which the population lacks immunity to, but because of how it overwhelms the system.

We have had a lot of people saying that all these people dying 'would have died anyway' which is clearly nonsense because we have a spike in 'excess deaths'.

SAGE have pointed this out repeatedly but I don't think this message has really been understood.

The problem with covid, isn't and never has been the death rate. Its how much it affects the overall health system and care of other conditions. Its how high the hospitalisation rate is - even if patients ultimately survive. Thats why bed issues and hospital capacity not deaths from / with covid are the thing people should be tracking obsessively. Except they aren't because they've missed the really important detail and don't understand all-cause mortality properly.

Once you are dead it doesn't matter whats on your death certificate. What you died of is ultimately irrelevant. What matters is whether your life was shortened compared to what you could have normally / reasonably expected.

And thats why all-cause mortality is far more important to this crisis than covid itself.

The bottom line is that when we look at treatments for health of all kinds we have to be conscious of whether we are actually increasing life expectancy or people have a point where their body gives up and they die of whatever. This isn't restricted to covid. We see it elsewhere in healthcare. Its a problem that is well documented as a blind spot in research as well as how we promote health care for some time. Its not something that is this crisis is likely to shine too much of a light on either unfortunately.

All-cause mortality generally is poorly understood, and we have something of an obsession with forgetting this in efforts to reduce deaths from a particular cause. For example there is a bias towards increasing screening for various other conditions without thinking about how treatment might not increase your life expectancy (and also decrease your quality of life) or how it might inadvertantly actually shorten it. The focus becomes about simply eliminating that cause of death from death certificates and the statistics because we have been conditioned to only see certain causes of death as problematic because they get the most PR and promotion. (Covid actually highlights this is a way: we are concerned about reducing deaths from cancer because we have lots of charities making a point about this, but dementia and deaths from flu are pretty overlooked areas of health care and almost seen as inevitable. This has a lot to do with a lack of publicity and awareness).

Unless you look at the bigger picture for health instead of focusing on one particular angle you could miss the real point/problem.

@RedToothBrush

Re: Died of v died with.

Its actually irrelevant.

For example it could be that if someone is in hospital with something else at the time of a peak in cases and they then get covid, they are going to get poorer care simply because the staff are over stretched and on their knees and staff having to have so much PPE has a detrimental effect too. Whether or not they actually died from covid therefore becomes irrelevant but it certainly could well be a contributary indirect factor.

What we should be doing over the next couple of years is the following:

1. Looking at a control group pre-covid and what deaths were like within that group for certain health related conditions like diabetes / or even the population as a whole
2. Then looking at a group who got covid and what happened to them.
3. Having a third group post-covid who were similar to group 2 but didn't get covid.

This isn't something we can do immediately, but something we can do long term. You look at all-cause mortality not just at 'deaths from covid'/'deaths with covid'.

What we will probably see is:

1. A higher life expectacy in group 1
2. A much reduced life expectacy in group 2
3. A life expectacy which is lower than group 1 (but possibly higher than group 2).

Why?

Because we know that covid is almost certainly causing deaths directly and indirectly and this is important to understand and acknowledge. Covid isn't deadly purely because of the disease but also because of the pressure it puts on the health care system elsewhere - all the cancelled regular appointments for other conditions and general monitoring of underlying health.

There isn't a lot of point in being classed as having 'survived the pandemic' because you've not caught covid, but you die during it of unrelated causes either during the pandemic or shortly after it, if had you been 5 years older, you could have expected 20 years more of life expectancy.

After we have a vaccine and the healthcare system is back to functioning as normal and we have caught up with a backlog of treating other health conditions THEN we could look at how deadly covid is for unvaccined people in a new light. THEN we can look at the case fatality rate of covid itself in an unbiased way. Once you have separated out the outside influences.

I've seen an awful lot of people already trying to look at the case fatality rate on MN and its massively premature and which frankly utterly pointless at this stage.

A pandemic is unique because its NOT a normal set of circumstances. Its significant not just because its a new disease which the population lacks immunity to, but because of how it overwhelms the system.

We have had a lot of people saying that all these people dying 'would have died anyway' which is clearly nonsense because we have a spike in 'excess deaths'.

SAGE have pointed this out repeatedly but I don't think this message has really been understood.

The problem with covid, isn't and never has been the death rate. Its how much it affects the overall health system and care of other conditions. Its how high the hospitalisation rate is - even if patients ultimately survive. Thats why bed issues and hospital capacity not deaths from / with covid are the thing people should be tracking obsessively. Except they aren't because they've missed the really important detail and don't understand all-cause mortality properly.

Once you are dead it doesn't matter whats on your death certificate. What you died of is ultimately irrelevant. What matters is whether your life was shortened compared to what you could have normally / reasonably expected.

And thats why all-cause mortality is far more important to this crisis than covid itself.

The bottom line is that when we look at treatments for health of all kinds we have to be conscious of whether we are actually increasing life expectancy or people have a point where their body gives up and they die of whatever. This isn't restricted to covid. We see it elsewhere in healthcare. Its a problem that is well documented as a blind spot in research as well as how we promote health care for some time. Its not something that is this crisis is likely to shine too much of a light on either unfortunately.

All-cause mortality generally is poorly understood, and we have something of an obsession with forgetting this in efforts to reduce deaths from a particular cause. For example there is a bias towards increasing screening for various other conditions without thinking about how treatment might not increase your life expectancy (and also decrease your quality of life) or how it might inadvertantly actually shorten it. The focus becomes about simply eliminating that cause of death from death certificates and the statistics because we have been conditioned to only see certain causes of death as problematic because they get the most PR and promotion. (Covid actually highlights this is a way: we are concerned about reducing deaths from cancer because we have lots of charities making a point about this, but dementia and deaths from flu are pretty overlooked areas of health care and almost seen as inevitable. This has a lot to do with a lack of publicity and awareness).

Unless you look at the bigger picture for health instead of focusing on one particular angle you could miss the real point/problem.

I disagree that died of vs died with is irrelevant. Or, at least, I think that's too simplistic a way to put it.

It is absolutely relevant to understand whether something is actually killing people or not. Otherwise, we open up the possibility of a stupid situation where we lockdown for the common cold because so many people die each year with a cold. If the cold isn't killing them then focusing on the cold as a public health emergency is just plain stupid.

I agree with what you're saying about the system being overwhelmed and I also agree about looking at the big picture but I would argue that looking at the big picture includes looking at illnesses and deaths - or the years of life expectancy you're taking away from people - that are caused by measures to contain covid. If you're preventing something that isn't killing people (or is killing an unknown number of people) and in doing so you're killing other people then that is obviously not a sensible solution. If you're fixated on reducing the number of people dying with covid, because that's all anyone cares about, without understanding whether covid is the cause of those deaths, while at the same time killing other people via mental health issues, undiagnosed cancer etc, then you're essentially causing death rather than preventing it.

Oh, another one.

Every year when Stand Up To Cancer is broadcast we get threads
“SUTC said that one in two of us will get cancer in our lifetime! Surely that figure used to be one in three! What terrible poison in our lifestyle is causing this terrifying escalating epidemic?”

What’s happening of course is that we’ve made massive strides in stopping people dying before their time from heart disease (or road or industrial accidents, but mostly heart disease).

Interesting point re the SUTC. Covid has simply shone a light on the lack of understanding of data and statistics and the problems of media reporting of science, which were always there.
The difference is that the lack of understanding is now writ large because everyone feels they have a little knowledge and want to share it very vocally and without nuance.

This isn't something we can do immediately, but something we can do long term.

Everyone wants to know everything NOW. I keep saying we won't know who 'won' in terms of strategy for about 20 years. Education, mental health, overall health. This takes a while to play out.

“SUTC said that one in two of us will get cancer in our lifetime! Surely that figure used to be one in three! What terrible poison in our lifestyle is causing this terrifying escalating epidemic?”

And, da-da-dum everyone dies of something. What we all care about is dying young or painfully of something. Yes, we all want to slip away painlessly in our sleep at 100 (or something like that) but the brutal fact is that something will get you and it probably won't be nice. We want to put it off!

Off the topic a bit but the SUTC comments remind me of a conversation I had with a relative recently. He is adamant that asthma doesn't exist and thinks it is "a fad". His evidence is that "Not a single kid had one of those bloody blue inhalers when I was at school, let me tell you".
I can't help suspecting that the fact that he left school several years before salbutamol was commercially available may have something to do with this.

@MrsAvocet

Off the topic a bit but the SUTC comments remind me of a conversation I had with a relative recently. He is adamant that asthma doesn't exist and thinks it is "a fad". His evidence is that "Not a single kid had one of those bloody blue inhalers when I was at school, let me tell you". I can't help suspecting that the fact that he left school several years before salbutamol was commercially available may have something to do with this.

😂. This is a great example of why statistics and the reporting of them - and subsequent SM discussion without any basis of understanding - can be so unhelpful. Funny though!

This thread is brilliant because I have simultaneously felt myself getting more intelligent, while also realising that I'm not intelligent at all.

@thenewaveragebear1983

This thread is brilliant because I have simultaneously felt myself getting more intelligent, while also realising that I'm not intelligent at all.

It is my firm belief that this is the point of education. To leave us in no doubt where our ignorance and intelligence lie so we can work on one while valuing the other.

Knowing what we don't know is vital.

Exponential growth.

The fact that everyday risk assessment always involves assessing consequences, not just the probability of something happening. Two people might agree completely that the objective risk of death from X is only 1 in 1000, but have quite different subjective opinions about how acceptable that risk is (based on the consequences for their particular family, say), and that's OK. They can both be right.

Also, the hospitalisations vs deaths issue highlighted by RTB.