Statisticians / medics please help me understand!

[Gladysthesphinx]

I’m trying to understand the various articles I’ve read about the reliability of PCR testing for coronavirus & am really struggling (not a scientist: no maths for 40 years). Can anyone help?

My understanding is that the PCR test has been assessed in the lab and has been rated for sensitivity (how many actual cases it picks up) and specificity (how well it avoids false positives). But this isn’t the end of the story - even if you know a test’s sensitivity and specificity, you need to know the prevalence of the relevant condition in the group you’re testing, in order to attribute a predicative value to an individual test result.

I’ve tried to work through some examples, on the basis of a test which has sensitivity of 80 percent and specificity of 95 per cent, and a test group of 10,000. (I don’t know if these are the PCR test ratings - I wanted to create an example to help me understand).

On the basis a 10 percent prevalence of the tested -for condition in the test group (ie 1000 people out of 10 000 actually have the condition), I got a 64 percent predictive value for a positive test, and a 97 percent predictive value for a negative test.

But when I tried the same sensitivity & specificity ratings on the assumption of a 1 percent prevalence of the tested -for condition in the test group (ie only 100 out of 10 000 people actually have it), I got a predictive value for a positive test of only 14 percent (ow!), and of 99.7 percent for a negative test.

This has left me really baffled. Is it the case that to know the predictive value of an individual test you need to know the prevalence of the tested for condition in the tested population? But if so surely the test cannot be used to show prevalence- you can’t infer prevalence from test results if you need to know the prevalence in order to get the test results! And in the case of Covid prevalence is what we actually want to know isn’t it?

Where I’ve got to I suppose is: is the PCR test actually pretty useless for ascertaining prevalence? Is it something that works as a diagnostic tool (for instance where someone is ill & you need to know how to treat them) but not as a population screening tool?

Baffled.

The only thing you are missing is the fact that the people being tested are not a random sample. They mainly either have symptoms, are close contacts of someone who has tested positive or are from door to door testing in high incidence neighbourhoods. This skews the sample to make the FPR lower than in random testing. The problem seems to be that either no one knows how many FPS there are or if government know they aren’t telling us. They could more or less eradicate them by testing positive samples again and getting people to isolate until the second test result came back.

This is a problem with the testing and one we don't seem to be talking about. Normally, when developing a new diagnostic test, you measure it's performance against a 'gold standard' test which you know tells you for sure if the tested person has the disease in question. Everyone in the sample gets the gold standard test and this is used to determine the prevalence.

Thanks both! This is where I was getting to overnight.

Say you’re testing for cancer. After the initial screening test, you do biopsies. A biopsy shows conclusively whether someone actually has the disease. Over the years, you use this information to estimate prevalence in society; and this estimate allows you to attribute a predicative value to a positive test. And this will also feed into your assessment of whether particular symptoms, for instance a lump, are in fact a sign that an individual’s positive test is of a higher predicative value than would be the case without the symptom - a big benefit.

Because we aren’t in this position with Covid (the biopsy equivalent) it is difficult to attribute a predicative value to a single positive test; and to determine the overall false positive rate. This uncertainty would be lessened by doing repeated tests (every 24 hours or so) on those who test positive, and using the results of those repeated tests (not just one test) to estimate prevalence.

Is this one of the reasons schools returning causes a spike? In my son’s class I know of 3 families who have had tests since school restarted: not because they had covid symptoms, but because they had cold symptoms (sneezing runny nose etc) and felt they couldn’t send children to school unless they had had a test. Is this a big part of the problem- increasing numbers of disease free people testing, for social reasons, and thereby altering the predicative value and FPR?

Does the gov not tell us the number of FPS because without repeat testing or another fairly conclusive test like hospitalisation or death it has no idea? That’s frightening.