COVID-19: Testing, Testing, Testing

When it comes to COVID-19, it seems that testing is discussed more than any other issue. Testing is important for a number of reasons:

• Testing allows us to identify those who are infected and for them to isolate themselves to reduce further spread of the virus
• Testing permits the identification of individuals who could have been exposed to COVID-19
• Testing is a valuable indicator of local and national infection rates, which are then used to determine appropriate health interventions.

 

In fact, testing may be the only topic the leader of the WHO and Patrick, an animated starfish, agree on.

However, despite the centrality of testing, there is significant confusion and misinformation about what types of tests are available, what those tests can and cannot tell us, and how accurate they are. We are going to shed light on some of these nuances today.

 

Types of COVID-19 Tests

There are two main types of tests currently available for COVID-19: viral detection tests and antibody detection tests.

• A viral test seeks to identify the presence of the virus in the body to determine if you are currently infected
• An antibody (or serology) test will tell you if you were previously infected by testing for the presence of antibodies in the body, which can take 1-3 weeks to develop following infection.

 

Both viral detection and antibody serology assays can serve as important inputs for clinical and public health decision making on how to respond to infection in an individual or a population. This blog will focus on viral detection tests, and next week will address antibody tests.

Interpreting a COVID-19 test result depends on two main factors: how accurate the test is (with respect to a gold standard, which we don’t truly have for COVID… but that’s a question for another time), and the probability of having disease in the population before testing.

How accurate are COVID-19 viral detection tests?

So far, the most common and reliable viral detection tests for COVID-19 gather a sample with a nasopharyngeal swab (or, more recently, saliva) and then use RT-PCR (Reverse Transcription Polymerase Chain Reaction), to identify the presence of SARS-CoV-2, the virus that causes COVID-19, in your body, using fluorescent dyes. There has been a lot news and social media discussion of the accuracy of these viral detection tests.

How do we judge accuracy?

It’s important to remember that no test is 100% accurate. The accuracy of tests is usually assessed by looking at a test’s sensitivity and specificity.

• Sensitivity measures how often a test correctly generates a positive result for people who have the condition (“true positive” rate). If a test is very sensitive, it will identify almost everyone with the disease and not generate many false-negative results. (Example: a 90% sensitive test will give a correct result for 9/10 people with the disease, but return a false negative for 1/10 people with the disease.)
• Specificity measures a test’s ability to correctly generate a negative result for people without the disease (“true negative” rate). A high-specificity test will correctly rule out almost everyone who doesn’t have the disease and will not give many false-positive results. (Example: a 90% specific test will return a “no disease” result for 9/10 people without the disease, but generate a false positive for 1/10 people without the disease)

 

Most testing is not a binary “yes” or “no”, but measures an underlying value of a biological marker: a hormone, a chemical, or mineral in the blood (for example, vitamin deficiencies). Often, we can increase or decrease the sensitivity and specificity of a test by determining the “cutoff”, or “threshold” between what we call a positive and a negative test.

Imagine you are measuring the amount of hormones in iron in the blood to determine if someone is pregnant using a pregnancy test, which measures HCG – or the human chorionic gonadotropin hormone levels. Say that the distribution of HCG levels for women who are not pregnant (yellow curve) and who are pregnant (blue curve) are as shown in the figure below. If we use a cutoff at 25, where < 25 means not pregnant and > 25 means pregnant, we will have good sensitivity and specificity, but neither would be perfect (there are some false negatives and false positives where the two curves overlap). However, if we shift our cutoff to 20, we will have 100% sensitivity – everyone we test who is pregnant will receive a positive test. But the consequence of this is that we increase the false positive rate – or decrease the specificity – and now everything to the right of our blue line will register a positive test.

Testing for COVID-19 is no different, and involves a careful balancing act between sensitivity and specificity.  To make a test more sensitive, you will usually lose some specificity (i.e. to catch all the ‘true’ positives, you will get need to generate some false positives).

But surveillance of infectious disease may have different priorities than pregnancy testing.  In particular, with COVID-19, the risk of wasting resources to track false positives may not be as significant as the risk of false negatives – because false negatives result in missing opportunities to find infected people. So, ideally a test for COVID-19 would have higher sensitivity, even if some false positives are generated along the way.

Ok – so how well does the COVID-19 viral test perform?

A review of RT-PCR tests where the same person tested negative and subsequently tested positive, identified false negative rates from 2%-29%, corresponding to a sensitivity of 71-98%. Another  study of 205 patients with confirmed COVID-19 showed that the highest sensitivity was for bronchoalveolar lavage (which can be done during intubation of hospitalized patients, and is highly invasive) at 93%, while nasal swabs had a sensitivity of just 63%. By contrast, the test’s specificity is close to 100% because the primer is designed to only detect the SARS-CoV-2 sequence (though false positives can occasionally occur due to contamination from other samples).

For details on the CDC’s PCR test itself, you can read more here.  So the real issue is that the current tests are less sensitive than what we would like, and much of this, as can be seen between the difference in sensitivity between bronchoalveolar lavage and nasal swabs – is due to how the sample is being collected, and whether there is enough virus to be detected.

So, what does this mean for interpreting COVID-19 tests?

Given that the specificity – or the true negative rate – of the RT-PCR tests is much higher than the sensitivity – or the true positive rate – a positive test result indicates high likelihood of infection, while negative test results do not give as much certainty that the person is truly negative (remember: low sensitivity means lots of false negatives). This figure from Watson et al. helps to show how sensitivity and specificity, combined with the pre-test probability of COVID-19 (how prevalent COVID-19 is in the population), interact to inform the post-test-probability of COVID-19 (how medical professionals will interpret the test result).

Someone with a pre-test probability of 50% who tests positive given these sensitivity and specificity values will have a post-test probability close to 100%.  A positive test means they almost certainly are positive for COVID-19.

However, for the same pre-test probability, a negative test only indicates about a 25% probability that they are negative for COVID-19. This means that these RT-PCR tests are likely missing cases – as some individuals who receive negative test results are actually positive for COVID-19.

 

What does it mean that RT-PCR tests are missing true COVID-19 cases?

Let’s go through an example.

Watson et al. illustrate the effects of imperfect test sensitivity and specificity with a discrete example: with a sample of 100 people, 80 of whom truly have COVID (80% pre-test probability), using the current RT-PCR test will significantly underestimate the number of true COVID infections. 

Using Watson et al.’s average COVID test (with 70% sensitivity and 95% specificity), we only see that 57 people test positive for COVID-19 – with 24 false negatives. That means that 25% of our population was falsely told that they do not have COVID-19, and do not need to self-isolate! These individuals can spread the virus and pose a risk to controlling COVID-19, especially because their being told they tested negative may embolden them to be less careful.

In our current environment for COVID-19 testing, low test sensitivity creates substantial challenges for our health systems to provide an effective response because true cases are being missed. Thus, contact tracing, estimates of prevalence and disease severity, and opportunities for early intervention all suffer as a result of low test sensitivity.

Given the low risk of staying home in self isolation (in the case of a false positive) compared to the risk of spreading the virus without knowing one is carrying the virus (in the case of a false negative) – a test with higher sensitivity – even if it came with lower specificity, would help slow the spread of this disease. 

Why is the sensitivity of RT-PCR COVID-19 viral tests so low?

A recent article from JAMA explains how to interpret different diagnostic tests for SARS-CoV-2, and the relationship between the timing of testing, type of test, and likelihood of a positive result.

As you can see from the figure, the blue and pink lines show the trajectory of likely detecting COVID-19 in a sample based on time of symptom onset. The authors found that most people with symptomatic COVID-19 could have a positive PCR test from a nasopharyngeal swab detected as early as the first day of symptoms – and the probability of a positive test peaks within the first week of having symptoms. By week 3, the likelihood of a positive PCR test goes down significantly, and becomes undetectable soon after. The authors also note that hospitalized patients might have positive tests for longer, though they caution that a positive test result doesn’t mean the virus can be transmitted (i.e. that they are still infectious), just that viral RNA was detected.

 

False negatives for the viral tests can often happen because the sample was collected too early or too late relative to the onset of symptoms, or due to poor sample collection technique — especially for nasopharyngeal swabs, which aren’t effective if they don’t get deep into the nasal cavity.

Main Takeaways:

• A positive RT-PCR test (nasal swab) for COVID-19 indicates high likelihood that you do, in fact, have COVID-19. So if you take a viral test for COVID-19 and that test is positive, it is extremely likely that you are currently infected.
• However, a negative RT-PCR test (nasal swab) may not rule out COVID infection due to the high false-negative rates.

I got a COVID swab last month, which was negative, but cannot know for sure if I was or was not infected. If you are symptomatic, continue to stay home, isolate, and act as if you are infected to protect your friends and family, even if your swab is negative!

For official guidelines from the CDC on how to interpret test results, consult this PDF.

 

This week, my lab mate, Cara Janusz, a fellow PhD candidate in Epidemiology at the University of Michigan, joins with editing contributions to the blog. She is excited to support evidence synthesis and communication about our immunization research and policy interests on this platform!  She is passionate about supporting evidence-based immunization policy and practice through her research.