The problem of false positives: Antibody tests in times of Corona

A few weeks or months from now, we could have a Covid-19 test kit sent to our home. Similar to a blood sugar test for diabetics, we would prick our finger, wait for a couple of minutes, and we will know whether we are immune or not. The general idea is that this would help in lessen the social distancing restrictions because those who are immune could in principle go back to a normal life.

But it is of course more complicated than that. In short: If an antibody test tells you that you are “positive” (i.e., you have developed antibodies and therefore are or should have been infected), the probability that you are actually immune is nonetheless very low because your test result is very likely a “false positive”.

Yesterday, I stumbled upon the following Twitter thread, which pinpoints at an important issue when it comes to antibody tests that should help to determine who was infected and who was not.

Antibody tests are coming online. Never before have humans needed to understand Bayes rule more. Let’s talk about why it’s critical NOT to assume you are immune to covid-19 when you have a positive antibody test. Seriously, people need to understand this to prevent many deaths.

Intuitively, most of us would probably think that the accuracy of a (antibody) test is primarily related to its precision. A test’s precision is denoted by its sensitivity (probability that a person tests positive given that they are indeed positive) and its specificity (probability that a person tests negative given that they are indeed negative). Cellex, which is the first company to get a rapid covid-19 antibody test approved by the FDA, has a sensitivity of 93.8% and a specificity of 95.6%.1 This sounds pretty good. Only 6.2% would be “false negatives” and only 4.4% would be “false positives”. But this oversimplifies the situation.

The problem is related to the known low probability of being or having been infected in the first place. From all (of course, biased) numbers that we know, the amount of people infected in a country is very low (e.g., in Germany, it would be 0.15% of the entire population at this moment). Even if unknown cases are ten times higher (e.g., cases without symptoms), we would still only have an infection rate of 1.5%. In Bayesian terms, this implies that our prior knowledge suggests that the probability of being “positive” is very low. Using Bayes theorem (which I won’t explain here), we can derive that it is very unlikely to be infected, even if the antibody test says that one is “positive”.2

Because this is a bit hard to grasp intuitively, let us simulate the problem. I therefore first define a function that:

1) creates a population with a certain percentage of infected people,
2) samples one person randomly, and
3) and conducts a test with a pre-defined sensitivity and specificity.

The resulting data frame now includes 1,000 tests per combination and a column that indicates whether the test was a false positive, false negative, correct positive or correct negative. We can see that the fifth case is already a false positive.
The following figure visualizes the results (click on image to enlarge!). I deliberately excluded all “correct negative” results as they make up the majority of the test results.

We can clearly see that the false positive rate increases strongly if a test has a lower specificity (as we would expect). More importantly, however, is the effect of the actual infection rate in the population. As long as the number of actually infected people in a population is very low (< 5%), a positive test results is more likely to be a “false positive” than a “correct positive”. Only with a very good specificity (>99%), the probability of being correctly identified as positive is as likely as being wrongly identified as positive (which is still pretty bad!). In the current situation, a single antibody test can thus hardly tell us whether we are actually immune. So what can we do? First, understand the problem. If you know that a positive result is likely to be a false positive, you will probably be more cautious in going back to normal life. Second, the probability of a “false positive” result decreases substantially if one takes more than one test. For example, obtaining a positive test result three times in a row should give you a greater confidence that you body actually has antibodies.

Take home message

A highly sensitive antibody test (e.g., Pr(tests+|is+) = 0.95) does not mean that the probability of being correctly identified as “positive” (i.e, Pr(is+|tests+)) is also high. In simple words, a large amount of people (the majority) who will get a positive test results are in fact not immune.

Your email address will not be published. Required fields are marked *

Comment

Name *

Email *

Website

Current ye@r *

Leave this field empty

Welcome!

I am a researcher in the field of communication science working at the Department of Communication at the Johannes Gutenberg University Mainz (Germany). Previously, I was a research assistant at the University of Hohenheim (Germany) and a visiting scholar at the Cornell University (Ithaca, USA). On this website, you can find information about my research interests, publications, teachings, and some personal background.

From time to time, I take pictures of things and people. You can find them here.

The cookie settings on this website are set to "allow cookies" to give you the best browsing experience possible. If you continue to use this website without changing your cookie settings or you click "Accept" below then you are consenting to this.