The accuracy of a rapid finger-prick antibody test for SARS-CoV-2, the virus responsible for covid-19 infection, may be considerably lower than previously suggested, finds a study published by The BMJ.

The results suggest that if 10% of people given the test had previously been infected, around 1 in 5 positive test results would be incorrect (false positive results).

These conclusions contrast with an earlier (not yet peer reviewed) study suggesting that the test gives no false positive results.

The findings suggest the test can deliver a sufficient degree of accuracy for surveillance studies of the population, but laboratory confirmation of positive results is likely to be needed if these tests are to be used to provide evidence of protection from the virus.

The AbC-19TM Rapid Test uses a drop of blood from a finger-prick to see if it’s likely that someone has previously been infected with SARS-CoV-2. It gives results in 20 minutes, without the need to go to a laboratory, and is approved for use by health professionals in the UK and EU.

The latest research was commissioned by the Department of Health and Social Care and conducted by scientists from Public Health England and the Universities of Bristol, Cambridge and Warwick.

Scientists tested blood samples in a laboratory from 2,847 key workers (healthcare, fire, and police staff) in England in June 2020.

Of these, 268 had a previous PCR (positive polymerase-chain reaction) positive result so were “known positives” while the remaining 2,579 had unknown previous infection status. A further 1,995 pre-pandemic blood samples were also tested as “known negatives.”

Based on a series of analyses, the researchers estimated the specificity of the AbC-19 test (ability to correctly identify a true negative sample) to be 97.9%, meaning that 2.1% of people who did not have a previous SARS-Cov-2 infection incorrectly tested positive.

They estimated the sensitivity of the AbC-19 test (ability to correctly identify a true positive sample) to be 92.5% based on PCR confirmed cases but considerably lower (84.7%) in people with unknown previous infection status prior to antibody testing.

This difference is probably due to the test being more sensitive when antibody levels are higher, explain the researchers. As people with a positive PCR result tended to have more severe disease, it is likely that they would have produced more antibodies.

They say the lower figure of 84.7% is probably a more realistic estimate of test sensitivity in the real world, if people were to choose to take the test to find out their own previous infection status. This means that 15.3% of people with a previous SARS-CoV-2 infection would be missed.

Putting these findings into context, the researchers say that, if 1 million people were tested with AbC-19, of whom 10% had been previously infected with SARS-CoV-2, there would be 18,900 false positive results. Overall, about one in five positive results would be wrong.

They also found that trained laboratory staff noted the test result band was often weak and disagreed on whether the result was positive or negative for almost 4% of AbC-19 tests. This implies that test accuracy could be lower still if the test was used at home by members of the public.

This is a large study, using data from individuals with both known and unknown previous infection status, but the authors do highlight some limitations.

For example, the test was evaluated in a laboratory, rather than having participants perform the test themselves, which may have overestimated performance, and the study included few people aged over 65 years, suggesting the need for further evaluation of the test in older ages when risk of severe covid-19 is substantially higher.

It is possible that other lateral flow devices detecting antibodies to SARS-CoV-2 may also work less well at lower antibody concentrations; while this study did not investigate this, the authors note that their work “highlights the scope for overestimation of SARS-CoV-2 antibody test sensitivity in other studies in which sensitivity has been estimated only from PCR confirmed cases.”

The UK Government has placed an order for one million AbC-19 tests for research purposes, to help build up a picture of how the virus has spread across the country.

In a linked editorial, Dipender Gill at Imperial College London and Mark Ponsford at Cardiff University, say this study “identifies notable limitations of the UK government’s antibody test of choice and provides good evidence that its specificity in a “real life” setting is highly unlikely to be 100%.”

They call for further work to clarify the relation between circulating antibody levels and immunity to SARS-Cov-2, and say “a clear message must be communicated to the public that positive results from these assays do not provide evidence of immunity.”

“Apart from limited surveillance to estimate the proportion of a population that has been infected, widespread use of this assay in any other role could risk considerable harm,” they conclude.