VERDICTThe aim is to develop a composite reference standard for COVID-19 diagnosis that will support a standardised approach across research groups to decrease the high false negative rate of rRT-PCR that could penalise the evaluation of diagnostic accuracy of new tests if rRT-PCR is used as a stand-alone reference standard. Currently there is a paucity of robust evidence for the use of blood biomarkers in COVID-19 diagnosis, therefore it is proposed that an extensive list of biomarkers is recorded during the diagnostic accuracy evaluations of the new COVID-19 tests in the summer months in order to help bridge this knowledge gap. To analyse this data, a temporary Composite Reference Standard (CRS) which includes radiological findings will be used to support the identification of biomarkers discriminatory for COVID-19. These will be included in the next iteration of CRS which could be used in the winter studies to discriminate COVID-19 from other respiratory diseases, highly prevalent in the winter months and characterized by similar radiological features to COVID-19.

BACKGROUNDClinical studies are in planning or currently underway and aim to evaluate the in-context diagnostic accuracy of novel diagnostic tests. These novel tests have the potential to be useful diagnostic tools during the next phases of the pandemic and for the differential diagnosis during the endemic phase. Real time reverse transcriptase polymerase chain reaction (rRT-PCR) is currently the reference standard for the diagnosis of COVID-19, but a high false negative rate has been reported.[1-3] Many of these diagnostic accuracy studies compare the new tests with rRT-PCR, an imperfect reference standard. The effect of this is underestimation of the specificity of the new tests.

The authors propose the development and use of a composite reference standard, with predefined diagnostic criteria or levels of diagnostic uncertainty, to allow a comparison with evidence from up to date literature with regards to diagnostic accuracy, but also with current clinical practice in disease management. Multiple composite reference standards can be developed/adapted for specific clinical settings in order to take into account feasibility of use in practice (e.g. evaluations in the community settings cannot rely on radiological examinations). However, the focus in this article was on the development of a composite reference standard to be used in hospital studies.

MethodsRapid reviews were carried out to identify signs and symptoms, biomarkers and radiology features that could support a diagnosis of COVID-19 (Appendix I). These reviews followed a Traditional Literature Review (TLR) approach [4] and supported the discussion of an expert panel who met on the 29th of April 2020 to identify the elements of the CRS for the diagnosis of COVID-19.

The bias was minimised in the development of the composite reference standard by aligning our methodology to the following recommendations:[5]

the use of the logical `OR` operator to link tests with perfect sensitivity, and the use of the `AND` operator to link tests with perfect specificity;

minimization of the number of tests needed;

maximization of independency among component tests (by choosing tests that measure biologically different aspect of the disease);

potentially assigning weights to tests in order to incorporate the information about their different clinical value and accuracy in the reference standard.

Presentation of evidence

Evidence from studies evaluating the potential different components of the reference standard were presented as the main biological categories of testing. Data were extracted from the BMJ-Best practice for COVID-19 diagnosis and identified through rapid literature reviews of primary and secondary diagnostic accuracy studies.[6] The areas of interest were:

Sign and Symptoms (Tables 1 and 2)

Blood Biomarkers (Table 3)

Radiological Examinations (Table 4)

Signs and SymptomsTable 1 Summary of evidence for signs and symptoms which are characteristic of COVID-19

The authors could not find articles reporting diagnostic accuracy of signs and symptoms in non-hospitalized populations. These would be beneficial in informing a reference standard for the community setting and help to understand how signs and symptoms develop during the course of the disease.

The only source identified is an article by Tostmann (2020) who reported signs and symptoms from hospital staff for early detection of COVID-19 (see Table 2):[14]

Table 2 Summary of signs and symptoms for early detection of COVID in a hospital staff population in Netherland (Table reprinted with permission from Tostmann, 2020).[14]

During the panel, clinicians reported that these symptoms are not specific to COVID-19 and the only one more characteristic of COVID-19 (i.e. anosmia) could, in future, be subject to bias due to extensive reporting and the potential for that symptom to be caused by nasal congestion of any aetiology. Indeed, since anosmia was recognised as a characteristic feature of COVID-19 and the information was disseminated in the general news, its discriminatory value is decreasing.

BiomarkersArticles evaluating the diagnostic accuracy of blood biomarkers are rare. A systematic review currently ongoing within our team has highlighted that the most comprehensive review to date being is by Soraya & Ulhaq, 2020.[15]

Clinicians commented that reduced leukocytes (WBC), lymphocytes and neutrophils are very common signs of infections that will not help differentiate COVID-19 from other infectious respiratory diseases during the winter months when these diseases are prevalent. Thrombocyte is a less common finding in infection and has the potential to be useful in the differential diagnosis.

Similarly, the features on the chest X-ray and chest CT scans are non-specific of COVID-19 but Ai (2020) reports that the positive rates of rRT-PCR and chest CT for patients diagnosed with COVID-19 were 59% (601/1014) and 88% (888/1014) respectively, with an overall sensitivity of chest CT of 97% compared to PCR.[16] Table 4 summarizes the current evidence:

Ai (2020) also showed the progression of ground glass opacity and consolidation in patients over time, indicating that the prevalence of a presenting feature is dependent upon the time at which the x-ray or CT is taken during the disease progression.

DiscussionThe expert panel decided that, even though the proposition of a composite reference standard for COVID-19 would be interesting to support standardisation of protocols for diagnostic accuracy studies and may provide better estimates of the accuracy of novel tests, there is currently not enough robust evidence around diagnostic accuracy of signs and symptoms and biomarkers for these to be used as part of a CRS. It was also recognized that during the winter, when other respiratory diseases are circulating, these symptoms and characteristics will be less specific. Therefore, a more comprehensive CRS will be needed for the winter season, with a simpler CRS currently more acceptable due to the lower prevalence of other respiratory illnesses in the summer months.

It was concluded that the best approach at this point of the pandemic is to use a temporary CRS, which includes radiological features and rRT-PCR, to support evaluations of new diagnostic tests during the summer season when there is a low prevalence of other respiratory conditions with similar radiological signature to COVID-19. During these evaluations robust, standardized data will be collected that will allow us in early Autumn to assess statistically which clinical features are discriminative of COVID-19. Another expert panel meeting will follow to develop another CRS, which incorporates the new results on clinical features and biomarkers, to be used prospectively during the winter season evaluations when other respiratory diseases are also circulating, and radiological evidence will not be as discriminatory.

Temporary Composite Reference Standard for the summer seasonThe panel agreed that the following CRS could be used until more robust evidence on symptoms and biomarkers emerge:

DEFINITE COVID-19: any positive rRT-PCR result during the course of the disease

POSSIBLE COVID-19: negative or undetermined PCR/ELISA AND any radiological evidence of pneumonia with characteristic symptoms of COVID-19

UNLIKELY COVID-19: negative or undetermined PCR/ELISA AND no radiological evidence of pneumonia even with characteristic symptoms of of COVID-19

It should be noted that this reference standard refers to a population of suspected COVID-19 based on a symptomatic presentation (i.e. sustained cough and/or fever).

Since a more widespread screening approach in the hospital seems to be the next stage in UK testing strategy, and with the emerging potential of new tests to capture the non-symptomatic population, it is important to include patients with positive molecular results but no symptoms characteristic of COVID-19 in diagnostic accuracy studies. It is advised to recruit and analyse this patient group separately with exploratory analysis.

The definite non-COVID-19 patients, i.e. patients with no symptoms and no rRT-PCR test positivity, are excluded in this late stage evaluation of the accuracy of the new tests because it is unlikely that future testing strategies will incorporate this population. Also, this population is usually included in earlier phases of evaluation (e.g. analytical validation studies).

Recommendations for researchWhen designing diagnostics accuracy studies for novel COVID-19 diagnostics, we advise researchers to collect the data, where possible, shown in Table 5 below (in addition to any molecular and serological testing results). This data combines both the observed indicators from the literature review and the listed symptoms, biomarkers and indicators laid out by ISARIC.[17]

Table 5 Recommended key characteristics/indications to record during the next round of diagnostic accuracy studies as aligned with ISARIC Case Reporting.[17]

In addition to the biomarkers and symptoms highlighted in the literature reviews, supplementary biomarkers and symptoms identified by ISARIC should also be captured where possible as shown in Table 6.

Inclusion of these additional data would allow for future design of the composite reference standard to pool data from multiple studies with the aim of improving its clinical application. Additionally, symptom onset and the time when the tests were performed must be included to allow direct comparison between studies and to gain a deeper understanding of the effect of the development of the disease and the change in the prevalence of these indicative characteristics over time.

Furthermore, as new serological testing for IgA and IgM is validated, an ELISA detection method has the potential to be used independently to confirm COVID-19 as part of a wider testing strategy in different settings. This technique could be particularly useful in the later stages of the disease when the detection rate of rRT-PCR falls.[18,19] The use of the IgG antibody as part of the reference standard needs to be carefully considered, especially if future studies confirm the presence of IgG antibodies for weeks or months after symptoms are resolved.[20]

However, until such tests are independently validated for clinical use, the current CRS will focus solely on the existing testing methods, with the aim of incorporating serological assays for the winter season.

CONCLUSIONSWe proposed a roadmap for diagnostic accuracy studies for COVID-19 classification, including the definition of a composite reference standard for ongoing studies, indications for data collection and considerations around the possible introduction of antibody testing in the reference standard. Recommendations were based on literature review and elicitation of judgements from a panel of experts in diagnostic research across UK. Follow-up systematic literature reviews are being performed to update the CRS and clinical guidance going forward if new evidence emerges.

SEARCHING STRATEGY

End.

Disclaimer: the article has not been peer-reviewed; it should not replace individual clinical judgement and the sources cited should be checked. The views expressed in this commentary represent the views of the authors and not necessarily those of the host institution, the NHS, the NIHR, or the Department of Health and Social Care. The views are not a substitute for professional medical advice.

Green, K., Allen, A., Suklan, J et al.. What Is The Role Of Imaging And Biomarkers Within The Current Testing Strategy For The Diagnosis Of Covid-19? – CEBM [online] CEBM. Available at: <https://www.cebm.net/covid-19/what-is-the-role-of-imaging-and-biomarkers-within-the-current-testing-strategy-for-the-diagnosis-of-covid-19/> [Accessed 13 May 2020].