PCR using blood for diagnosis of invasive pneumococcal disease: systematic review and meta-analysis

Avni T, Mansur N, Leibovici L, Paul M

CRD summary

This review concluded that available methods for polymerase chain reaction with blood samples for the diagnosis of invasive pneumococcal disease lacked the sensitivity and specificity necessary for clinical practice. These conclusions reflected the data presented and are likely to be reliable.

Authors' objectives

To assess the accuracy of polymerase chain reaction (PCR) assays on blood samples for the diagnosis of invasive pneumococcal disease and to analyze the reasons for differences in performance between different tests, infections, and populations.

Searching

PubMed was searched to October 2009 and the Cochrane database, LILACS, National Library of Medicine (NLM) and KoreaMed were searched to December 2008. In addition, the proceedings of theEuropean Congress of Clinical Microbiology and Infectious Diseases, and the Interscience Conference on Antimicrobial Agents and Chemotherapy, were searched between 2000 and 2008. Search terms were reported. Bibliographies of included studies and websites of commercial products were scanned for additional articles.

Study selection

Prospective cohort studies, prospective nested case-control studies, and retrospective case-control studies, that assessed the accuracy of any polymerase chain reaction (PCR)-based molecular method performed with blood samples for the diagnosis of invasive pneumococcal disease in adults or children, were eligible for inclusion. Invasive pneumococcal disease was defined as primary bacteraemia (signs and symptoms of sepsis without a documented source of infection), pneumonia, empyema, or meningitis caused by Streptococcus pneumoniae; studies that assessed only sinusitis and otitis media were excluded. PCR, nested PCR, reverse transcription-PCR, real-time PCR, and multiplex PCR assays were included, whilst PCR testing of blood cultures after incubation or after the first identification of growth was excluded. The reference standard was defined as follows: level I - growth of S. pneumoniae in blood cultures; level II - growth of S. pneumoniae in blood cultures or specimens from other sterile sites (cerebrospinal fluid, pleural fluid, lung biopsy specimens); level III -growth of S. pneumoniae as detailed for level II by the use of pneumococcal antigen-based tests with blood or urine or serological tests for S. pneumoniae; level IV was any of the methods detailed for level III or a clinical rule that defined invasive pneumococcal disease. Cultures of non-sterile upper airway samples (sputum, nasopharynx, throat, etc.) were not considered an appropriate reference standard.

Of the 29 included studies, ten assessed only children, seven assessed only adults, and the remaining populations were unspecified. Fourteen studies targeted any invasive pneumococcal disease, 13 studies evaluated pneumococcal pneumonia alone, two studies evaluated meningitis alone, and one study evaluated children with otitis media in addition to invasive pneumococcal disease. Standard, real-time, and nested PCR assays were each used in approximately one-third of the included studies for whole blood, serum, plasma, or buffy coat specimens; three studies used fresh samples, and the remainder used frozen samples. Most studies (22) used primers for the pneumococcal ply gene, 10 studies used primers for the lytA gene, two studies used primers for the gene for PBP 2b, and primers for psaA and SPN9802 were used in one study each.

Two reviewers independently assessed studies for inclusion and disagreements were resolved by consensus, or consultation with a third reviewer.

Assessment of study quality

Methodological quality was assessed using the QUADAS (Quality Assessment of Diagnostic Accuracy Studies) checklist, adapted for this review (details reported in supplemental online material, see URL for Additional Data), which included items on appropriateness of participant spectrum, use of an acceptable reference standard, avoidance of disease progression bias, verification bias and review biases, and handling of uninterpretable results and withdrawals. QUADAS items were assigned a score of 2 for yes (a low risk for bias), 1 for unknown or unclear, and 0 for no (a high risk for bias).

The authors did not state how many reviewers performed the quality assessment.

Two reviewers independently extracted data and disagreements were resolved by consensus, or consultation with a third reviewer.

Methods of synthesis

Where studies reported results for more than one gene-specific primer, the results obtained with the ply gene were used for the primary analysis.

Sensitivity and specificity were plotted in receiver operating characteristic space and the hierarchical summary receiver operating characteristic method was used to fit a summary receiver operating characteristic curve and to calculate summary estimates for sensitivity, specificity, and hence positive and negative likelihood ratios, with 95% confidence intervals (CIs) and a 95% prediction region. A value of 0.5 was assigned to zero cells.

The effects of several covariates on test performance (as indicated by the diagnostic odds ratio DOR) were investigated using univariate meta-regression: blood sample type (whole blood, serum, plasma, or other); PCR method (nested, real time, or other); pneumococcal gene(s) used as the primer; study design (cohort versus case-control); age (studies recruiting adults versus studies recruiting children); source of infection (studies assessing pneumonia only versus studies assessing any invasive pneumococcal disease); study year; DNA extraction method (methods with commercial kits or enzymatic or mechanical methods); use of serial dilution; and use of a protocol to circumvent the effects of inhibitors in blood) on test performance. Adjustment for multiple testing was performed by using a univariable permutation test. Where the unadjusted meta-regression results indicated a significant effect of a variable on diagnostic odds ratio, summary hierarchical summary receiver operating characteristic sensitivity and specificity values were presented separately by category.

Results of the review

Twenty-nine studies (total number of participants not reported) were included in the review. Seventeen studies were prospective cohort studies; three studies were considered nested case-control studies (a prospective cohort of participants evaluated for invasive pneumococcal disease was assessed, patients with proven invasive pneumococcal disease from that cohort served as cases, while a separate cohort of healthy participants served as controls); nine studies were classified as case-control studies (used stored clinical samples from patients with invasive pneumococcal disease for comparison with samples from a cohort of healthy people or patients with infections caused by bacteria other than S. pneumoniae). There was variation between studies in recruitment of consecutive participants, timing of polymerase chain reaction (PCR) tests in relation to the time of collection of blood for culture, application of reference standard to all participants, and the description of the availability of clinical information to test interpreters. Only one study reported withdrawals. Reported quality scores ranged from 11 to 22 points.

Pneumococcal bacteraemia: When participants with disease were defined as patients with pneumococcal bacteraemia, and participants without disease were defined as healthy people or patients with bacteraemia caused by other bacteria (22 studies), the summary estimate for sensitivity was 57.1% (95% CI 45.7 to 67.8) and specificity was 98.6% (95% CI, 96.4 to 99.5). Case-control studies showed higher sensitivities than cohort studies, and pneumonia as the source of infection was associated with a slightly lower sensitivity than other types of invasive pneumococcal disease. When the control groups were patients suspected of having invasive pneumococcal disease without pneumococcal bacteraemia (26 studies), estimates of 66.4% (95% CI 55.9 to 75.6%) for sensitivity and 87.8% (95% CI 79.5 to 93.1%) for specificity; only the type of PCR effected test performance (nested PCR had lower sensitivity and specificity than other types of PCR).

Any culture-proven invasive pneumococcal disease: When participants with disease were defined as those with a clinical syndrome suggestive of invasive pneumococcal disease and proof of pneumococcal infection by culture of a specimen from a sterile site (including blood), and participants without disease were defined a healthy people or patients with infections bacteriologically proven to be caused by other bacteria (22 studies), the summary estimate for sensitivity was 56.1% (95% CI 47.1 to 64.7%) and for specificity was 98.6% (95% CI, 96.2 to 99.4%). No covariate affected test performance.

Any proven invasive pneumococcal disease: When participants with disease were defined as positive on culture of specimens from sterile sites, serology, or antigen testing, and participants without disease were defined as healthy people or patients who did not have invasive pneumococcal disease (21 studies), the summary estimates for sensitivity was 46.6% (95% CI 34.8 to 58.8) and specificity was 98.8% (95% CI 96.6 to 99.6). Studies that assessed pneumococcal pneumonia had higher sensitivities than studies that included any invasive pneumococcal disease; studies that included only children reported lower sensitivities than studies that included adults or mixed populations.

Possible invasive pneumococcal disease: Six studies used a clinical definition of invasive pneumococcal disease, in addition to conventional diagnostic techniques. The summary estimate of sensitivity for these studies was 31.7% (95% CI 16 to 53), and the summary estimate of specificity was 98.3% (95% CI, 80.9 to 100).

The lack of an appropriate reference standard may have caused underestimation of the performance of the PCR test.

Estimates of predictive ability and a sensitivity analysis (pooled estimates of sensitivity for studies selected on the basis of covariates identified as significant in the regression model) were also reported.

Authors' conclusions

Available methods for PCR with blood samples for the diagnosis of invasive pneumococcal disease lacked the sensitivity and specificity necessary for clinical practice.

CRD commentary

This review addressed a clearly stated question and detailed inclusion criteria were defined. A number of sources were searched to identify relevant studies. No search restrictions were reported. Measures were taken, during study selection and data extraction, to minimise the potential for error and/or bias, but it was unclear whether a similar approach was applied to quality assessment.

The authors reported heterogeneity in all data sets, (details of individual studies reported in online supplementary material). The meta-analyses undertaken were appropriate and clearly reported.

The authors' conclusions reflected the data presented and are likely to be reliable.

Implications of the review for practice and research

Practice: The authors stated that, given the available data, PCR directly with blood samples cannot be considered for use in clinical practice.

Research: The authors stated that analyses should be conducted to assess the performance of PCR applied to blood samples for the detection of other bacterial infections, and these results should be considered prior to the introduction of novel costly technologies. They further stated that guidelines for a uniform design and method of reporting for studies assessing the diagnostic accuracies of molecular techniques for the diagnosis of infections are needed.

Funding

Partially supported by a research grant (for optimal antibiotic treatment of moderate to severe bacterial infections: integration of PCR technology and medical informatics/artificial intelligence) from the Rothschild-Caesarea Foundation.

This is a critical abstract of a systematic review that meets the criteria for inclusion on DARE. Each critical abstract contains a brief summary of the review methods, results and conclusions followed by a detailed critical assessment on the reliability of the review and the conclusions drawn.