Of gastro and the gold standard: evaluation and policy implications of norovirus test performance for outbreak detection.

Fisman DN, Greer AL, Brouhanski G, Drews SJ - J Transl Med (2009)

Bottom Line:
If all specimens contained norovirus, RT2-PCR or EIA would be associated with > 99.9% likelihood of at least one test being positive after three specimens tested.Testing of more than 5 true negative specimens with RT2-PCR would be associated with a greater than 50% likelihood of a false positive test.Given risks of false positive test results, it is reasonable to limit the number of specimens tested when RT2-PCR or EIA are available.

Background: The norovirus group (NVG) of caliciviruses are the etiological agents of most institutional outbreaks of gastroenteritis in North America and Europe. Identification of NVG is complicated by the non-culturable nature of this virus, and the absence of a diagnostic gold standard makes traditional evaluation of test characteristics problematic.

Results: Latent class modelling estimated sensitivities of RT2-PCR, EIA, and EM as 100%, 86%, and 17% respectively; specificities were 84%, 92%, and 100%; estimates obtained using a composite reference standard were similar. If all specimens contained norovirus, RT2-PCR or EIA would be associated with > 99.9% likelihood of at least one test being positive after three specimens tested. Testing of more than 5 true negative specimens with RT2-PCR would be associated with a greater than 50% likelihood of a false positive test.

Conclusion: Our findings support the characterization of EM as lacking sensitivity for NVG outbreaks. The high sensitivity of RT2-PCR and EIA permit identification of NVG outbreaks with testing of limited numbers of clinical specimens. Given risks of false positive test results, it is reasonable to limit the number of specimens tested when RT2-PCR or EIA are available.

Figure 3: Probability of One or More Positive Test Results by Specimens Tested, Under Varying Assumptions Regarding Proportion of True Positive Specimens. Curves are constructed based on a binomial distribution. Each contour represents a different proportion of true positive test specimens. Graph (A) represents estimates generated based on high (100%) sensitivity estimated for real-time reverse-transcriptase polymerase chain reaction using both latent class modeling (LCM) and composite reference standard (CRS) methods. Graph (B) presents estimates generated using LCM estimates for enzyme immunoassay (EIA) sensitivity (86%). A graph using EIA sensitivity estimates from CRS would be similar to graph (A) due to high (97%) sensitivity estimates using the latter approach.

Mentions:
We assessed the likelihood that an individual specimen contained NVG material by comparing submitted specimen numbers in identified outbreaks to the number of specimens testing positive by RT2-PCR in those same outbreaks (Table 3). Depending on the presence or absence of EM confirmation of a given outbreak, the proportion of specimens testing positive in apparent outbreaks varied from approximately 58–72% (with 95% confidence intervals as low as 54% and as high as 76%). As such, it would be estimated that using highly sensitive methods such as RT2-PCR an outbreak will be identified with greater than 98% certainty with the submission of five stool specimens during an outbreak investigation, even if only 50% of specimens contain detectable norovirus. With slightly less sensitive but more specific test methods such as EIA, similar projections are generated (Figures 3A and 3B).

Figure 3: Probability of One or More Positive Test Results by Specimens Tested, Under Varying Assumptions Regarding Proportion of True Positive Specimens. Curves are constructed based on a binomial distribution. Each contour represents a different proportion of true positive test specimens. Graph (A) represents estimates generated based on high (100%) sensitivity estimated for real-time reverse-transcriptase polymerase chain reaction using both latent class modeling (LCM) and composite reference standard (CRS) methods. Graph (B) presents estimates generated using LCM estimates for enzyme immunoassay (EIA) sensitivity (86%). A graph using EIA sensitivity estimates from CRS would be similar to graph (A) due to high (97%) sensitivity estimates using the latter approach.

Mentions:
We assessed the likelihood that an individual specimen contained NVG material by comparing submitted specimen numbers in identified outbreaks to the number of specimens testing positive by RT2-PCR in those same outbreaks (Table 3). Depending on the presence or absence of EM confirmation of a given outbreak, the proportion of specimens testing positive in apparent outbreaks varied from approximately 58–72% (with 95% confidence intervals as low as 54% and as high as 76%). As such, it would be estimated that using highly sensitive methods such as RT2-PCR an outbreak will be identified with greater than 98% certainty with the submission of five stool specimens during an outbreak investigation, even if only 50% of specimens contain detectable norovirus. With slightly less sensitive but more specific test methods such as EIA, similar projections are generated (Figures 3A and 3B).

Bottom Line:
If all specimens contained norovirus, RT2-PCR or EIA would be associated with > 99.9% likelihood of at least one test being positive after three specimens tested.Testing of more than 5 true negative specimens with RT2-PCR would be associated with a greater than 50% likelihood of a false positive test.Given risks of false positive test results, it is reasonable to limit the number of specimens tested when RT2-PCR or EIA are available.

Background: The norovirus group (NVG) of caliciviruses are the etiological agents of most institutional outbreaks of gastroenteritis in North America and Europe. Identification of NVG is complicated by the non-culturable nature of this virus, and the absence of a diagnostic gold standard makes traditional evaluation of test characteristics problematic.

Results: Latent class modelling estimated sensitivities of RT2-PCR, EIA, and EM as 100%, 86%, and 17% respectively; specificities were 84%, 92%, and 100%; estimates obtained using a composite reference standard were similar. If all specimens contained norovirus, RT2-PCR or EIA would be associated with > 99.9% likelihood of at least one test being positive after three specimens tested. Testing of more than 5 true negative specimens with RT2-PCR would be associated with a greater than 50% likelihood of a false positive test.

Conclusion: Our findings support the characterization of EM as lacking sensitivity for NVG outbreaks. The high sensitivity of RT2-PCR and EIA permit identification of NVG outbreaks with testing of limited numbers of clinical specimens. Given risks of false positive test results, it is reasonable to limit the number of specimens tested when RT2-PCR or EIA are available.