Of gastro and the gold standard: evaluation and policy implications of norovirus test performance for outbreak detection.

Fisman DN, Greer AL, Brouhanski G, Drews SJ - J Transl Med (2009)

Bottom Line:
If all specimens contained norovirus, RT2-PCR or EIA would be associated with > 99.9% likelihood of at least one test being positive after three specimens tested.Testing of more than 5 true negative specimens with RT2-PCR would be associated with a greater than 50% likelihood of a false positive test.Given risks of false positive test results, it is reasonable to limit the number of specimens tested when RT2-PCR or EIA are available.

Background: The norovirus group (NVG) of caliciviruses are the etiological agents of most institutional outbreaks of gastroenteritis in North America and Europe. Identification of NVG is complicated by the non-culturable nature of this virus, and the absence of a diagnostic gold standard makes traditional evaluation of test characteristics problematic.

Results: Latent class modelling estimated sensitivities of RT2-PCR, EIA, and EM as 100%, 86%, and 17% respectively; specificities were 84%, 92%, and 100%; estimates obtained using a composite reference standard were similar. If all specimens contained norovirus, RT2-PCR or EIA would be associated with > 99.9% likelihood of at least one test being positive after three specimens tested. Testing of more than 5 true negative specimens with RT2-PCR would be associated with a greater than 50% likelihood of a false positive test.

Conclusion: Our findings support the characterization of EM as lacking sensitivity for NVG outbreaks. The high sensitivity of RT2-PCR and EIA permit identification of NVG outbreaks with testing of limited numbers of clinical specimens. Given risks of false positive test results, it is reasonable to limit the number of specimens tested when RT2-PCR or EIA are available.

Figure 2: Empirical Estimate of Cumulative Specimens Tested for One or More Positive Test Results in Documented Norovirus Gastroenteritis Outbreaks. Specimens are numbered in the order in which they were accessioned by the laboratory. Solid line represents outbreaks without confirmation by electron microscopy; dashed line represents outbreaks identified by real-time reverse-transcriptase polymerase chain reaction (RT2-PCR) alone.

Mentions:
Specimens submitted for evaluation in the context of outbreak investigations are likely to contain a mixture of truly positive and truly negative specimens; in this context, we used Kaplan-Meier methods to evaluate the relationship between specimen submissions and the identification of at least one positive specimen in PCR-positive outbreaks with and without EM confirmation. Even with a test with approximately 100% sensitivity (i.e., PCR) and in the context of a true-positive (EM-confirmed) outbreak, 3 specimens needed to be tested before a single positive test result is identified with a probability > 95%. For EM-negative outbreaks, 95% of outbreaks had been identified after testing of two specimens (Figure 2).

Figure 2: Empirical Estimate of Cumulative Specimens Tested for One or More Positive Test Results in Documented Norovirus Gastroenteritis Outbreaks. Specimens are numbered in the order in which they were accessioned by the laboratory. Solid line represents outbreaks without confirmation by electron microscopy; dashed line represents outbreaks identified by real-time reverse-transcriptase polymerase chain reaction (RT2-PCR) alone.

Mentions:
Specimens submitted for evaluation in the context of outbreak investigations are likely to contain a mixture of truly positive and truly negative specimens; in this context, we used Kaplan-Meier methods to evaluate the relationship between specimen submissions and the identification of at least one positive specimen in PCR-positive outbreaks with and without EM confirmation. Even with a test with approximately 100% sensitivity (i.e., PCR) and in the context of a true-positive (EM-confirmed) outbreak, 3 specimens needed to be tested before a single positive test result is identified with a probability > 95%. For EM-negative outbreaks, 95% of outbreaks had been identified after testing of two specimens (Figure 2).

Bottom Line:
If all specimens contained norovirus, RT2-PCR or EIA would be associated with > 99.9% likelihood of at least one test being positive after three specimens tested.Testing of more than 5 true negative specimens with RT2-PCR would be associated with a greater than 50% likelihood of a false positive test.Given risks of false positive test results, it is reasonable to limit the number of specimens tested when RT2-PCR or EIA are available.

Background: The norovirus group (NVG) of caliciviruses are the etiological agents of most institutional outbreaks of gastroenteritis in North America and Europe. Identification of NVG is complicated by the non-culturable nature of this virus, and the absence of a diagnostic gold standard makes traditional evaluation of test characteristics problematic.

Results: Latent class modelling estimated sensitivities of RT2-PCR, EIA, and EM as 100%, 86%, and 17% respectively; specificities were 84%, 92%, and 100%; estimates obtained using a composite reference standard were similar. If all specimens contained norovirus, RT2-PCR or EIA would be associated with > 99.9% likelihood of at least one test being positive after three specimens tested. Testing of more than 5 true negative specimens with RT2-PCR would be associated with a greater than 50% likelihood of a false positive test.

Conclusion: Our findings support the characterization of EM as lacking sensitivity for NVG outbreaks. The high sensitivity of RT2-PCR and EIA permit identification of NVG outbreaks with testing of limited numbers of clinical specimens. Given risks of false positive test results, it is reasonable to limit the number of specimens tested when RT2-PCR or EIA are available.