Similar presentations

2 BY THE END OF THIS SESSION, YOU SHOULD BE ABLE TO ….describe and illustrate key measures of diagnostic test performancedescribe some less commonly quoted measures of diagnostic test performancerepresent diagnostic test performance in at least four different ways (five if time allows!)

3 METHOD 1: NATURAL FREQUENCIES GRIDPerson without the diseasePerson with the diseasePerson who tests positivePerson who tests negativeTrue positive on the testFalse positive on the testTrue negative on the testFalse negative on the testAssume that the prevalence of the disease is 4%

4 Assume that of the 4 people with the disease, 3 are picked up by the testPerson without the diseasePerson with the diseasePerson who tests positivePerson who tests negativeTrue positive on the testFalse positive on the testTrue negative on the testFalse negative on the test

5 Assume that of the test is positive for a further 7 people who don’t have the diseasePerson without the diseasePerson with the diseasePerson who tests positivePerson who tests negativeTrue positive on the testFalse positive on the testTrue negative on the testFalse negative on the testPerson without the diseasePerson with the diseasePerson who tests positivePerson who tests negativeTrue positive on the testFalse positive on the testTrue negative on the testFalse negative on the test

6 The remainder of the sample are negative on the testPerson without the diseasePerson with the diseasePerson who tests positivePerson who tests negativeTrue positive on the testFalse positive on the testTrue negative on the testFalse negative on the testPerson without the diseasePerson with the diseasePerson who tests positivePerson who tests negativeTrue positive on the testFalse positive on the testTrue negative on the testFalse negative on the test

7 SENSITIVITYPerson without the diseasePerson with the diseasePerson who tests positivePerson who tests negativeTrue positive on the testFalse positive on the testTrue negative on the testFalse negative on the testPerson without the diseasePerson with the diseasePerson who tests positivePerson who tests negativeTrue positive on the testFalse positive on the testTrue negative on the testFalse negative on the testSENSITIVITY is the proportion of people with the disease correctly identified by the testIt measures the proportion of false NEGATIVES

8 SENSITIVITY In this case, sensitivity is ¾ or 75%Person without the diseasePerson with the diseasePerson who tests positivePerson who tests negativeTrue positive on the testFalse positive on the testTrue negative on the testFalse negative on the testPerson without the diseasePerson with the diseasePerson who tests positivePerson who tests negativeTrue positive on the testFalse positive on the testTrue negative on the testFalse negative on the testIn this case, sensitivity is ¾ or 75%

9 SPECIFICITYPerson without the diseasePerson with the diseasePerson who tests positivePerson who tests negativeTrue positive on the testFalse positive on the testTrue negative on the testFalse negative on the testPerson without the diseasePerson with the diseasePerson who tests positivePerson who tests negativeTrue positive on the testFalse positive on the testTrue negative on the testFalse negative on the testSPECIFICITY is the proportion of people without the disease correctly identified by the testIt measures the proportion of false POSITIVES

10 SPECIFICITY In this case, specificity is (96-7)/96 or 93%Person without the diseasePerson with the diseasePerson who tests positivePerson who tests negativeTrue positive on the testFalse positive on the testTrue negative on the testFalse negative on the testPerson without the diseasePerson with the diseasePerson who tests positivePerson who tests negativeTrue positive on the testFalse positive on the testTrue negative on the testFalse negative on the testIn this case, specificity is (96-7)/96 or 93%

11 If someone is positive on the test, what are the chances that he/she has the disease?Person without the diseasePerson with the diseasePerson who tests positivePerson who tests negativeTrue positive on the testFalse positive on the testTrue negative on the testFalse negative on the testPerson without the diseasePerson with the diseasePerson who tests positivePerson who tests negativeTrue positive on the testFalse positive on the testTrue negative on the testFalse negative on the testProbability = 3/10 = 30%This is the POSITIVE PREDICTIVE VALUE (the value of the test in predicting a positive result)

12 If someone is negative on the test, what are the chances that he/she does not have the disease?Person without the diseasePerson with the diseasePerson who tests positivePerson who tests negativeTrue positive on the testFalse positive on the testTrue negative on the testFalse negative on the testPerson without the diseasePerson with the diseasePerson who tests positivePerson who tests negativeTrue positive on the testFalse positive on the testTrue negative on the testFalse negative on the testProbability = 89/90 = 99%This is the NEGATIVE PREDICTIVE VALUE (the value of the test in predicting a negative result)

13 SENSITIVITY, SPECIFICITY AND PREDICTIVE VALUESFor sensitivity and specificity, the reference variable (‘denominator) is the DISEASEFor predictive value, the reference variable (‘denominator’) is the TEST

15 IN EVERY 100 PEOPLE, 4 WILL HAVE THE DISEASEPopulation100Disease +4Disease -96If these 100 people are representative of the population at risk, the assessed rate of those with the disease (4%) represents the PREVALENCE of the disease – it can also be considered the PRE-TEST PROBABILITY of having the disease

16 OF THE 4 PEOPLE WITH THE DISEASE, THE TEST WILL DETECT 3Population100Disease +4Disease -96In other words, the sensitivity is 75%Test +3Test -1

17 AMONG THE 96 PEOPLE WITHOUT THE DISEASE, 7 WILL TEST POSITIVEIn other words, the specificity is 93%Population100Disease +4Disease -96Test +3Test -1Test +7Test -89

18 AMONG THOSE WHO TEST POSITIVE, 3 IN 10 WILL ACTUALLY HAVE THE DISEASEThis is also the POST-TEST PROB- ABILITY of having the diseasePopulation100Disease +4Disease -96POSITIVEPREDICTIVEVALUE = 30%Test +3Test +7Test -1Test -89

19 AMONG THOSE WHO TEST NEGATIVE, 89 OF 90 WILL NOT HAVE THE DISEASEPopulation100Disease +4Disease -96Test +3Test +7NEGATIVEPREDICTIVEVALUE = 99%Test -1Test -89

20 CONVERSELY, IF SOMEONE TESTS NEGATIVE, THE CHANCE OF HAVING THE DISEASE IS ONLY 1 IN 90Population100Disease +4Disease -96Test +3Test +7Test -1Test -89

21 PREDICTIVE VALUES AND CHANGING PREVALENCEPopulation1000Disease +4Disease -996Prevalence reduced by an order of magnitude from 4% to 0.4%

25 PREDICTION OF LOW PREVALENCE EVENTSEven highly specific tests, when applied to low prevalence events, yield a high number of false positive resultsBecause of this, under such circumstances, the Positive Predictive Value of a test is lowHowever, this has much less influence on the Negative Predictive Value

26 RELATIONSHIP BETWEEN PREVALENCE AND PREDICTIVE VALUEDifference between PPV and NPV relatively smallDifference between PPV and NPV relatively largeBased on a test with 90% sensitivity and 82% specificity

27 RELATIONSHIP BETWEEN PREVALENCE AND PREDICTIVE VALUEBased on a test with 75% sensitivity and 93% specificity

29 LIKELIHOOD Population 100 Disease + 4 Test + 3 Test - 1The likelihood that someone with the disease will have a positive test is ¾ or 75%This is the same as the sensitivityTest +3Test -1

30 LIKELIHOOD II Population 100 Disease - 96 Test + 7 Test - 89The likelihood that someone without the disease will have a positive test is 7/96 or 7%This is the same as the (1-specificity)Test +7Test -89

31 LIKELIHOOD RATIO LIKELIHOOD OF POSITIVE TEST GIVEN THE DISEASE=LIKELIHOOD OF POSITIVE TESTIN THE ABSENCE OF THE DISEASESENSITIVITY1- SPECIFICITY0.750.07=== 10.7A Likelihood Ratio of 1.0 indicates an uninformative test (occurs when sensitivity and specificity are both 50%)The higher the Likelihood Ratio, the better the test (other factors being equal)

35 PRE-TEST ODDSIn the sample as a whole, the odds of having the disease are 4 to 96 or 4% (the PRE-TEST ODDS)

36 POST-TEST ODDSIn the sample as a whole, the odds of having the disease are 4 to 96 or 4% (the PRE-TEST ODDS)In those who score positive on the test, the odds of having the disease are 3 to 7 or 43% (the POST-TEST ODDS)

37 POST-TEST ODDSIn the sample as a whole, the odds of having the disease are 4 to 96 or 4% (the PRE-TEST ODDS)In those who score positive on the test, the odds of having the disease are 3 to 7 or 43% (the POST-TEST ODDS)In those who score negative on the test, the odds of having the disease are 1 to 89 or approximately 1%

38 DIAGNOSTIC ODDS RATIOPotentially useful as an overall summary measure, but only in conjunction with other measures (LR, sensitivity, specificity)The Diagnostic Odds Ratio is the ratio of odds of having the diagnosis given a positive test to those of having the diagnosis given a negative test

40 LIKELIHOOD RATIO AND PRE- AND POST-TEST PROBABILITIESFor a given test with a given likelihood ratio, the post-test probability will depend on the pre-test probability (that is, the prevalence of the condition in the sample being assessed)

42 SENSITIVITY ANALYSIS OF A DIAGNOSTIC TESTValue95% CIPre-test probability35%26% to 44%Likelihood ratio5.03.0 to 8.5Applying the 95% confidence intervals above to the nomogram, the post-test probability is likely to lie in the range 55-85%

43 RECEIVER OPERATING CHARACTERISTIC CURVEThe diagonal line (representing Sensitivity=0.5 and Specificity=0.5) represents performance no better than chanceRECEIVER OPERATING CHARACTERISTIC CURVEOverall shape is predicted by the reciprocal relationship between sensitivity and specificityThe closer the curve gets to Sensitivity=1 and Specificity=1, the better the overall performance of the testTRUE POSITIVE RATE (Sensitivity)Hence the area under the curve gives a measure of the test’s performanceFALSE POSITIVE RATE (1-Specificity)

44 AREA UNDER ROC CURVESSensitivity and specificity both 100% - TEST PERFECTAREA=1.0Sensitivity and specificity both 50% - TEST USELESSThe area under a ROC curve will be between 0.5 and 1.0AREA=0.5

45 AREA UNDER ROC CURVESArea = 0.7 (between 0.5 and 1.0)Consider (hypothetically) two patients drawn randomly from the DISEASE+ and DISEASE- groups respectivelyIf the test is used to guess which patient is from the DISEASE+ group, it will be right 70% of the time

46 APPLYING A DIAGNOSTIC TEST IN DIFFERENT SETTINGSThe Positive Predictive Value of a test will vary (according to the prevalence of the condition in the chosen setting)Sensitivity and Specificity are usually considered properties of the test rather than the setting, and are therefore usually considered to remain constantHowever, sensitivity and specificity are likely to be influenced by complexity of differential diagnoses and a multitude of other factors (cf spectrum bias)

48 METHOD 4: A TEST WITH NORMALLY DISTRIBUTED VALUESAssessing the performance of the test assumes that these two distributions remain constant. However, each of them will vary (particularly through spectrum or selection bias)Test cut-off% of GroupNON-DESEASEDDISEASEDNegativePositiveDegree of ‘positivity’ on test

49 PERFORMANCE OF A DIAGNOSTIC TESTNON-CASESCASESFALSE NEGATIVESTest cut-offFALSE POSITIVES% of GroupNON-DESEASEDDISEASEDNegativePositiveDegree of ‘positivity’ on test

50 MINIMISING FALSE NEGATIVES: A SENSITIVE TESTNON-CASESCASESCut-off shifted to minimise false negatives ie to optimise sensitivityCONSEQUENCES:Specificity reducedA Negative result from a seNsitive test rules out the diagnosis - snNoutTest cut-off% of GroupNON-DESEASEDDISEASEDNegativePositiveDegree of ‘positivity’ on test

51 MINIMISING FALSE POSITIVES: A SPECIFIC TESTCut-off shifted to minimise false positives ie to optimise specificityCONSEQUENCES:Sensitivity reducedA Positive result from a sPecific test rules in the diagnosis - spPinTest cut-off% of GroupNON-DESEASEDDISEASEDNegativePositiveDegree of ‘positivity’ on test