Sensitivity and specificity: Imperfect predictors of guideline utility in radiology

Objective: Many "guideline-development studies" have presented the sensitivity (SN) and specificity (SP) of a new decision tool to describe the potential improvements in utilisation of imaging techniques as a result of adopting the new guideline. However, SN and SP are measures designed to assess how well a new guideline compares with a gold standard. These measures do not evaluate how many patients with a positive test actually have the disease; nor do they evaluate how many patients with a negative test do not have the disease. To evaluate these characteristics of a decision tool, other measures, namely the positive predictive value (PPV) and negative predictive value (NPV), should be calculated. This report highlights some of the main methodological challenges in interpretation of the studies that attempt to evaluate the development of an imaging guideline and the effectiveness of an imaging guideline in real world practice. Methods: We define four key measures of a decision tool: SN, SP, PPV and NPV. Using data from two hypothetical populations, we explain how these measures can be calculated and interpreted. We place special emphasis on the purpose of and differences between the SN-SP and PPV-NPV. Results: Borrowing information from two studies, we demonstrate how these measures should be used in the radiology healthcare services research to evaluate decision guidelines. Conclusion: The use of appropriate measures for the specific question at hand will ensure the guidelines are useful, safe, cost reducing and effective clinical tools.