A Systematic Review of Studies Comparing Diagnostic Clinical Prediction Rules with Clinical Judgment

RESEARCH ARTICLE A Systematic Review of Studies Comparing Diagnostic Clinical Prediction Rules with Clinical Judgment Sharon Sanders*, Jenny Doust, Paul Glasziou The Centre for Research in Evidence Based Practice, Bond University, Gold Coast, 4226, Australia * Abstract Background OPEN ACCESS Citation: Sanders S, Doust J, Glasziou P (2015) A Systematic Review of Studies Comparing Diagnostic Clinical Prediction Rules with Clinical Judgment. PLoS ONE 10(6): e0128233. doi:10.1371/journal. pone.0128233 Academic Editor: Robert S. Phillips, University of York, UNITED KINGDOM Received: December 1, 2014 Accepted: April 24, 2015 Published: June 3, 2015 Copyright: © 2015 Sanders et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Data Availability Statement: All relevant data are within the paper and its Supporting Information files. Funding: SS received funding from an Australian Postgraduate Award scholarship and the Screening and diagnostic Test Evaluation Program which is supported by a National Health and Medical Research Council Program Grant (https://www. nhmrc.gov.au/). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Competing Interests: The authors have declared that no competing interests exist. Diagnostic clinical prediction rules (CPRs) are developed to improve diagnosis or decrease diagnostic testing. Whether, and in what situations diagnostic CPRs improve upon clinical judgment is unclear. Methods and Findings We searched MEDLINE, Embase and CINAHL, with supplementary citation and reference checking for studies comparing CPRs and clinical judgment against a current objective reference standard. We report 1) the proportion of study participants classified as not having disease who hence may avoid further testing and or treatment and 2) the proportion, among those classified as not having disease, who do (missed diagnoses) by both approaches. 31 studies of 13 medical conditions were included, with 46 comparisons between CPRs and clinical judgment. In 2 comparisons (4%), CPRs reduced the proportion of missed diagnoses, but this was offset by classifying a larger proportion of study participants as having disease (more false positives). In 36 comparisons (78%) the proportion of diagnoses missed by CPRs and clinical judgment was similar, and in 9 of these, the CPRs classified a larger proportion of participants as not having disease (fewer false positives). In 8 comparisons (17%) the proportion of diagnoses missed by the CPRs was greater. This was offset by classifying a smaller proportion of participants as having the disease (fewer false positives) in 2 comparisons. There were no comparisons where the CPR missed a smaller proportion of diagnoses than clinical judgment and classified more participants as not having the disease. The design of the included studies allows evaluation of CPRs when their results are applied independently of clinical judgment. The performance of CPRs, when implemented by clinicians as a support to their judgment may be different. Conclusions In the limited studies to date, CPRs are rarely superior to clinical judgment and there is generally a trade-off between the proportion classified as not having disease and the proportion PLOS ONE | DOI:10.1371/journal.pone.0128233 June 3, 2015 1 / 25 Diagnostic Clinical Prediction Rules versus Clinical Judgment of missed diagnoses. Differences between the two methods of judgment are likely the result of different diagnostic thresholds for positivity. Which is the preferred judgment method for a particular clinical condition depends on the relative benefits and harms of true positive and false positive diagnoses. Introduction Diagnostic clinical prediction rules (CPRs) are tools designed to improve clinical decision making [1]. Theoretically, CPRs, by providing objective estimates of the probability of the presence or absence of disease derived from the statistical analysis of cases with known outcomes and or by suggesting a clinical course of action, can improve the accuracy of diagnosis and or decision making. Understanding whether and in what situations CPRs improve upon clinical judgment is an important step in the evaluation of CPRs and for the acceptance of CPRs by clinicians [2]. Existing research, which has focused on the comparative performance of CPRs and clinical judgment when both judgment methods are viewed as competing alternatives, is difficult to interpret. One body of research on the relative merits of clinical and statistical prediction has consistently reported the superior accuracy of statistical models over a clinicians ability to integrate the same data and to collect and integrate their preferred data [3–5], while another, more recent body of research has found that heuristics – proposed as models of human judgment, are on occasions more accurate than statistical models [6]. It is also difficult to know how to apply the general findings of this research to clinical practice. Many of the reviews of comparative accuracy have summarised findings from diverse professional fields including finance, medicine, psychology and education. Further, judging the clinical utility of clinical judgment and CPRs requires consideration of not just overall accuracy but the consequences of missed diagnoses (false negative) and false positive results. Results of the existing comparative research are generally not reported in a way that allows such evaluation. We conducted a systematic review of studies that compared the performance of diagnostic CPRs with clinical judgment or the performance of the combination of CPR and clinical judgment versus either alone in the same study participants against a current and objective reference standard. Methods This review was performed following methods detailed in the systematic review protocol (S1 Table– Study protocol) and is reported in line with the PRISMA Statement (S2 Table – PRISMA checklist). Data sources and searches We searched MEDLINE, Embase and CINAHL from inception to January 2012, with an updated MEDLINE search to March 2013 (S3 Table – Electronic database search strategies). No limits were applied to the database searches. We also searched for systematic reviews of diagnostic CPRs using PubMed Clinical Queries. The reference lists of systematic reviews and the included studies were checked. We conducted forward searches of included studies using Science Citation Index Expanded in Web of Science and checked related citations using PubMed's Related Citations link. PLOS ONE | DOI:10.1371/journal.pone.0128233 June 3, 2015 2 / 25 Diagnostic Clinical Prediction Rules versus Clinical Judgment Study selection We included studies that compared the CPRs with clinical judgment (...truncated)