Framework for the impact analysis and implementation of Clinical Prediction Rules (CPRs)

Wallace et al. BMC Medical Informatics and Decision Making 2011, 11:62 http://www.biomedcentral.com/1472-6947/11/62 CORRESPONDENCE Open Access Framework for the impact analysis and implementation of Clinical Prediction Rules (CPRs) Emma Wallace1*, Susan M Smith1, Rafael Perera-Salazar2, Paul Vaucher3, Colin McCowan4, Gary Collins5, Jan Verbakel6, Monica Lakhanpaul7 and Tom Fahey1, for (Members of the International Diagnostic and Prognosis Prediction (IDAPP) group) Abstract Clinical Prediction Rules (CPRs) are tools that quantify the contribution of symptoms, clinical signs and available diagnostic tests, and in doing so stratify patients according to the probability of having a target outcome or need for a specified treatment. Most focus on the derivation stage with only a minority progressing to validation and very few undergoing impact analysis. Impact analysis studies remain the most efficient way of assessing whether incorporating CPRs into a decision making process improves patient care. However there is a lack of clear methodology for the design of high quality impact analysis studies. We have developed a sequential four-phased framework based on the literature and the collective experience of our international working group to help researchers identify and overcome the specific challenges in designing and conducting an impact analysis of a CPR. There is a need to shift emphasis from deriving new CPRs to validating and implementing existing CPRs. The proposed framework provides a structured approach to this topical and complex area of research. Background The International Diagnosis and Prognosis Prediction (IDAPP) group has recently been established. This collaborative group includes researchers and clinicians with an interest in Clinical Prediction Rules (CPRs). One of its objectives is to enhance the analysis and reporting of CPR research. One area of interest is the impact analysis of CPRs. An obstacle to this type of research is the lack of clear and well-disseminated methodology for the design of high quality impact studies. At a recent IDAPP workshop a sequential four-phased framework was developed to help researchers identify and overcome the specific challenges in designing and conducting an impact analysis of a CPR. This paper presents an overview of this framework. A CPR has been defined as a tool that uses a combination of history, clinical examination and diagnostic tests to stratify a patient in terms of the probability of having a target outcome [1]. CPRs may relate to * Correspondence: 1 Department of General Practice, Royal College of Surgeons in Ireland, (123 Stephen’s green) Dublin 2, Republic of Ireland Full list of author information is available at the end of the article diagnosis, prognosis or treatment and include scoring systems which predict outcomes or inform management decisions, risk calculators and may also encompass screening questionnaires. There are an increasing number of CPRs included in clinical guidelines and implemented in clinical management systems such as GP software [2]. CPRs may be assistive and therefore designed to calculate probabilities without recommending decisions or directive and designed to give specific management recommendations (Figure 1 and Figure 2). There is a widely accepted methodology for the development of CPRs [1,3]. The derivation of a CPR is the first of three steps required before it can be disseminated and used in practice. This is followed by internal and external validation (Step Two) before finally testing the impact (Step Three) of its use on clinical outcomes. These steps require cumulative levels of evidence and the adoption of several types of study designs to answer the relevant research and clinical questions (Figure 3). The increasing number of CPRs reported in the literature have a tendency to focus on the derivation stage with only a minority progressing to validation and very few undergoing impact analysis [4]. Nevertheless, impact © 2011 Wallace et al; licensee BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Wallace et al. BMC Medical Informatics and Decision Making 2011, 11:62 http://www.biomedcentral.com/1472-6947/11/62 Page 2 of 7 Risk score interpretation (probability of DVT): >/=3 points: high risk (75%); 1 to 2 points: moderate risk (17%); <1 point: low risk (3%). Wells PS, Anderson DR, Bormanis J, Guy F, Mitchell M, Gray L, Clement C, Robinson KS, Lewandowski B. Value of assessment of pretest probability of deep-vein thrombosis in clinical management. Lancet 1997,350 :1795-8 Figure 1 Alternative formats and functions of Clinical Prediction Rules (CPRs). Assistive CPR; Wells CPR for Deep Venous Thrombosis (DVT) Centor RM, Witherspoon JM, Dalton HP, Brody CE, Link K. The diagnosis of strep throat in adults in the emergency room. Med Decis Making. 1981;1(3):239-246. Figure 2 Alternative formats and functions of Clinical Prediction Rules (CPRs). Directive CPR; Centor score for sore throat Wallace et al. BMC Medical Informatics and Decision Making 2011, 11:62 http://www.biomedcentral.com/1472-6947/11/62 Page 3 of 7 Aim Implementation Broad Validation Statistical Derivation Narrow Validation Theory Identify potential predictive factors Derivate predictive model Causal Effects Validate CPR with similar conditions as derivation cohort Validate CPR in multiple different settings or for different populations Systematic review Cross-sectional Case-control Qualitative Cohort Cross-sectional Sytematic review Cohort Derivation Validation Measure effectiveness of CPR on clinical relevant outcomes using an experimental design Cluster randomised trial (CRT) Impact analysis Long term dissemination and implementation of CPR Survey Cohort CRT Description Study type Dissemination Steps Increasing level of evidence Figure 3 Theoretical framework for study designs from theory to implementation of CPRs. analysis studies remain the most valid way of assessing whether incorporating CPRs into a decision making process improves patient outcomes. There is a need to change emphasis from deriving new CPRs to validating and implementing existing CPRs. The integration of a validated CPR into routine clinical practice presents a number of challenges. These include measuring the acceptability of the CPR to clinicians, deciding how it will be delivered at the point of care and the applicability of a CPR derived in one setting to a new setting. As a result, we have developed a tailored four-phased framework based on the literature and the collective experience of our working group (Figure 4) [4-6]. Although the phases in the framework are designed to be sequential, there may be a requirement to adopt an iterative process where findin (...truncated)