Does GEM-encoding clinical practice guidelines improve the quality of knowledge bases? A study with the rule-based formalism.

Does GEM-Encoding Clinical Practice Guidelines Improve the Quality of Knowledge Bases? A Study with the Rule-Based Formalism Gersende Georg, Brigitte Séroussi, and Jacques Bouaud Mission Recherche en Sciences et Technologies de l’Information Médicale, DPA / DSI / AP-HP, Paris, France & INSERM ERM 202, UFR Broussais - Hôtel-Dieu, Université Paris 6, Paris, France The aim of this work was to determine whether the GEM-encoding step could improve the representation of clinical practice guidelines as formalized knowledge bases. We used the 1999 Canadian recommendations for the management of hypertension, chosen as the knowledge source in the ASTI project. We first clarified semantic ambiguities of therapeutic sequences recommended in the guideline by proposing an interpretative framework of therapeutic strategies. Then, after a formalization step to standardize the terms used to characterize clinical situations, we created the GEM-encoded instance of the guideline. We developed a module for the automatic derivation of a rule base, BR-GEM, from the instance. BR-GEM was then compared to the rule base, BR-ASTI, embedded within the critic mode of ASTI, and manually built by two physicians from the same Canadian guideline. As compared to BR-ASTI, BR-GEM is more specific and covers more clinical situations. When evaluated on 10 patient cases, the GEM-based approach led to promising results. INTRODUCTION Clinical practice guidelines (CPGs) have been elaborated to reduce practice variations among physicians and thus improve the quality of care. They are originally textual documents usually structured as a set of specific clinical situations for which evidence-based therapies are recommended. As the simple dissemination of guidelines had no impact on physician compliance with recommendations,1 guideline knowledge is currently embedded within knowledge bases (KBs) of computer-based decision support systems (DSSs) that provide patient-specific recommendations at the point-of-care. Original CPGs are expressed in natural language and usually suffer from incompleteness, ambiguities and imprecision. These drawbacks result in interpretation variations of guideline content during the formalization step of CPGs prior to the development of KBs. ASTI2 (“Aide à la Stratégie Thérapeutique Informatisée”) is a French project which aim is to develop a guideline-based DSS to be used in primary care. It has been first applied to the management of hypertension. The KB used in the critic mode is modeled as a set of production rules that has been manually encoded by two physicians from the 1999 Canadian recommendations for the management of hypertension.3 We have used the Guideline Elements Model4 (GEM), proposed as a document-based model, to develop a new rule base from the same CPG. The aim of our work is to compare GEM-based production rules to those manually encoded by physicians to check whether the GEM-encoding step has an impact on the quality of the rule base produced. BACKGROUND The translation of medical knowledge, originally expressed in textual CPGs to KBs is currently manually processed. Once formalized, guideline knowledge may be easily represented. A variety of representation models have been published to facilitate computer-based implementation of guideline knowledge. The oldest one, and the most widely used, is the Arden Syntax5 in which Medical Logic Modules (MLMs) support clinical decision by the generation of alerts and reminders. More recently, the Guideline Interchange Format6 (GLIF) proposes to model guideline content as a flowchart of structured steps representing clinical actions and decisions. However, the formalization step relies on a human interpretation of the guideline which is subject to variations according to the developer's experience, competence, and medical expertise.7 A study using GLIF showed that representations encoded by different subjects were different both in content and structure. Intended to serve as a document model of CPGs, GEM4 proposed to make direct use of the guideline document structure to improve guideline content interpretation. By describing pertinent concepts to guideline representation, attributes of these concepts and relationships among them, GEM aims at promoting translation of textual guidelines into a format that can be processed by computers. However, substantial variation is still observed in the AMIA 2003 Symposium Proceedings − Page 254 creation of a GEM-encoded instance from a given CPG by different subjects.8 Few works have been published to propose a methodology to formally compare KBs. KBs are often simply analyzed in terms of coverage, and level of specificity, e.g. quantitative information.9 For instance, Del Fiol et al.10 proposed an evaluation of two drug KBs developed in different academic medical centers. The same inference module was applied to the two KBs to check for drug interactions in a database of drug prescriptions. The aim of our work is to measure the impact of GEM-encoding. We thus compare two KBs represented as production rules and built from the same guideline document, e.g. the 1999 Canadian recommendations for the management of hypertension.3 The first KB has been classically manually encoded to be used within the critic mode of ASTI. The second KB has been automatically derived from the GEM-encoded instance of the guideline document. MATERIAL ASTI project The ASTI2 project aims at designing a guidelinebased DSS to enable general practitioners to avoid prescription errors and to improve compliance with best therapeutic practices. The "critic mode" operates as a background process and corrects the physician's prescription on the basis of automatically triggered rules that account for isolated guideline recommendations. The KB is formalized as “IFTHEN” production rules, and has been manually built from the Canadian CPGs3 by two physicians of the project. IF-parts of the rules represent clinical situations descriptions. They are composed of a set of inclusion criteria, e.g. patient state, pathology, and current therapy, and, exclusion criteria, e.g. pathologies that the patient is not suffering from, as well as the current therapeutic level of intention, e.g. the rank of the current treatment step in the therapeutic strategy. THEN-parts correspond to the set of recommended actions and include the grade of the recommendation. 1999 Canadian recommendations for the management of hypertension The 1999 Canadian recommendations for the management of hypertension3 is the guideline chosen by the ASTI project as the knowledge source for the development of KBs. It is a textual guideline document, well structured in chapters that correspond to specific clinical situations for which an ordered sequence of therapeutic recommendations is proposed. As it is usually the case, the guideline suffers from incompleteness, e.g. no recommendation for complex poly-pathological patient conditions, and ambiguities, e.g. the terms (...truncated)