Assessing Complex Emergency Management with Clinical Case-Vignettes: A Validation Study

RESEARCH ARTICLE Assessing Complex Emergency Management with Clinical Case-Vignettes: A Validation Study Anne Rousseau1,2☯*, Patrick Rozenberg1,3☯, Philippe Ravaud2,4☯ 1 Department of Obstetrics and Gynecology, Poissy-Saint Germain Hospital, Poissy, France, 2 INSERM U1153 Research Unit, Paris Descartes-Sorbonne Paris Cité University, Paris, France, 3 Research unit EA 7285, Versailles-St Quentin University, Saint Quentin en Yvelines, France, 4 Assistance Publique-Hôpitaux de Paris, Centre d’Epidémiologie Clinique, Hôpital Hôtel-Dieu, Paris, France ☯ These authors contributed equally to this work. * Abstract Objective OPEN ACCESS Citation: Rousseau A, Rozenberg P, Ravaud P (2015) Assessing Complex Emergency Management with Clinical Case-Vignettes: A Validation Study. PLoS ONE 10(9): e0138663. doi:10.1371/journal. pone.0138663 Editor: Sheila Alexander, University of Pittsburgh, UNITED STATES Received: January 22, 2015 Accepted: September 2, 2015 Published: September 18, 2015 Copyright: © 2015 Rousseau et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Data Availability Statement: All relevant data are within the paper and its Supporting Information files. Funding: The authors have no support or funding to report. Competing Interests: The authors have declared that no competing interests exist. To evaluate whether responses to dynamic case-vignettes accurately reflect actual practices in complex emergency situations. We hypothesized that when obstetricians were faced with vignette of emergency situation identical to one they previously managed, they would report the management strategy they actually used. On the other hand, there is no reason to suppose that their response to a vignette based on a source case managed by another obstetrician would be the same as the actual management. Methods A multicenter vignette-based study was used in 7 French maternity units. We chose the example of severe postpartum hemorrhage (PPH) to study the use of case-vignettes for assessing the management of complex situations. We developed dynamic case-vignettes describing incidents of PPH in several steps, using documentation in patient files. Vignettes described the postpartum course and included multiple-choice questions detailing proposed clinical care. Each participating obstetrician was asked to evaluate 4 case-vignettes: 2 directly derived from cases they previously managed and 2 derived from other obstetricians’ cases. We compared the final treatment decision in vignette responses to those documented in the source-case by the overall agreement and the Kappa coefficient, both for the cases the obstetricians previously managed and the cases of others. Results Thirty obstetricians participated. Overall agreement between final treatment decisions in case-vignettes and documented care for cases obstetricians previously managed was 82% (Kappa coefficient: 0.75, 95% CI [0.62–0.88]). Overall agreement between final treatment decisions in case-vignettes and documented care in vignettes derived from other PLOS ONE | DOI:10.1371/journal.pone.0138663 September 18, 2015 1 / 12 Vignettes to Assess Complex Emergency Management obstetricians’ cases was only 48% (Kappa coefficient: 0.30, 95% CI [0.12–0.48]). Final agreement with documented care was significantly better for cases based on their own previous cases than for others (p<0.001). Conclusions Dynamic case-vignettes accurately reflect actual practices in complex emergency situations. Therefore, they can be used to assess the quality of management in these situations. Introduction The first step in improving clinical practices is assessing their quality, preferably by a simple, feasible, and accurate method. Several methods have been used: standardized patients (trained actors), high fidelity simulation, chart abstraction, clinical audits, and clinical case vignettes. Clinical vignettes are short, clear texts that describe realistic clinical situations so that physicians can assess identical scenarios. Most often, vignettes have been used to survey practices, or to assess opinions or preferences [1–4]. Vignettes are intended to assess both physicians’ knowledge and their actual practices [5,6]. Peabody et al [7,8] have concluded that vignettes are a valid tool for measuring the quality of clinical practice, compared with standardized (actor) patients or chart abstraction. They measure quality of care better than chart abstraction does. Vignettes are easy to use and more cost-effective than standardized patients, high fidelity simulation, or even clinical audits [7–10]. Moreover, it is easier to control case-mix variation in vignettes than in data sets. They have thus been widely used across countries, health care systems, and specialties [11–13]. Case-vignettes have been used and validated to analyze practices such as screening, diagnosis, care, assessment of prognosis, and ethical decision making [3,5,6,14–17]. They have not, however, been validated to assess the complex management strategies often observed in emergency situations. Dynamic multistage vignettes can be constructed to assess clinical practices in such situations. The objective of our study was to determine whether such vignettes accurately reflect what physicians do in real complex emergency situations. We focused on the situation of severe postpartum hemorrhage (PPH) because it is a common, complex emergency situation that occurs in about 1 to 2% of deliveries in developed countries [18,19]. Materials and Methods This multicenter cross-sectional study took place in October to November 2012. Our purpose was to determine whether responses to dynamic vignettes reflect actual practices in managing complex PPH. We hypothesized that when obstetricians were faced with a vignette of a situation of severe PPH identical to one they had previously managed, they would report the management strategy they had actually used. Conversely, we hypothesized that in cases not identical to those they had handled, they would not reproduce a strategy identical to that of the other obstetrician who did manage it. The study was intentionally conducted independently of quality of practices because our aim was simply to verify that obstetricians use the same practices in actual situations as they report in vignettes. Vignette Construction Vignettes were developed by abstracting information from patient files. We retrospectively selected patient files from 7 maternity units in the Paris area—6 public university hospitals, and 1 public non-university hospital. We identified the names and number of senior PLOS ONE | DOI:10.1371/journal.pone.0138663 September 18, 2015 2 / 12 Vignettes to Assess Complex Emergency Management obstetricians in each unit and then reviewed each hospital’s birth register to a (...truncated)