Augmenting existing deterioration indices with chest radiographs to predict clinical deterioration

PLOS ONE RESEARCH ARTICLE Augmenting existing deterioration indices with chest radiographs to predict clinical deterioration Emily Mu ID1*, Sarah Jabbour ID2, Adrian V. Dalca1,3, John Guttag1, Jenna Wiens2,4, Michael W. Sjoding4,5 a1111111111 a1111111111 a1111111111 a1111111111 a1111111111 1 Department of Computer Science and Electrical Engineering, Massachusetts Institute of Technology, Cambridge, MA, United States of America, 2 Division of Computer Science and Engineering, University of Michigan College of Engineering, Ann Arbor, MI, United States of America, 3 Martinos Center, Massachusetts General Hospital, Harvard Medical School, Boston, MA, United States of America, 4 Institute for Healthcare Policy and Innovation, University of Michigan, Ann Arbor, MI, United States of America, 5 Department of Internal Medicine, University of Michigan Medical School, Ann Arbor, MI, United States of America * Abstract OPEN ACCESS Importance Citation: Mu E, Jabbour S, Dalca AV, Guttag J, Wiens J, Sjoding MW (2022) Augmenting existing deterioration indices with chest radiographs to predict clinical deterioration. PLoS ONE 17(2): e0263922. https://doi.org/10.1371/journal. pone.0263922 When hospitals are at capacity, accurate deterioration indices could help identify low-risk patients as potential candidates for home care programs and alleviate hospital strain. To date, many existing deterioration indices are based entirely on structured data from the electronic health record (EHR) and ignore potentially useful information from other sources. Editor: Gopal Krishna Dhali, School of Digestive & Liver Diseases, Institute of Post Graduate Medical Education & Research, INDIA Objective To improve the accuracy of existing deterioration indices by incorporating unstructured imaging data from chest radiographs. Received: August 5, 2021 Accepted: January 29, 2022 Design, setting, and participants Published: February 15, 2022 Machine learning models were trained to predict deterioration of patients hospitalized with acute dyspnea using existing deterioration index scores and chest radiographs. Models were trained on hospitalized patients without coronavirus disease 2019 (COVID-19) and then subsequently tested on patients with COVID-19 between January 2020 and December 2020 at a single tertiary care center who had at least one radiograph taken within 48 hours of hospital admission. Copyright: © 2022 Mu et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Data Availability Statement: Human data for this study from Michigan Medicine is restricted and not made publicly available due to legal and privacy restrictions. A limited, de-identified version could be made available to other researchers from accredited research institutions after entering into a data use agreement with the University of Michigan. Requests will be reviewed on an individual basis by the University of Michigan Medicine Data Office for Clinical and Translation Research () led by Erin Main outcomes and measures Patient deterioration was defined as the need for invasive or non-invasive mechanical ventilation, heated high flow nasal cannula, IV vasopressor administration or in-hospital mortality at any time following admission. The EPIC deterioration index was augmented with unstructured data from chest radiographs to predict risk of deterioration. We compared discriminative performance of the models with and without incorporating chest radiographs using area under the receiver operating curve (AUROC), focusing on comparing the fraction and total patients identified as low risk at different negative predictive values (NPV). PLOS ONE | https://doi.org/10.1371/journal.pone.0263922 February 15, 2022 1 / 13 PLOS ONE Augmenting existing deterioration indices with chest radiographs to predict clinical deterioration Kaleba (). Request to access the data can be sent to Michael Sjoding (). Our machine learning models are available upon request. Requests to access the model repository can be sent to Emily Mu () The MIMICCXR pretraining dataset is available from Massachusetts Institute of Technology and available with restricted access at: https:// physionet.org/content/mimic-cxr/2.0.0/. The CheXpert pretraining dataset is publicly available and can be downloaded from: https:// stanfordmlgroup.github.io/competitions/chexpert/. Results Funding: This work was supported by NIH grants K01 HL136687 (MWS) and R01 LM013325 (JW, MWS), and MM Precision Health Award (MWS). This work was also supported by the generosity of Quanta Computer (JG, EM). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Augmenting existing deterioration indices with chest radiographs results in better identification of low-risk patients. The model augmentation strategy could be used in the future to incorporate other forms of unstructured data into existing disease models. Competing interests: The authors have declared that no competing interests exist. Data from 6278 hospitalizations were analyzed, including 5562 hospitalizations without COVID-19 (training cohort) and 716 with COVID-19 (216 in validation, 500 in held-out test cohort). At a NPV of 0.95, the best-performing image-augmented deterioration index identified 49 more (9.8%) individuals as low-risk compared to the deterioration index based on clinical data alone in the first 48 hours of admission. At a NPV of 0.9, the EPIC image-augmented deterioration index identified 26 more individuals (5.2%) as low-risk compared to the deterioration index based on clinical data alone in the first 48 hours of admission. Conclusion and relevance Introduction An essential characteristic of risk models used in medical decision-making is predictive performance, and new strategies should therefore prioritize the inclusion of all available data that could improve this accuracy. However, the vast majority of risk models analyze only structured data from the electronic health record (EHR) when making predictions [1–7]. In contrast, clinicians often synthesize information from multiple sources, such as radiological imaging and findings, laboratory test results, and clinical observations, when making clinical decisions [8]. Moreover, when radiologists are not given access to essential clinical information, their diagnostic decisions are negatively impacted [9, 10]. Because chest radiology data may contain critical information about a patient’s risk of deterioration in COVID-19 [11, 12], they are often used by clinicians. Augmenting existing deterioration indices with additional features extracted from chest radiographs might therefore have a significant impact on their ability to differentiate low and high risk patients. Large influxes of hospitalize (...truncated)