Manual and automated methods for identifying potentially preventable readmissions: a comparison in a large healthcare system

BMC Medical Informatics and Decision Making, Apr 2014

Background Identification of potentially preventable readmissions is typically accomplished through manual review or automated classification. Little is known about the concordance of these methods. Methods We manually reviewed 459 30-day, all-cause readmissions at 18 Kaiser Permanente Northern California hospitals, determining potential preventability through a four-step manual review process that included a chart review tool, interviews with patients, their families, and treating providers, and nurse reviewer and physician evaluation of findings and determination of preventability on a five-point scale. We reassessed the same readmissions with 3 M’s Potentially Preventable Readmission (PPR) software. We examined between-method agreement and the specificity and sensitivity of the PPR software using manual review as the reference. Results Automated classification and manual review respectively identified 78% (358) and 47% (227) of readmissions as potentially preventable. Overall, the methods agreed about the preventability of 56% (258) of readmissions. Using manual review as the reference, the sensitivity of PPR was 85% and specificity was 28%. Conclusions Concordance between methods was not high enough to replace manual review with automated classification as the primary method of identifying preventable 30-day, all-cause readmission for quality improvement purposes.

A PDF file should load here. If you do not see its contents the file may be temporarily unavailable at the journal website or you do not have a PDF plug-in installed and enabled in your browser.

Alternatively, you can download the file locally and open with any standalone PDF reader:

http://www.biomedcentral.com/content/pdf/1472-6947-14-28.pdf

Manual and automated methods for identifying potentially preventable readmissions: a comparison in a large healthcare system

Ana H Jackson 0 Emily Fireman 0 Paul Feigenbaum 2 Estee Neuwirth 0 Patricia Kipnis 1 Jim Bellows 0 0 Care Management Institute, Kaiser Permanente , Oakland, California , USA 1 Management Information and Analysis, Kaiser Permanente Northern California , Oakland, California , USA 2 The Permanente Medical Group , Oakland, California , USA Background: Identification of potentially preventable readmissions is typically accomplished through manual review or automated classification. Little is known about the concordance of these methods. Methods: We manually reviewed 459 30-day, all-cause readmissions at 18 Kaiser Permanente Northern California hospitals, determining potential preventability through a four-step manual review process that included a chart review tool, interviews with patients, their families, and treating providers, and nurse reviewer and physician evaluation of findings and determination of preventability on a five-point scale. We reassessed the same readmissions with 3 M's Potentially Preventable Readmission (PPR) software. We examined between-method agreement and the specificity and sensitivity of the PPR software using manual review as the reference. Results: Automated classification and manual review respectively identified 78% (358) and 47% (227) of readmissions as potentially preventable. Overall, the methods agreed about the preventability of 56% (258) of readmissions. Using manual review as the reference, the sensitivity of PPR was 85% and specificity was 28%. Conclusions: Concordance between methods was not high enough to replace manual review with automated classification as the primary method of identifying preventable 30-day, all-cause readmission for quality improvement purposes. - Background Hospital readmissions are expensive and may reflect poor quality care. Under the new Readmissions Reduction Program, the U.S. Centers for Medicare and Medicaid Services reduces payments to hospitals with excess 30-day readmission rates [1]. Many hospitals are therefore interested in identifying preventable readmissions and understanding how they can be prevented. Classifying readmissions as potentially preventable or not preventable can be used to improve hospital performance. Administrators can sort potentially preventable readmissions into categories that are actionable for improvement. They can identify trends over time or across reporting units. Classifying readmissions as potentially preventable or not preventable can also be used to establish accountability across reporting units and reward top performers. In a recent meta-analysis of 16 studies, the median proportion of 30-day readmissions that were judged as avoidable was 21.6% [2]. The range was 5% to 59% [2-4]. The methods used to measure potential preventability vary greatly, but most involve manual chart review by at least one reviewer [2,5]. Manual review is labor intensive and subjective. To address these shortcomings, automated software classification programs have been developed that rely on administrative data to identify potential preventability [5,6]. Automated classification offers the prospect of greater efficiency and consistency. However, automated classification has been found to identify more readmissions as potentially preventable than does manual review, so its validity has been questioned [5,7]. Although studies have compared manual review to automated classification, no published evidence describes the extent of agreement between methods applied to the same readmissions. We assessed the concordance between manual review and automated classification on the same set of readmissions to determine if automated classification could more efficiently identify preventable readmissions for quality improvement purposes. Methods Design We compared a manual review of readmissions to automated classification by the Potentially Preventable Readmission (PPR) software from 3 M. Manual review consisted of a multi-step process that has been described in more detail elsewhere, which was conducted to identify missed opportunities to prevent readmissions [8]. The first step was a detailed chart review conducted by trained nurse reviewers, based loosely on an expanded version of a readmissions diagnostic tool from the Institute for Healthcare Improvement [9]. Chart review data came from KP HealthConnectTM, the electronic health record (EHR). Interviews with treating physicians followed, and guided topics included their assessment of the preventability of readmission. We also interviewed patients and family caregivers in 73% of readmissions, again using an interview guide and soliciting an assessment of preventability. The same nurse reviewer conducted the chart review and interviews for each patient. In the final step of manual review, the nurse reviewer partnered with a physician reviewer to review and assess information and opinions from the chart review and interviews. They identified factors representing missed opportunities to (...truncated)


This is a preview of a remote PDF: http://www.biomedcentral.com/content/pdf/1472-6947-14-28.pdf

Ana H Jackson, Emily Fireman, Paul Feigenbaum, Estee Neuwirth, Patricia Kipnis, Jim Bellows. Manual and automated methods for identifying potentially preventable readmissions: a comparison in a large healthcare system, BMC Medical Informatics and Decision Making, 2014, pp. 28, 14, DOI: 10.1186/1472-6947-14-28