Manual and automated methods for identifying potentially preventable readmissions: a comparison in a large healthcare system
Ana H Jackson
0
Emily Fireman
0
Paul Feigenbaum
2
Estee Neuwirth
0
Patricia Kipnis
1
Jim Bellows
0
0
Care Management Institute, Kaiser Permanente
,
Oakland, California
,
USA
1
Management Information and Analysis, Kaiser Permanente Northern California
,
Oakland, California
,
USA
2
The Permanente Medical Group
,
Oakland, California
,
USA
Background: Identification of potentially preventable readmissions is typically accomplished through manual review or automated classification. Little is known about the concordance of these methods. Methods: We manually reviewed 459 30-day, all-cause readmissions at 18 Kaiser Permanente Northern California hospitals, determining potential preventability through a four-step manual review process that included a chart review tool, interviews with patients, their families, and treating providers, and nurse reviewer and physician evaluation of findings and determination of preventability on a five-point scale. We reassessed the same readmissions with 3 M's Potentially Preventable Readmission (PPR) software. We examined between-method agreement and the specificity and sensitivity of the PPR software using manual review as the reference. Results: Automated classification and manual review respectively identified 78% (358) and 47% (227) of readmissions as potentially preventable. Overall, the methods agreed about the preventability of 56% (258) of readmissions. Using manual review as the reference, the sensitivity of PPR was 85% and specificity was 28%. Conclusions: Concordance between methods was not high enough to replace manual review with automated classification as the primary method of identifying preventable 30-day, all-cause readmission for quality improvement purposes.
-
Background
Hospital readmissions are expensive and may reflect poor
quality care. Under the new Readmissions Reduction
Program, the U.S. Centers for Medicare and Medicaid Services
reduces payments to hospitals with excess 30-day
readmission rates [1]. Many hospitals are therefore interested in
identifying preventable readmissions and understanding
how they can be prevented.
Classifying readmissions as potentially preventable or not
preventable can be used to improve hospital performance.
Administrators can sort potentially preventable
readmissions into categories that are actionable for improvement.
They can identify trends over time or across reporting
units. Classifying readmissions as potentially preventable or
not preventable can also be used to establish accountability
across reporting units and reward top performers.
In a recent meta-analysis of 16 studies, the median
proportion of 30-day readmissions that were judged as
avoidable was 21.6% [2]. The range was 5% to 59% [2-4]. The
methods used to measure potential preventability vary
greatly, but most involve manual chart review by at least
one reviewer [2,5]. Manual review is labor intensive and
subjective. To address these shortcomings, automated
software classification programs have been developed that
rely on administrative data to identify potential
preventability [5,6]. Automated classification offers the prospect
of greater efficiency and consistency. However, automated
classification has been found to identify more
readmissions as potentially preventable than does manual review,
so its validity has been questioned [5,7]. Although studies
have compared manual review to automated classification,
no published evidence describes the extent of agreement
between methods applied to the same readmissions. We
assessed the concordance between manual review and
automated classification on the same set of readmissions to
determine if automated classification could more efficiently
identify preventable readmissions for quality improvement
purposes.
Methods
Design
We compared a manual review of readmissions to
automated classification by the Potentially Preventable
Readmission (PPR) software from 3 M. Manual review
consisted of a multi-step process that has been described in
more detail elsewhere, which was conducted to identify
missed opportunities to prevent readmissions [8]. The first
step was a detailed chart review conducted by trained nurse
reviewers, based loosely on an expanded version of a
readmissions diagnostic tool from the Institute for Healthcare
Improvement [9]. Chart review data came from KP
HealthConnectTM, the electronic health record (EHR). Interviews
with treating physicians followed, and guided topics
included their assessment of the preventability of
readmission. We also interviewed patients and family caregivers in
73% of readmissions, again using an interview guide and
soliciting an assessment of preventability. The same nurse
reviewer conducted the chart review and interviews for
each patient. In the final step of manual review, the nurse
reviewer partnered with a physician reviewer to review and
assess information and opinions from the chart review and
interviews. They identified factors representing missed
opportunities to (...truncated)