Value of electronic alerts for acute kidney injury in high-risk wards: a pilot randomized controlled trial
Value of electronic alerts for acute kidney injury in high‑risk wards: a pilot randomized controlled trial
Yanhua Wu 0 1 2
Yuanhan Chen 0 1 2
Shaowen Li 0 1 2
Wei Dong 0 1 2
Huaban Liang 0 1 2
Miaoyi Deng 0 1 2
Yingnan Chen 0 1 2
Shixin Chen 0 1 2
Xinling Liang 0 1 2
0 Division of Preventive Medicine, School of Public Health, Guangzhou Medical University , 195, Dongfeng West Road, Guangzhou City 510182, Guangdong Province , China
1 Division of Information Management, Guangdong General Hospital, Guangdong Academy of Medical Sciences , 106, Zhong Shan Road 2, Guangzhou City 510080, Guangdong Province , China
2 Division of Nephrology, Guangdong General Hospital, Guangdong Academy of Medical Sciences , 106, Zhong Shan Road 2, Guangzhou City 510080, Guangdong Province , China
3 Xinling Liang
Purpose To investigate the application value of “electronic alerts” (“e-alerts”) for acute kidney injury (AKI) among highrisk wards of hospitals. Methods A prospective, randomized, controlled study was conducted. We developed an e-alert system for AKI and ran the system in intensive care units and divisions focusing on cardiovascular disease. The e-alert system diagnosed AKI automatically based on serum creatinine levels. Patients were assigned randomly to an e-alert group (467 patients) or non-e-alert group (408 patients). Only the e-alert group could receive pop-up messages. Results The sensitivity, specificity, Youden Index and accuracy of the AKI e-alert system were 99.8, 97.7, 97.5 and 98.1%, respectively. The prevalence of the diagnosis for AKI and expanded-AKI (AKI or multiple-organ failure) in the e-alert group was higher than that in the non-e-alert group (AKI 7.9 and 2.7%, P = 0.001; expanded-AKI 16.3 and 6.1%, P < 0.001). The prevalence of nephrology consultation in the e-alert group was higher than that in the non-e-alert group (9.0 and 3.7%, P = 0.001). There was no significant difference in the prevalence dialysis, rehabilitation of renal function or death in the two groups. Conclusion The e-alert system described here was a reliable tool to make an accurate diagnosis of AKI.
Acute kidney injury; e-alert; Information system; Surveillance; Diagnosis
Approximately 21% of hospitalized patients have acute
kidney injury (AKI) [
]. A multicenter epidemiologic study
showed that 1.4–2.9 million hospitalized patients developed
AKI in China in 2013 [
]. AKI is associated with adverse
outcomes. AKI may develop into chronic kidney disease
(CKD) and even end-stage renal disease. AKI is associated
with mortality and increased health-care costs and is a
financial burden to families and societies [
]. Small changes
to serum levels of creatinine (sCr) are not detected by
clinicians, so early intervention is unlikely .
In 2015, The International Society of Nephrology (ISN)
proposed the “0 by 25” initiative for AKI. “Electronic alerts”
(“e-alerts”) are key interventions in patients with AKI [
In 2016, the Acute Dialysis Quality Initiative (ADQI) stated
the importance of an early diagnosis of AKI and suggested
to build e-alerts to improve the diagnosis of AKI [
studies have shown that e-alerts can affect the behavior of
clinicians, reduce the use of nephrotoxic drugs and improve
the prognosis of AKI [
]. Wilson et al. [
] found that e-alerts
for AKI did not improve clinical outcomes among
hospitalized patients. The value of e-alerts for AKI is not clear. Here,
we evaluated the clinical value of e-alerts for AKI patients.
This was a prospective, randomized, controlled study. This
study is registered with clinicaltrials.gov (NCT02793167).
Ethical approval of the study protocol was obtained from
Guangdong General Hospital (GDREC2016164H;
The e-alert system was based on automatic
measurement of sCr levels using computer software. The e-alert
was based on Kidney Disease: Improving Global
Outcomes (KDIGO) criteria. Patients were divided randomly
into two groups: “e-alert” and “non-e-alert.” Only the
e-alert group could receive pop-up windows on dashboard
of the instrument.
During the study, trained researchers prospectively
collected the clinical data of all patients: diagnosis upon
admission and discharge from hospital; medical history;
consultation records; dialysis records; final outcome in
Interpretation and related definitions of AKI
In the present study, three classifications of AKI were
used. The first was “e-alert-confirmed,” whereby the
diagnosis was based on the sCr value. The second was
“researcher-confirmed AKI,” whereby the researcher
confirmed AKI. This type of diagnosis was considered the
“gold standard” because e-alert data and clinical data were
satisfied simultaneously. The final classification type was
“discharge-diagnosis AKI,” which was based on medical
records, and was one of the main endpoints of our study.
In this system, each sCr value could be compared with
the baseline sCr value. AKI was diagnosed on the e-alert
according to 2012 KDIGO-AKI guidelines [
baseline sCr value was based on three rules: (1) a sCr value had
been obtained in the last 2 days, and the latest value was
higher by 26.5 µmol/L than the baseline sCr value; (2) a
sCr value had not been obtained in the last 2 days, but a
sCr value had been obtained in the last 7 days and, using
the lowest value as the baseline, the latest sCr value was
50% higher than the baseline sCr value; (3) a sCr value
had not been obtained in the last 7 days, but a sCr value
had been obtained in the last 30, 90 or 365 days; using the
lowest sCr value in the above time range as the baseline,
the latest sCr value was 50% higher than the baseline sCr
value. The AKI stage was classified as 1, 2 or 3 according
to the maximum value of sCr (Table 1).
The clinical data of all cases were collected
prospectively by trained researchers. Researchers confirmed AKI
after ruling out the following conditions in the e-alert:
(1) baseline sCr > 353.6 µmol/L; (2) a history of stage-5
chronic kidney disease (CKD) or maintenance
hemodialysis; (3) kidney transplantation; (4) amputation; (5) no
clinical evidence to support a diagnosis of AKI.
For non-e-alert patients, the researchers screened for
AKI-related disease processes (e.g., oliguria) and confirmed
them to be non-AKI. The sensitivity, specificity, Youden
Index (sensitivity + specificity− 1) and accuracy of e-alerts
were calculated using researcher-confirmed AKI as the gold
Discharge-diagnosis AKI was judged according to the
International Classification of Disease (tenth revision,
clinical modification, ICD-10). Relevant codes were N17 and
N10 x00 (acute renal tubular interstitial nephritis), N14.102
(contrast nephropathy), N18.80001 (acute exacerbation of
CKD), N99.000 (kidney failure after operation) and N99.001
(kidney failure after surgery). Two ICDs for multiple-organ
dysfunction syndrome (MODS) were included in
dischargediagnosis AKI: R65.101 (infectious multiple-organ
dysfunction syndrome) and R65.301 (multiple-organ dysfunction
syndrome). The diagnostic prevalence of AKI and
expandedAKI was obtained by calculating the ratio of
discharge-diagnosis AKI and researcher-confirmed AKI.
The recovery of kidney function among discharged
patients was defined as shown in Table 2.
Development of the e‑alert system for AKI
The e-alert system was developed by the divisions of
Nephrology and Information Management of Guangdong
General Hospital (Patent Application 201610001950.5). The
e-alert system consisted of a patient filter, sCr extractor,
AKI automatic interpretation, random-number generation
and distribution, and pop-up window generator. The system
generated the random allocation sequence according to the
time of acute kidney injury.
The working principle of the e-alert system was
ingenious. First, the system screened adult patients aged
≥ 18 years who could be alerted. Upon hospitalization, the
system would compare the sCr values (including the results
of previous hospitalizations or outpatient visits). Then, the
system made a diagnosis of AKI according to KDIGO
criteria. Based on the generation of random numbers, the
system divided patients into an e-alert group and non-e-alert
group. Only the e-alert group would receive pop-up windows
(Fig. 1). The participants were blinded after assignment to
Operation of the e‑alert for AKI
The e-alert system was run in intensive care units (ICUs),
cardiology divisions (eight divisions), coronary care units
(CCUs) and cardiac surgery divisions (four wards) in
Guangdong General Hospital from July 1 to November 31, 2016.
Analyses were done using SPSS v20.0 (IBM, Armonk, NY,
USA). Continuous data are presented as the mean (standard
deviation) or median (interquartile range) as appropriate,
and categorical variables as n (%). We compared groups
using the Student’s t test for continuous variables and χ2 test
for categorical variables. For ordered classification variables
and non-normally distributed measurement data, we used the
rank sum test. All tests were two-tailed, and P < 0.05 was
During the operation of the e-alert system, 5308 patients
were screened in our hospital. There were 975 (18.3%)
e-alerts according to 2012 KDIGO criteria. At random, 513
patients received an e-alert, and the remainder (462 patients)
did not. According to the exclusion criteria, 467 patients
(in the e-alert group) and 408 patients (in the non-e-alert
group) were researcher-confirmed AKI. From 4333
non-ealert patients, clinical evidence (e.g., oliguria) was used to
diagnose AKI in two patients (Fig. 2).
There was no significant difference between the e-alert
group and non-e-alert group with regard to sex, age, age
group, baseline sCr value or Charlson score (Table 3).
Authenticity of e‑alerts
Using researcher-confirmed AKI as the gold standard, we
evaluated the authenticity of the e-alert system for AKI.
The total number of e-alerts was 976, of which 875 were
researcher-confirmed AKI. The total number of non-e-alerts
was 4333, of which 2 were researcher-confirmed AKI.
Evaluation of e-alerts revealed the sensitivity to be 99.8%,
specificity to be 97.7%, Youden Index to be 97.5% and accuracy
to be 98.1%.
Value of e‑alerts for improving the prevalence of AKI diagnosis
The prevalence of the diagnosis of AKI in the e-alert group
was higher than that for the non-e-alert group, and the
difference was significant (7.9 and 2.7%, P = 0.001). The
prevalence of the diagnosis of expanded-AKI was also higher
in the e-alert group than that in the non-e-alert group, and
the difference was significant (16.3 and 6.1%, P< 0.001).
In patients with stage-1 AKI, the prevalence of the
diagnosis of AKI and expanded-AKI was higher in the e-alert
group than in the non-alert group (AKI: 3.5 and 0.7%,
P = 0.048; expanded-AKI: 7.7 and 1.8%, P = 0.003).
However, for stage-2 and stage-3 AKI, a significant difference
was not observed in the two groups (AKI: P = 1.000, 0.181;
expanded-AKI: P = 0.555, 0.306). In a ward-stratified study,
e-alerts improved the prevalence of the diagnosis of AKI
and expanded-AKI in cardiology divisions (AKI: 5.0 and
0.0%, P = 0.011; expanded-AKI: 8.3 and 0.0%, P < 0.001)
and cardiac surgery divisions (AKI: 9.0 and 2.7%, P = 0.019;
expanded-AKI: 17.5 and 6.8%, P = 0.004). There was no
Fig. 2 Trial profile
Relationship between e‑alerts and outcome
The prevalence of nephrology consultation in the e-alert
group was higher than that in the non-e-alert group (9.0
and 3.7%, P = 0.001). There was no significant difference
in the prevalence of dialysis, rehabilitation of renal function
AKI is a common and serious clinical syndrome, especially
in the ICU. Due to the insufficiency of cardiac function,
the use of contrast agents and extracorporeal circulation
in cardiac surgery, [
] cardiology divisions become the
“frontline” in the battle against AKI.
The ISN and ADQI suggested that e-alerts should be
constructed to resolve the early diagnosis of AKI. However,
Wilson et al. [
] found that e-alerts did not improve the
clinical endpoints of patients with AKI. One of the
limitations they considered was that the role of e-alerts in different
wards might be different. Therefore, we applied e-alerts to
the high-risk wards of AKI and evaluated its value.
An accurate and timely diagnosis is prerequisites for
effective interventions. In ICUs, if AKI or other types of organ
dysfunction occur, clinicians are more likely to diagnose MODS.
Hence, we included MODS in the diagnosis of
expandedAKI. The prevalence of the diagnosis for AKI and
expandedAKI was higher in the e-alert group than that of the non-alert
group. In 2013, AKI was documented in 1.4–2.9 million
hospitalized patients [
]. The e-alert used in the present study
increased the prevalence of the diagnosis of AKI from 2.7 to
7.9%. This means that it can reduce the missed diagnosis of
72,800–150,800 cases nationwide per year in China.
Research has shown that even the clinical
manifestations of low-level AKI or “transient” AKI are associated
with dialysis, cardiovascular disease and death [
]. In the
present study, e-alerts were more effective in improving the
prevalence of the diagnosis of patients with stage-1 AKI.
AKI can occur in different hospital divisions. The
AKIrelated knowledge of physicians in different specialties is
not optimal, especially in non-ICU and surgery divisions.
The preliminary results of our study showed that e-alerts
can improve the prevalence of the diagnosis in cardiology
and cardiac surgery divisions.
In the present study, use of e-alerts reduced the prevalence
of the missed diagnosis of AKI, but the overall prevalence
was very low. There was no improvement in the prevalence
of dialysis or renal-function recovery in survivors. There
could have been three main reasons for this observation.
First, the high prevalence of AKI in high-risk wards was
not matched with monitoring of renal function, especially in
non-ICU and surgery wards. Second, some clinicians did not
understand the meaning of e-alerts. This information system
is convenient for medical management, but, with increasing
use of e-alerts for different diseases, “e-alert fatigue” may
]. Third, nephrologists did not intervene actively.
Early intervention and follow-up by nephrologists can
improve the prognosis of AKI [
]. An alert system alone is
not adequate to improve the effectiveness of AKI
management. Therefore, it is necessary to strengthen the education
of clinicians. Through meetings and lectures, we can explain
the purpose, importance and methods of the present study.
There were two study limitations. First, we lacked
follow-up data such as long-term rehospitalization and other
adverse outcomes. Second, because preadmission and in
hospital medical records are not combined in most Chinese
hospitals, it is difficult to distinguish community-acquired
AKI with e-alert.
The present study evaluated, in a preliminary fashion,
the value of application of e-alerts in AKI high-risk wards.
e-alerts reduced the prevalence of a missed diagnosis of AKI
to some extent, but there was no improvement in the main
endpoints. Our research team intends to strengthen the
training of physicians in relevant divisions of our hospital. In the
future, we will reevaluate the value of e-alerts based not only
on the inhospital outcomes, but also on long-term prognosis.
Acknowledgements This work was funded by Science and Technology
Program of Guangzhou, China (201604020037).
Author’s contribution YW designed the study, contributed to the
database, performed the statistical analysis and drafted the manuscript. YC
conceived of the study and participated in the analysis, interpreted the
data and drafted the manuscript. SL, MD and YC designed the electric
system. HL, WD and SC participated in clinical data collection and
statistics. XL edited the final manuscript.
Compliance with ethical standards
Conflict of interest The authors declare that they have no competing
Open Access This article is distributed under the terms of the
Creative Commons Attribution 4.0 International License (http://creativeco
mmons.org/licenses/by/4.0/), which permits unrestricted use,
distribution, and reproduction in any medium, provided you give appropriate
credit to the original author(s) and the source, provide a link to the
Creative Commons license, and indicate if changes were made.
1. Susantitaphong P , Cruz DN , Cerda J et al ( 2013 ) World incidence of AKI: a meta-analysis . Clin J Am Soc Nephrol 8 ( 9 ): 1482 - 1493
2. Yang L , Xing G , Wang L et al ( 2015 ) Acute kidney injury in China: a cross-sectional survey . Lancet 386 ( 10002 ): 1465 - 1471
3. Coca SG , Singanamala S , Parikh CR ( 2012 ) Chronic kidney disease after acute kidney injury: a systematic review and metaanalysis . Kidney Int 81 ( 5 ): 442 - 448
4. Kerr M , Bedford M , Matthews B et al ( 2014 ) The economic impact of acute kidney injury in England . Nephrol Dial Transplant 29 ( 7 ): 1362 - 1368
5. Aitken E , Carruthers C , Gall L et al ( 2013 ) Acute kidney injury: outcomes and quality of care . QJM 106 ( 4 ): 323 - 332
6. Mehta RL , Cerda J , Burdmann EA et al ( 2015 ) International Society of Nephrology's 0 by 25 initiative for acute kidney injury (zero preventable deaths by 2025): a human rights case for nephrology . Lancet 385 ( 9987 ): 2616 - 2643
7. James MT , Hobson CE , Darmon M et al ( 2016 ) Applications for detection of acute kidney injury using electronic medical records and clinical information systems: workgroup statements from the 15th ADQI consensus conference . Can J Kidney Health Dis 3 : 100
8. Mccoy AB , Waitman LR , Gadd CS et al ( 2010 ) A computerized provider order entry intervention for medication safety during acute kidney injury: a quality improvement report . Am J Kidney Dis 56 ( 5 ): 832 - 841
9. Wilson FP , Shashaty M , Testani J et al ( 2015 ) Automated, electronic alerts for acute kidney injury: a single-blind, parallel-group, randomised controlled trial . Lancet 385 ( 9981 ): 1966 - 1974
10. Kidney Disease: Improving Global Outcomes (KDIGO) Acute Kidney Injury Work Group ( 2012 ) KDIGO clinical practice guideline for acute kidney injury . Kidney Int Suppl 2 ( 1 ): 1 - 13
11. Uchino S , Bellomo R , Bagshaw SM et al ( 2010 ) Transient azotaemia is associated with a high risk of death in hospitalized patients . Nephrol Dial Transplant 25 ( 6 ): 1833 - 1839
12. van der Sijs H , Aarts J , Vulto A et al ( 2006 ) Overriding of drug safety alerts in computerized physician order entry . J Am Med Inform Assoc 13 ( 2 ): 138 - 147
13. Harel Z , Wald R , Bargman JM et al ( 2013 ) Nephrologist follow-up improves all-cause mortality of severe acute kidney injury survivors . Kidney Int 83 ( 5 ): 901 - 908