Cognitive screening among acute respiratory failure survivors: a cross-sectional evaluation of the Mini-Mental State Examination
Pfoh et al. Critical Care
Cognitive screening among acute respiratory failure survivors: a cross-sectional evaluation of the Mini-Mental State Examination
Elizabeth R Pfoh
Kitty S Chan
Victor D Dinglas
Timothy D Girard
James C Jackson
Peter E Morris
Catherine L Hough
Pedro A Mendez-Tellez
E Wesley Ely
Dale M Needham
Ramona O Hopkins 0 1
with the NIH NHLBI ARDS Network
0 Psychology Department and Neuroscience Center, Brigham Young University , 1022 SWKT, Provo, UT 84602 , USA
1 Division of Pulmonary and Critical Care Medicine, Intermountain Medical Center , 5121 Cottonwood Street, Murray, UT 84107 , USA
Introduction: The Mini-Mental State Examination (MMSE) is a common cognitive screening test, but its utility in identifying impairments in survivors of acute respiratory failure is unclear. The purpose of this study was to evaluate MMSE performance versus a concurrently administered detailed neuropsychological test battery in survivors of acute respiratory failure. Methods: This cross-sectional analysis used data from the ARDSNet Long Term Outcomes Study (ALTOS) and Awakening and Breathing Controlled Trial (ABC). Participants were 242 survivors of acute respiratory failure. The MMSE and detailed neuropsychological tests were administered at 6 and 12 months post-hospital discharge for the ALTOS study, and at hospital discharge, 3 and 12 months for the ABC study. Overall cognitive impairment identified by the MMSE (score <24) was compared to impairments identified by the neuropsychological tests. We also matched orientation, registration, attention, memory and language domains on the MMSE to the corresponding neuropsychological test. Pairwise correlations, sensitivity, specificity, positive and negative predictive values, and agreement were assessed. Results: Agreement between MMSE and neuropsychological tests for overall cognitive impairment was fair (42 to 80%). Specificity was excellent (≥93%), but sensitivity was poor (19 to 37%). Correlations between MMSE domains and corresponding neuropsychological tests were weak to moderate (6 months: r = 0.11 to 0.28; 12 months: r = 0.09 to 0.34). The highest correlation between the MMSE and neuropsychological domains was for attention at 6 months (r = 0.28) and language at 12 months (r = 0.34). Conclusions: In acute respiratory failure survivors, the MMSE has poor sensitivity in detecting cognitive impairment compared with concurrently administered detailed neuropsychological tests. MMSE results in this population should be interpreted with caution.
Many survivors of acute respiratory failure experience
long-term cognitive impairments across a variety of
cognitive domains, including attention, memory, mental
processing speed and executive function [1-6]. To evaluate
such impairments, comprehensive neuropsychological test
batteries are commonly used [7-11]. However, such
batteries require trained personnel, expensive licensing fees and
several hours per patient for test administration and
scoring. Hence, valid cognitive screening tests that are
inexpensive, brief and easy to administer would be invaluable
in identifying which survivors may develop long-term
cognitive impairment . Furthermore, cognitive screening
tests may aid in understanding risk factors and trajectories
of cognitive impairments [5,12], and may facilitate the
development and evaluation of targeted interventions to
address cognitive impairments [13,14].
The Mini-Mental State Examination (MMSE) is a
commonly used screening test for cognitive impairment
in both clinical practice [15,16], and research [17-24].
Among older adults, the MMSE is the most widely used
cognitive screening test, with a pooled sensitivity of 88%
and specificity of 86% . Moreover, the MMSE has
good performance in identifying mild cognitive
impairment in older adults , identifying subtypes of mild
cognitive impairment  and predicting cognitive
impairment in patients with post-operative delirium .
As a result of these favorable performance
characteristics, the MMSE has been commonly used to assess
cognitive outcomes in critical care populations [17-22,24].
However, the performance of the MMSE has not been
specifically evaluated in acute respiratory failure
survivors. Understanding whether the MMSE can accurately
screen for cognitive impairment in this patient
population would provide important new insights. Hence, the
objective of this study is to assess whether the MMSE
can detect cognitive impairment, as assessed by a
concurrently administered, detailed neuropsychological test
battery, in survivors of acute respiratory failure.
Additionally, this study explored whether the timing of patient
follow-up assessment after hospital discharge or patient
characteristics influenced the relationship between the
MMSE and the neuropsychological test battery.
We conducted a cross-sectional secondary analysis of
data from two prospective studies of acute respiratory
failure patients requiring mechanical ventilation in an
ICU. The data from the first study came from the
National Institutes of Health-funded ARDS Network Long
Term Outcomes Study (ALTOS), which evaluated 6 and
12 month outcomes from patients enrolled in multi-site
randomized trials conducted by the ARDS Network
from July 2008 to May 2012 [19,27-30]. The patients
from these trials had similar eligibility criteria and were
enrolled from the same study sites during a similar time
frame; as such the data were pooled for this study.
Patients in the ALTOS study who completed the MMSE
and a concurrent neuropsychological test battery at
either 6- or 12-month follow-up (N = 200) were included
in the current study.
To increase the generalizability of this evaluation, data
from a second study were evaluated. The Awaken and
Breathing Controlled (ABC) study evaluated long-term
neurocognitive outcomes in patients enrolled at a single
hospital in Nashville, Tennessee, between October 2003
and March 2006 . Patients from a sub-study of the
ABC study patients were included here . The
following baseline and ICU data were obtained: age, gender,
race, years of education, employment status, Charlson
comorbidity index  and psychiatric comorbidity (as
described further below), severity of illness, mechanical
ventilation duration, and ICU and hospital length of stay.
In the ALTOS study, patients were administered the
MMSE and a standardized neuropsychological test
battery at 6- and 12-month follow-up . The MMSE was
administered via telephone using a validated 26-item
version , while the neuropsychological battery was
administered in person by research assistants who
underwent detailed training and quality assurance evaluations
by RH. The telephone version of the MMSE differs from
the in-person MMSE in that it excludes the following
items: 1) A question about orientation to place, which asks
‘What floor of the building are we on?’; 2) A question
related to following commands which asks the patient to
‘close your eyes’; 3) A question related to simple sentence
construction that asks the patient to ‘write a sentence’ 4)
A question related to visuospatial construction that asks
the patient to ‘copy overlapping pentagons’. Additionally,
the telephone version uses only one naming item, as
opposed to three items in the standard MMSE . The
MMSE telephone score was converted to the standard
30-point scale of the standard MMSE using previously
published methods . The MMSE assesses overall
cognitive impairment based on five cognitive domains: 1)
orientation (orientation to time and place), 2) registration
(registration of three words), 3) attention (backward
spelling of ‘WORLD’), 4) memory (recall of three words)
and 5) language (naming, repetition and following a
three-stage command) .
Similar cognitive domains were identified from the
neuropsychological test battery, which is made up of
widely used neuropsychological tests with known
reliability and validity [35-37]. The domains included: 1)
orientation from the Neurobehavioral Cognitive Status
Examination orientation total score ; 2) registration
from the Digit Span’s raw forward score from the
Wechsler Adult Intelligence Scale-Third Edition [39,40];
3) attention from the Digit Span’s total age-adjusted
score and the raw forward and backward scores from
the Wechsler Adult Intelligence Scale-Third Edition
[39,40]; 4) memory from the Logical Memory I and
Logical Memory II age-adjusted scores from the Wechsler
Memory Scale-Third Edition [39,40] and 5) language via
the Verbal Fluency test . MMSE scores for each
domain were compared to scores for the corresponding
neuropsychological test (recognizing there is not
necessarily a one-to-one correspondence between the tests in
the domains of interest between the MMSE and
neuropsychological test battery). We also compared overall
cognitive impairment between the MMSE and the
neuropsychological test battery.
Presence of baseline psychiatric comorbidity was
present if the patient had any of the following (based on
medical chart review): current or past excessive alcohol use,
illicit drug use or drug rehabilitation, diagnosis or
treatment for psychiatric disorder or documented history of
depression or anxiety [19,42]. Depression and anxiety
were assessed at 6 and 12 months using the Hospital
Anxiety and Depression Scale, which provides separate
subscale scores for depression and anxiety. Scores of eight or
higher indicate probable depression or anxiety [19,43].
In the ABC study, patients were administered the MMSE
and a detailed neuropsychological test battery in person at
hospital discharge, and at 3- and 12-month follow-up, by
a single neuropsychologist . To examine
generalizability of the primary ALTOS study analyses, we
evaluated the ability of the MMSE to detect overall cognitive
impairment compared to a similar neuropsychological
test battery in the ABC study . The ABC study test
battery included: 1) Digit Span and Digit Symbol from
the Wechsler Adult Intelligence Scale-Third Edition [39,40];
2) the Rey Auditory Verbal Learning Test (RAVLT) ;
3) Rey-Osterreith Complex Figure-Copy and Delayed Re
call ; 3) Trail Making Test Parts A and B  and 4)
Verbal Fluency test . We also used data from the ABC
study to compare the MMSE attention domain with the
Digit Span scores .
Cross-sectional data analysis compared MMSE scores
with neuropsychological test scores at 3, 6 and 12 months.
Cognitive impairment was defined for the MMSE as total
scores <27 and <24, which represent widely used, highly
sensitive and specific cutoff scores for cognitive
impairment [17,48]. Sensitivity and specificity were similar when
comparing total MMSE scores of <27 and <24 versus
neuropsychological test scores; therefore, we only report
the more conservative cutoff of <24. For the
neuropsychological test batteries, cognitive impairment was
conservatively defined, as done in prior studies [19,27,31], as
having at least one cognitive test ≥2 standard deviations
below population norms (bottom 2.5%), or at least two
tests ≥1.5 standard deviations below population norms
(bottom 6.7% for both tests). The ability of the MMSE to
detect cognitive impairment was evaluated using
sensitivity, specificity, positive and negative predictive values, and
agreement. The area under the receiver operating
characteristics curves was also used to examine the ability of the
total MMSE score to discriminate between survivors with
and without cognitive impairment, as defined by the
neuropsychological test battery [19,27-31].
Pearson correlations were used to examine the
relationship between MMSE cognitive domains and the
corresponding cognitive domains from the
neuropsychological test results. We examined whether these
relationships were affected by patient characteristics. Correlations
were calculated within relevant patient subgroups defined
by age (dichotomized at the mean age of 50 years), gender,
education level (dichotomized at the mean of 12 years,
that is, high school graduation), depression and anxiety
status at 6 and 12 months using the Hospital Anxiety and
Depression Scale , and pre-existing psychiatric
comorbidity prior to acute respiratory failure. Statistical
significance was defined as P ≤0.05 and statistical analyses were
completed using STATA 12.0 (StataCorp, College Station,
TX, USA). This study was conducted in accordance with
the amended Declaration of Helsinki. All studies obtained
informed consent from participants, and were approved
by relevant institutional review boards (ALTOS: Johns
Hopkins School of Medicine Institutional Review Board-5
approval number: NA_00013113; ABC: Vanderbilt
University and Saint Thomas Hospital Institutional Review
Board number: 030803).
In the ALTOS study, of the 200 eligible patients, 181
and 174 patients completed both the MMSE and the
neuropsychological tests at 6 and 12 months,
respectively. In the ABC study, 61, 69 and 47 participants
completed both the MMSE and neuropsychological tests at
hospital discharge, 3 months and 12 months,
respectively. Respondents in both studies were predominately
white, middle-aged and male, with a mean of
approximately 12 years of education (Table 1). Pre-ICU
psychiatric comorbidity was common, with a prevalence of
42% in the ALTOS study and 74% in the ABC study. All
patients were mechanically ventilated, and the mean
ICU length of stay was 14 and 10 days in the ALTOS
and ABC studies, respectively. For the ALTOS study,
12% and 10% of patients had MMSE scores <24 at 6 and
12 months, respectively; while 38% and 25% of patients
had cognitive impairments based on the
neuropsychological test scores at 6 and 12 months. In the ABC study,
the percentage of patients with an MMSE score <24 was
33%, 17% and 13% at discharge, 3 months and 12 months
respectively; while 90%, 79% and 71% had cognitive
impairments based on the neuropsychological test scores at
these same time points, respectively.
Comparing overall cognitive impairment
For the ALTOS study, at 6 and 12 months, overall
agreement was 67% and 80%, respectively. Kappa indicated
poor concordance after accounting for chance
agreement (0.19 and 0.34). At 6 and 12 months, the specificity
of the MMSE was excellent (93% and 97%, respectively);
however, sensitivity was poor (24% and 30%,
respectively), and positive predictive values (67% and 76%,
Table 1 Patient characteristics
Age, mean (SD) years
Education years, mean (SD)
Employed, n (%)
Charlson Comorbidity Index, mean (SD)
Psychiatric comorbidity, n (%)
Severity of illness score, mean (SD)
Mechanical ventilation duration,
mean (SD) days
ICU length of stay, mean (SD) days
Hospital length of stay, mean (SD) days
ICU, Intensive Care Unit; SD, standard deviation; n, number.
respectively) and negative predictive values (67% and
81%, respectively) were fair to moderate (Table 2). The
area under the receiver operating characteristics curve
was 0.66 (95% CI: 0.58 to 0.74) at 6 months and 0.76
(95% CI: 0.67 to 0.85) at 12 months, indicating fair
discrimination (Figure 1).
For the ABC study data, overall agreement and kappa
scores were low (ranging from 38 to 42%, and 0.09 to
0.12, respectively) when assessing concordance between
screening positive for cognitive impairment on the
MMSE, and having cognitive impairment based on the
neuropsychological test scores. Specificity and positive
3 Months MMSE <24 12 Months MMSE <24
predictive value of the MMSE was excellent, at 100% for
all three time points. Sensitivity was poor, ranging from
19 to 37%. Negative predictive values were also poor at
all three time points: 13% at discharge, 25% at 3 months
and 33% at 12 months (Table 2).
Comparing specific cognitive domains
Correlations between cognitive domains in the MMSE
compared to neuropsychological domains were weak to
moderate (6 months: 0.11 to 0.28, Table 3; 12 months:
0.09 to 0.34, Table 4), with the highest correlations
observed for attention at 6 months (r = 0.28) and language
at 12 months (r = 0.34). There was no consistent
influence of patient characteristics (gender, age, education
level, concurrent depression and anxiety status, or
preexisting psychiatric comorbidity) on the correlations in
any cognitive domain. The majority of correlations in
the subgroup analyses were ≤0.30.
Compared to the ALTOS study, the ABC study data
had slightly stronger correlations for attention at all time
periods (<0.5 versus <0.3). However, in both the ALTOS
and ABC studies, there was no consistent influence of
gender and age on correlations between MMSE and
neuropsychological attention scores (Table 5).
In cross-sectional analyses of two prospective
longitudinal studies of acute respiratory failure survivors, we
found that the MMSE had fair to moderate agreement,
both for overall cognitive impairment and for specific
Table 2 Overall cognitive impairment at discharge, 3, 6 and 12 months for MMSE versus neuropsychological test battery
Positive predictive value
Negative predictive value
MMSE, mini-mental status examination; SD standard deviation.
Impairment on neuropsychological test battery = any battery test ≤2 SD or 2+ tests ≤ 1.5 SD.
For ALTOS study data: At 6 months: N = 181; True positives = 9%; False positives = 4%; True negatives = 58%; False negatives = 29%. At 12 months: N = 174; True
positives = 7%; False positives = 2%; True negatives = 73%; False negatives = 17%.
For ABC study data: At discharge: N = 78; True positives = 33%; False positives = 0%; True negatives = 9%; False negatives = 58%. At 3 months: N = 76; True
positives =17%; False positives = 0%; True negatives = 21%; False negatives = 62%. At 12 months: N = 52; True positives = 13%; False positives = 0%; True
negatives = 29%; False negatives = 58%.
A ALTOS − 6 month data
B ALTOS − 12 month data
Figure 1 Receiver operating characteristics curve for the detection of overall cognitive impairment using the MMSE versus a detailed
neuropsychological test battery. A. ALTOS study 6-month data and B. ALTOS study 12-month data. ALTOS, ARDS Network Long Term Outcomes
Study; MMSE, Mini-Mental State Examination.
cognitive domains, compared to more detailed
neuropsychological tests. These findings were not influenced
by patient characteristics or timing of longitudinal
followup assessment. Indeed, even when accounting for gender,
age, education level, concurrent depression and anxiety
status, and pre-existing psychiatric comorbidity prior to
ICU admission, MMSE domain scores correlations were
weak to moderate with neuropsychological test scores.
The ability of the MMSE to discriminate overall cognitive
impairment was only fair based on the receiver operating
characteristics curve analysis; expressly, the MMSE had low
sensitivity. Even using a conservative cutoff score of <24,
the MMSE failed to identify a substantial portion of
survivors who had cognitive impairment identified on
neuropsychological tests, with poor to moderate negative
predictive values. Generally, a screening tool should have
an area under the ROC curve of at least >0.8 to
demonstrate excellent diagnostic accuracy , and the MMSE
failed to meet this threshold at both 6 and 12 months.
These results are similar to previous work in cardiac
surgery  and bariatric surgery  patients. Understanding
the limited extent of the MMSE’s agreement with
neuropsychological test scores administered concurrently has
important implications. Specifically, screening negative on the
MMSE for cognitive impairment does not rule out the
presence of cognitive impairment for acute respiratory
failure survivors at 3, 6 and 12 months.
There is great interest in screening for cognitive
impairment in ICU survivors given the frequent and long-lasting
cognitive impairments observed . Our study provides
additional evidence that the MMSE may not be a valid
cognitive screening tool for use in this patient population.
Evaluation of other cognitive screening instruments (such
as the Montreal Cognitive Assessment) in this population
is needed to determine if they will better identify cognitive
impairments . Recent work found the Montreal
Cognitive Assessment is more sensitive in accurately
differentiating mild cognitive impairment from normal cognitive
function . Another approach could include augmenting
the MMSE with more targeted instruments. For example,
adding a measure of executive function to the MMSE can
improve the detection of cognitive deficits .
Depression Anxiety Psychiatric
Depression Anxiety Psychiatric
Table 5 Correlations of MMSE domain scores with corresponding neuropsychological test by demographic variables,
ABC study data
Corresponding neuropsychological test
(n = 19)
(n = 10)
(n = 44)
(N = 61)
(n = 68–69)
(n = 37)
(n = 31–32)
(n = 49–50)
(n = 45–47)
(n = 21–22)
(n = 24–25)
(n = 35–37)
3 Months: Attention 12 Months: Attention
Digit Span total score
Digit Span forward
Digit Span backward
Digit Span total score
Digit Span forward
Digit Span backward
Digit Span total score
Digit Span forward
Digit Span backward
The MMSE is characterized by highly selective coverage
of cognitive domains and does not evaluate learning,
delayed memory, processing speed and executive function,
among other cognitive domains. Many of these omitted
cognitive domains reflect the frontal-subcortical white
matter-mediated impairments known to occur in critical
illness, and commonly identified in ICU survivors
[4,14,27,31,56]. Hence, their omission in a short screening
test is problematic. As we learn more about the granular
and specific aspects of cognitive impairment in acute
respiratory failure survivors, better cognitive screening tools
tailored to this population could be developed.
Our study is among the first to test the validity of the
MMSE in acute respiratory failure survivors. Strengths
of our study include the ability to examine overall and
domain-specific performance of the MMSE compared to
a battery of neuropsychological tests administered
concurrently. The ALTOS study provided a relatively large
multicenter sample, which employed both a screening
and neuropsychological test battery, and data from two
time points. Inclusion of the ABC study allowed us to
test the generalizability of our findings in an
independent cohort using similar neuropsychological tests. Our
study also has potential limitations. First, we were not
able to fully replicate all of the analyses from the ALTOS
data using the ABC study data due to the different
follow-up time points and neuropsychological tests used.
However, the available comparisons (overall cognitive
impairment and attention) demonstrated similar results
to the ALTOS study, helping increase confidence in
the generalizability of our findings, especially since the
studies used different neuropsychological tests. Further,
participants in the ABC study had a high baseline
psychiatric comorbidity, which is higher than the
prevalence observed in other ICU cohorts [57,58]. However,
while the prevalence of baseline psychiatric comorbidities
was different between the two studies (42% for the ALTOS
study and 74% for the ABC study), the results of the
analyses of the MMSE were similar. Second, despite the
relatively large sample size, secondary analyses
conducted using binary patient categories had smaller
sample sizes and may have reduced power to detect
true difference between subgroups. Third, the MMSE
and neuropsychological tests were administered using
different methods in the ALTOS study (phone versus
in-person), potentially contributing to the differences.
However, the MMSE is validated for telephone
administration, minimizing this concern . Moreover,
inperson administration was used for all testing in the
ABC study and demonstrated similar results to the
ALTOS study. Finally, the MMSE total score is
validated to identify cognitive impairment; however, the
domain scores have not been validated as standalone
measures. Our assessment of the correlations between
the MMSE domains and the corresponding
neuropsychological tests should be taken within context of
overall study findings, and not as a recommendation to
only administer specific domains of the MMSE.
Identifying valid, reliable and feasible screening tools is vital
for evaluating long-term cognitive impairments commonly
occurring in acute respiratory failure survivors. The MMSE,
one of the most commonly used cognitive screening tools,
demonstrated only fair agreement and poor sensitivity when
compared to concurrently administered neuropsychological
test batteries in acute respiratory failure survivors during
their first year of recovery. Our findings do not support use
of the MMSE as a screening tool for cognitive impairment
in this patient population. Evaluation of alternative cognitive
screening tests, either alone or in combination with other
measures, is needed in acute respiratory failure survivors.
The Mini-Mental Status Examination (MMSE) is
widely used as a screening tool for cognitive
impairment in other patient populations, but its
ability to detect impairment in survivors of acute
respiratory failure is unclear.
In survivors of acute respiratory failure, the MMSE
(compared to a detailed neuropsychological test
battery) demonstrated moderate agreement in
detecting overall cognitive impairment, with
excellent specificity but poor sensitivity.
The area under the receiver operating characteristics
curve indicates the MMSE had only fair
discrimination in detecting cognitive impairment at
6 and 12 months.
Our findings were not influenced by patient
characteristics (age, gender or education),
psychiatric disorders or timing of longitudinal
MMSE results in survivors of acute respiratory
failure should be interpreted with caution.
TD Girard has received payment for lectures from Hospira. C Hough
received consulting fees from High Point Pharmaceuticals, High Point, NC.
EW Ely received consulting fees from Hospira, Abbott Laboratories, and
Orion. The authors declare that they have no other competing interests.
All authors contributed to the conception and/or design of this study. DMN,
VDD, TDG, PEM, CLH, JCJ, PAM-T, EW and ROH contributed to the acquisition
of data. ERP, MH and KSC contributed to the analysis of data and all authors
contributed to the interpretation of data. ERP and ROH drafted the manuscript,
and all authors critically revised it for important intellectual content and
approved the final version to be submitted.
ERP takes responsibility for the content of the manuscript including the data
This research was supported by the NIH (grant numbers: R24HL111895,
R01HL091760, R01HL091760-02S1, R01HL096504 and K23AG034257), the
Johns Hopkins Institute for Clinical and Translational Research (ICTR)
(grant number: UL1 TR 000424–06) and the ALTA and EDEN/OMEGA
trials (contracts for sites participating in this study: HSN268200536170C,
HHSN268200536171C, HHSN268200536173C, HHSN268200536174C,
HSN268200536175C and HHSN268200536179C). The sponsors had no
role in the data acquisition, analysis or preparation of the manuscript.
We thank all patients and their proxies who participated in the study. We
acknowledge our dedicated research staff, including the following who
assisted with data collection, training and quality assurance and/or data
management: Lindsay Anderson, Ellen Caldwell, Nancy Ciesla, William
Flickinger, Jacqueline Flynn, Jonathan Gellar, Stephanie Gundel, John
Keenan, Christopher Mayhew, Melissa McCullough, Jessica McCurley,
Mardee Merrill, Laura Methvin, Kristin Sepulveda, Kelly Swanson, Elizabeth
Vayda and Cassie Wicken.
Investigators and research staff from the National Heart, Lung, and Blood
Institute Acute Respiratory Distress Syndrome (ARDS) Clinical Trials Network
sites that participated in this follow-up study: University of Washington,
Harborview (*L Hudson, S Gundel, *C Hough, M Neff, K Sims, A Ungar, T
Watkins); Johns Hopkins University (*R Brower, H Fessler, D Hager, P
MendezTellez, D Needham, K Oakjones); Johns Hopkins Bayview Medical Center (J
Sevransky, A Workneh); University of Maryland (C Shanholtz, D Herr, H Howes,
G Netzer, P Rock, A Sampaio, J Titus); Union Memorial Hospital (P Sloane, T
Beck, D Highfield, S King); Washington Hospital Center (B Lee, N Bolouri);
Vanderbilt University (*AP Wheeler, GR Bernard, M Hays, S Mogan, TW Rice);
Wake Forest University (*RD Hite, A Harvey, PE Morris, M Ragusky);
Intermountain Medical Center (*A Morris, *C Grissom, A Austin, S Barney, S
Brown, J Ferguson, H Gallo, T Graydon, E Hirshberg, A Jephson, N Kumar, M
Lanspa, R Miller, D Murphy, J Orme, A Stowe, L Struck, F Thomas, D Ward,);
LDS Hospital (P Bailey, W Beninati, L Bezdjian, T Clemmer, S Rimkus, R Tanaka,
L Weaver); McKay Dee Hospital (C Lawton, D Hanselman); Utah Valley
Regional Medical Center (K Sundar, W Alward, C Bishop, D Eckley, D Harris, T
Hill, B Jensen, K Ludwig, D Nielsen, M Pearce). Clinical Coordinating Center:
Massachusetts General Hospital and Harvard Medical School (*D Schoenfeld,
N Dong, M Guha, E Hammond, P Lazar, R Morse, C Oldmixon, N Ringwood, E
Smoot, BT Thompson, R Wilson). National Heart, Lung and Blood Institute: A
Harabin, S Bredow, M Waclawiw, G Weinmann. Data and Safety Monitoring
Board: RG Spragg (chair), A Slutsky, M Levy, B Markovitz, E Petkova, C Weijer.
Protocol Review Committee: J Sznajder (chair), M Begg, L Gilbert-McClain, E
Israel, J Lewis, S McClave, P Parsons.
1. Needham DM , Davidson J , Cohen H , Hopkins RO , Weinert C , Wunsch H , et al. Improving long-term outcomes after discharge from intensive care unit: report from a stakeholders' conference . Crit Care Med . 2012 ; 40 : 502 - 9 .
2. Elliott D , Davidson JE , Harvey MA , Bemis-Dougherty A , Hopkins RO , Iwashyna TJ , et al. Exploring the scope of post-intensive care syndrome therapy and care: engagement of non-critical care providers and survivors in a second stakeholders meeting . Crit Care Med . 2014 ; 42 : 2518 - 26 .
3. Wolters AE , Slooter AJ , van der Kooi AW , van Dijk D. Cognitive impairment after intensive care unit admission: a systematic review . Intensive Care Med . 2013 ; 39 : 376 - 86 .
4. Pandharipande PP , Girard TD , Jackson JC , Morandi A , Thompson JL , Pun BT , et al. Long-term cognitive impairment after critical illness . N Engl J Med . 2013 ; 369 : 1306 - 16 .
5. Iwashyna TJ , Ely EW , Smith DM , Langa KM . Long-term cognitive impairment and functional disability among survivors of severe sepsis . JAMA . 2010 ; 304 : 1787 - 94 .
6. Ehlenbach WJ , Hough CL , Crane OK , Haneuse SJ , Carson SS , Curtis JR , et al. Association between acute care and critical illness hospitalization and cognitive function in older adults . JAMA . 2010 ; 303 : 763 - 70 .
7. Richards JE , Guillamondegui OD , Archer KR , Jackson JC , Ely EW , Obremskey WT . The association of reamed intramedullary nailing and long-term cognitive impairment . J Orthop Trauma . 2011 ; 25 : 707 - 13 .
8. Jackson JC , Obremskey W , Bauer R , Greevy R , Cotton BA , Anderson V , et al. Longterm cognitive, emotional, and functional outcomes in trauma intensive care unit survivors without intracranial hemorrhage . J Trauma . 2007 ; 62 : 80 - 8 .
9. Jackson JC , Archer KR , Bauer R , Abraham CM , Song Y , Greevey R , et al. A prospective investigation of long-term cognitive impairment and psychological distress in moderately versus severely injured trauma intensive care unit survivors without intracranial hemorrhage . J Trauma . 2011 ; 71 : 860 - 6 .
10. Guillamondegui OD , Richards JE , Ely EW , Jackson JC , Archer KR , Norris PR , et al. Does hypoxia affect intensive care unit delirium or long-term cognitive impairment after multiple trauma without intracranial hemorrhage ? J Trauma . 2011 ; 70 : 910 - 5 .
11. Girard TD , Jackson JC , Pandharipande PP , Pun BT , Pun BT , Thompson JL , et al. Delirium as a predictor of long-term cognitive impairment in survivors of critical illness . Crit Care Med . 2010 ; 38 : 1513 - 20 .
12. Iwashyna TJ . Trajectories of recovery and dysfunction after acute illness, with implications for clinical trial design . Am J Respir Crit Care Med . 2012 ; 186 : 302 - 4 .
13. Brummel NE , Girard TD , Ely EW , Pandharipande PP , Morandi A , Hughes CG , et al. Feasibility and safety of early combined cognitive and physical therapy for critically ill medical and surgical patients: the Activity and Cognitive Therapy in ICU (ACT-ICU) trial . Intensive Care Med . 2014 ; 40 : 370 - 9 .
14. Jackson JC , Ely EW , Morey MC , Morey MC , Anderson VM , Denne LB , et al. Cognitive and physical rehabilitation of intensive care unit survivors: results of the RETURN randomized controlled pilot investigation . Crit Care Med . 2012 ; 40 : 1088 - 97 .
15. Lin JS , O'Connor E , Rossom RC , Perdue LA , Eckstrom E. Screening for cognitive impairment in older adults: a systematic review for the U .S. Preventive Services Task Force. Ann Intern Med . 2013 ; 159 : 601 - 12 .
16. Tangalos EG , Smith GE , Ivnik RJ , Petersen RC , Kokmen E , Kurland LT , et al. The Mini-Mental State Examination in general medical practice: clinical utility and acceptance . Mayo Clin Proc . 1996 ; 71 : 829 - 37 .
17. Woon FL , Dunn CB , Hopkins RO . Predicting cognitive sequelae in survivors of critical illness with cognitive screening tests . Am J Respir Crit Care Med . 2012 ; 186 : 333 - 40 .
18. Jackson JC , Hart RP , Gordon SM , Shintani A , Truman B , May L , et al. Sixmonth neuropsychological outcome of medical intensive care unit patients . Crit Care Med . 2003 ; 31 : 1226 - 34 .
19. Needham DM , Dinglas VD , Bienvenu OJ , Colantuoni E , Wozniak AW , Rice TW , et al. One year outcomes in patients with acute lung injury randomised to initial trophic or full enteral feeding: prospective follow-up of EDEN randomised trial . BMJ . 2013 ; 346 :f1532.
20. Peek GJ , Mugford M , Tiruvoipati R , Wilson A , Allen E , Thalanany MM , et al. Efficacy and economic assessment of conventional ventilatory support versus extracorporeal membrane oxygenation for severe adult respiratory failure (CESAR): a multicentre randomised controlled trial . Lancet . 2009 ; 374 : 1351 - 63 .
21. Sacanella E , Pérez-Castejón JM , Nicolás JM , Masanés F , Navarro M , Castro P , et al. Functional status and quality of life 12 months after discharge from a medical ICU in healthy elderly patients: a prospective observational study . Crit Care . 2011 ; 15 :R105.
22. Unroe M , Kahn JM , Carson SS , Govert JA , Martinu T , Sathy SJ , et al. One-year trajectories of care and resource utilization for recipients of prolonged mechanical ventilation: a cohort study . Ann Intern Med . 2010 ; 153 : 167 - 75 .
23. Radtke FM , Franck M , Herbig TS , Papkalla N , Kleinwaechter R , Kork F , et al. Incidence and risk factors for cognitive dysfunction in patients with severe systemic disease . J Int Med Res . 2012 ; 40 : 612 - 20 .
24. Torgersen J , Hole JF , Kvåle R , Wentzel-Larsen T , Flaatten H. Cognitive impairments after critical illness . Acta Anaesthesiol Scand . 2011 ; 55 : 1044 - 51 .
25. Kim SY , Lim TS , Lee HY , Moon SY . Clustering mild cognitive impairment by mini-mental state examination . Neurol Sci . 2014 ; 35 : 1353 - 8 .
26. Saczynski JS , Marcantonio ER , Quach L , Fong TG , Gross A , Inouye SK , et al. Cognitive trajectories after postoperative delirium . N Engl J Med . 2012 ; 367 : 30 - 9 .
27. Needham DM , Dinglas VD , Morris PE , Jackson JC , Hough CL , Mendez-Tellez PA , et al. Physical and cognitive performance of patients with acute lung injury 1 year after initial trophic versus full enteral feeding . EDEN trial follow-up . Am J Respir Crit Care Med . 2013 ; 188 : 567 - 76 .
28. National Heart , Lung, and Blood Institute Acute Respiratory Distress Syndrome (ARDS) Clinical Trials Network , Matthay MA , Brower RG , Carson S , Douglas IS , Eisner M , et al. Randomized, placebo-controlled clinical trial of an aerosolized β2-agonist for treatment of acute lung injury . Am J Respir Crit Care Med . 2011 ; 184 : 561 - 8 .
29. Rice TW , Wheeler AP , Thompson BT , de Boisblanc BP , Steingrub J , Rock P , et al. Enteral omega-3 fatty acid, gamma-linolenic acid, and antioxidant supplementation in acute lung injury . JAMA . 2011 ; 306 : 1574 - 81 .
30. Heart N , Lung, and Blood Institute Acute Respiratory Distress Syndrome (ARDS) Clinical Trials Network , Rice TW , Wheeler AP , Thompson BT , Steingrub J , Hite RD , et al. Initial trophic vs full enteral feeding in patients with acute lung injury: the EDEN randomized trial . JAMA . 2012 ; 307 : 795 - 803 .
31. Jackson JC , Girard TD , Gordon SM , Thompson JL , Shintani AK , Thomason JW , et al. Long-term cognitive and psychological outcomes in the awakening and breathing controlled trial . Am J Respir Crit Care Med . 2010 ; 182 : 183 - 91 .
32. Charlson ME , Pompei P , Ales KL , MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation . J Chronic Dis . 1987 ; 40 : 373 - 83 .
33. Newkirk LA , Kim JM , Thompson JM , Tinklenberg JR , Yesavage JA , Taylor JL . Validation of a 26-point telephone version of the Mini-Mental State Examination . J Geriatr Psychiatry Neurol . 2004 ; 17 : 81 - 7 .
34. Folstein MF , Folstein SE , McHugh PR. Mini-mental state. A practical method for grading the cognitive state of patients for the clinician . J Psychiatr Res . 1975 ; 12 : 189 - 98 .
35. Tulsky D , Zhu J. WAIS-III WMS-III Technical manual . San Antonio, TX: The Psychological Corporation, Harcourt Brace & Company; 1977 .
36. Ross TP , Calhoun E , Cox T , Wenner C , Kono W , Pleasant M. The reliability and validity of qualitative scores for the Controlled Oral Word Association Test . Arch Clin Neuropsychol . 2007 ; 22 : 475 - 88 .
37. Ruff RM , Light RH , Parker SB , Levin HS . Benton Controlled Oral Word Association Test: reliability and updated norms . Arch Clin Neuropsychol . 1996 ; 11 : 329 - 38 .
38. Kiernan RJ , Mueller J , Langston JW , Van Dyke C. The Neurobehavioral Cognitive Status Examination: a brief but quantitative approach to cognitive assessment . Ann Intern Med . 1987 ; 107 : 481 - 5 .
39. Wechsler D. Wechsler Memory Scale . 3rd ed . San Antonio, TX: The Psychological Corporation; 1997 .
40. Heaton RK , Miller SW , Taylor JR , Grant I. Comprehensive norms for an expanded Halstead Reitan Battery: demographically adjusted neuropsychological norms for African American and Caucasian adults . Professional Manual. Lutz, FL: Psychological Assessment Resources Inc .; 2004 .
41. Lezak MD . Neuropsychological assessment . 3rd ed . New York : Oxford University Press ; 1995 .
42. Bienvenu OJ , Colantuoni E , Mendez-Tellez PA , Dinglas VD , Shanholtz C , Husain N , et al. Depressive symptoms and impaired physical function after acute lung injury: a 2-year longitudinal study . Am J Respir Crit Care Med . 2012 ; 185 : 517 - 24 .
43. Zigmond AS , Snaith RP . The hospital anxiety and depression scale . Acta Psychiatr Scand . 1983 ; 67 : 361 - 70 .
44. Rey A. L'examen psychologique dans les cas d'encephalopathie traumatique . Arch Psychol . 1941 ; 28 : 286 - 340 .
45. Rey A. L'examen clinique en psychologie . Paris: Presses Universitaites de France; 1964 .
46. Reitan RM , Wolfson D. The Halstead Reitan neuropsychological test battery . Tuscon , AZ: Neuropsychology Press ; 1985 .
47. Spreen O , Benton AL . Neurosensory Center Comprehensive Examination for Aphasis (NCCEA ). Victoria: University of Victoria Neuropsychological Laboratory; 1969 .
48. Kukull WA , Larson EB , Teri L , Bowen J , McCormick W , Pfanschmidt ML . The Mini-Mental State Examination score and the clinical diagnosis of dementia . J Clin Epidemiol . 1994 ; 47 : 1061 - 7 .
49. Kamath PS , Wiesner RH , Malinchoc M , Kremers W , Therneau TM , Kosberg CL , et al. A model to predict survival in patients with end-stage liver disease . Hepatology . 2001 ; 33 : 464 - 70 .
50. Burker EJ , Blumenthal JA , Feldman M , Thyrum E , Mahanna E , White W , et al. The Mini Mental State Exam as a predictor of neuropsychological functioning after cardiac surgery . Int J Psychiatry Med . 1995 ; 25 : 263 - 76 .
51. Galioto R , Garcia S , Spitznagel MB , Strain G , Devlin M , Crosby RD , et al. The Mini-Mental State Exam (MMSE) is not sensitive to cognitive impairment in bariatric surgery candidates . Surg Obes Relat Dis . 2014 ; 10 : 553 - 7 .
52. Wilcox ME , Brummel NE , Archer K , Ely EW , Jackson JC , Hopkins RO . Cognitive dysfunction in ICU patients: risk factors, predictors, and rehabilitation interventions . Crit Care Med . 2013 ; 41 : S81 - 98 .
53. Nasreddine ZS , Phillips NA , Bédirian V , Charbonneau S , Whitehead V , Collin I , et al. The Montreal Cognitive Assessment, MoCA: a brief screening tool for mild cognitive impairment . J Am Geriatr Soc . 2005 ; 53 : 695 - 9 .
54. Roalf DR , Moberg PJ , Xie SX , Wolk DA , Moelter ST , Arnold SE . Comparative accuracies of two common screening instruments for classification of Alzheimer's disease, mild cognitive impairment, and healthy aging . Alzheimers Dement . 2013 ; 9 : 529 - 37 .
55. Kim JW , Lee DY , Seo EH , Sohn BK , Choe YM , Kim SG , et al. Improvement of screening accuracy of Mini-Mental State Examination for mild cognitive impairment and non-Alzheimer's disease dementia by supplementation of verbal fluency performance . Psychiatry Investig . 2014 ; 11 : 44 - 51 .
56. Hopkins RO , Weaver LK , Collingridge D , Parkinson RB , Chan KJ , Orme Jr JF . Two-year cognitive, emotional, and quality-of-life outcomes in acute respiratory distress syndrome . Am J Respir Crit Care Med . 2005 ; 171 : 340 - 7 .
57. Jackson JC , Pandharipande PP , Girard TD , Brummel NE , Thompson JL , Hughes CG , et al. Depression, post-traumatic stress disorder, and functional disability in survivors of critical illness in the BRAIN-ICU study: a longitudinal cohort study . Lancet Respir Med . 2014 ; 2 : 369 - 79 .
58. Dowdy DW , Bienvenu OJ , Dinglas VD , Mendez-Tellez PA , Sevransky J , Shanholtz C , et al. Are intensive care factors associated with depressive symptoms 6 months after acute lung injury? Crit Care Med . 2009 ; 37 : 1702 - 7 .