Test Validity and Performance Validity: Considerations in Providing a Framework for Development of an Ability-Focused Neuropsychological Test Battery

Archives of Clinical Neuropsychology 29 (2014) 695– 714 Test Validity and Performance Validity: Considerations in Providing a Framework for Development of an Ability-Focused Neuropsychological Test Battery Glenn J. Larrabee* Independent Practice, Sarasota, FL, USA *Corresponding author at: 2650 Bahia Vista Street, # 308, Sarasota, FL 34242, USA. Tel.: +1-941-955-9596; fax: +1-941-957-3485. E-mail address: (G. J. Larrabee). Accepted 8 September 2014 Abstract Literature on test validity and performance validity is reviewed to propose a framework for specification of an ability-focused battery (AFB). Factor analysis supports six domains of ability: first, verbal symbolic; secondly, visuoperceptual and visuospatial judgment and problem solving; thirdly, sensorimotor skills; fourthly, attention/working memory; fifthly, processing speed; finally, learning and memory (which can be divided into verbal and visual subdomains). The AFB should include at least three measures for each of the six domains, selected based on various criteria for validity including sensitivity to presence of disorder, sensitivity to severity of disorder, correlation with important activities of daily living, and containing embedded/derived measures of performance validity. Criterion groups should include moderate and severe traumatic brain injury, and Alzheimer’s disease. Validation groups should also include patients with left and right hemisphere stroke, to determine measures sensitive to lateralized cognitive impairment and so that the moderating effects of auditory comprehension impairment and neglect can be analyzed on AFB measures. Keywords: Assessment; Test construction; Meta-analysis; Head injury; Traumatic brain injury; Alzheimer’s disease; Cerebrovascular disease/accident and stroke Introduction Bauer (2000) distinguished between two major approaches to neuropsychological assessment: first, the fixed battery approach, wherein everyone receives the same comprehensive battery of tests, regardless of the referral question or patient clinical presentation and second, a flexible battery, wherein there is a limited core set of procedures administered to all, to provide a basis for generating clinical hypotheses about the patient’s neuropsychological status for purposes of additional evaluation. Bauer also discusses an intermediate approach, which he characterizes as multiple fixed battery, which can characterize populationspecific batteries constructed for specific clinical disorders, for example, multiple sclerosis, traumatic brain injury (TBI), or domain-specific batteries, constructed for extensive evaluation of a particular process such as language or memory (e.g., Multilingual Aphasia Examination, Benton, Hamsher, & Sivan, 1994; Wechsler Memory Scale-IV; WMS-IV, Wechsler, 2009). Surveys have established that the flexible battery approach has been the major practice orientation in clinical neuropsychology for several years (Rabin, Barr, & Burton, 2005; Sweet, Meyer, Nelson, & Moberg, 2011). In the most recent survey (Sweet et al., 2011), 78% of neuropsychologists endorsed using a flexible battery (core set of procedures with additional testing based on clinical history and core test findings), compared with 18% using a completely flexible approach (test selection governed entirely by referral question and patient clinical presentation), and 5% using a fixed standardized battery (e.g., Halstead-Reitan, Reitan & Wolfson, 1993; Luria-Nebraska, Golden, Purisch, & Hammeke, 1985; or Neuropsychological Assessment Battery, NAB, Stern & White, 2003; [note the NAB also has a feature allowing for more flexible application, with administration of a screening battery, which can be followed by more in depth examination of core areas of ability]). The assessment practice survey by Rabin et al. (2005) reflects high test use frequencies for select procedures such as 63.1% using the WAIS-R/III (Wechsler, 1981, 1997a), # The Author 2014. Published by Oxford University Press. All rights reserved. For permissions, please e-mail: . doi:10.1093/arclin/acu049 Advance Access publication on 3 October 2014 696 G.J. Larrabee / Archives of Clinical Neuropsychology 29 (2014); 695–714 42.7% using WMS-R/WMS-III (Wechsler, 1987, 1997b), 17.6% using the Trail Making Test (Reitan & Wolfson, 1993), and 17.3% using the CVLT/CVLT-II (Delis, Kramer, Kaplan, & Ober, 1987, 2000). This survey did not elicit data on frequency of combinations of tests, for example, the percentage of clinicians using the WAIS-R, WMS-R, CVLT, and Trail Making Test. Despite the popularity of the flexible battery approach, there is no common set of tests comprising the core of a flexible battery, which is universally used. The purpose of the current paper is to lay out a framework for composing a standard neuropsychological test battery that can serve as the core for a flexible battery, supported by construct and criterion validity, and which also contains embedded/derived measures of performance validity (Performance Validity Tests, PVTs; Larrabee, 2012a). The reader should be aware that Meyers (Meyers & Rohling, 2004) has developed a 212 h core for a flexible battery, the Meyers Neuropsychological Battery (MNB), comprised of 22 tests, which also includes 11 embedded/derived PVTs (Meyers et al., 2014; Meyers & Volbrecht, 2003). While the MNB does follow some of the validity guidelines I will be presenting, it differs, largely in three ways: first, originally selecting tests on the basis of sensitivity to presence of brain dysfunction in a mixed sample of neurologic cases (Meyers, Miller, & Tuita, 2013); second, using only one test each to represent the domains of motor function, verbal and visual learning and memory; finally including tests not in common use (with the exception of those clinicians using the MNB): 1-Minute Estimation, and Dichotic Listening, as well as administering two tests that are usually only administered during evaluation of acquired aphasia (Sentence Repetition and the Token Test). The MNB does, however, present a model for how the core of a flexible battery can be composed of individually normed tests that in many ways functions as well as more extensive batteries of co-normed tests. This is likely because the MNB contains sensitive measures of memory and processing speed such as the Auditory Verbal Learning Test (AVLT), Complex Figure Test and Trail Making Test (cf. Rohling, Meyers, & Millis, 2003). As others have shown, tests of processing speed and memory are among the tests most sensitive to acquired brain impairment. This is particularly true when the brain damage is of a diffuse nature in conditions such as Alzheimer’s disease (AD) or moderate and severe TBI (Backman, Jones, Berger, Laukka, & Small, 2005; Christensen, Griffiths, Mackinnon, & Jacomb, 1997; Dikmen, Machamer, Winn, & Temkin, 1995; Larrabee, Millis, & Meyers, 2008; Miller, Fichtenberg, & Millis, 2010; Powell, Cripe, & Dodrill, 1991). In the following sect (...truncated)