Assessment of goal-directed behavior with the 3D videogame EPELI: Psychometric features in a web-based adult sample

PLOS ONE RESEARCH ARTICLE Assessment of goal-directed behavior with the 3D videogame EPELI: Psychometric features in a web-based adult sample Jussi Jylkkä ID1*, Liisa Ritakallio1, Liya Merzon2, Suvi Kangas ID3, Matthias Kliegel4,5, Sascha Zuber5,6, Alexandra Hering4,7, Juha Salmi2,8, Matti Laine1 a1111111111 a1111111111 a1111111111 a1111111111 a1111111111 1 Department of Psychology, Åbo Akademi University, Turku, Finland, 2 Department of Neuroscience and Biomedical Engineering, Aalto University, Espoo, Finland, 3 Department of Psychology, University of Helsinki, Helsinki, Finland, 4 Faculty of Psychology and Educational Sciences, University of Geneva, Geneva, Switzerland, 5 Centre for the Interdisciplinary Study of Gerontology and Vulnerability, University of Geneva, Geneva, Switzerland, 6 Swiss National Center of Competences in Research LIVES—Overcoming Vulnerability: Life Course Perspectives, Lausanne & Geneva, Switzerland, 7 Department of Developmental Psychology, Tilburg University, Tilburg, The Netherlands, 8 MAGICS, Aalto University, Espoo, Finland * Abstract OPEN ACCESS Citation: Jylkkä J, Ritakallio L, Merzon L, Kangas S, Kliegel M, Zuber S, et al. (2023) Assessment of goal-directed behavior with the 3D videogame EPELI: Psychometric features in a web-based adult sample. PLoS ONE 18(3): e0280717. https://doi. org/10.1371/journal.pone.0280717 Editor: Gabriel G. De La Torre, University of Cadiz: Universidad de Cadiz, SPAIN Received: August 9, 2022 Accepted: January 5, 2023 Published: March 21, 2023 Copyright: © 2023 Jylkkä et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Data Availability Statement: All data is available at the Open Science Framework, https://osf.io/3y5je/. Funding: JS was supported by the Academy of Finland (#325981 and #328954); ML was supported by Academy of Finland (#323251). LR was supported by the Finnish Cultural Foundation. Competing interests: The authors have declared that no competing interests exist. EPELI (Executive Performance in Everyday LIving) is a recently developed gaming tool for objective assessment of goal-directed behavior and prospective memory (PM) in everyday contexts. This pre-registered study examined psychometric features of a new EPELI adult online version, modified from the original child version and further developed for self-administered web-based testing at home. A sample of 255 healthy adults completed EPELI where their task was to perform household chores instructed by a virtual character. The participants also filled out PM-related questionnaires and a diary and performed two conventional PM tasks and an intelligence test. We expected that the more “life-like” EPELI task would show stronger associations with conventional PM questionnaires and diary-based everyday PM reports than traditional PM tasks would do. This hypothesis did not receive support. Although EPELI was rated as more similar to everyday tasks, performance in it was not associated with the questionnaires and the diary. However, there were associations between time-monitoring behavior in EPELI and the traditional PM tasks. Taken together, online adult-EPELI was found to be a reliable method with high ecological face validity, but its convergent validity requires further research. Introduction An important aspect of any psychological assessment method is its ability to predict real-world behaviors, that is, its ecological validity. Cognitive tasks have been criticized in this respect as they are often simplified and strictly controlled, putting emphasis on internal rather than external validity [1–4]. Moreover, traditional cognitive tasks typically use highly controlled, but therefore low-dimensional and static stimuli, which strongly stand in contrast to dynamic and complex real-world environments [5]. This reflects the common underlying assumption PLOS ONE | https://doi.org/10.1371/journal.pone.0280717 March 21, 2023 1 / 24 PLOS ONE Assessment of goal-directed behavior with the 3D videogame EPELI in traditional cognitive testing that cognitive functions are invariant across contexts, which implies that the same cognitive processes are at play in laboratory tasks and in real life [6]. This idea of generalizability is challenged by findings indicating that performances in laboratory tasks often correlate weakly with real-life behaviors, which suggests that cognitive processes may be context-sensitive [7]. This motivates a move towards a ‘cognitive ethology’ approach where cognition is tested in natural contexts [6,8]. Such studies have indeed been conducted in neuropsychology (e.g., Multiple Errands Test; [9]) but they face several practical limitations, lacking experimental control and being difficult to implement [10]. However, the development of video games and virtual reality (VR) tools has renewed interest in the development of more complex, ‘life-like’ cognitive tasks [11]. Another important aspect that has not been previously addressed in naturalistic studies is the scalability of the measurement methods. Obtaining large enough sample sizes when studying human goal-directed behavior in lifelike contexts is highly relevant: in complex task situations one can expect considerable inter-individual performance variation, and identifying factors that underlie this variation is difficult in limited samples. In addition, various background factors like age and cognitive ability, as well as the highly variable employment of spontaneous strategies could mediate the performance in complex tasks [12,13]. In the present study, we set out to examine the internal consistency as well as convergent and ecological validity of a new, Internet browser-based 3D video game called EPELI (Executive Performance in Everyday Living) in a large sample of healthy adults. A VR version of this task has been recently validated in children with ADHD versus normally developing controls [14]. However, VR methods typically require laboratory testing and significant human resources because most persons do not possess the necessary equipment. The present webbased version of EPELI enables scalable, home-based testing. Although several similar tasks have been conducted in laboratory context (for a review, see [15]), to our knowledge this is the first time when a naturalistic video game to assess cognitive processes related to goal-directed behavior is utilized remotely through the Internet. Many of the early VR testing games were in practice implementations of laboratory tasks, such as the Wisconsin Card Sorting test [16] or the Continuous Performance Test (CPT, [17]; for a review, see [11]). However, these VR implementations suffer from many of the same limitations as traditional laboratory-based tasks, such as employing abstract, unrealistic and static stimulus materials, (...truncated)