Multisensory Self-Motion Compensation During Object Trajectory Judgments

Cerebral Cortex March 2015;25:619–630 doi:10.1093/cercor/bht247 Advance Access publication September 22, 2013 Multisensory Self-Motion Compensation During Object Trajectory Judgments Kalpana Dokka1, Paul R. MacNeilage2, Gregory C. DeAngelis3 and Dora E. Angelaki1 1 Department of Neuroscience, Baylor College of Medicine, Houston, TX 77030, USA, 2German Center for Vertigo and Balance Disorders, University Hospital of Munich, Munich, Germany and 3Department of Brain and Cognitive Sciences, University of Rochester, Rochester, NY 14627, USA Address correspondence to: Dora E. Angelaki, Department of Neuroscience, Room S740, MS: BCM295, Baylor College of Medicine, One Baylor Plaza, Houston, TX 77030, USA. Email: Kalpana Dokka and Paul R. MacNeilage contributed equally to this work. Keywords: flow parsing, object motion, optic flow, self-motion, vestibular Introduction We exist in a dynamic environment in which objects constantly move around us, so accurate judgment of object trajectories is essential. Assuming that the eyes and head remain stationary relative to the body, object trajectory relative to the observer can be recovered directly from retinal image motion associated with the object. We refer to this as observer-relative object motion because object motion is represented in observercentered (i.e., egocentric) coordinates. If the observer is stationary, this estimate also conveys accurate information about how the object is moving in world (i.e., allocentric) coordinates, which we refer to as world-relative object motion. Even when the observer is moving, observer-relative object motion is sufficient for simple interception and avoidance tasks, such as catching a ball while running or avoiding collision with another moving car on the highway. However, in many situations, we need to compute world-relative object motion while we ourselves are moving. Consider a parent walking a few steps behind a toddler in a crowded pedestrian environment. The moving parent must judge the trajectories of the child, other pedestrians, and street-fixed obstacles to protect the child from collision. To do this, the © The Author 2013. Published by Oxford University Press. All rights reserved. For Permissions, please e-mail: parent must compensate for self-motion to accurately judge object trajectories in world coordinates because retinal image motion is determined by the combination of object motion in the world and self-motion (Fig. 1A,B). If self-motion is not properly accounted for, then judgments of world-relative object motion will be biased. While compensating for self-motion may be beneficial for some tasks, we hypothesize that this compensation comes at the cost of decreased precision in judging object motion, because compensation should add noise associated with estimation of self-motion. Previous studies most often report partial compensation for self-motion during judgments of object motion, but the relationship between accuracy and precision of these judgments has not been evaluated systematically. One group of studies has evaluated compensation by measuring perception of object speed. Results often show that the object had to move in the same direction as the observer to appear stationary in the world, and thus, self-motion compensation was incomplete. This is true whether the cue to self-motion was nonvisual-only (Gogel and Tietz 1977; Gogel 1982; Gogel and Tietz 1992; Mesland et al. 1996; Wexler 2003; Dyde and Harris 2008; Dupin and Wexler 2013) or visual-only (Matsumiya and Ando 2009; Dupin and Wexler 2013). During combined visual and nonvisual selfmotion, more complete compensation has been reported, especially when self-motion is actively generated (Dyde and Harris 2008; Dupin and Wexler 2013). Misperception of object speed in the world has been attributed to misperception of object distance (Gogel and Tietz 1977; Gogel 1981, 1982; Gogel and Tietz 1992), misperception of self-motion velocity (Dyde and Harris 2008; Dupin and Wexler 2013), or a default tendency to judge object motion in retinal coordinates (Shebilske and Proffitt 1981). Another group of studies has evaluated compensation by measuring perception of object trajectory. In these studies, self-motion is most often simulated based on visual cues only (but see Fajen and Matthis 2013). These studies have concluded that object direction judgments are biased in a manner consistent with the operation of a “flow-parsing” mechanism that subtracts the global component of optic flow associated with self-motion from the retinal image to estimate object motion (Gray et al. 2004; Warren and Rushton 2007, 2008; Matsumiya and Ando 2009; Warren and Rushton 2009a, 2009b; Fajen and Matthis 2013). Unfortunately, results from most of these experiments do not allow one to directly assess the degree of compensation (but see Matsumiya and Ando 2009), and thus, the nature of the representation of object trajectory (i.e., world-relative vs. observer-relative) remains unclear. Judging object trajectory during self-motion is a fundamental ability for mobile organisms interacting with their environment. This fundamental ability requires the nervous system to compensate for the visual consequences of self-motion in order to make accurate judgments, but the mechanisms of this compensation are poorly understood. We comprehensively examined both the accuracy and precision of observers’ ability to judge object trajectory in the world when self-motion was defined by vestibular, visual, or combined visual– vestibular cues. Without decision feedback, subjects demonstrated no compensation for self-motion that was defined solely by vestibular cues, partial compensation (47%) for visually defined self-motion, and significantly greater compensation (58%) during combined visual– vestibular self-motion. With decision feedback, subjects learned to accurately judge object trajectory in the world, and this generalized to novel self-motion speeds. Across conditions, greater compensation for self-motion was associated with decreased precision of object trajectory judgments, indicating that self-motion compensation comes at the cost of reduced discriminability. Our findings suggest that the brain can flexibly represent object trajectory relative to either the observer or the world, but a world-centered representation comes at the cost of decreased precision due to the inclusion of noisy selfmotion signals. Materials and Methods Figure 1. Schematic illustration of interactions between object and observer motion and the experimental protocol. (A) An object (black circle) located to the left of the observer-fixed fixation point moves downward in the world, while the subject is translated rightward. (B) This stimulus results in a retinal motion vector that can be computed by the vector sum of components associated with object movement (speed Vo) and observer movement (speed Vs). The direction of retinal motion (θ) is tan−1 (Vs/Vo). (C) The task was a rightward/lef (...truncated)