Dynamical latent state computation in the male macaque posterior parietal cortex

Article https://doi.org/10.1038/s41467-023-37400-4 Dynamical latent state computation in the male macaque posterior parietal cortex Received: 13 February 2022 Accepted: 15 March 2023 1234567890():,; 1234567890():,; Check for updates Kaushik J. Lakshminarasimhan Dora E. Angelaki2,6,7 1 , Eric Avila2, Xaq Pitkow 3,4,5,7 & Success in many real-world tasks depends on our ability to dynamically track hidden states of the world. We hypothesized that neural populations estimate these states by processing sensory history through recurrent interactions which reflect the internal model of the world. To test this, we recorded brain activity in posterior parietal cortex (PPC) of monkeys navigating by optic flow to a hidden target location within a virtual environment, without explicit position cues. In addition to sequential neural dynamics and strong interneuronal interactions, we found that the hidden state - monkey’s displacement from the goal - was encoded in single neurons, and could be dynamically decoded from population activity. The decoded estimates predicted navigation performance on individual trials. Task manipulations that perturbed the world model induced substantial changes in neural interactions, and modified the neural representation of the hidden state, while representations of sensory and motor variables remained stable. The findings were recapitulated by a task-optimized recurrent neural network model, suggesting that task demands shape the neural interactions in PPC, leading them to embody a world model that consolidates information and tracks task-relevant hidden states. Imagine you are driving on a busy highway and wish to change lanes. To safely do so, you need to mentally track the pattern of traffic behind you even when not looking into the rear-view mirror. Many everyday tasks require maintaining and updating beliefs about state variables that are not directly observable. This can be computationally hard especially if the latent world states are continuousvalued, i.e., assume a range of values, and dynamic (vary in time); these properties are typically true in the real-world1. Mechanisms underlying sensory perception and movement generation have been extensively investigated under a wide variety of conditions, such that we are converging on good computational models that are consistent with neural data2–5. In contrast, we do not understand how the intermediate, continuous-valued, time-varying, latent states - the stuff of thoughts - are represented in the brain, nor the mechanisms used to compute those states6,7. Filling this void is essential to building a complete picture of neural computations in the sensorimotor loop. The past few decades have seen the emergence of two distinct approaches in the study of neural representation of latent world states. These have contributed significantly to our understanding in complementary ways. One approach, following the tradition of sensory neuroscience, uses binary decision-making tasks (e.g., motion direction discrimination) in which participants gradually integrate sensory evidence over time and then report one perceived outcome (e.g., dots moving to the left or right)8–11. The high degree of experimental control afforded by this paradigm has helped reveal a tight link between the neural activity in the posterior parietal cortex and the time course of decision variables that guide behavior12,13. However, because the latent world states themselves tend to be discrete and/or static in such tasks, it is difficult to fully extrapolate those insights to continuous, 1 Center for Theoretical Neuroscience, Columbia University, New York City, NY, USA. 2Center for Neural Science, New York University, New York City, NY, USA. Department of Neuroscience, Baylor College of Medicine, Houston, TX, USA. 4Center for Neuroscience and Artificial Intelligence, Baylor College of Medicine, Houston, TX, USA. 5Electrical & Computer Engineering, Rice University, Houston, TX, USA. 6Department of Mechanical and Aerospace Engineering, New York e-mail: University, New York City, NY, USA. 7These authors jointly supervised this work: Xaq Pitkow, Dora E. Angelaki. 3 Nature Communications | (2023)14:1832 1 Article interactive behaviors where those latent states change continually as a consequence of one’s own actions. The alternative approach, which emerged from cognitive psychology, has sought to characterize neural correlates of continuously changing latent world states (e.g., position, heading of freely foraging animals)14,15. This has led to a rich description of neural maps in the hippocampal formation that can potentially be used for computing latent world states. However, because neither sensory input nor behavior is controlled in this approach, it is difficult to determine the precise relationship between neural activity and the animal’s momentary beliefs in such settings. To overcome these limitations, we took an approach that combined the desirable elements of both approaches by using a task that was ecologically valid, yet welldefined and controllable. Our goal was threefold: (i) to characterize the neural representation of the repertoire of sensory, latent, and motor variables in a naturalistic closed-loop task featuring action-perception loops, (ii) to test whether the latent states computed by the neural population influence behavior, and (iii) to constrain the space of possible mechanisms that create the neural representation of the latent states. We created a virtual environment in which monkeys used a joystick to steer to a transiently cued, random target location by integrating sparse optic flow cues16. To successfully perform the task, monkeys had to continuously update an internal estimate of the relative target location (the latent state) by integrating their own movement velocity inferred from the sparse optic flow cues. Brain regions in the posterior parietal cortex (PPC) have been implicated in various aspects of this computation such as optic flow processing17,18, working memory19–21, as well as planning of spatial movements22,23. Because we are primarily interested in understanding the mechanisms of latent state computation rather than optic flow processing per se, we wanted to record neural activity in a region within PPC that likely already receives abstract velocity signals, such that it may serve as the locus of latent state computation in our task. There are several properties that make area 7a a more ideal candidate than other parts of PPC. First, anatomical tracing studies have consistently found a pattern of interareal connectivity that places area 7a at the top of the motionprocessing (‘dorsal stream’) hierarchy24,25. Moreover, it is one of the few areas in PPC that directly projects to the hippocampal formation26,27, with lesions to area 7a affecting navigation performance28,29. Second, area 7a neurons are known to have large, bilateral receptive fields (15–25 degrees) and activated by the full-field mot (...truncated)