Computational Framework Explains How Animals Select Actions with Rewarding Outcomes

Citation: Weaver J (2015) Computational Framework Explains How Animals Select Actions with Rewarding Outcomes. PLoS Biol 13(1): e1002035. doi:10.1371/journal.pbio.1002035 Computational Framework Explains How Animals Select Actions with Rewarding Outcomes Janelle Weaver 0 0 Freelance Science Writer , Carbondale, Colorado , United States of America Fig. 1. Learning how to make decisions: a computational model bridges the gap between the intricate subtleties of individual neuronal connections and the behavior of the whole animal. Image credit: Left: Iwan Gabovitch, openclipart; Center: Kevin Gurney; Right: Alexey Krasavin, Flickr. doi:10.1371/journal.pbio.1002035.g001 - A key component of survival is learning to associate rewarding outcomes with specific actions, such as searching for food or avoiding predators. Actions are represented in the cortexthe brains outer layer of neural tissueand rewarding outcomes activate neurons that release a brain chemical called dopamine. These neuronal signals are sent to the striatum the input station for a collection of brain structures called the basal ganglia, which play an important role in action selection. Collectively, this evidence suggests that dopamine signals change the strength of connections between cortical and striatal neurons, thereby determining which action is appropriate for a specific set of environmental circumstances. But until now, no model had integrated these strands of evidence to test this widely held hypothesis. In a study published this week in PLOS Biology, University of Sheffield researchers Kevin Gurney and Peter Redgrave teamed up with Mark Humphries of the University of Manchester to build a computational model that shows how the brains internal signal for outcome changes the strength of neuronal connections, leading to the selection of rewarded actions and the suppression of unrewarded actions. By bridging the gap between the intricate subtleties of individual neuronal connections and the behavior of the whole animal, the model reveals how several brain signals work together to shape the input from the cortex to the basal ganglia at the interface between actions and their outcomes. The researchers developed a network model of the whole basal ganglia based on previous electrophysiological studies that investigated the activity of two types of dopamine-responsive cells called D1 and D2 striatal medium spiny neurons. In addition to this action selection model, they developed an independent plasticity Selected PLOS Biology research articles are accompanied by a synopsis written for a general audience to provide non-experts with insight into the significance of the published work. model by incorporating experimental data from a previous study to show how the strength of neuronal connections, called synapses, is affected by three factors: the timing of neuronal activity, the type of medium spiny neuron, and dopamine level. Then they linked the two models, testing whether plasticity rules at single synapses between cortical and striatal neurons could give rise to the predicted changes in the activity of the two types of medium spiny neurons, resulting in successful learning of the association between actions and outcomes. Achieving a remarkable convergence between vastly different scales of space and time, the computational framework not only replicates experimental data on cortico-striatal plasticity but also accounts for behavioral data on learning the association between actions and outcomes (Fig. 1). The model revealed that the relative strength of cortical inputs that represent different possible actions to the two populations of medium spiny neurons determines whether an action is selected or suppressed. Moreover, the timing of neuronal activity, the type of medium spiny neuron, and dopamine level are necessary for generating neuronal activity patterns that result in successful learning. According to the dominant conceptual model of the basal ganglia, D1 and D2 medium spiny neuron populations project through two competing pathways that either promote or suppress an action. This model has been used to explain the motor symptoms of Parkinsons disease, Huntingtons disease, and other neurological disorders associated with basal ganglia dysfunction. However, the new network model revealed that D1 and D2 medium spiny neurons coding the same action cooperate to produce optimal action selection, suggesting that the dominant model is not true and needs to be reevaluated. Competing Interests: The author has declared that no competing interests exist. Taken together, the results of the study provide strong support for the hypothesis that cortical inputs to dopamine-releasing neurons in the striatum are crucial for learning the association between action and outcome. Moving forward, the model provides a common framework in which to place new findings on all aspects of learning from outcomes. In the clinical realm, it could also reveal novel (...truncated)