Memory and decision making interact to shape the value of unchosen options

ARTICLE https://doi.org/10.1038/s41467-021-24907-x OPEN Memory and decision making interact to shape the value of unchosen options 1234567890():,; Natalie Biderman 1 ✉ & Daphna Shohamy1,2 ✉ The goal of deliberation is to separate between options so that we can commit to one and leave the other behind. However, deliberation can, paradoxically, also form an association in memory between the chosen and unchosen options. Here, we consider this possibility and examine its consequences for how outcomes affect not only the value of the options we chose, but also, by association, the value of options we did not choose. In five experiments (total n = 612), including a preregistered experiment (n = 235), we found that the value assigned to unchosen options is inversely related to their chosen counterparts. Moreover, this inverse relationship was associated with participants’ memory of the pairs they chose between. Our findings suggest that deciding between options does not end the competition between them. Deliberation binds choice options together in memory such that the learned value of one can affect the inferred value of the other. 1 Department of Psychology and Mortimer B. Zuckerman Mind, Brain, Behavior Institute, Columbia University, New York, NY, USA. 2 The Kavli Institute for Brain Science, Columbia University, New York, NY, USA. ✉email: ; NATURE COMMUNICATIONS | (2021)12:4648 | https://doi.org/10.1038/s41467-021-24907-x | www.nature.com/naturecommunications 1 ARTICLE T NATURE COMMUNICATIONS | https://doi.org/10.1038/s41467-021-24907-x he Latin origin of the verb ‘to decide’ literally means ‘to cut off’ (‘de’ = off, ‘caedere’ = cut). The act of deciding is supposed to cut off the deliberation process to settle on one’s choice. Yet, often the deliberation does not seem to end when a decision is made. Instead, the unchosen option continues to linger in one’s mind. Over the years, research has shown that people continue to think about counterfactuals1–6. Yet, little is known about the mechanism that allows unchosen options to linger in our minds and the consequences of such lingering for the value of unchosen options. Here we aim to address this gap. The starting point of our inquiry is that thoughts about unchosen alternatives do not come from thin air, but that they are instead tied to the chosen option7–9. For example, consider a decision you had to make, say about where to go on vacation. If the location you ended up choosing did not meet your expectations, this would of course lead you to devalue the choice you made10. But often in such situations, we also find ourselves automatically thinking back to the unchosen option, which we may now evaluate as a better option than it seemed at the time. Such post-decision experiences suggest that the options we deliberate between remain linked in our minds long after the decision was made. Indeed, deliberation is a comparative process, wherein options are evaluated simultaneously in relation to each other11–13. In this sense, deliberation provides a temporal and conceptual context shared by choice alternatives. Extensive memory research has shown that shared context creates an association between disparate elements, binding them to each other in memory14–17. These findings suggest that rather than ‘cutting off’ the unchosen option, deliberation may, paradoxically, tie the options together. The possibility of choice options remaining associated in memory could have substantial consequences for how learning shapes value once the outcomes of a choice are revealed. Many studies have investigated the mechanisms by which outcomes of one’s choice drive learning about the chosen option. According to prominent reinforcement learning models, learning the outcomes of chosen options leads to their value being updated based on the difference between the expected value and the actual experienced value10. The mechanisms of learning about unchosen options, however, are less clear. Studies of counterfactual learning have focused on how people learn from explicit information about what they could have gained if they had chosen the other option18–24. Yet, in most cases, people are not exposed to the outcomes of their forgone alternatives, leaving open questions about if, and what, people learn about unchosen options afterwards, when they experience outcomes of the choice that they made. In these situations, is the value of the unchosen options updated as well? Here we hypothesize that the value of unchosen options is updated through their association with the chosen options. Central to this hypothesis is the role of memory. Recent advances suggest that memory associations can facilitate value inference and generalization. The basic idea is that once an association is formed, encountering one item leads to reactivation of the associated items25–27. In a recent study rats were required to make navigation decisions, and when they learned they made the wrong choice and their expected reward was nowhere to be seen, they reactivated the unchosen location9. Moreover, studies in humans have shown that if a reward is given to one of two associated items, the value of that reward can spread towards the associated items by reactivating a mnemonic network28–30. This reactivation mechanism has been shown to account for updating of chosen options, however, it remains unknown if it also affects the value of unchosen options. The current study sought to determine whether the act of deliberation creates a memory association between the 2 deliberated options and to explore the consequences of this association for later value learning. We hypothesized that learning about the outcomes of the choice leads to reactivation, in memory, of the unchosen option and this in turn leads to a change in the unchosen option’s value. Unlike previous studies showing direct value transfer among associated items28–30, choice options are associated within the deliberation context, which involves a contrast between the options. We, therefore, expect the value to transfer in the opposite direction. That is, if chosen items are explicitly rewarded (or unrewarded), we expect unchosen items to be inferred as unrewarded (or rewarded), a behavioral tendency we refer to as inverse inference of value. Thus, we hypothesize that deliberation may have a somewhat paradoxical role: While it is meant to dissociate choice options, deliberation binds them in our memory. When this bond is reactivated for the purpose of value updating, it continues to serve the deliberation goal of teasing the value of options apart. Our prediction of an inverse transfer of value between choice options is based upon previous behavioral findings showing that participants continue to separate the value of options even after the decision was terminated. In studies of choice-supportive memory, participants exhibited a bias to better learn and remember the positive aspects of chosen options and the negative aspe (...truncated)