Accessing long-term memory representations during visual change detection

Melissa R. Beck Amanda E. van Lamsweerde In visual change detection tasks, providing a cue to the change location concurrent with the test image (postcue) can improve performance, suggesting that, without a cue, not all encoded representations are automatically accessed. Our studies examined the possibility that postcues can encourage the retrieval of representations stored in long-term memory (LTM). Participants detected changes in images composed of familiar objects. Performance was better when the cue directed attention to the post-change object. Supporting the role of LTM in the cue effect, the effect was similar regardless of whether the cue was presented during the inter-stimulus interval, concurrent with the onset of the test image, or after the onset of the test image. Furthermore, the post-cue effect and LTM performance were similarly influenced by encoding time. These findings demonstrate that monitoring the visual world for changes does not automatically engage LTM retrieval. - It is generally agreed that between two and five attended objects, depending on the objects complexity, are stored in visual working memory (VWM) and that change detection This research was funded by a State of Louisiana Board of Regents support fund, Research Competitiveness Subprogram Grant, to the first author. failures are caused by this storage limit in VWM (Alvarez & Cavanagh, 2004; Eng, Chen & Jiang, 2005; Irwin, 1992; Irwin & Andrews, 1996; Levin, Simons, Angelone & Chabris, 2002; O'Regan, 1992; Rensink, 2002; Simons, 1996; Simons & Levin, 1997; for review see Simons & Rensink, 2005). However, several studies have demonstrated that change detection tasks may not accurately assess all of the information encoded into memory (Angelone, Levin & Simons, 2003; Beck, Peterson & Angelone, 2007; Beck & Levin, 2003; Griffin & Nobre, 2003; Hollingworth, 2005; Hollingworth & Henderson, 2002; Landman, Spekreijse & Lamme, 2003; Lepsien, Griffin, Devlin & Nobre, 2005; Makovski, Sussman & Jiang, 2008; Sligte, Scholte, & Lamme, 2008; Varakin & Levin, 2006). For example, change detection performance can be improved when a cue (an arrow pointing at the postchange object) draws attention to the post-change object (Hollingworth, 2003). Because the cue is presented after encoding, this post-cue effect is potentially aiding retrieval from long-term memory (LTM). However, several studies have not found post-cue effects (Becker, Pashler, & Anstis, 2000; Landman et al., 2003; Luck & Vogel, 1997; Sligte et al., 2008; Wheeler & Treisman, 2002), suggesting that LTM representations are not automatically encoded and available for retrieval in VWM tasks. Our studies examined the conditions under which a post-cue may encourage a collaborative relationship between VWM and LTM. In order to determine why some studies report a post-cue effect while others do not, it is important to understand the stages of processing involved in maintaining and updating visual representations (Simons, 2000; Simons & Rensink, 2005). First, a representation of the pre-change object must be encoded and maintained in VWM and/or LTM. The post-change object must be attended, allowing for a comparison process between the pre- and post-change objects. If the representation of the pre-change object was encoded into LTM but is no longer available in VWM, a retrieval process must occur (Unsworth & Engle, 2006) before comparison can occur. Finally, a decision must be made as to whether or not a change has occurred. A cue could improve change detection by improving processing in VWM (maintaining a durable representation or improved comparison), encouraging LTM retrieval, and/or reducing error in the decision process. It is important to note that these are not mutually exclusive possibilities. Cue effects on VWM If the cue improves processing in VWM, the cue may be orienting attention to the pre-change representation before it has faded or been overwritten by newly attended information (representation volatility hypothesis). It has been demonstrated that changes are often undetected because the pre-change representations are disrupted or lost (Beck & Levin, 2003; Becker & Pashler, 2002). The cue can be effective in stabilizing a representation that will otherwise fade with time or be overwritten by the test image (Averbach & Coriell, 1961; Griffin & Nobre, 2003; Landman et al., 2003; Lepsien et al., 2005; Makovski & Jiang, 2007; Matsukura, Luck, & Vecera, 2007; Sligte et al., 2008; Sperling, 1960). In these studies a cue effect was generally found when a cue was presented during the inter-stimulus interval (ISI), but not when the cue was presented with the test image, suggesting that the test image can overwrite the pre-change representation. In experiment 2 of our studies we varied the onset time of the cue to test the representation volatility hypothesis. If the representation volatility hypothesis is supported, the cue effect should be strongest when the cue occurs more closely in time to when the pre-change representation was encoded and before attention has been directed to other objects in the test image. A post-cue may also improve processing in VWM by encouraging a more complete comparison process (comparison hypothesis). Incomplete comparison processes are more likely to result in a failure to detect small changes than large changes (Awh, Barton, & Vogel, 2007). This suggests that detecting a small change requires a more effortful comparison than detecting a large change. If the cue acts to increase the amount of effort devoted to the comparison process, the post-cue effect should be larger for small changes. However, in Hollingworths (2003) study, the post-cue effect was similar for both rotation changes (smaller changes) and identity changes (larger changes). Furthermore, recent research proposes that the process of comparing information in VWM to the current perceptual information occurs automatically and is preattentive, implying that an attentional cue should not affect the comparison process (Hyun, Woodman, Vogel, Hollingworth, & Luck, 2009). In experiment 1, size of change was quantified across several dimensions (size, average color, color variation, and orientation). If the comparison hypothesis is supported, the post-cue effect should be larger for small changes than for large changes. Cue effects on LTM The evidence for whether LTM representations are used to improve change detection performance has been mixed. Experts perform better than novices on change detection tasks using stimuli within their area of expertise (Reingold, Charness, Pomplun, & Stampe, 2001; Werner & Thies, 2000), and change detection for famous faces is better than change detection for unfamiliar faces (Jackson & Raymond, 2008). These results are attributed to the ability to rely partially on LTM representations to improve performance. However, LTM is not necessarily used to improve change detection performance (Chen, Eng, & Jiang, 2006). For e (...truncated)