1
|
Jones HM, Thyer WS, Suplica D, Awh E. Cortically Disparate Visual Features Evoke Content-Independent Load Signals during Storage in Working Memory. J Neurosci 2024; 44:e0448242024. [PMID: 39327004 PMCID: PMC11529812 DOI: 10.1523/jneurosci.0448-24.2024] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/08/2024] [Revised: 08/30/2024] [Accepted: 09/05/2024] [Indexed: 09/28/2024] Open
Abstract
It is well established that holding information in working memory (WM) elicits sustained stimulus-specific patterns of neural activity. Nevertheless, here we provide evidence for a distinct class of neural activity that tracks the number of individuated items in working memory, independent of the type of visual features stored. We present two EEG studies of young adults of both sexes that provide robust evidence for a signal tracking the number of individuated representations in working memory, regardless of the specific feature values stored. In Study 1, subjects maintained either colors or orientations across separate blocks in a single session. We found near-perfect generalization of the load signal between these two conditions, despite being able to simultaneously decode which feature had been voluntarily stored. In Study 2, participants attended to two features with very distinct cortical representations: color and motion coherence. We again found evidence for a neural load signal that robustly generalized across these distinct visual features, even though cortically disparate regions process color and motion coherence. Moreover, representational similarity analysis provided converging evidence for a content-independent load signal, while simultaneously showing that unique variance in EEG activity tracked the specific features that were stored. We posit that this load signal reflects a content-independent "pointer" operation that binds objects to the current context while parallel but distinct neural signals represent the features that are stored for each item in memory.
Collapse
Affiliation(s)
- Henry M Jones
- Department of Psychology, The University of Chicago, Chicago, Illinois 60637
- Institute for Mind and Biology, The University of Chicago, Chicago, Illinois 60637
| | - William S Thyer
- Department of Psychology, The University of Chicago, Chicago, Illinois 60637
| | - Darius Suplica
- Department of Psychology, The University of Chicago, Chicago, Illinois 60637
| | - Edward Awh
- Department of Psychology, The University of Chicago, Chicago, Illinois 60637
- Institute for Mind and Biology, The University of Chicago, Chicago, Illinois 60637
| |
Collapse
|
2
|
Wei N, Song J, Zhang H, Zhou T. Unravelling the object-based nature of visual working memory: insight from pointers. Mem Cognit 2024:10.3758/s13421-024-01643-3. [PMID: 39384731 DOI: 10.3758/s13421-024-01643-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 09/21/2024] [Indexed: 10/11/2024]
Abstract
Visual working memory (VWM) plays a crucial role in temporarily storing and processing visual information, but the nature of stored representations and their interaction with new inputs has long been unclear. The pointer system refers to how VWM links new sensory inputs to stored information using specific cues. This study aimed to investigate whether the pointer system is based on spatial, feature-based, or object-based cues by employing the repetition benefit effect, where memory performance improves with repeated memory items. Across three experiments, we manipulated spatial positions, shapes, and colors as pointer cues to determine how these features affect VWM consolidation and updating. The results showed that while spatial location serves as a strong pointer cue, shape and color features can also effectively reestablish object correspondence in VWM. These findings support the view that the pointer system in VWM is flexible and object-based, utilizing various feature cues to maintain memory continuity. This study provides new insights into how VWM connects new inputs with stored information through the pointer system.
Collapse
Affiliation(s)
- Ning Wei
- China National Clinical Research Center for Neurological Diseases, Beijing Tiantan Hospital, Capital Medical University, Beijing, China.
| | - Jintao Song
- State Key Laboratory of Brain and Cognitive Science, Institute of Biophysics, Chinese Academy of Sciences, Beijing, China
- Hefei Comprehensive National Science Center, Institute of Artificial Intelligence, Hefei, China
- University of Chinese Academy of Sciences, Beijing, China
| | - Hongyi Zhang
- State Key Laboratory of Brain and Cognitive Science, Institute of Biophysics, Chinese Academy of Sciences, Beijing, China
- Hefei Comprehensive National Science Center, Institute of Artificial Intelligence, Hefei, China
- University of Chinese Academy of Sciences, Beijing, China
| | - Tiangang Zhou
- State Key Laboratory of Brain and Cognitive Science, Institute of Biophysics, Chinese Academy of Sciences, Beijing, China.
- Hefei Comprehensive National Science Center, Institute of Artificial Intelligence, Hefei, China.
- University of Chinese Academy of Sciences, Beijing, China.
| |
Collapse
|
3
|
Friedman S, Drew T, Luria R. The effect of context on pointer allocation in visual working memory. Cortex 2024; 177:170-179. [PMID: 38865761 DOI: 10.1016/j.cortex.2024.04.019] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/26/2023] [Revised: 03/21/2024] [Accepted: 04/30/2024] [Indexed: 06/14/2024]
Abstract
Visual working memory (VWM) can hold a limited amount of visual information and manipulate it. It encodes this information and forms representations of each one of the relevant objects. When an object changes, VWM can either update or reset its representation to account for this change. To access a specific representation VWM relies on a pointer system associating each representation with the corresponding object in the environment. While previous studies described these processes as reacting to a change in the object status, this study investigated the adaptability of the pointer system to the task context. We measured the contralateral delay activity (CDA; an electrophysiological marker of VWM) as a marker of updating and resetting. In two experiments we used a shape change detection task (similar to Balaban & Luria, 2017) and manipulated the proportion of the resetting and updating trials to create different task contexts. Experiment 1 indicated that VWM can adapt to a resetting mode in which it performs resetting in conditions that triggered updating in previous studies. However, Experiment 2 revealed that the pointer system cannot adapt to an updating mode and perform updating in conditions that trigger resetting. These results suggest that VWM can strategically perform resetting, but once a pointer is lost, it's impossible to update the representation and a resetting process is mandatory triggered regardless of the context.
Collapse
Affiliation(s)
- Shani Friedman
- The School of Psychological Sciences, Tel Aviv University, Tel Aviv, Israel.
| | - Trafton Drew
- Department of Psychology, University of Utah, Salt Lake City, UT, USA
| | - Roy Luria
- The School of Psychological Sciences, Tel Aviv University, Tel Aviv, Israel; The Sagol School of Neuroscience, Tel Aviv University, Tel Aviv, Israel.
| |
Collapse
|
4
|
Abstract
The body of research on visual working memory (VWM)-the system often described as a limited memory store of visual information in service of ongoing tasks-is growing rapidly. The discovery of numerous related phenomena, and the many subtly different definitions of working memory, signify a challenge to maintain a coherent theoretical framework to discuss concepts, compare models and design studies. A lack of robust theory development has been a noteworthy concern in the psychological sciences, thought to be a precursor to the reproducibility crisis (Oberauer & Lewandowsky, Psychonomic Bulletin & Review, 26, 1596-1618, 2019). I review the theoretical landscape of the VWM field by examining two prominent debates-whether VWM is object-based or feature-based, and whether discrete-slots or variable-precision best describe VWM limits. I share my concerns about the dualistic nature of these debates and the lack of clear model specification that prevents fully determined empirical tests. In hopes of promoting theory development, I provide a working theory map by using the broadly encompassing memory for latent representations model (Hedayati et al., Nature Human Behaviour, 6, 5, 2022) as a scaffold for relevant phenomena and current theories. I illustrate how opposing viewpoints can be brought into accordance, situating leading models of VWM to better identify their differences and improve their comparison. The hope is that the theory map will help VWM researchers get on the same page-clarifying hidden intuitions and aligning varying definitions-and become a useful device for meaningful discussions, development of models, and definitive empirical tests of theories.
Collapse
Affiliation(s)
- William Xiang Quan Ngiam
- Department of Psychology, University of Chicago, Chicago, Illinois, USA.
- Institute of Mind and Biology, University of Chicago, Chicago, Illinois, USA.
| |
Collapse
|
5
|
Yu X, Li J, Zhu H, Tian X, Lau E. Electrophysiological hallmarks for event relations and event roles in working memory. Front Neurosci 2024; 17:1282869. [PMID: 38328555 PMCID: PMC10847304 DOI: 10.3389/fnins.2023.1282869] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/25/2023] [Accepted: 12/22/2023] [Indexed: 02/09/2024] Open
Abstract
The ability to maintain events (i.e., interactions between/among objects) in working memory is crucial for our everyday cognition, yet the format of this representation is poorly understood. The current ERP study was designed to answer two questions: How is maintaining events (e.g., the tiger hit the lion) neurally different from maintaining item coordinations (e.g., the tiger and the lion)? That is, how is the event relation (present in events but not coordinations) represented? And how is the agent, or initiator of the event encoded differently from the patient, or receiver of the event during maintenance? We used a novel picture-sentence match-across-delay approach in which the working memory representation was "pinged" during the delay, replicated across two ERP experiments with Chinese and English materials. We found that maintenance of events elicited a long-lasting late sustained difference in posterior-occipital electrodes relative to non-events. This effect resembled the negative slow wave reported in previous studies of working memory, suggesting that the maintenance of events in working memory may impose a higher cost compared to coordinations. Although we did not observe significant ERP differences associated with pinging the agent vs. the patient during the delay, we did find that the ping appeared to dampen the ongoing sustained difference, suggesting a shift from sustained activity to activity silent mechanisms. These results suggest a new method by which ERPs can be used to elucidate the format of neural representation for events in working memory.
Collapse
Affiliation(s)
- Xinchi Yu
- Program of Neuroscience and Cognitive Science, University of Maryland, College Park, MD, United States
- Department of Linguistics, University of Maryland, College Park, MD, United States
| | - Jialu Li
- Division of Arts and Sciences, New York University Shanghai, Shanghai, China
- Shanghai Key Laboratory of Brain Functional Genomics (Ministry of Education), School of Psychology and Cognitive Science, East China Normal University, Shanghai, China
- NYU-ECNU Institute of Brain and Cognitive Science at NYU Shanghai, Shanghai, China
| | - Hao Zhu
- Division of Arts and Sciences, New York University Shanghai, Shanghai, China
- Shanghai Key Laboratory of Brain Functional Genomics (Ministry of Education), School of Psychology and Cognitive Science, East China Normal University, Shanghai, China
- NYU-ECNU Institute of Brain and Cognitive Science at NYU Shanghai, Shanghai, China
| | - Xing Tian
- Division of Arts and Sciences, New York University Shanghai, Shanghai, China
- Shanghai Key Laboratory of Brain Functional Genomics (Ministry of Education), School of Psychology and Cognitive Science, East China Normal University, Shanghai, China
- NYU-ECNU Institute of Brain and Cognitive Science at NYU Shanghai, Shanghai, China
| | - Ellen Lau
- Program of Neuroscience and Cognitive Science, University of Maryland, College Park, MD, United States
- Department of Linguistics, University of Maryland, College Park, MD, United States
| |
Collapse
|
6
|
Balaban H, Drew T, Luria R. Dissociable online integration processes in visual working memory. Cereb Cortex 2023; 33:11420-11430. [PMID: 37814362 DOI: 10.1093/cercor/bhad378] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/06/2023] [Revised: 09/16/2023] [Accepted: 09/17/2023] [Indexed: 10/11/2023] Open
Abstract
Visual working memory has severe capacity limits, creating a bottleneck for active processing. A key way of mitigating this limitation is by chunking, i.e. compressing several pieces of information into one visual working memory representation. However, despite decades of research, chunking efficiency remains debated because of mixed evidence. We propose that there are actually 2 integration mechanisms: Grouping combines several objects to one representation, and object-unification merges the parts of a single object. Critically, we argue that the fundamental distinction between the 2 processes is their differential use of the pointer system, the indexing process connecting visual working memory representations with perception. In grouping, the objects that are represented together still maintain independent pointers, making integration costly but highly flexible. Conversely, object-unification fuses the pointers as well as the representations, with the single pointer producing highly efficient integration but blocking direct access to individual parts. We manipulated integration cues via task-irrelevant movement, and monitored visual working memory's online electrophysiological marker. Uniquely colored objects were flexibly grouped and ungrouped via independent pointers (experiment 1). If objects turned uniformly black, object-integration could not be undone (experiment 2), requiring visual working memory to reset before re-individuation. This demonstrates 2 integration levels (representational-merging versus pointer-compression) and establishes the dissociation between visual working memory representations and their underlying pointers.
Collapse
Affiliation(s)
- Halely Balaban
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, 02139 MA, United States
- Psychology Department, Harvard University, Cambridge, 02138 MA, United States
| | - Trafton Drew
- Psychology Department, University of Utah, Salt Lake City, 84112 UT, United States
| | - Roy Luria
- Sagol School of Neuroscience, Tel Aviv University, Tel Aviv 6997801, Israel
- The School of Psychological Sciences, Tel Aviv University, Tel Aviv 6997801, Israel
| |
Collapse
|
7
|
Roy Y, Faubert J. Is the Contralateral Delay Activity (CDA) a robust neural correlate for Visual Working Memory (VWM) tasks? A reproducibility study. Psychophysiology 2023; 60:e14180. [PMID: 36124370 PMCID: PMC10078237 DOI: 10.1111/psyp.14180] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/18/2022] [Revised: 08/03/2022] [Accepted: 08/22/2022] [Indexed: 01/07/2023]
Abstract
Visual working memory (VWM) allows us to actively store, update, and manipulate visual information surrounding us. While the underlying neural mechanisms of VWM remain unclear, contralateral delay activity (CDA), a sustained negativity over the hemisphere contralateral to the positions of visual items to be remembered, is often used to study VWM. To investigate if the CDA is a robust neural correlate for VWM tasks, we reproduced eight CDA-related studies with a publicly accessible EEG data set. We used the raw EEG data from these eight studies and analyzed all of them with the same basic pipeline to extract CDA. We were able to reproduce the results from all the studies and show that with a basic automated EEG pipeline we can extract a clear CDA signal. We share insights from the trends observed across the studies and raise some questions about the CDA decay and the CDA during the recall phase, which surprisingly, none of the eight studies did address. Finally, we also provide reproducibility recommendations based on our experience and challenges in reproducing these studies.
Collapse
Affiliation(s)
- Yannick Roy
- Faubert Lab, Université de Montréal, Montréal, Canada
| | | |
Collapse
|
8
|
Thyer W, Adam KCS, Diaz GK, Velázquez Sánchez IN, Vogel EK, Awh E. Storage in Visual Working Memory Recruits a Content-Independent Pointer System. Psychol Sci 2022; 33:1680-1694. [PMID: 36006809 PMCID: PMC9630722 DOI: 10.1177/09567976221090923] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/26/2021] [Accepted: 02/24/2022] [Indexed: 11/17/2022] Open
Abstract
Past work has shown that storage in working memory elicits stimulus-specific neural activity that tracks the stored content. Here, we present evidence for a distinct class of load-sensitive neural activity that indexes items without representing their contents per se. We recorded electroencephalogram (EEG) activity while adult human subjects stored varying numbers of items in visual working memory. Multivariate analysis of the scalp topography of EEG voltage enabled precise tracking of the number of individuated items stored and robustly predicted individual differences in working memory capacity. Critically, this signature of working memory load generalized across variations in both the type and number of visual features stored about each item, suggesting that it tracked the number of individuated memory representations and not the content of those memories. We hypothesize that these findings reflect the operation of a capacity-limited pointer system that supports on-line storage and attentive tracking.
Collapse
Affiliation(s)
- William Thyer
- Department of Psychology, The
University of Chicago
- Institute for Mind and Biology, The
University of Chicago
| | - Kirsten C. S. Adam
- Department of Psychology, University of
California San Diego
- Institute for Neural Computation,
University of California San Diego
| | - Gisella K. Diaz
- Department of Psychology, The
University of Chicago
- Institute for Mind and Biology, The
University of Chicago
| | - Itzel N. Velázquez Sánchez
- Department of Psychology, The
University of Chicago
- Institute for Mind and Biology, The
University of Chicago
| | - Edward K. Vogel
- Department of Psychology, The
University of Chicago
- Institute for Mind and Biology, The
University of Chicago
| | - Edward Awh
- Department of Psychology, The
University of Chicago
- Institute for Mind and Biology, The
University of Chicago
| |
Collapse
|
9
|
Li AY, Fukuda K, Barense MD. Independent features form integrated objects: Using a novel shape-color “conjunction task” to reconstruct memory resolution for multiple object features simultaneously. Cognition 2022; 223:105024. [DOI: 10.1016/j.cognition.2022.105024] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/19/2021] [Revised: 12/17/2021] [Accepted: 01/13/2022] [Indexed: 11/16/2022]
|
10
|
Abstract
Area V4-the focus of this review-is a mid-level processing stage along the ventral visual pathway of the macaque monkey. V4 is extensively interconnected with other visual cortical areas along the ventral and dorsal visual streams, with frontal cortical areas, and with several subcortical structures. Thus, it is well poised to play a broad and integrative role in visual perception and recognition-the functional domain of the ventral pathway. Neurophysiological studies in monkeys engaged in passive fixation and behavioral tasks suggest that V4 responses are dictated by tuning in a high-dimensional stimulus space defined by form, texture, color, depth, and other attributes of visual stimuli. This high-dimensional tuning may underlie the development of object-based representations in the visual cortex that are critical for tracking, recognizing, and interacting with objects. Neurophysiological and lesion studies also suggest that V4 responses are important for guiding perceptual decisions and higher-order behavior.
Collapse
Affiliation(s)
- Anitha Pasupathy
- Department of Biological Structure, University of Washington, Seattle, Washington 98195, USA; ,
- Washington National Primate Research Center, University of Washington, Seattle, Washington 98121, USA
| | - Dina V Popovkina
- Department of Psychology, University of Washington, Seattle, Washington 98105, USA;
| | - Taekjun Kim
- Department of Biological Structure, University of Washington, Seattle, Washington 98195, USA; ,
- Washington National Primate Research Center, University of Washington, Seattle, Washington 98121, USA
| |
Collapse
|