1
|
Huang J, Wang T, Dai W, Li Y, Yang Y, Zhang Y, Wu Y, Zhou T, Xing D. Neuronal representation of visual working memory content in the primate primary visual cortex. SCIENCE ADVANCES 2024; 10:eadk3953. [PMID: 38875332 PMCID: PMC11177929 DOI: 10.1126/sciadv.adk3953] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/19/2023] [Accepted: 05/10/2024] [Indexed: 06/16/2024]
Abstract
The human ability to perceive vivid memories as if they "float" before our eyes, even in the absence of actual visual stimuli, captivates the imagination. To determine the neural substrates underlying visual memories, we investigated the neuronal representation of working memory content in the primary visual cortex of monkeys. Our study revealed that neurons exhibit unique responses to different memory contents, using firing patterns distinct from those observed during the perception of external visual stimuli. Moreover, this neuronal representation evolves with alterations in the recalled content and extends beyond the retinotopic areas typically reserved for processing external visual input. These discoveries shed light on the visual encoding of memories and indicate avenues for understanding the remarkable power of the mind's eye.
Collapse
Affiliation(s)
- Jiancao Huang
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, China
| | - Tian Wang
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, China
- College of Life Sciences, Beijing Normal University, Beijing 100875, China
| | - Weifeng Dai
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, China
| | - Yang Li
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, China
| | - Yi Yang
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, China
| | - Yange Zhang
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, China
| | - Yujie Wu
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, China
| | - Tingting Zhou
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, China
| | - Dajun Xing
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, China
| |
Collapse
|
2
|
Bays PM, Schneegans S, Ma WJ, Brady TF. Representation and computation in visual working memory. Nat Hum Behav 2024; 8:1016-1034. [PMID: 38849647 DOI: 10.1038/s41562-024-01871-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/29/2022] [Accepted: 03/22/2024] [Indexed: 06/09/2024]
Abstract
The ability to sustain internal representations of the sensory environment beyond immediate perception is a fundamental requirement of cognitive processing. In recent years, debates regarding the capacity and fidelity of the working memory (WM) system have advanced our understanding of the nature of these representations. In particular, there is growing recognition that WM representations are not merely imperfect copies of a perceived object or event. New experimental tools have revealed that observers possess richer information about the uncertainty in their memories and take advantage of environmental regularities to use limited memory resources optimally. Meanwhile, computational models of visuospatial WM formulated at different levels of implementation have converged on common principles relating capacity to variability and uncertainty. Here we review recent research on human WM from a computational perspective, including the neural mechanisms that support it.
Collapse
Affiliation(s)
- Paul M Bays
- Department of Psychology, University of Cambridge, Cambridge, UK
| | | | - Wei Ji Ma
- Center for Neural Science and Department of Psychology, New York University, New York, NY, USA
| | - Timothy F Brady
- Department of Psychology, University of California, San Diego, La Jolla, CA, USA.
| |
Collapse
|
3
|
Kumle L, Võ MLH, Nobre AC, Draschkow D. Multifaceted consequences of visual distraction during natural behaviour. COMMUNICATIONS PSYCHOLOGY 2024; 2:49. [PMID: 38812582 PMCID: PMC11129948 DOI: 10.1038/s44271-024-00099-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/09/2023] [Accepted: 05/15/2024] [Indexed: 05/31/2024]
Abstract
Visual distraction is a ubiquitous aspect of everyday life. Studying the consequences of distraction during temporally extended tasks, however, is not tractable with traditional methods. Here we developed a virtual reality approach that segments complex behaviour into cognitive subcomponents, including encoding, visual search, working memory usage, and decision-making. Participants copied a model display by selecting objects from a resource pool and placing them into a workspace. By manipulating the distractibility of objects in the resource pool, we discovered interfering effects of distraction across the different cognitive subcomponents. We successfully traced the consequences of distraction all the way from overall task performance to the decision-making processes that gate memory usage. Distraction slowed down behaviour and increased costly body movements. Critically, distraction increased encoding demands, slowed visual search, and decreased reliance on working memory. Our findings illustrate that the effects of visual distraction during natural behaviour can be rather focal but nevertheless have cascading consequences.
Collapse
Affiliation(s)
- Levi Kumle
- Department of Experimental Psychology, University of Oxford, Oxford, UK
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK
| | - Melissa L.-H. Võ
- Department of Psychology, Goethe University Frankfurt, Frankfurt, Germany
| | - Anna C. Nobre
- Department of Experimental Psychology, University of Oxford, Oxford, UK
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK
- Wu Tsai Institute and Department of Psychology, Yale University, New Haven, CT USA
| | - Dejan Draschkow
- Department of Experimental Psychology, University of Oxford, Oxford, UK
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK
| |
Collapse
|
4
|
Santo-Angles A, Temudo A, Babushkin V, Sreenivasan KK. Effective connectivity of working memory performance: a DCM study of MEG data. Front Hum Neurosci 2024; 18:1339728. [PMID: 38501039 PMCID: PMC10944968 DOI: 10.3389/fnhum.2024.1339728] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/16/2023] [Accepted: 02/12/2024] [Indexed: 03/20/2024] Open
Abstract
Visual working memory (WM) engages several nodes of a large-scale network that includes frontal, parietal, and visual regions; however, little is understood about how these regions interact to support WM behavior. In particular, it is unclear whether network dynamics during WM maintenance primarily represent feedforward or feedback connections. This question has important implications for current debates about the relative roles of frontoparietal and visual regions in WM maintenance. In the current study, we investigated the network activity supporting WM using MEG data acquired while healthy subjects performed a multi-item delayed estimation WM task. We used computational modeling of behavior to discriminate correct responses (high accuracy trials) from two different types of incorrect responses (low accuracy and swap trials), and dynamic causal modeling of MEG data to measure effective connectivity. We observed behaviorally dependent changes in effective connectivity in a brain network comprising frontoparietal and early visual areas. In comparison with high accuracy trials, frontoparietal and frontooccipital networks showed disrupted signals depending on type of behavioral error. Low accuracy trials showed disrupted feedback signals during early portions of WM maintenance and disrupted feedforward signals during later portions of maintenance delay, while swap errors showed disrupted feedback signals during the whole delay period. These results support a distributed model of WM that emphasizes the role of visual regions in WM storage and where changes in large scale network configurations can have important consequences for memory-guided behavior.
Collapse
Affiliation(s)
- Aniol Santo-Angles
- Division of Science and Mathematics, New York University Abu Dhabi, Abu Dhabi, United Arab Emirates
- Center for Brain and Health, New York University Abu Dhabi, Abu Dhabi, United Arab Emirates
| | - Ainsley Temudo
- Division of Science and Mathematics, New York University Abu Dhabi, Abu Dhabi, United Arab Emirates
| | - Vahan Babushkin
- Division of Science and Mathematics, New York University Abu Dhabi, Abu Dhabi, United Arab Emirates
| | - Kartik K. Sreenivasan
- Division of Science and Mathematics, New York University Abu Dhabi, Abu Dhabi, United Arab Emirates
- Center for Brain and Health, New York University Abu Dhabi, Abu Dhabi, United Arab Emirates
| |
Collapse
|
5
|
Teng C, Kaplan SM, Shomstein S, Kravitz DJ. Assessing the interaction between working memory and perception through time. Atten Percept Psychophys 2023; 85:2196-2209. [PMID: 37740152 DOI: 10.3758/s13414-023-02785-3] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 08/31/2023] [Indexed: 09/24/2023]
Abstract
Content maintained in visual working memory changes concurrent visual processing, suggesting that visual working memory may recruit an overlapping neural representation with visual perception. However, it remains unclear whether visual working memory representations persist as a sensory code through time, or are recoded later into an abstract code. Here, we directly contrasted a temporal decay + visual code account and a temporal decay + abstract code account within the temporal dynamics of the interaction between working memory and perception. By manipulating the ISI (inter-stimulus interval) between working memory encoding and a perceptual discrimination task, we found that task-relevant and therefore actively maintained perceptual information parametrically altered participants' ability to discriminate perceptual stimuli even 4 s after encoding, whereas task-irrelevant information caused only an acutely transient effect. While continuously present, the size of this shift in discrimination thresholds gradually decreased over time. Concomitantly, the size of the bias in working memory reports increased over time. The opposing directions of threshold and bias effects are consistent with the local maintenance of information in perceptual areas, explained by a temporal decay + visual code account. As the maintained representation decays over time, its ability to alter incoming perceptual signals decreases (reduced threshold effects) while its likelihood of being impacted by those same signals increases (increased bias effects). Altogether, these results suggest that the readout of working memory relies on a sensory representation at a cost of increased interference by ongoing perception.
Collapse
Affiliation(s)
- Chunyue Teng
- Department of Neuroscience, Lawrence University, Appleton, WI, USA.
| | - Simon M Kaplan
- Department of Psychological and Brain Sciences, George Washington University, Washington, DC, USA
| | - Sarah Shomstein
- Department of Psychological and Brain Sciences, George Washington University, Washington, DC, USA
| | - Dwight J Kravitz
- Department of Psychological and Brain Sciences, George Washington University, Washington, DC, USA
- Directorate for Social, Behavioral, and Economic Sciences, National Science Foundation, Arlington, VA, USA
| |
Collapse
|
6
|
Yu X, Zhou Z, Becker SI, Boettcher SEP, Geng JJ. Good-enough attentional guidance. Trends Cogn Sci 2023; 27:391-403. [PMID: 36841692 DOI: 10.1016/j.tics.2023.01.007] [Citation(s) in RCA: 10] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/02/2022] [Revised: 01/24/2023] [Accepted: 01/25/2023] [Indexed: 02/27/2023]
Abstract
Theories of attention posit that attentional guidance operates on information held in a target template within memory. The template is often thought to contain veridical target features, akin to a photograph, and to guide attention to objects that match the exact target features. However, recent evidence suggests that attentional guidance is highly flexible and often guided by non-veridical features, a subset of features, or only associated features. We integrate these findings and propose that attentional guidance maximizes search efficiency based on a 'good-enough' principle to rapidly localize candidate target objects. Candidates are then serially interrogated to make target-match decisions using more precise information. We suggest that good-enough guidance optimizes the speed-accuracy-effort trade-offs inherent in each stage of visual search.
Collapse
Affiliation(s)
- Xinger Yu
- Center for Mind and Brain, University of California Davis, Davis, CA, USA; Department of Psychology, University of California Davis, Davis, CA, USA
| | - Zhiheng Zhou
- Center for Mind and Brain, University of California Davis, Davis, CA, USA
| | - Stefanie I Becker
- School of Psychology, University of Queensland, Brisbane, QLD, Australia
| | | | - Joy J Geng
- Center for Mind and Brain, University of California Davis, Davis, CA, USA; Department of Psychology, University of California Davis, Davis, CA, USA.
| |
Collapse
|
7
|
Phylactou P, Shimi A, Konstantinou N. Causal evidence for the role of the sensory visual cortex in visual short-term memory maintenance. ROYAL SOCIETY OPEN SCIENCE 2023; 10:230321. [PMID: 37090966 PMCID: PMC10113812 DOI: 10.1098/rsos.230321] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 03/21/2023] [Accepted: 03/27/2023] [Indexed: 05/03/2023]
Abstract
The role of the sensory visual cortex during visual short-term memory (VSTM) remains controversial. This controversy is possibly due to methodological issues in previous attempts to investigate the effects of transcranial magnetic stimulation (TMS) on VSTM. The aim of this study was to use TMS, while covering previous methodological deficits. Sixty-four young adults were recruited to participate in two experiments (Experiment 1: n = 36; Experiment 2: n = 28) using a VSTM orientation change-detection task under TMS. Monocular vision was ensured using red-blue goggles combined with red-blue stimuli. Double-pulse TMS was delivered at different times (Experiment 1: 0, 200 or 1000 ms; Experiment 2: 200, 1000 ms) during a 2 s maintenance phase, on one side of the occipital hemisphere. In Experiment 2, a sham TMS condition was introduced. Decreased detection sensitivity (d') in the ipsilateral occipital hemisphere to visual hemifield, and in the real TMS (compared with sham TMS) condition indicated inhibitory TMS effects, and thus, a causal involvement of the sensory visual cortex during early (200 ms) and late (1000 ms) maintenance in VSTM. These findings are aligned with sensory recruitment, which proposes that both perceptual and memory processes rely upon the same neural substrates in the sensory visual cortex. The methods used in this study were preregistered and had received in-principle acceptance on 6 June 2022 (Stage 1 protocol can be found in: https://doi.org/10.17605/OSF.IO/EMPDT).
Collapse
Affiliation(s)
- Phivos Phylactou
- Department of Rehabilitation Sciences, Faculty of Health Sciences, Cyprus University of Technology, Limassol 3041, Cyprus
| | - Andria Shimi
- Department of Psychology, Faculty of Social Sciences and Education, University of Cyprus, CY-1678 Nicosia, Cyprus
| | - Nikos Konstantinou
- Department of Rehabilitation Sciences, Faculty of Health Sciences, Cyprus University of Technology, Limassol 3041, Cyprus
| |
Collapse
|
8
|
Ahveninen J, Uluç I, Raij T, Nummenmaa A, Mamashli F. Spectrotemporal content of human auditory working memory represented in functional connectivity patterns. Commun Biol 2023; 6:294. [PMID: 36941477 PMCID: PMC10027691 DOI: 10.1038/s42003-023-04675-8] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/10/2022] [Accepted: 03/07/2023] [Indexed: 03/23/2023] Open
Abstract
Recent research suggests that working memory (WM), the mental sketchpad underlying thinking and communication, is maintained by multiple regions throughout the brain. Whether parts of a stable WM representation could be distributed across these brain regions is, however, an open question. We addressed this question by examining the content-specificity of connectivity-pattern matrices between subparts of cortical regions-of-interest (ROI). These connectivity patterns were calculated from functional MRI obtained during a ripple-sound auditory WM task. Statistical significance was assessed by comparing the decoding results to a null distribution derived from a permutation test considering all comparable two- to four-ROI connectivity patterns. Maintained WM items could be decoded from connectivity patterns across ROIs in frontal, parietal, and superior temporal cortices. All functional connectivity patterns that were specific to maintained sound content extended from early auditory to frontoparietal cortices. Our results demonstrate that WM maintenance is supported by content-specific patterns of functional connectivity across different levels of cortical hierarchy.
Collapse
Affiliation(s)
- Jyrki Ahveninen
- Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Charlestown, MA, USA.
- Department of Radiology, Harvard Medical School, Boston, MA, USA.
| | - Işıl Uluç
- Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Charlestown, MA, USA
- Department of Radiology, Harvard Medical School, Boston, MA, USA
| | - Tommi Raij
- Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Charlestown, MA, USA
- Department of Radiology, Harvard Medical School, Boston, MA, USA
| | - Aapo Nummenmaa
- Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Charlestown, MA, USA
- Department of Radiology, Harvard Medical School, Boston, MA, USA
| | - Fahimeh Mamashli
- Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Charlestown, MA, USA
- Department of Radiology, Harvard Medical School, Boston, MA, USA
| |
Collapse
|
9
|
Quilty-Dunn J, Porot N, Mandelbaum E. The best game in town: The reemergence of the language-of-thought hypothesis across the cognitive sciences. Behav Brain Sci 2022; 46:e261. [PMID: 36471543 DOI: 10.1017/s0140525x22002849] [Citation(s) in RCA: 12] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/12/2022]
Abstract
Mental representations remain the central posits of psychology after many decades of scrutiny. However, there is no consensus about the representational format(s) of biological cognition. This paper provides a survey of evidence from computational cognitive psychology, perceptual psychology, developmental psychology, comparative psychology, and social psychology, and concludes that one type of format that routinely crops up is the language-of-thought (LoT). We outline six core properties of LoTs: (i) discrete constituents; (ii) role-filler independence; (iii) predicate-argument structure; (iv) logical operators; (v) inferential promiscuity; and (vi) abstract content. These properties cluster together throughout cognitive science. Bayesian computational modeling, compositional features of object perception, complex infant and animal reasoning, and automatic, intuitive cognition in adults all implicate LoT-like structures. Instead of regarding LoT as a relic of the previous century, researchers in cognitive science and philosophy-of-mind must take seriously the explanatory breadth of LoT-based architectures. We grant that the mind may harbor many formats and architectures, including iconic and associative structures as well as deep-neural-network-like architectures. However, as computational/representational approaches to the mind continue to advance, classical compositional symbolic structures - that is, LoTs - only prove more flexible and well-supported over time.
Collapse
Affiliation(s)
- Jake Quilty-Dunn
- Department of Philosophy and Philosophy-Neuroscience-Psychology Program, Washington University in St. Louis, St. Louis, MO, USA. , sites.google.com/site/jakequiltydunn/
| | - Nicolas Porot
- Africa Institute for Research in Economics and Social Sciences, Mohammed VI Polytechnic University, Rabat, Morocco. , nicolasporot.com
| | - Eric Mandelbaum
- Departments of Philosophy and Psychology, The Graduate Center & Baruch College, CUNY, New York, NY, USA. , ericmandelbaum.com
| |
Collapse
|
10
|
Wang XJ. Theory of the Multiregional Neocortex: Large-Scale Neural Dynamics and Distributed Cognition. Annu Rev Neurosci 2022; 45:533-560. [PMID: 35803587 DOI: 10.1146/annurev-neuro-110920-035434] [Citation(s) in RCA: 19] [Impact Index Per Article: 9.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/02/2023]
Abstract
The neocortex is a complex neurobiological system with many interacting regions. How these regions work together to subserve flexible behavior and cognition has become increasingly amenable to rigorous research. Here, I review recent experimental and theoretical work on the modus operandi of a multiregional cortex. These studies revealed several general principles for the neocortical interareal connectivity, low-dimensional macroscopic gradients of biological properties across cortical areas, and a hierarchy of timescales for information processing. Theoretical work suggests testable predictions regarding differential excitation and inhibition along feedforward and feedback pathways in the cortical hierarchy. Furthermore, modeling of distributed working memory and simple decision-making has given rise to a novel mathematical concept, dubbed bifurcation in space, that potentially explains how different cortical areas, with a canonical circuit organization but gradients of biological heterogeneities, are able to subserve their respective (e.g., sensory coding versus executive control) functions in a modularly organized brain.
Collapse
Affiliation(s)
- Xiao-Jing Wang
- Center for Neural Science, New York University, New York, NY, USA;
| |
Collapse
|
11
|
Sensory recruitment in visual short-term memory: A systematic review and meta-analysis of sensory visual cortex interference using transcranial magnetic stimulation. Psychon Bull Rev 2022; 29:1594-1624. [PMID: 35606595 DOI: 10.3758/s13423-022-02107-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 04/13/2022] [Indexed: 11/08/2022]
Abstract
Sensory visual areas are involved in encoding information in visual short-term memory (VSTM). Yet it remains unclear whether sensory visual cortex is a necessary component of the brain network for maintenance of information in VSTM. Here, we aimed to systematically review studies that have investigated the role of the sensory visual cortex in VSTM using transcranial magnetic stimulation (TMS) and to quantitatively explore these effects using meta-analyses. Fourteen studies were identified and reviewed. Eight studies provided sufficient data for meta-analysis. Two meta-analyses, one regarding the VSTM encoding phase (17 effect sizes) and one regarding the VSTM maintenance phase (15 effect sizes), two meta-regressions (32 effect sizes in each), and one exploratory meta-analysis were conducted. Our results indicate that the sensory visual cortex is similarly involved in both the encoding and maintenance VSTM phase. We suggest that some cases where evidence did not show significant TMS effects was due to low memory or perceptual task demands. Overall, these findings support the idea that sensory visual areas are part of the brain network responsible for successfully maintaining information in VSTM.
Collapse
|
12
|
Henderson MM, Rademaker RL, Serences JT. Flexible utilization of spatial- and motor-based codes for the storage of visuo-spatial information. eLife 2022; 11:e75688. [PMID: 35522567 PMCID: PMC9075954 DOI: 10.7554/elife.75688] [Citation(s) in RCA: 15] [Impact Index Per Article: 7.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/19/2021] [Accepted: 04/24/2022] [Indexed: 01/26/2023] Open
Abstract
Working memory provides flexible storage of information in service of upcoming behavioral goals. Some models propose specific fixed loci and mechanisms for the storage of visual information in working memory, such as sustained spiking in parietal and prefrontal cortex during working memory maintenance. An alternative view is that information can be remembered in a flexible format that best suits current behavioral goals. For example, remembered visual information might be stored in sensory areas for easier comparison to future sensory inputs, or might be re-coded into a more abstract action-oriented format and stored in motor areas. Here, we tested this hypothesis using a visuo-spatial working memory task where the required behavioral response was either known or unknown during the memory delay period. Using functional magnetic resonance imaging (fMRI) and multivariate decoding, we found that there was less information about remembered spatial position in early visual and parietal regions when the required response was known versus unknown. Furthermore, a representation of the planned motor action emerged in primary somatosensory, primary motor, and premotor cortex during the same task condition where spatial information was reduced in early visual cortex. These results suggest that the neural networks supporting working memory can be strategically reconfigured depending on specific behavioral requirements during a canonical visual working memory paradigm.
Collapse
Affiliation(s)
- Margaret M Henderson
- Neurosciences Graduate Program, University of California, San DiegoSan DiegoUnited States
- Department of Machine Learning, Carnegie Mellon UniversityPittsburghUnited States
- Neuroscience Institute, Carnegie Mellon UniversityPittsburghUnited States
| | - Rosanne L Rademaker
- Department of Psychology, University of California, San DiegoSan DiegoUnited States
- Ernst Strüngmann Institute (ESI) for Neuroscience in Cooperation with Max Planck SocietyFrankfurtGermany
| | - John T Serences
- Neurosciences Graduate Program, University of California, San DiegoSan DiegoUnited States
- Department of Psychology, University of California, San DiegoSan DiegoUnited States
- Kavli Foundation for the Brain and Mind, University of California, San DiegoSan DiegoUnited States
| |
Collapse
|
13
|
Wang B, Knapen T, Olivers CNL. Visual Working Memory Adapts to the Nature of Anticipated Interference. J Cogn Neurosci 2022; 34:1148-1163. [PMID: 35468211 DOI: 10.1162/jocn_a_01853] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/04/2022]
Abstract
Visual working memory has been proven to be relatively robust against interference. However, little is known on whether such robust coding is obligatory, or can be flexibly recruited depending on its expected usefulness. To address this, participants remembered both the color and orientation of a grating. During the maintenance, we inserted a secondary color/orientation memory task, interfering with the primary task. Crucially, we varied the expectations of the type of interference by varying the probability of the two types of intervening task. Behavioral data indicate that to-be-remembered features for which interference is expected are bolstered, whereas to-be-remembered features for which no interference is expected are left vulnerable. This was further supported by fMRI data obtained from visual cortex. In conclusion, the flexibility of visual working memory allows it to strengthen memories for which it anticipates the highest risk of interference.
Collapse
Affiliation(s)
- Benchi Wang
- South China Normal University, China.,Cognition and Education Sciences (South China Normal University), China.,Vrije Universiteit Amsterdam, The Netherlands
| | | | | |
Collapse
|
14
|
Delay activity during visual working memory: A meta-analysis of 30 fMRI experiments. Neuroimage 2022; 255:119204. [PMID: 35427771 DOI: 10.1016/j.neuroimage.2022.119204] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/17/2021] [Revised: 04/04/2022] [Accepted: 04/08/2022] [Indexed: 01/22/2023] Open
Abstract
Visual working memory refers to the temporary maintenance and manipulation of task-related visual information. Recent debate on the underlying neural substrates of visual working memory has focused on the delay period of relevant tasks. Persistent neural activity throughout the delay period has been recognized as a correlate of working memory, yet regions demonstrating sustained hemodynamic responses show inconsistency across individual studies. To develop a more precise understanding of delay-period activations during visual working memory, we conducted a coordinate-based meta-analysis on 30 fMRI experiments involving 515 healthy adults with a mean age of 25.65 years. The main analysis revealed a widespread frontoparietal network associated with delay-period activity, as well as activation in the right inferior temporal cortex. These findings were replicated using different meta-analytical algorithms and were shown to be robust against between-study heterogeneity and publication bias. Further meta-analyses on different subgroups of experiments with specific task demands and stimulus types revealed similar delay-period networks, with activations distributed across the frontal and parietal cortices. The roles of prefrontal regions, posterior parietal regions, and inferior temporal areas are reviewed and discussed in the context of content-specific storage. We conclude that cognitive operations that occur during the unfilled delay period in visual working memory tasks can be flexibly expressed across a frontoparietal-temporal network depending on experimental parameters.
Collapse
|
15
|
Gresch D, Boettcher SEP, van Ede F, Nobre AC. Shielding working-memory representations from temporally predictable external interference. Cognition 2021; 217:104915. [PMID: 34600356 PMCID: PMC8543071 DOI: 10.1016/j.cognition.2021.104915] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/05/2021] [Revised: 09/01/2021] [Accepted: 09/21/2021] [Indexed: 12/15/2022]
Abstract
Protecting working-memory content from distracting external sensory inputs and intervening tasks is an ubiquitous demand in daily life. Here, we ask whether and how temporal expectations about external events can help mitigate effects of such interference during working-memory retention. We manipulated the temporal predictability of interfering items that occurred during the retention period of a visual working-memory task and report that temporal expectations reduce the detrimental influence of external interference on subsequent memory performance. Moreover, to determine if the protective effects of temporal expectations rely on distractor suppression or involve shielding of internal representations, we compared effects after irrelevant distractors that could be ignored vs. interrupters that required a response. Whereas distractor suppression may be sufficient to confer protection from predictable distractors, any benefits after interruption are likely to involve memory shielding. We found similar benefits of temporal expectations after both types of interference. We conclude that temporal expectations may play an important role in safeguarding behaviour based on working memory - acting through mechanisms that include the shielding of internal content from external interference.
Collapse
Affiliation(s)
- Daniela Gresch
- Department of Experimental Psychology, University of Oxford, Oxford, UK; Oxford Centre for Human Brain Activity, Wellcome Centre for Integrative Neuroimaging, Department of Psychiatry, University of Oxford, Oxford, UK.
| | - Sage E P Boettcher
- Department of Experimental Psychology, University of Oxford, Oxford, UK; Oxford Centre for Human Brain Activity, Wellcome Centre for Integrative Neuroimaging, Department of Psychiatry, University of Oxford, Oxford, UK
| | - Freek van Ede
- Institute for Brain and Behavior Amsterdam, Department of Experimental and Applied Psychology, Vrije Universiteit Amsterdam, The Netherlands
| | - Anna C Nobre
- Department of Experimental Psychology, University of Oxford, Oxford, UK; Oxford Centre for Human Brain Activity, Wellcome Centre for Integrative Neuroimaging, Department of Psychiatry, University of Oxford, Oxford, UK
| |
Collapse
|
16
|
Hallenbeck GE, Sprague TC, Rahmati M, Sreenivasan KK, Curtis CE. Working memory representations in visual cortex mediate distraction effects. Nat Commun 2021; 12:4714. [PMID: 34354071 PMCID: PMC8342709 DOI: 10.1038/s41467-021-24973-1] [Citation(s) in RCA: 27] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/20/2021] [Accepted: 07/13/2021] [Indexed: 11/17/2022] Open
Abstract
Although the contents of working memory can be decoded from visual cortex activity, these representations may play a limited role if they are not robust to distraction. We used model-based fMRI to estimate the impact of distracting visual tasks on working memory representations in several visual field maps in visual and frontoparietal association cortex. Here, we show distraction causes the fidelity of working memory representations to briefly dip when both the memorandum and distractor are jointly encoded by the population activities. Distraction induces small biases in memory errors which can be predicted by biases in neural decoding in early visual cortex, but not other regions. Although distraction briefly disrupts working memory representations, the widespread redundancy with which working memory information is encoded may protect against catastrophic loss. In early visual cortex, the neural representation of information in working memory and behavioral performance are intertwined, solidifying its importance in visual memory. The relative roles of visual, parietal, and frontal cortex in working memory have been actively debated. Here, the authors show that distraction impacts visual working memory representations in primary visual areas, indicating that these regions play a key role in the maintenance of working memory.
Collapse
Affiliation(s)
| | - Thomas C Sprague
- Department of Psychology, New York University, New York, NY, USA.,Department of Psychological and Brain Sciences, University of California, Santa Barbara, CA, USA
| | - Masih Rahmati
- Department of Psychology, New York University, New York, NY, USA.,Center for Neural Science, New York University, New York, NY, USA
| | - Kartik K Sreenivasan
- Division of Science and Mathematics, New York University Abu Dhabi, Abu Dhabi, UAE
| | - Clayton E Curtis
- Department of Psychology, New York University, New York, NY, USA. .,Center for Neural Science, New York University, New York, NY, USA.
| |
Collapse
|
17
|
Teng C, Postle BR. Understanding occipital and parietal contributions to visual working memory: Commentary on Xu (2020). VISUAL COGNITION 2021; 29:401-408. [PMID: 34335071 DOI: 10.1080/13506285.2021.1883171] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/22/2022]
Abstract
In her commentary, Xu (2020) admonishes the reader that "To have a full understanding of the cognitive mechanisms underlying VWM [visual working memory], both behavioral and neural evidence needs to be taken into account. This is a must, and not a choice, for any study that attempts to capture the nature of VWM" (p. 11). Although we don't disagree with this statement, our overall assessment of this commentary is that it, itself, fails to satisfy several "musts" and, consequently, does not pose a serious challenge for the sensory recruitment framework for understanding visual working memory. These "musts" include accurately characterizing the framework being critiqued, not favoring verbal models and intuition at the expense of formal quantitative models, and providing even-handed interpretation of the work of others. We'll conclude with a summary of how the sensory recruitment framework can be incorporated into a broader working model of visual working memory.
Collapse
Affiliation(s)
- Chunyue Teng
- Department of Psychiatry, University of Wisconsin-Madison
| | - Bradley R Postle
- Department of Psychiatry, University of Wisconsin-Madison.,Department of Psychology, University of Wisconsin-Madison
| |
Collapse
|
18
|
Xu Y. Towards a better understanding of information storage in visual working memory. VISUAL COGNITION 2021; 29:437-445. [DOI: 10.1080/13506285.2021.1946230] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/21/2022]
Affiliation(s)
- Yaoda Xu
- Department of Psychology, Yale University, New Haven, USA
| |
Collapse
|
19
|
Iamshchinina P, Christophel TB, Gayet S, Rademaker RL. Essential considerations for exploring visual working memory storage in the human brain. VISUAL COGNITION 2021. [DOI: 10.1080/13506285.2021.1915902] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/21/2022]
Affiliation(s)
- Polina Iamshchinina
- Department of Education and Psychology, Freie Universität Berlin, Germany
- Berlin School of Mind and Brain, Humboldt-Universität zu Berlin, Berlin, Germany
| | - Thomas B. Christophel
- Bernstein Center for Computational Neuroscience and Berlin Center for Advanced Neuroimaging and Clinic for Neurology, Charité Universitätsmedizin, Freie Universität Berlin, Humboldt Universität zu Berlin, Berlin Institute of Health, Berlin, Germany
- Department of Psychology, Humboldt-Universität zu Berlin, Berlin, Germany
| | - Surya Gayet
- Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, Netherlands
| | - Rosanne L. Rademaker
- Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, Netherlands
- Department of Psychology, University of California, La Jolla, CA, USA
- Ernst Strüngmann Institute (ESI) for Neuroscience in Cooperation with Max Planck Society, Frankfurt, Germany
| |
Collapse
|
20
|
Lorenc ES, Sreenivasan KK. Reframing the debate: The distributed systems view of working memory. VISUAL COGNITION 2021. [DOI: 10.1080/13506285.2021.1899091] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/14/2022]
Affiliation(s)
| | - Kartik K. Sreenivasan
- Division of Science and Mathematics, New York University Abu Dhabi, Abu Dhabi, United Arab Emirates
| |
Collapse
|
21
|
Chota S, Van der Stigchel S. Dynamic and flexible transformation and reallocation of visual working memory representations. VISUAL COGNITION 2021. [DOI: 10.1080/13506285.2021.1891168] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/22/2022]
Affiliation(s)
- Samson Chota
- Department of Experimental Psychology, Helmholtz Institute, Utrecht University, Utrecht, The Netherlands
| | - Stefan Van der Stigchel
- Department of Experimental Psychology, Helmholtz Institute, Utrecht University, Utrecht, The Netherlands
| |
Collapse
|
22
|
Lorenc ES, Mallett R, Lewis-Peacock JA. Distraction in Visual Working Memory: Resistance is Not Futile. Trends Cogn Sci 2021; 25:228-239. [PMID: 33397602 PMCID: PMC7878345 DOI: 10.1016/j.tics.2020.12.004] [Citation(s) in RCA: 53] [Impact Index Per Article: 17.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/30/2020] [Revised: 11/30/2020] [Accepted: 12/04/2020] [Indexed: 01/19/2023]
Abstract
Over half a century of research focused on understanding how working memory is capacity constrained has overshadowed the fact that it is also remarkably resistant to interference. Protecting goal-relevant information from distraction is a cornerstone of cognitive function that involves a multifaceted collection of control processes and storage mechanisms. Here, we discuss recent advances in cognitive psychology and neuroscience that have produced new insights into the nature of visual working memory and its ability to resist distraction. We propose that distraction resistance should be an explicit component in any model of working memory and that understanding its behavioral and neural correlates is essential for building a comprehensive understanding of real-world memory function.
Collapse
Affiliation(s)
- Elizabeth S Lorenc
- Department of Psychology, University of Texas at Austin, Austin, TX 78712, USA.
| | - Remington Mallett
- Department of Psychology, University of Texas at Austin, Austin, TX 78712, USA
| | | |
Collapse
|
23
|
Olmos-Solis K, van Loon AM, Olivers CNL. Content or status: Frontal and posterior cortical representations of object category and upcoming task goals in working memory. Cortex 2020; 135:61-77. [PMID: 33360761 DOI: 10.1016/j.cortex.2020.11.011] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/10/2019] [Revised: 10/13/2020] [Accepted: 11/03/2020] [Indexed: 11/18/2022]
Abstract
To optimize task sequences, the brain must differentiate between current and prospective goals. We previously showed that currently and prospectively relevant object representations in working memory can be dissociated within object-selective cortex. Based on other recent studies indicating that a range of brain areas may be involved in distinguishing between currently relevant and prospectively relevant information in working memory, here we conducted multivoxel pattern analyses of fMRI activity in additional posterior areas (specifically early visual cortex and the intraparietal sulcus) as well as frontal areas (specifically the frontal eye fields and lateral prefrontal cortex). We assessed whether these areas represent the memory content, the current versus prospective status of the memory, or both. On each trial, participants memorized an object drawn from three different categories. The object was the target for either a first task (currently relevant), a second task (prospectively relevant), or for neither task (irrelevant). The results revealed a division of labor across brain regions: While posterior areas preferentially coded for content (i.e., the category), frontal areas carried information about the current versus prospective relevance status of the memory, irrespective of the category. Intraparietal sulcus revealed both strong category- and status-sensitivity, consistent with its hub function of combining stimulus and priority signals. Furthermore, cross-decoding analyses revealed that while current and prospective representations were similar prior to search, they became dissimilar during search, in posterior as well as frontal areas. The findings provide further evidence for a dissociation between content and control networks in working memory.
Collapse
Affiliation(s)
- Katya Olmos-Solis
- Institute for Brain and Behaviour, Vrije Universiteit Amsterdam, the Netherlands
| | - Anouk M van Loon
- Department of General Practice & Elderly Care Medicine, Amsterdam Public Health Research Institute, Amsterdam University Medical Center, Location VUmc, Amsterdam, the Netherlands
| | | |
Collapse
|
24
|
Olivers CNL, Van der Stigchel S. Future steps in visual working memory research. VISUAL COGNITION 2020. [DOI: 10.1080/13506285.2020.1833478] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/23/2022]
Affiliation(s)
- Christian N. L. Olivers
- Faculty of Behavioural and Movement Sciences, Institute of Brain and Behavior Amsterdam, Experimental and Applied Psychology, Vrije Universiteit, Amsterdam, Netherlands
| | | |
Collapse
|