Short-term memory for spatial, sequential and duration information

The role of recollection and familiarity in visual working memory: A mixture of threshold and signal detection processes

Whether working memory reflects a thresholded recollection process whereby only a limited number of items are maintained in memory, or a signal detection process in which each studied item is increased in familiarity strength, is a topic of considerable debate. A review of visual working memory studies that have examined receiver operating characteristics (ROCs) across a broad set of materials and test conditions indicates that both signal detection and threshold processes contribute to working memory. In addition, the role that these two processes play varies systematically across conditions, such that a threshold process plays a particularly critical role when binary old/new judgments are required, when changes are relatively discrete, and when the hippocampus does not contribute to performance. In contrast, a signal detection process plays a greater role when confidence judgments are required, when the materials or the changes are global in nature, and when the hippocampus contributes to performance. In addition, the ROC results indicate that in standard single-probe tests of working memory, items that are maintained in an active recollected state support both recall-to-accept and recall-to-reject responses; whereas in complex-probe tests, recollection preferentially supports recall-to-reject; and in item-recognition tests it preferentially supports recall-to-accept. Moreover, there is growing evidence that these threshold and strength-based processes are related to distinct states of conscious awareness whereby they support perceiving- and sensing-based responses, respectively. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Precision in spatial working memory examined with mouse pointing

The capacity of visuospatial working memory (VSWM) is limited. However, there is continued debate surrounding the nature of this capacity limitation. The resource model (Bays et al., 2009) proposes that VSWM capacity is limited by the precision with which visuospatial features can be retained. In one of the few studies of spatial working memory, Schneegans and Bays (2016) report that memory guided pointing responses show a monotonic decrease in precision as set size increases, consistent with resource models. Here we report two conceptual replications of this study that use mouse responses rather than pointing responses. Overall results are consistent with the resource model, as there was an exponential increase in localisation error and monotonic increases in the probability of misbinding and guessing with increases in set size. However, an unexpected result of Experiment One was that, unlike Schneegans and Bays (2016), imprecision did not increase between set sizes of 2 and 8. Experiment Two replicated this effect and ruled out the possibility that the invariance of imprecision at set sizes greater than 2 was a product of oculomotor strategies during recall. We speculate that differences in imprecision are related to additional visuomotor transformations required for memory-guided mouse localisation compared to memory-guided manual pointing localisation. These data demonstrate the importance of considering the nature of the response modality when interpreting VSWM data.

Microsaccades Track Location-Based Object Rehearsal in Visual Working Memory

Besides controlling eye movements, the brain's oculomotor system has been implicated in the control of covert spatial attention and the rehearsal of spatial information in working memory. We investigated whether the oculomotor system also contributes to rehearsing visual objects in working memory when object location is never asked about. To address this, we tracked the incidental use of locations for mnemonic rehearsal via directional biases in microsaccades while participants maintained two visual objects (colored oriented gratings) in working memory. By varying the stimulus configuration (horizontal, diagonal, and vertical) at encoding, we could quantify whether microsaccades were more aligned with the configurational axis of the memory contents, as opposed to the orthogonal axis. Experiment 1 revealed that microsaccades continued to be biased along the axis of the memory content several seconds into the working memory delay. In Experiment 2, we confirmed that this directional microsaccade bias was specific to memory demands, ruling out lingering effects from passive and attentive encoding of the same visual objects in the same configurations. Thus, by studying microsaccade directions, we uncover oculomotor-driven rehearsal of visual objects in working memory through their associated locations.

No obligatory trade-off between the use of space and time for working memory

Space and time can each act as scaffolds for the individuation and selection of visual objects in working memory. Here we ask whether there is a trade-off between the use of space and time for visual working memory: whether observers will rely less on space, when memoranda can additionally be individuated through time. We tracked the use of space through directional biases in microsaccades after attention was directed to memory contents that had been encoded simultaneously or sequentially to the left and right of fixation. We found that spatial gaze biases were preserved when participants could (Experiment 1) and even when they had to (Experiment 2) additionally rely on time for object individuation. Thus, space remains a profound organizing medium for working memory even when other organizing sources are available and utilized, with no evidence for an obligatory trade-off between the use of space and time.

The precuneus as a central node in declarative memory retrieval

Both, the hippocampal formation and the neocortex are contributing to declarative memory, but their functional specialization remains unclear. We investigated the differential contribution of both memory systems during free recall of word lists. In total, 21 women and 17 men studied the same list but with the help of different encoding associations. Participants associated the words either sequentially with the previous word on the list, with spatial locations on a well-known path, or with unique autobiographical events. After intensive rehearsal, subjects recalled the words during functional magnetic resonance imaging (fMRI). Common activity to all three types of encoding associations was identified in the posterior parietal cortex, in particular in the precuneus. Additionally, when associating spatial or autobiographical material, retrosplenial cortex activity was elicited during word list recall, while hippocampal activity emerged only for autobiographically associated words. These findings support a general, critical function of the precuneus in episodic memory storage and retrieval. The encoding-retrieval repetitions during learning seem to have accelerated hippocampus-independence and lead to direct neocortical integration in the sequentially associated and spatially associated word list tasks. During recall of words associated with autobiographical memories, the hippocampus might add spatiotemporal information supporting detailed scenic and contextual memories.

Oculomotor rehearsal in visuospatial working memory

The neural and cognitive mechanisms of spatial working memory are tightly coupled with the systems that control eye movements, but the precise nature of this coupling is not well understood. It has been argued that the oculomotor system is selectively involved in rehearsal of spatial but not visual material in visuospatial working memory. However, few studies have directly compared the effect of saccadic interference on visual and spatial memory, and there is little consensus on how the underlying working memory representation is affected by saccadic interference. In this study we aimed to examine how working memory for visual and spatial features is affected by overt and covert attentional interference across two experiments. Participants were shown a memory array, then asked to either maintain fixation or to overtly or covertly shift attention in a detection task during the delay period. Using the continuous report task we directly examined the precision of visual and spatial working memory representations and fit psychophysical functions to investigate the sources of recall error associated with different types of interference. These data were interpreted in terms of embodied theories of attention and memory and provide new insights into the nature of the interactions between cognitive and motor systems.

Role of time in binding features in visual working memory

Previous research on feature binding in visual working memory has supported a privileged role for location in binding an object's nonspatial features. However, humans are able to correctly recall feature conjunctions of objects that occupy the same location at different times. In a series of behavioral experiments, we investigated binding errors under these conditions, and specifically tested whether ordinal position can take the role of location in mediating feature binding. We performed two dual report experiments in which participants had to memorize three colored shapes presented sequentially at the screen center. When participants were cued with the ordinal position of one item and had to report its shape and color, report errors for the two features were largely uncorrelated. In contrast, when participants were cued, for example, with an item's shape and reported an incorrect ordinal position, they had a high chance of making a corresponding error in the color report. This pattern of error correlations closely matched the predictions of a model in which color and shape are bound to each other only indirectly via an item's ordinal position. In a third experiment, we directly compared the roles of location and sequential position in feature binding. Participants viewed a sequence of colored disks displayed at different locations and were cued either by a disk's location or its ordinal position to report its remaining properties. The pattern of errors supported a mixed strategy with individual variation, suggesting that binding via either time or space could be used for this task. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Caught in the ACTS: Defining Abstract Cognitive Task Sequences as an Independent Process

Cognitive neuroscience currently conflates the study of serial responses (e.g., delay match to sample/nonsample, n-back) with the study of sequential operations. In this essay, our goal is to define and disentangle the latter, termed abstract cognitive task sequences (ACTS). Existing literatures address tasks requiring serial events, including procedural learning of implicit motor responses, statistical learning of predictive relationships, and judgments of attributes. These findings do not describe the behavior and underlying mechanism required to succeed at remembering to evaluate color, then shape; or to multiply, then add. A new literature is needed to characterize these sorts of second-order cognitive demands of studying a sequence of operations. Our second goal is to characterize gaps in knowledge related to ACTS that merit further investigation. In the following sections, we define more precisely what we mean by ACTS and suggest research questions' further investigation would be positioned to address.

Recruitment of a long-term memory supporting neural network during repeated maintenance of a multi-item abstract visual image in working memory

Humans can flexibly transfer information between different memory systems. Information in visual working memory (VWM) can for instance be stored in long-term memory (LTM). Conversely, information can be retrieved from LTM and temporarily held in WM when needed. It has previously been suggested that a neural transition from parietal- to midfrontal activity during repeated visual search reflects transfer of information from WM to LTM. Whether this neural transition indeed reflects consolidation and is also observed when memorizing a rich visual scene (rather than responding to a single target), is not known. To investigate this, we employed an EEG paradigm, in which abstract six-item colour-arrays were repeatedly memorized and explicitly visualized, or merely attended to. Importantly, we tested the functional significance of a potential neural shift for longer-term consolidation in a subsequent recognition task. Our results show a gradually enhanced- and sustained modulation of the midfrontal P170 component and a decline in parietal CDA, during repeated WM maintenance. Improved recollection/visualization of memoranda upon WM-cueing, was associated with contralateral parietal- and right temporal activity. Importantly, only colour-arrays previously held in WM, induced a greater midfrontal P170-response, together with left temporal- and late centro-parietal activity, upon re-exposure. These findings provide evidence for recruitment of an LTM-supporting neural network which facilitates visual WM maintenance.

Working memory and active sampling of the environment: Medial temporal contributions

Working memory (WM) refers to the ability to maintain and actively process information-either derived from perception or long-term memory (LTM)-for intelligent thought and action. This chapter focuses on the contributions of the temporal lobe, particularly medial temporal lobe (MTL) to WM. First, neuropsychological evidence for the involvement of MTL in WM maintenance is reviewed, arguing for a crucial role in the case of retaining complex relational bindings between memorized features. Next, MTL contributions at the level of neural mechanisms are covered-with a focus on WM encoding and maintenance, including interactions with ventral temporal cortex. Among WM use processes, we focus on active sampling of environmental information, a key input source to capacity-limited WM. MTL contributions to the bidirectional relationship between active sampling and memory are highlighted-WM control of active sampling and sampling as a way of selecting input to WM. Memory-based sampling studies relying on scene and object inspection, visual-based exploration behavior (e.g., vicarious behavior), and memory-guided visual search are reviewed. The conclusion is that MTL serves an important function in the selection of information from perception and transfer from LTM to capacity-limited WM.

Location-independent feature binding in visual working memory for sequentially presented objects

Spatial location is believed to have a privileged role in binding features held in visual working memory. Supporting this view, Pertzov and Husain (Attention, Perception, & Psychophysics, 76(7), 1914–1924, 2014) reported that recall of bindings between visual features was selectively impaired when items were presented sequentially at the same location compared to sequentially at different locations. We replicated their experiment, but additionally tested whether the observed impairment could be explained by perceptual interference during encoding. Participants viewed four oriented bars in highly discriminable colors presented sequentially either at the same or different locations, and after a brief delay were cued with one color to reproduce the associated orientation. When we used the same timing as the original study, we reproduced its key finding of impaired binding memory in the same-location condition. Critically, however, this effect was significantly modulated by the duration of the inter-stimulus interval, and disappeared if memoranda were presented with longer delays between them. In a second experiment, we tested whether the effect generalized to other visual features, namely reporting of colors cued by stimulus shape. While we found performance deficits in the same-location condition, these did not selectively affect binding memory. We argue that the observed effects are best explained by encoding interference, and that memory for feature binding is not necessarily impaired when memoranda share the same location.

Static internal representation of dynamic situations reveals time compaction in human cognition

Introduction

The human brain has evolved under the constraint of survival in complex dynamic situations. It makes fast and reliable decisions based on internal representations of the environment. Whereas neural mechanisms involved in the internal representation of space are becoming known, entire spatiotemporal cognition remains a challenge. Growing experimental evidence suggests that brain mechanisms devoted to spatial cognition may also participate in spatiotemporal information processing.

Objectives

The time compaction hypothesis postulates that the brain represents both static and dynamic situations as purely static maps. Such an internal reduction of the external complexity allows humans to process time-changing situations in real-time efficiently. According to time compaction, there may be a deep inner similarity between the representation of conventional static and dynamic visual stimuli. Here, we test the hypothesis and report the first experimental evidence of time compaction in humans.

Methods

We engaged human subjects in a discrimination-learning task consisting in the classification of static and dynamic visual stimuli. When there was a hidden correspondence between static and dynamic stimuli due to time compaction, the learning performance was expected to be modulated. We studied such a modulation experimentally and by a computational model.

Results

The collected data validated the predicted learning modulation and confirmed that time compaction is a salient cognitive strategy adopted by the human brain to process time-changing situations. Mathematical modelling supported the finding. We also revealed that men are more prone to exploit time compaction in accordance with the context of the hypothesis as a cognitive basis for survival.

Conclusions

The static internal representation of dynamic situations is a human cognitive mechanism involved in decision-making and strategy planning to cope with time-changing environments. The finding opens a new venue to understand how humans efficiently interact with our dynamic world and thrive in nature.

Incidental encoding of visual information in temporal reference frames in working memory

Visual events are structured in space and time, yet models of visual working memory (VWM) have largely relied on tasks emphasizing spatial aspects. Here, we show that temporal properties of visual events are incidentally encoded along with spatial properties. In five experiments, participants performed change-detection tasks, in which items had unique spatial and temporal coordinates at encoding. Crucially, neither space nor time was task-relevant. The key manipulation concerned the retrieval context: The test array was identical to the memory array either in its entire spatiotemporal structure, or only its spatial or temporal structure. Removing spatial or temporal information at retrieval resulted in costs, indicating that memory relied on both spatial and temporal context in which items were initially perceived. Encoding of spatiotemporal structure occurred incidentally, not strategically, as it was robust even when the retrieval context was perfectly predictable. However, spatial and temporal inter-item spacings influenced the weighting of spatial and temporal information: It favoured the domain in which items were more widely spaced, facilitating their individuation and, likely, access to representations. Across individuals, the weighting of spatial and temporal information varied substantially, but it remained consistent across sessions, suggesting stable preferences for coding in the spatial or temporal domain. No comparable incidental encoding occurred for other task-irrelevant feature dimensions (size or colour). We propose that temporal structure serves as fundamental a function in VWM as spatial structure, scaffolding events that unfold over time.

Working Memory in Unilateral Spatial Neglect: Evidence for Impaired Binding of Object Identity and Object Location

Working memory (WM) is known to be impaired in patients with stroke experiencing unilateral spatial neglect (USN). Here, we examined in a systematic manner three WM components: memory of object identity, memory of object location, and binding between object identity and location. Moreover, we used two different retention intervals to isolate maintenance from other mnemonic and perceptual processes. Fourteen USN first-event stroke patients with right-hemisphere damage were tested in two different WM experiments using long and short retention intervals and an analog response scale. Patients exhibited more identification errors for items displayed on the contralesional side. Localization errors were also more prominent in the contralesional side, especially after a long retention interval. These localization errors were often a result of swap errors, that is, erroneous localizations of correctly identified contralesional objects in correctly memorized locations of ipsilesional objects. We conclude that a key WM deficit in USN is a lateralized impairment in binding between the identity of an object and its spatial tag.

Rapid forgetting of faces in congenital prosopagnosia

Congenital prosopagnosia (CP) is a life-long impairment in face recognition that occurs in the absence of any known brain damage. It is still unclear whether this disorder is related to a visual deficit, or to an impairment in encoding, maintaining or retrieving a face from memory. We tested CPs and matched neurotypical controls using a delayed estimation task in which a target face was shown either upright or inverted. Participants were asked to select the target face out of a cyclic space of morphed faces that could either resemble the target face, or not. The inclusion of upright and inverted faces enabled to examine the extent of the face inversion effect, a well-known face specific effect often associated with holistic processing. To enable disentangling visual from mnemonic processing, reports were required either following 1 and 6 sec retention interval, or simultaneously while the target face was still visible. Controls showed slower forgetting of upright compared to inverted faces. In contrast, CPs exhibited rapid forgetting of upright faces that was comparable to their performance and to performance of controls on inverted faces. Such forgetting was evident in random errors in which the selected faces did not resemble the face in memory, implying a time related decrease in the probability to access the correct face in memory. Importantly, CPs exhibited no inversion effect across all retention intervals, including the simultaneous one, suggesting that their abnormal rapid forgetting could be explained by an impairment in holistic visual processing of upright faces.

Genetic manipulation of cyclic nucleotide signaling during hippocampal neuroplasticity and memory formation

Decades of research have underscored the importance of cyclic nucleotide signaling in memory formation and synaptic plasticity. In recent years, several new genetic techniques have expanded the neuroscience toolbox, allowing researchers to measure and modulate cyclic nucleotide gradients with high spatiotemporal resolution. Here, we will provide an overview of studies using genetic approaches to interrogate the role cyclic nucleotide signaling plays in hippocampus-dependent memory processes and synaptic plasticity. Particular attention is given to genetic techniques that measure real-time changes in cyclic nucleotide levels as well as newly-developed genetic strategies to transiently manipulate cyclic nucleotide signaling in a subcellular compartment-specific manner with high temporal resolution.

Scale-invariant Characteristics of Forgetting: Toward a Unifying Account of Hippocampal Forgetting across Short and Long Timescales

After over 100 years of relative silence in the cognitive literature, recent advances in the study of the neural underpinnings of memory-specifically, the hippocampus-have led to a resurgence of interest in the topic of forgetting. This review draws a theoretically driven picture of the effects of time on forgetting of hippocampus-dependent memories. We review evidence indicating that time-dependent forgetting across short and long timescales is reflected in progressive degradation of hippocampal-dependent relational information. This evidence provides an important extension to a growing body of research accumulated in recent years, showing that-in contrast to the once prevailing view that the hippocampus is exclusively involved in memory and forgetting over long timescales-the role of the hippocampus also extends to memory and forgetting over short timescales. Thus, we maintain that similar rules govern not only remembering but also forgetting of hippocampus-dependent information over short and long timescales.

Contralateral delay activity during temporal order memory

In everyday life, we constantly need to remember the temporal sequence of visual events over short periods of time, for example, when making sense of others' actions or watching a movie. While there is increasing knowledge available on neural mechanisms underlying visual working memory (VWM) regarding the identity and spatial location of objects, less is known about how the brain encodes and retains information on temporal sequences. Here, we investigate whether the contralateral-delay activity (CDA), a well-studied electroencephalographic (EEG) component associated with VWM of object identity, also reflects the encoding and retention of temporal order. In two independent experiments, we presented participants with a sequence of four or six images, followed by a 1 s retention period. Participants judged temporal order by indicating whether a subsequently presented probe image was originally displayed during the first or the second half of the sequence. As a main novel result, we report the emergence of a contralateral negativity already following the presentation of the first item of the sequence, which increases over the course of a trial with every presented item, up to a limit of four items. We further observed no differences in the CDA during the temporal-order task compared to one obtained during a task concerning the spatial location of the presented items. Since the characteristics of the CDA appear to be highly similar between different encoded feature dimensions and increases as additional items are being encoded, we suggest this component might be a general characteristic of various types of VWM.

Spectral Imprints of Working Memory for Everyday Associations in the Frontoparietal Network

How does the human brain rapidly process incoming information in working memory? In growing divergence from a single-region focus on the prefrontal cortex (PFC), recent work argues for emphasis on how distributed neural networks are rapidly coordinated in support of this central neurocognitive function. Previously, we showed that working memory for everyday “what,” “where,” and “when” associations depends on multiplexed oscillatory systems, in which signals of different frequencies simultaneously link the PFC to parieto-occipital and medial temporal regions, pointing to a complex web of sub-second, bidirectional interactions. Here, we used direct brain recordings to delineate the frontoparietal oscillatory correlates of working memory with high spatiotemporal precision. Seven intracranial patients with electrodes simultaneously localized to prefrontal and parietal cortices performed a visuospatial working memory task that operationalizes the types of identity and spatiotemporal information we encounter every day. First, task-induced oscillations in the same delta-theta (2–7 Hz) and alpha-beta (9–24 Hz) frequency ranges previously identified using scalp electroencephalography (EEG) carried information about the contents of working memory. Second, maintenance was linked to directional connectivity from the parietal cortex to the PFC. However, presentation of the test prompt to cue identity, spatial, or temporal information changed delta-theta coordination from a unidirectional, parietal-led system to a bidirectional, frontoparietal system. Third, the processing of spatiotemporal information was more bidirectional in the delta-theta range than was the processing of identity information, where alpha-beta connectivity did not exhibit sensitivity to the contents of working memory. These findings implicate a bidirectional delta-theta mechanism for frontoparietal control over the contents of working memory.

Dynamic frontotemporal systems process space and time in working memory

How do we rapidly process incoming streams of information in working memory, a cognitive mechanism central to human behavior? Dominant views of working memory focus on the prefrontal cortex (PFC), but human hippocampal recordings provide a neurophysiological signature distinct from the PFC. Are these regions independent, or do they interact in the service of working memory? We addressed this core issue in behavior by recording directly from frontotemporal sites in humans performing a visuospatial working memory task that operationalizes the types of identity and spatiotemporal information we encounter every day. Theta band oscillations drove bidirectional interactions between the PFC and medial temporal lobe (MTL; including the hippocampus). MTL theta oscillations directed the PFC preferentially during the processing of spatiotemporal information, while PFC theta oscillations directed the MTL for all types of information being processed in working memory. These findings reveal an MTL theta mechanism for processing space and time and a domain-general PFC theta mechanism, providing evidence that rapid, dynamic MTL–PFC interactions underlie working memory for everyday experiences.

How do we rapidly process incoming streams of information in working memory? Dominant views of working memory focus on the prefrontal cortex (PFC), but other data suggest a role for the medial temporal lobe (MTL). To delineate whether (and how) these brain regions interact during working memory, we recorded directly from PFC and MTL sites in humans performing a task that tests working memory for the types of “what,” “where,” and “when” information encountered every day. MTL oscillations in the theta band (3–7 Hz) directed PFC activity during the processing of spatiotemporal information, while PFC theta oscillations directed MTL activity for all types of information. This functional dissociation provides the first demonstration of bidirectional communication between the PFC and MTL during working memory. Our findings reveal that rapid, dynamic interactions between these two regions underlie working memory for everyday experiences, challenging dominant views on the central role of the PFC.

The Persistence of Memory: How the Brain Encodes Time in Memory

Time and memory are inextricably linked, but it is far from clear how event durations and temporal sequences are encoded in memory. In this review, we focus on resource allocation models of working memory which suggest that memory resources can be flexibly distributed amongst several items such that the precision of working memory decreases with the number of items to be encoded. This type of model is consistent with human performance in working memory tasks based on visual, auditory as well as temporal stimulus patterns. At the neural-network level, we focus on excitatory-inhibitory oscillatary processes that are able to encode both interval timing and working memory in a coupled excitatory-inhibitory network. This modification of the striatal beat-frequency model of interval timing shows how memories for multiple time intervals are represented by neural oscillations and can also be used to explain the mechanisms of resource allocation in working memory.