Output planning at the input stage in visual working memory

A Neural Decision Signal during Internal Sampling from Working Memory in Humans

How humans transform sensory information into decisions that steer purposeful behavior is a central question in psychology and neuroscience that is traditionally investigated during the sampling of external environmental signals. The decision-making framework of gradual information sampling toward a decision has also been proposed to apply when sampling internal sensory evidence from working memory. However, neural evidence for this proposal remains scarce. Here we show (using scalp EEG in male and female human volunteers) that sampling internal visual representations from working memory elicits a scalp EEG potential associated with gradual evidence accumulation-the central parietal positivity. Consistent with an evolving decision process, we show how this signal (1) scales with the time participants require to reach a decision about the cued memory content and (2) is amplified when having to decide among multiple contents in working memory. These results bring the electrophysiology of decision-making into the domain of working memory and suggest that variability in memory-guided behavior may be driven (at least in part) by variations in the sampling of our inner mental contents.

Neural Signatures of Competition between Voluntary and Involuntary Influences over the Focus of Attention in Visual Working Memory

Adaptive behavior relies on the selection and prioritization of relevant sensory inputs from the external environment as well as from among internal sensory representations held in working memory. Recent behavioral evidence suggests that the classic distinction between voluntary (goal-driven) and involuntary (stimulus-driven) influences over attentional allocation also applies to the selection of internal representations held in working memory. In the current EEG study, we set out to investigate the neural dynamics associated with the competition between voluntary and involuntary control over the focus of attention in visual working memory. We show that when voluntary and involuntary factors compete for the internal focus of attention, prioritization of the appropriate item is delayed-as reflected both in delayed gaze biases that track internal selection and in delayed neural beta (15-25 Hz) dynamics that track the planning for the upcoming memory-guided manual action. We further show how this competition is paralleled-possibly resolved-by an increase in frontal midline theta (4-8 Hz) activity that, moreover, predicts the speed of ensuing memory-guided behavior. Finally, because theta increased following retrocues that effectively reduced working-memory load, our data unveil how frontal theta activity during internal attentional focusing tracks demands on cognitive control over and above working-memory load. Together, these data yield new insight into the neural dynamics that govern the focus of attention in visual working memory, and disentangle the contributions of frontal midline theta activity to the processes of control versus retention in working memory.

Goal-Dependent Use of Temporal Regularities to Orient Attention under Spatial and Action Uncertainty

The temporal regularities in our environments support the proactive dynamic anticipation of relevant events. In visual attention, one important outstanding question is whether temporal predictions must be linked to predictions about spatial locations or motor plans to facilitate behaviour. To test this, we developed a task for manipulating temporal expectations and task relevance of visual stimuli appearing within rapidly presented streams, while stimulus location and responding hand remained uncertain. Differently coloured stimuli appeared in one of two concurrent (left and right) streams with distinct temporal probability structures. Targets were defined by colour on a trial-by-trial basis and appeared equiprobably in either stream, requiring a localisation response. Across two experiments, participants were faster and more accurate at detecting temporally predictable targets compared to temporally unpredictable targets. We conclude that temporal expectations learned incidentally from temporal regularities can be called upon flexibly in a goal-driven manner to guide behaviour. Moreover, we show that visual temporal attention can facilitate performance in the absence of concomitant spatial or motor expectations in dynamically unfolding contexts.

Internal Representations Are Prioritized by Frontoparietal Theta Connectivity and Suppressed by alpha Oscillation Dynamics: Evidence from Concurrent Transcranial Magnetic Stimulation EEG and Invasive EEG

Control over internal representations requires the prioritization of relevant information and suppression of irrelevant information. The frontoparietal network exhibits prominent neural oscillations during these distinct cognitive processes. Yet, the causal role of this network-scale activity is unclear. Here, we targeted theta-frequency frontoparietal coherence and dynamic alpha oscillations in the posterior parietal cortex using online rhythmic transcranial magnetic stimulation (TMS) in women and men while they prioritized or suppressed internally maintained working memory (WM) representations. Using concurrent high-density EEG, we provided evidence that we acutely drove the targeted neural oscillation and TMS improved WM capacity only when the evoked activity corresponded with the desired cognitive process. To suppress an internal representation, we increased the amplitude of lateralized alpha oscillations in the posterior parietal cortex contralateral to the irrelevant visual field. For prioritization, we found that TMS to the prefrontal cortex increased theta-frequency connectivity in the prefrontoparietal network contralateral to the relevant visual field. To understand the spatial specificity of these effects, we administered the WM task to participants with implanted electrodes. We found that theta connectivity during prioritization was directed from the lateral prefrontal to the superior posterior parietal cortex. Together, these findings provide causal evidence in support of a model where a frontoparietal theta network prioritizes internally maintained representations and alpha oscillations in the posterior parietal cortex suppress irrelevant representations.

Mapping visual working memory models to a theoretical framework

The body of research on visual working memory (VWM)-the system often described as a limited memory store of visual information in service of ongoing tasks-is growing rapidly. The discovery of numerous related phenomena, and the many subtly different definitions of working memory, signify a challenge to maintain a coherent theoretical framework to discuss concepts, compare models and design studies. A lack of robust theory development has been a noteworthy concern in the psychological sciences, thought to be a precursor to the reproducibility crisis (Oberauer & Lewandowsky, Psychonomic Bulletin & Review, 26, 1596-1618, 2019). I review the theoretical landscape of the VWM field by examining two prominent debates-whether VWM is object-based or feature-based, and whether discrete-slots or variable-precision best describe VWM limits. I share my concerns about the dualistic nature of these debates and the lack of clear model specification that prevents fully determined empirical tests. In hopes of promoting theory development, I provide a working theory map by using the broadly encompassing memory for latent representations model (Hedayati et al., Nature Human Behaviour, 6, 5, 2022) as a scaffold for relevant phenomena and current theories. I illustrate how opposing viewpoints can be brought into accordance, situating leading models of VWM to better identify their differences and improve their comparison. The hope is that the theory map will help VWM researchers get on the same page-clarifying hidden intuitions and aligning varying definitions-and become a useful device for meaningful discussions, development of models, and definitive empirical tests of theories.

Be prepared for interruptions: EEG correlates of anticipation when dealing with task interruptions and the role of aging

Dealing with task interruptions requires the flexible use of working memory and attentional control mechanisms, which are prone to age-related changes. We investigated effects of age on dealing with task interruptions and potential advantages of anticipating an interruption using EEG and a retrospective cueing (retro-cue) paradigm. Thirty-two young (18-30 years) and 28 older (55-70 years) participants performed a visual working memory task, where they had to report the orientation of a target following a retro-cue. Within blocks of 10 trials, they were always, never, or randomly interrupted with an arithmetic task before the onset of the retro-cue. The interruption-induced decline in primary task performance was more pronounced in older participants, while only these benefited from anticipation. The EEG analysis revealed reduced theta and alpha/beta response to the retro-cue following interruptions, especially for the older participants. In both groups, anticipated interruptions were associated with increased theta and alpha/beta power prior and during the interruption, and stronger beta suppression to the retro-cue. The results indicate that interruptions impede the refocusing of attention on the task-relevant representation of the primary task, especially in older people, while anticipation facilitates preparation for the interruption task and resumption of the primary task.

Alpha-Band Lateralization and Microsaccades Elicited by Exogenous Cues Do Not Track Attentional Orienting

We explore the world by constantly shifting our focus of attention toward salient stimuli and then disengaging from them in search of new ones. The alpha rhythm (8-13 Hz) has been suggested as a pivotal neural substrate of these attentional shifts, due to its local synchronization and desynchronization that suppress irrelevant cortical areas and facilitate relevant areas, a phenomenon called alpha lateralization. Whether alpha lateralization tracks the focus of attention from orienting toward a salient stimulus to disengaging from it is still an open question. We addressed it by leveraging the phenomenon of inhibition of return (IOR), consisting of an initial facilitation in response times (RTs) for stimuli appearing at an exogenously cued location, followed by a suppression of that location. Our behavioral data from human participants showed a typical IOR effect with both early facilitation and subsequent inhibition. In contrast, alpha lateralized in the cued direction after the behavioral facilitation effect and never re-lateralized compatibly with the behavioral inhibition. Furthermore, we analyzed the interaction between alpha lateralization and microsaccades: while alpha was lateralized toward the cued location, microsaccades were mostly oriented away from it. Crucially, the two phenomena showed a significant positive correlation. These results indicate that alpha lateralization reflects primarily the processing of salient stimuli, challenging the view that alpha lateralization is directly involved in exogenous attentional orienting per se. We discuss the relevance of the present findings for an oculomotor account of alpha lateralization as a modulator of cortical excitability in preparation of a saccade.

The reactivation of task rules triggers the reactivation of task-relevant items

Working memory (WM) describes the temporary storage of task-relevant items and procedural rules to guide action. Despite its central importance for goal-directed behavior, the interplay between WM and long-term memory (LTM) remains poorly understood. Recent studies have shown that repeated use of the same task-relevant item in WM results in a hand-off of the storage of that item to LTM, and switching to a new item reactivates WM. To further elucidate the rules governing WM-LTM interactions, we here planned to probe whether a change in task rules, independent of a switch in task-relevant items, would also lead to WM reactivation of maintained items. To this end, we used scalp-recorded electroencephalogram (EEG) data, specifically the contralateral delay activity (CDA), to track WM item storage while manipulating repetitions and changes in task rules and task-relevant items across trials in a visual WM task. We tested two rival hypotheses: If changes in task rules result in a reactivation of the target item representation, then the CDA should increase when a task change is cued even when the same target has been repeated across trials. However, if the reactivation of a task-relevant item only depends on the mnemonic availability of the item itself instead of the task it is used for, then only the changes in task-relevant items should reactivate the representations. Accordingly, the CDA amplitude should decrease for repeated task-relevant items independently of a task change. We found a larger CDA on task-switch compared to task-repeat trials, suggesting that the reactivation of task rules triggers the reactivation of task-relevant items in WM. By demonstrating that WM reactivation of LTM is interdependent for task rules and task-relevant items, this study informs our understanding of visual WM and its interplay with LTM. PREREGISTERED STAGE 1 PROTOCOL: https://osf.io/zp9e8 (date of in-principle acceptance: 19/12/2021).

Prospection of Potential Actions during Visual Working Memory Starts Early, Is Flexible, and Predicts Behavior

For visual working memory to serve upcoming behavior, it is crucial that we prepare for the potential use of working-memory contents ahead of time. Recent studies have demonstrated how the prospection and planning for an upcoming manual action starts early after visual encoding, and occurs alongside visual retention. Here, we address whether such "output planning" in visual working memory flexibly adapts to different visual-motor mappings, and occurs even when an upcoming action will only potentially become relevant for behavior. Human participants (female and male) performed a visual-motor working memory task in which they remembered one or two colored oriented bars for later (potential) use. We linked, and counterbalanced, the tilt of the visual items to specific manual responses. This allowed us to track planning of upcoming behavior through contralateral attenuation of β band activity, a canonical motor-cortical EEG signature of manual-action planning. The results revealed how action encoding and subsequent planning alongside visual working memory (1) reflect anticipated task demands rather than specific visual-motor mappings, (2) occur even for actions that will only potentially become relevant for behavior, and (3) are associated with faster performance for the encoded item, at the expense of performance to other working-memory content. This reveals how the potential prospective use of visual working memory content is flexibly planned early on, with consequences for the speed of memory-guided behavior.SIGNIFICANCE STATEMENT It is increasingly studied how visual working memory helps us to prepare for the future, in addition to how it helps us to hold onto the past. Recent studies have demonstrated that the planning of prospective actions occurs alongside encoding and retention in working memory. We show that such early "output planning" flexibly adapts to varying visual-motor mappings, occurs both for certain and potential actions, and predicts ensuing working-memory guided behavior. These results highlight the flexible and future-oriented nature of visual working memory, and provide insight into the neural basis of the anticipatory dynamics that translate visual representations into adaptive upcoming behavior.

Temporally Dissociable Mechanisms of Spatial, Feature, and Motor Selection during Working Memory-guided Behavior

Working memory (WM) is a capacity- and duration-limited system that forms a temporal bridge between fleeting sensory phenomena and possible actions. But how are the contents of WM used to guide behavior? A recent high-profile study reported evidence for simultaneous access to WM content and linked motor plans during WM-guided behavior, challenging serial models where task-relevant WM content is first selected and then mapped on to a task-relevant motor response. However, the task used in that study was not optimized to distinguish the selection of spatial versus nonspatial visual information stored in memory, nor to distinguish whether or how the chronometry of selecting nonspatial visual information stored in memory might differ from the selection of linked motor plans. Here, we revisited the chronometry of spatial, feature, and motor selection during WM-guided behavior using a task optimized to disentangle these processes. Concurrent EEG and eye position recordings revealed clear evidence for temporally dissociable spatial, feature, and motor selection during this task. Thus, our data reveal the existence of multiple WM selection mechanisms that belie conceptualizations of WM-guided behavior based on purely serial or parallel visuomotor processing.

Capacity and selection in immersive visual working memory following naturalistic object disappearance

Visual working memory-holding past visual information in mind for upcoming behavior-is commonly studied following the abrupt removal of visual objects from static two-dimensional (2D) displays. In everyday life, visual objects do not typically vanish from the environment in front of us. Rather, visual objects tend to enter working memory following self or object motion: disappearing from view gradually and changing the spatial relation between memoranda and observer. Here, we used virtual reality (VR) to investigate whether two classic findings from visual working memory research-a capacity of around three objects and the reliance on space for object selection-generalize to more naturalistic modes of object disappearance. Our static reference condition mimicked traditional laboratory tasks whereby visual objects were held static in front of the participant and removed from view abruptly. In our critical flow condition, the same visual objects flowed by participants, disappearing from view gradually and behind the observer. We considered visual working memory performance and capacity, as well as space-based mnemonic selection, indexed by directional biases in gaze. Despite vastly distinct modes of object disappearance and altered spatial relations between memoranda and observer, we found comparable capacity and comparable gaze signatures of space-based mnemonic selection. This finding reveals how classic findings from visual working memory research generalize to immersive situations with more naturalistic modes of object disappearance and with dynamic spatial relations between memoranda and observer.

Microsaccades transiently lateralise EEG alpha activity

The lateralisation of 8-12 Hz alpha activity is a canonical signature of human spatial cognition that is typically studied under strict fixation requirements. Yet, even during attempted fixation, the brain produces small involuntary eye movements known as microsaccades. Here we report how spontaneous microsaccades - made in the absence of incentives to look elsewhere - can themselves drive transient lateralisation of EEG alpha power according to microsaccade direction. This transient lateralisation of posterior alpha power occurs similarly following start and return microsaccades and is, at least for start microsaccades, driven by increased alpha power ipsilateral to microsaccade direction. This reveals new links between spontaneous microsaccades and human electrophysiological brain activity. It highlights how microsaccades are an important factor to consider in studies relating alpha activity - including spontaneous fluctuations in alpha activity - to spatial cognition, such as studies on visual attention, anticipation, and working memory.

Attention in flux

Selective attention comprises essential infrastructural functions supporting cognition-anticipating, prioritizing, selecting, routing, integrating, and preparing signals to guide adaptive behavior. Most studies have examined its consequences, systems, and mechanisms in a static way, but attention is at the confluence of multiple sources of flux. The world advances, we operate within it, our minds change, and all resulting signals progress through multiple pathways within the dynamic networks of our brains. Our aim in this review is to raise awareness of and interest in three important facets of how timing impacts our understanding of attention. These include the challenges posed to attention by the timing of neural processing and psychological functions, the opportunities conferred to attention by various temporal structures in the environment, and how tracking the time courses of neural and behavioral modulations with continuous measures yields surprising insights into the workings and principles of attention.

Good-enough attentional guidance

Theories of attention posit that attentional guidance operates on information held in a target template within memory. The template is often thought to contain veridical target features, akin to a photograph, and to guide attention to objects that match the exact target features. However, recent evidence suggests that attentional guidance is highly flexible and often guided by non-veridical features, a subset of features, or only associated features. We integrate these findings and propose that attentional guidance maximizes search efficiency based on a 'good-enough' principle to rapidly localize candidate target objects. Candidates are then serially interrogated to make target-match decisions using more precise information. We suggest that good-enough guidance optimizes the speed-accuracy-effort trade-offs inherent in each stage of visual search.

Working memory is supported by learning to represent items as actions

Working memory is typically described as a set of processes that allow for the maintenance and manipulation of information for proximal actions, yet the "action" portion of this construct is commonly overlooked. In contrast, neuroscience-informed theories of working memory have emphasized the hierarchical nature of memory representations, including both goals and sensory representations. These two representational domains are combined for the service of actions. Here, we tested whether, as it is commonly measured (i.e., with computer-based stimuli and button-based responses), working memory involved the planning of motor actions (i.e., specific button presses). Next, we examined the role of motor plan learning in successful working memory performance. Results showed that visual working memory performance was disrupted by unpredictable motor mappings, indicating a role for motor planning in working memory. Further, predictable motor mappings were in fact learned over the course of the experiment, thereby causing the measure of working memory to be partially a measure of participants' ability to learn arbitrary associations between visual stimuli and motor responses. Such learning was not highly specific to certain mappings; in sequences of short tasks, participants improved in their abilities to learn to represent items as actions in working memory. We discuss implications for working memory theories in light of hierarchical structure learning and ecological validity.

Changes in behavioral priority influence the accessibility of working memory content

Evolving behavioral goals require the existence of selection mechanisms that prioritize task-relevant working memory (WM) content for action. Selecting an item stored in WM is known to blunt and/or reverse information loss in stimulus-specific representations of that item reconstructed from human brain activity, but extant studies have focused on all-or-none circumstances that allow or disallow an agent to select one of several items stored in WM. Conversely, behavioral studies suggest that humans can flexibly assign different levels of priority to different items stored in WM, but how doing so influences neural representations of WM content is unclear. One possibility is that assigning different levels of priority to items in WM influences the quality of those representations, resulting in more robust neural representations of high- vs. low-priority WM content. A second - and non-exclusive - possibility is that asymmetries in behavioral priority influence how rapidly neural representations of high- vs. low-priority WM content can be selected and reported. We tested these possibilities in two experiments by decoding high- and low-priority WM content from EEG recordings obtained while human volunteers performed a retrospectively cued WM task. Probabilistic changes in the behavioral relevance of a remembered item had no effect on our ability to decode it from EEG signals; instead, these changes influenced the latency at which above-chance decoding performance was reached. Thus, our results indicate that probabilistic changes in the behavioral relevance of WM content influence the ease with which memories can be selected independently of their strength.

Interference between items stored for distinct tasks in visual working memory

The action perspective on working memory suggests that memory representations are coded according to their specific temporal and behavioral task demands. This stands in contrast to theories that assume representations are stored in a task-agnostic format within a "common workspace". Here, we tested whether visual items that are memorized for different tasks are stored separately from one another or show evidence of inter-item interference during concurrent maintenance, indicating a common storage. In two experiments, we combined a framing memory task (memorize a motion direction for continuous direction report) with an embedded memory task (memorize a motion direction for a binary direction discrimination) that was placed within the retention period of the framing task. Even though the temporal and action demands were item specific, we observed two types of interference effects between the items: The embedded motion direction was (1) repulsed away and (2) degraded in precision by the motion direction of the item in the framing task. Repulsion and precision degradation increased with item similarity when both items were concurrently held in working memory. In contrast, perceptual and iconic memory control conditions revealed weaker repulsion overall and no interference effect on precision during the stimulus processing stages prior to working memory consolidation. Thus, additional inter-item interference arose uniquely within working memory. Together, our results present evidence that items that are stored for distinct tasks to be performed at distinct points in time, reside in a common workspace in working memory.

Turning Attention Inside Out: How Working Memory Serves Behavior

Flexible behavior requires guidance not only by sensations that are available immediately but also by relevant mental contents carried forward through working memory. Therefore, selective-attention functions that modulate the contents of working memory to guide behavior (inside-out) are just as important as those operating on sensory signals to generate internal contents (outside-in). We review the burgeoning literature on selective attention in the inside-out direction and underscore its functional, flexible, and future-focused nature. We discuss in turn the purpose (why), targets (what), sources (when), and mechanisms (how) of selective attention inside working memory, using visual working memory as a model. We show how the study of internal selective attention brings new insights concerning the core cognitive processes of attention and working memory and how considering selective attention and working memory together paves the way for a rich and integrated understanding of how mind serves behavior.

Dynamics Are the Only Constant in Working Memory

In this short perspective, we reflect upon our tendency to use oversimplified and idiosyncratic tasks in a quest to discover general mechanisms of working memory. We discuss how the work of Mark Stokes and collaborators has looked beyond localized, temporally persistent neural activity and shifted focus toward the importance of distributed, dynamic neural codes for working memory. A critical lesson from this work is that using simplified tasks does not automatically simplify the neural computations supporting behavior (even if we wish it were so). Moreover, Stokes' insights about multidimensional dynamics highlight the flexibility of the neural codes underlying cognition and have pushed the field to look beyond static measures of working memory.

Opening Questions in Visual Working Memory

In this reflective piece on visual working memory, I depart from the laboriously honed skills of writing a review. Instead of integrating approaches, synthesizing evidence, and building a cohesive perspective, I scratch my head and share niggles and puzzlements. I expose where my scholarship and understanding are stumped by findings and standard views in the literature.

Temporal expectation based on the duration variability modulates alpha oscillations during working memory retention

While maintaining information over a delay of time, working memory (WM) also allows individuals to prepare the mnemonic contents for prospective utilisation. However, it remains unclear whether the expectation of the time of WM test could modulate neural responses during the retention interval of WM and subsequent performance. Here, we investigated whether temporal expectations based on the variability of delay duration can modulate 9-13 Hz alpha oscillations during WM retention and whether the expectation-induced alpha activity was associated with WM performance. Participants performed a retro-cueing WM task with magnetoencephalography (MEG) (Experiment 1) and a standard WM task with electroencephalography (EEG) (Experiment 2). The expectation of the timing of the WM test was manipulated by the temporal structure of the tasks with small or large variability in the delay durations. We showed that alpha oscillations during retention interval and WM performance varied with duration variability in both of the MEG and EEG experiments. The novel finding was greater alpha-power attenuation over the left frontal and parietal regions during WM retention when the duration variability was small and the test onset was predictable, compared to when the duration variability was large and the test onset was less predictable. Importantly, we observed a positive relationship in variability difference between the response benefit and alpha-power attenuation in the left posterior parietal regions at both MEG-source and EEG-electrode levels. Finally, we confirmed the behavioural benefit when a condition with a fixed delay-duration was included in a behavioural experiment (Experiment 3). When conjoined, the delay duration enables individuals to anticipate when the relevant information would be put to work, and alpha oscillations track the anticipatory states during WM maintenance.

Visual working memory representations bias attention more when they are the target of an action plan

Attention has frequently been regarded as an emergent property of linking sensory representations to action plans. It has recently been proposed that similar mechanisms may operate within visual working memory (VWM), such that linking an object in VWM to an action plan strengthens its sensory memory representation, which then expresses as an attentional bias. Here we directly tested this hypothesis by comparing attentional biases induced by VWM representations which were the target of a future action, to those induced by VWM representations that were equally task-relevant, but not the direct target of action. We predicted that the first condition would result in a more prioritized memory state and hence stronger attentional biases. Specifically, participants memorized a geometric shape for a subsequent memory test. At test, in case of a match, participants either had to perform a grip movement on the matching object (action condition), or perform the same movement, but on an unrelated object (control condition). To assess any attentional biases, during the delay period between memorandum and test, participants performed a visual selection task in which either the target was surrounded by the memorized shape (congruent trials) or a distractor (incongruent trials). Eye movements were measured as a proxy for attentional priority. We found a significant interaction for saccade latencies between action condition and shape congruency, reflecting more pronounced VWM-based attentional biases in the action condition. Our results are consistent with the idea that action plans prioritize sensory representations in VWM.

Explainable Machine-Learning-Based Characterization of Abnormal Cortical Activities for Working Memory of Restless Legs Syndrome Patients

Restless legs syndrome (RLS) is a sensorimotor disorder accompanied by a strong urge to move the legs and an unpleasant sensation in the legs, and is known to accompany prefrontal dysfunction. Here, we aimed to clarify the neural mechanism of working memory deficits associated with RLS using machine-learning-based analysis of single-trial neural activities. A convolutional neural network classifier was developed to discriminate the cortical activities between RLS patients and normal controls. A layer-wise relevance propagation was applied to the trained classifier in order to determine the critical nodes in the input layer for the output decision, i.e., the time/location of cortical activities discriminating RLS patients and normal controls during working memory tasks. Our method provided high classification accuracy (~94%) from single-trial event-related potentials, which are known to suffer from high inter-trial/inter-subject variation and low signal-to-noise ratio, after strict separation of training/test/validation data according to leave-one-subject-out cross-validation. The determined critical areas overlapped with the cortical substrates of working memory, and the neural activities in these areas were correlated with some significant clinical scores of RLS.

Preparing for the unknown: How working memory provides a link between perception and anticipated action

What mechanisms underlie the transfer of a working memory representation into a higher-level code for guiding future actions? Electrophysiological correlates of attentional selection and motor preparation processes within working memory were investigated in two retrospective cuing tasks. In the first experiment, participants stored the orientation and location of a grating. Subsequent feature cues (selective vs. neutral) indicated which feature would be the target for later report. The oscillatory response in the mu and beta frequency range with an estimated source in the sensorimotor cortex contralateral to the responding hand was used as correlate of motor preparation. Mu/beta suppression was stronger following the selective feature cues compared to the neutral cue, demonstrating that purely feature-based selection is sufficient to form a prospective motor plan. In the second experiment, another retrospective cue was included to study whether knowledge of the task at hand is necessary to initiate motor preparation. Following the feature cue, participants were cued to either compare the stored feature(s) to a probe stimulus (recognition task) or to adjust the memory probe to match the target feature (continuous report task). An analogous suppression of mu oscillations was observed following a selective feature cue, even ahead of task specification. Further, a subsequent selective task cue again elicited a mu/beta suppression, which was stronger after a continuous report task cue. This indicates that working memory is able to flexibly store different types of information in higher-level mental codes to provide optimal prerequisites for all required action possibilities.

Grounding Context in Embodied Cognitive Robotics

Biological agents are context-dependent systems that exhibit behavioral flexibility. The internal and external information agents process, their actions, and emotions are all grounded in the context within which they are situated. However, in the field of cognitive robotics, the concept of context is far from being clear with most studies making little to no reference to it. The aim of this paper is to provide an interpretation of the notion of context and its core elements based on different studies in natural agents, and how these core contextual elements have been modeled in cognitive robotics, to introduce a new hypothesis about the interactions between these contextual elements. Here, global context is categorized as agent-related, environmental, and task-related context. The interaction of their core elements, allows agents to first select self-relevant tasks depending on their current needs, or for learning and mastering their environment through exploration. Second, to perform a task and continuously monitor its performance. Third, to abandon a task in case its execution is not going as expected. Here, the monitoring of prediction error, the difference between sensorimotor predictions and incoming sensory information, is at the core of behavioral flexibility during situated action cycles. Additionally, monitoring prediction error dynamics and its comparison with the expected reduction rate should indicate the agent its overall performance on executing the task. Sensitivity to performance evokes emotions that function as the driving element for autonomous behavior which, at the same time, depends on the processing of the interacting core elements. Taking all these into account, an interactionist model of contexts and their core elements is proposed. The model is embodied, affective, and situated, by means of the processing of the agent-related and environmental core contextual elements. Additionally, it is grounded in the processing of the task-related context and the associated situated action cycles during task execution. Finally, the model proposed here aims to guide how artificial agents should process the core contextual elements of the agent-related and environmental context to give rise to the task-related context, allowing agents to autonomously select a task, its planning, execution, and monitoring for behavioral flexibility.

Planning the Potential Future during Multi-item Visual Working Memory

Working memory allows us to retain visual information to guide upcoming future behavior. In line with this future-oriented purpose of working memory, recent studies have shown that action planning occurs during encoding and retention of a single visual item, for which the upcoming action is certain. We asked whether and how this extends to multi-item visual working memory, when visual representations serve the potential future. Human participants performed a visual working-memory task with a memory-load manipulation (one/two/four items) and a delayed orientation-reproduction report (of one item). We measured EEG to track 15- to 25-Hz beta activity in electrodes contralateral to the required response hand-a canonical marker of action planning. We show an attenuation of beta activity, not only in Load 1 (with one certain future action) but also in Load 2 (with two potential future actions), compared with Load 4 (with low prospective-action certainty). Moreover, in Load 2, potential action planning occurs regardless whether both visual items afford similar or dissimilar manual responses, and it predicts the speed of ensuing memory-guided behavior. This shows that potential action planning occurs during multi-item visual working memory and brings the perspective that working memory helps us prepare for the potential future.

Flexible utilization of spatial- and motor-based codes for the storage of visuo-spatial information

Working memory provides flexible storage of information in service of upcoming behavioral goals. Some models propose specific fixed loci and mechanisms for the storage of visual information in working memory, such as sustained spiking in parietal and prefrontal cortex during working memory maintenance. An alternative view is that information can be remembered in a flexible format that best suits current behavioral goals. For example, remembered visual information might be stored in sensory areas for easier comparison to future sensory inputs, or might be re-coded into a more abstract action-oriented format and stored in motor areas. Here, we tested this hypothesis using a visuo-spatial working memory task where the required behavioral response was either known or unknown during the memory delay period. Using functional magnetic resonance imaging (fMRI) and multivariate decoding, we found that there was less information about remembered spatial position in early visual and parietal regions when the required response was known versus unknown. Furthermore, a representation of the planned motor action emerged in primary somatosensory, primary motor, and premotor cortex during the same task condition where spatial information was reduced in early visual cortex. These results suggest that the neural networks supporting working memory can be strategically reconfigured depending on specific behavioral requirements during a canonical visual working memory paradigm.

Unveiling the abstract format of mnemonic representations

Working memory (WM) enables information storage for future use, bridging the gap between perception and behavior. We hypothesize that WM representations are abstractions of low-level perceptual features. However, the neural nature of these putative abstract representations has thus far remained impenetrable. Here, we demonstrate that distinct visual stimuli (oriented gratings and moving dots) are flexibly recoded into the same WM format in visual and parietal cortices when that representation is useful for memory-guided behavior. Specifically, the behaviorally relevant features of the stimuli (orientation and direction) were extracted and recoded into a shared mnemonic format that takes the form of an abstract line-like pattern. We conclude that mnemonic representations are abstractions of percepts that are more efficient than and proximal to the behaviors they guide.

Shielding working-memory representations from temporally predictable external interference

Protecting working-memory content from distracting external sensory inputs and intervening tasks is an ubiquitous demand in daily life. Here, we ask whether and how temporal expectations about external events can help mitigate effects of such interference during working-memory retention. We manipulated the temporal predictability of interfering items that occurred during the retention period of a visual working-memory task and report that temporal expectations reduce the detrimental influence of external interference on subsequent memory performance. Moreover, to determine if the protective effects of temporal expectations rely on distractor suppression or involve shielding of internal representations, we compared effects after irrelevant distractors that could be ignored vs. interrupters that required a response. Whereas distractor suppression may be sufficient to confer protection from predictable distractors, any benefits after interruption are likely to involve memory shielding. We found similar benefits of temporal expectations after both types of interference. We conclude that temporal expectations may play an important role in safeguarding behaviour based on working memory - acting through mechanisms that include the shielding of internal content from external interference.