Parietal and Frontal Cortex Encode Stimulus-Specific Mnemonic Representations during Visual Working Memory

Representational geometry explains puzzling error distributions in behavioral tasks

Measuring and interpreting errors in behavioral tasks is critical for understanding cognition. Conventional wisdom assumes that encoding/decoding errors for continuous variables in behavioral tasks should naturally have Gaussian distributions, so that deviations from normality in the empirical data indicate the presence of more complex sources of noise. This line of reasoning has been central for prior research on working memory. Here, we reassess this assumption and find that even in ideal observer models with Gaussian encoding noise, the error distribution is generally non-Gaussian, contrary to the commonly held belief. Critically, we find that the shape of the error distribution is determined by the geometrical structure of the encoding manifold via a simple rule. In the case of a high-dimensional geometry, the error distributions naturally exhibit flat tails. Using this insight, we apply our theory to visual short-term memory tasks, and find that it can account for a large array of experimental data with only two free parameters. Our results challenge the dominant view in the mechanisms and capacity constraints of working memory systems. They instead suggest that the Bayesian framework, which explains various aspects of perceptual behavior, also provides an excellent account of working memory. Overall, our results establish a direct connection between neural manifold geometry and behavior, and call attention to the geometry of the representation as a critically important, yet underappreciated factor in determining the character of errors in human behavior.

Verbal Encoding Strategies in Visuo-Spatial Working Memory

Visual working memory and verbal storage are often investigated independently of one another. However, a growing body of evidence suggests that naming visual stimuli can provide an advantage in performance during visual working memory tasks. On the other hand, there is also evidence that labeling could lead to biases in recall. Here, we present an exploratory investigation of verbal labels associated with the memorization of simple visuo-spatial stimuli, and how the use of these labels informs recall behavior of the same stimuli in a separate working memory task. English-speaking participants performed a working memory task with orientation and location stimuli, followed by a separate naming task featuring the same stimuli. We found a diverse set of labels employed frequently and with a consistent distribution across stimulus types, the stimulus space, and among participants. The use of individual spatial words, predicted class 1 cardinal biases in memory (i.e. the observation that cardinal stimuli are more accurately recalled than non-cardinal ones). Conversely, words expressing uncertainty (e.g. 'slightly', 'near') predicted class 2 cardinal bias (i.e. recall biases away from the cardinal planes). This relationship between word use and recall biases is consistent with shared representational resources that are used for both visuo-spatial and verbal working memory.

Advancing working memory research through cortico-cortical transcranial magnetic stimulation

The neural underpinnings of working memory (WM) have been of continuous scientific interest for decades. As the understanding of WM progresses and new theories, such as the distributed view of WM, develop, the need to advance the methods used to study WM also arises. This perspective discusses how building from the state-of-the-art in the field of transcranial magnetic stimulation (TMS), and utilising cortico-cortical TMS, may pave the way for testing some of the predictions proposed by the distributed WM view. Further, after briefly discussing current barriers that need to be overcome for implementing cortico-cortical TMS for WM research, examples of how cortico-cortical TMS may be employed in the context of WM research are provided, guided by the ongoing debate on the sensory recruitment framework.

The representation of abstract goals in working memory is supported by task-congruent neural geometry

Successful goal-directed behavior requires the maintenance and implementation of abstract task goals on concrete stimulus information in working memory. Previous working memory research has revealed distributed neural representations of task information across cortex. However, how the distributed task representations emerge and communicate with stimulus-specific information to implement flexible goal-directed computations is still unclear. Here, leveraging electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) in human participants along with state space analyses, we provided converging evidence in support of a low-dimensional neural geometry of goal information congruent with a designed task space, which first emerged in frontal cortex during goal maintenance and then transferred to posterior cortex through frontomedial-to-posterior theta coherence for implementation on stimulus-specific representations. Importantly, the fidelity of the goal geometry was associated with memory performance. Collectively, our findings suggest that abstract goals in working memory are represented in an organized, task-congruent neural geometry for communications from frontal to posterior cortex to enable computations necessary for goal-directed behaviors.

Anatomical Connectivity Constrains Dynamic Functional Connectivity among Neural Systems: Implications for Cognition and Behavior

Leslie Ungerleider had a tremendous impact across many different areas of cognitive neuroscience. Her ideas and her approach, as well as her findings, will continue to impact the field for generations to come. One of the most impactful aspects of her approach was her focus on the ways that anatomical connections constrain functional communications among brain regions. Furthermore, she emphasized that changes in these functional communications, whether from lesions to the anatomical connections or temporary modulations of the efficacy of information transmission resulting from selective attention, have consequences for cognition and behavior. By necessity, this short review cannot cover the vast amount of research that contributed to or benefited from Leslie's work. Rather, we focus on one line of research that grew directly from some of Leslie's early work and her mentoring on these important concepts. This research and the many other lines of research that arose from these same origins has helped develop our understanding of the visual system, and cognitive systems more generally, as collections of highly organized, specialized, dynamic, and interacting subsystems.

Maintenance of Bodily Expressions Modulates Functional Connectivity Between Prefrontal Cortex and Extrastriate Body Area During Working Memory Processing

Background/Objectives: As a form of visual input, bodily expressions can be maintained and manipulated in visual working memory (VWM) over a short period of time. While the prefrontal cortex (PFC) plays an indispensable role in top-down control, it remains largely unclear whether this region also modulates the VWM storage of bodily expressions during a delay period. Therefore, the two primary goals of this study were to examine whether the emotional bodies would elicit heightened brain activity among areas such as the PFC and extrastriate body area (EBA) and whether the emotional effects subsequently modulate the functional connectivity patterns for active maintenance during delay periods. Methods: During functional magnetic resonance imaging (fMRI) scanning, participants performed a delayed-response task in which they were instructed to view and maintain a body stimulus in working memory before emotion categorization (happiness, anger, and neutral). If processing happy and angry bodies consume increased cognitive demands, stronger PFC activation and its functional connectivity with perceptual areas would be observed. Results: Results based on univariate and multivariate analyses conducted on the data collected during stimulus presentation revealed an enhanced processing of the left PFC and left EBA. Importantly, subsequent functional connectivity analyses performed on delayed-period data using a psychophysiological interaction model indicated that functional connectivity between the PFC and EBA increases for happy and angry bodies compared to neutral bodies. Conclusions: The emotion-modulated coupling between the PFC and EBA during maintenance deepens our understanding of the functional organization underlying the VWM processing of bodily information.

Sensory stimulation enhances visual working memory capacity

Visual working memory (vWM) plays a crucial role in visual information processing and higher cognitive functions; however, it has a very limited capacity. Recently, several studies have successfully modulated vWM capacity in humans using entrainment with transcranial alternate current stimulation (tACS) by targeting parietal theta in a frequency-specific manner. In the current study, we aim to expand upon these findings by utilizing sensory instead of electrical stimulation. Across six behavioral experiments (combined N = 209), we applied rhythmic visual and auditory sensory stimulation at 4 Hz and 7 Hz, aiming to modulate vWM capacity. Collectively, the results showed an overall robust improvement with sensory stimulation at either frequency, compared to baseline. However, contrary to our prediction, 7 Hz stimulation tended to slightly outperform 4 Hz stimulation. Importantly, the observed facilitatory effect was mainly driven by the low-capacity sub-group of participants. Follow-up experiments using the Attention Network Test (ANT) and pupillometry measures did not find evidence that this effect could be directly attributed to modulation of phasic or tonic arousal. We speculate that our results differed from those obtained with tACS due to targeting functionally different theta oscillations, or the modulation of participants' temporal expectations.

A gradual transition toward categorical representations along the visual hierarchy during working memory, but not perception

The ability to stably maintain visual information over brief delays is central to healthy cognitive functioning, as is the ability to differentiate such internal representations from external inputs. One possible way to achieve both is via multiple concurrent mnemonic representations along the visual hierarchy that differ systematically from the representations of perceptual inputs. To test this possibility, we examine orientation representations along the visual hierarchy during perception and working memory. Human participants directly viewed, or held in mind, oriented grating patterns, and the similarity between fMRI activation patterns for different orientations was calculated throughout retinotopic cortex. During direct viewing of grating stimuli, similarity was relatively evenly distributed amongst all orientations, while during working memory the similarity was higher around oblique orientations. We modeled these differences in representational geometry based on the known distribution of orientation information in the natural world: The "veridical" model uses an efficient coding framework to capture hypothesized representations during visual perception. The "categorical" model assumes that different "psychological distances" between orientations result in orientation categorization relative to cardinal axes. During direct perception, the veridical model explained the data well. During working memory, the categorical model gradually gained explanatory power over the veridical model for increasingly anterior retinotopic regions. Thus, directly viewed images are represented veridically, but once visual information is no longer tethered to the sensory world there is a gradual progression to more categorical mnemonic formats along the visual hierarchy.

Differential Mnemonic Contributions of Cortical Representations during Encoding and Retrieval

Several recent fMRI studies of episodic and working memory representations converge on the finding that visual information is most strongly represented in occipito-temporal cortex during the encoding phase but in parietal regions during the retrieval phase. It has been suggested that this location shift reflects a change in the content of representations, from predominantly visual during encoding to primarily semantic during retrieval. Yet, direct evidence on the nature of encoding and retrieval representations is lacking. It is also unclear how the representations mediating the encoding-retrieval shift contribute to memory performance. To investigate these two issues, in the current fMRI study, participants encoded pictures (e.g., picture of a cardinal) and later performed a word recognition test (e.g., word "cardinal"). Representational similarity analyses examined how visual (e.g., red color) and semantic representations (e.g., what cardinals eat) support successful encoding and retrieval. These analyses revealed two novel findings. First, successful memory was associated with representational changes in cortical location (from occipito-temporal at encoding to parietal at retrieval) but not with changes in representational content (visual vs. semantic). Thus, the representational encoding-retrieval shift cannot be easily attributed to a change in the nature of representations. Second, in parietal regions, stronger representations predicted encoding failure but retrieval success. This encoding-retrieval "flip" in representations mimics the one previously reported in univariate activation studies. In summary, by answering important questions regarding the content and contributions to the performance of the representations mediating the encoding-retrieval shift, our findings clarify the neural mechanisms of this intriguing phenomenon.

Manipulating attentional priority creates a trade-off between memory and sensory representations in human visual cortex

People often remember visual information over brief delays while actively engaging with ongoing inputs from the surrounding visual environment. Depending on the situation, one might prioritize mnemonic contents (i.e., remembering details of a past event), or preferentially attend sensory inputs (i.e., minding traffic while crossing a street). Previous fMRI work has shown that early sensory regions can simultaneously represent both mnemonic and passively viewed sensory information. Here we test the limits of such simultaneity by manipulating attention towards sensory distractors during a working memory task performed by human subjects during fMRI scanning. Participants remembered the orientation of a target grating while a distractor grating was shown during the middle portion of the memory delay. Critically, there were several subtle changes in the contrast and the orientation of the distractor, and participants were cued to either ignore the distractor, detect a change in contrast, or detect a change in orientation. Despite sensory stimulation being matched in all three conditions, the fidelity of memory representations in early visual cortex was highest when the distractor was ignored, intermediate when participants attended distractor contrast, and lowest when participants attended the orientation of the distractor during the delay. In contrast, the fidelity of distractor representations was lowest when ignoring the distractor, intermediate when attending distractor-contrast, and highest when attending distractor-orientation. These data suggest a trade-off in early sensory representations when engaging top-down feedback to attend both seen and remembered features and may partially explain memory failures that occur when subjects are distracted by external events.

Similar patterns of brain activity for holistic representation of working memory: a study in children and adults

Working memory is the fundamental function of the various cognitive processes and abilities in the overall trajectory of development. Significant advances in multivariate analysis of human functional magnetic resonance imaging data have converged functional segregation models toward integrated representation-based models. However, due to the inherent limitations of the multi-voxel pattern analysis method, we are unable to determine whether the underlying neural representations are spatially similar in the brain. Our study attempts to answer this question by examining the spatial similarity of brain activity during the working memory task in children and adults. Our results reveal similar patterns of activity between the regions involved in working memory. This functional network of similar spatial patterns was observed in both normally developing children and adults. However, the between-region similarity was more pronounced in adults than in children and associated with better performance. We propose an exchange of similar information flows through the brain at an integrated level of working memory processes, underpinning the holistic nature of working memory representation.

EEG decoding reveals task-dependent recoding of sensory information in working memory

Working memory (WM) supports future behavior by retaining perceptual information obtained in the recent past. The present study tested the hypothesis that WM recodes sensory information in a format that better supports behavioral goals. We recorded EEG while participants performed color delayed-estimation tasks where the colorwheel for the response was either randomly rotated or held fixed across trials. Accordingly, observers had to remember the exact colors in the Rotation condition, whereas they could prepare for a response based on the fixed mapping between the colors and their corresponding locations on the colorwheel in the No-Rotation condition. Results showed that the color reports were faster and more precise in the No-Rotation condition even when exactly the same set of colors were tested in both conditions. To investigate how the color information was maintained in the brain, we decoded the color using a multivariate EEG classification method. The decoding was limited to the stimulus encoding period in the Rotation condition, whereas it continued to be significant during the maintenance period in the No-Rotation condition, indicating that the color information was actively maintained in the condition. Follow-up analyses suggested that the prolonged decoding was not merely driven by the covert shift of attention but rather by the recoding of sensory information into an action-oriented response format. Together, these results provide converging evidence that WM flexibly recodes sensory information depending on the specific task context to optimize subsequent behavioral performance.

Altered EEG Theta and Alpha Band Functional Connectivity in Mild Cognitive Impairment During Working Memory Coding

Individuals with mild cognitive impairment (MCI), the preclinical stage of Alzheimer disease (AD), suffer decline in their visual working memory (WM) functions. Using large-scale network analysis of electroencephalography (EEG), the current study intended to investigate if there are differences in functional connectivity properties extracted during visual WM coding stages between MCI patients and normal controls (NC). A total of 21 MCI patients and 20 NC performed visual memory tasks of load four, while 32-channel EEG recordings were acquired. The functional connectivity properties were extracted from the acquired EEGs by the directed transform function (DTF) via spectral Granger causal analysis. Brain network analyses revealed distinctive brain network patterns between the two groups during the WM coding stage. Compared with the NC, MCI patients exhibited a reduced visual network connectivity of the frontal-temporal in θ (4-7Hz) band. A likely compensation mechanism was observed in MCI patients, with a strong brain functional connectivity of the frontal-occipital and parietal-occipital in both θ and α (8-13Hz) band. Further analyses of the network core node properties based on the differential brain network showed that, in θ band, there was a significant difference in the out-degree of the frontal lobe and parietal lobe between the two groups, while in α band, such difference was located only in the parietal lobe. The current study found that, in MCI patients, dysconnectivity is found from the prefrontal lobe to bilateral temporal lobes, leading to increased recruitment of functional connectivity in the frontal-occipital and parietal-occipital direction. The dysconnectivity pattern of MCI is more complex and primarily driven by core nodes Pz and Fz. These results significantly expanded previous knowledge of MCI patients' EEG dynamics during WM tasks and provide new insights into the underpinning neural mechanism MCI. It further provided a potential therapeutic target for clinical interventions of the condition.

Dysfunction of large-scale brain networks underlying cognitive impairments in shift work disorder

It has been demonstrated that shift work can affect cognitive functions. Several neuroimaging studies have revealed altered brain function and structure for patients with shift work disorder (SWD). However, knowledge on the dysfunction of large-scale brain networks underlying cognitive impairments in shift work disorder is limited. This study aims to identify altered functional networks associated with cognitive declines in shift work disorder, and to assess their potential diagnostic value. Thirty-four patients with shift work disorder and 36 healthy controls (HCs) were recruited to perform the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS) and resting-state functional scans. After surface-based preprocessing, we calculated within- and between-network functional connectivity (FC) using the Dosenbach atlas. Moreover, correlation analysis was done between altered functional connectivity of large-scale brain networks and scores of cognitive assessments in patients with shift work disorder. Finally, we established a classification model to provide features for patients with shift work disorder concerning the disrupted large-scale networks. Compared with healthy controls, increased functional connectivity within-networks across the seven brain networks, and between-networks involving ventral attention network (VAN)-subcortical network (SCN), SCN-frontoparietal network (FPN), and somatosensory network (SMN)-SCN were observed in shift work disorder. Decreased functional connectivity between brain networks was found in shift work disorder compared with healthy controls, including visual network (VN)-FPN, VN-default mode network (DMN), SMN-DMN, dorsal attention network (DAN)-DMN, VAN-DMN, and FPN-DMN. Furthermore, the altered functional connectivity of large-scale brain networks was significantly correlated with scores of immediate memory, visuospatial, and delayed memory in patients with shift work disorder, respectively. Abnormal functional connectivity of large-scale brain networks may play critical roles in cognitive dysfunction in shift work disorder. Our findings provide new evidence to interpret the underlying neural mechanisms of cognitive impairments in shift work disorder.

Shared structure facilitates working memory of multiple sequences

Daily experiences often involve the processing of multiple sequences, yet storing them challenges the limited capacity of working memory (WM). To achieve efficient memory storage, relational structures shared by sequences would be leveraged to reorganize and compress information. Here, participants memorized a sequence of items with different colors and spatial locations and later reproduced the full color and location sequences one after another. Crucially, we manipulated the consistency between location and color sequence trajectories. First, sequences with consistent trajectories demonstrate improved memory performance and a trajectory correlation between reproduced color and location sequences. Second, sequences with consistent trajectories show neural reactivation of common trajectories, and display spontaneous replay of color sequences when recalling locations. Finally, neural reactivation correlates with WM behavior. Our findings suggest that a shared common structure is leveraged for the storage of multiple sequences through compressed encoding and neural replay, together facilitating efficient information organization in WM.

Goal-directed attention transforms both working and long-term memory representations in the human parietal cortex

The abundance of distractors in the world poses a major challenge to our brain's limited processing capacity, but little is known about how selective attention modulates stimulus representations in the brain to reduce interference and support durable target memory. Here, we collected functional magnetic resonance imaging (fMRI) data in a selective attention task in which target and distractor pictures of different visual categories were simultaneously presented. Participants were asked to selectively process the target according to the effective cue, either before the encoding period (i.e., perceptual attention) or the maintenance period (i.e., reflective attention). On the next day, participants were asked to perform a memory recognition task in the scanner in which the targets, distractors, and novel items were presented in a pseudorandom order. Behavioral results showed that perceptual attention was better at enhancing target memory and reducing distractor memory than reflective attention, although the overall memory capacity (memory for both target and distractor) was comparable. Using multiple-voxel pattern analysis of the neural data, we found more robust target representation and weaker distractor representation in working memory for perceptual attention than for reflective attention. Interestingly, perceptual attention partially shifted the regions involved in maintaining the target representation from the visual cortex to the parietal cortex. Furthermore, the targets and distractors simultaneously presented in the perceptual attention condition showed reduced pattern similarity in the parietal cortex during retrieval compared to items not presented together. This neural pattern repulsion positively correlated with individuals' recognition of both targets and distractors. These results emphasize the critical role of selective attention in transforming memory representations to reduce interference and improve long-term memory performance.

Neuronal representation of visual working memory content in the primate primary visual cortex

The human ability to perceive vivid memories as if they "float" before our eyes, even in the absence of actual visual stimuli, captivates the imagination. To determine the neural substrates underlying visual memories, we investigated the neuronal representation of working memory content in the primary visual cortex of monkeys. Our study revealed that neurons exhibit unique responses to different memory contents, using firing patterns distinct from those observed during the perception of external visual stimuli. Moreover, this neuronal representation evolves with alterations in the recalled content and extends beyond the retinotopic areas typically reserved for processing external visual input. These discoveries shed light on the visual encoding of memories and indicate avenues for understanding the remarkable power of the mind's eye.

Visual working memories are abstractions of percepts

During perception, decoding the orientation of gratings depends on complex interactions between the orientation of the grating, aperture edges, and topographic structure of the visual map. Here, we aimed to test how aperture biases described during perception affect working memory (WM) decoding. For memoranda, we used gratings multiplied by radial and angular modulators to generate orthogonal aperture biases for identical orientations. Therefore, if WM representations are simply maintained sensory representations, they would have similar aperture biases. If they are abstractions of sensory features, they would be unbiased and the modulator would have no effect on orientation decoding. Neural patterns of delay period activity while maintaining the orientation of gratings with one modulator (e.g. radial) were interchangeable with patterns while maintaining gratings with the other modulator (e.g. angular) in visual and parietal cortex, suggesting that WM representations are insensitive to aperture biases during perception. Then, we visualized memory abstractions of stimuli using models of visual field map properties. Regardless of aperture biases, WM representations of both modulated gratings were recoded into a single oriented line. These results provide strong evidence that visual WM representations are abstractions of percepts, immune to perceptual aperture biases, and compel revisions of WM theory.

Concurrent attention to hetero-depth surfaces in 3-D visual space is governed by theta rhythm

When simultaneously confronted with multiple attentional targets, visual system employs a time-multiplexing approach in which each target alternates for prioritized access, a mechanism broadly known as rhythmic attentional sampling. For the past decade, rhythmic attentional sampling has received mounting support from converging behavioral and neural findings. However, so compelling are these findings that a critical test ground has been long overshadowed, namely the 3-D visual space where attention is complicated by extraction of the spatial layout of surfaces extending beyond 2-D planes. It remains unknown how attentional deployment to multiple targets is accomplished in the 3-D space. Here, we provided a time-resolved portrait of the behavioral and neural dynamics when participants concurrently attended to two surfaces defined by motion-depth conjunctions. To characterize the moment-to-moment attentional modulation effects, we measured perceptual sensitivity to the hetero-depth surface motions on a fine temporal scale and reconstructed their neural representations using a time-resolved multivariate inverted encoding model. We found that the perceptual sensitivity to the two surface motions rhythmically fluctuated over time at ~4 Hz, with one's enhancement closely tracked by the other's diminishment. Moreover, the behavioral pattern was coupled with an ongoing periodic alternation in strength between the two surface motion representations in the same frequency. Together, our findings provide the first converging evidence of an attentional "pendulum" that rhythmically traverses different stereoscopic depth planes and are indicative of a ubiquitous attentional time multiplexor based on theta rhythm in the 3-D visual space.

When remembering less is more: Unfiltered items are associated with reduced memory fidelity in visual short-term memory

Visual short-term memory (VSTM), the ability to store information no longer visible, is essential for human behavior. VSTM limits vary across the population and are correlated with overall cognitive ability. It has been proposed that low-memory individuals are unable to select only relevant items for storage and that these limitations are greatest when memory demands are high. However, it is unknown whether these effects simply reflect task difficulty and whether they impact the quality of memory representations. Here we varied the number of items presented, or set size, to investigate the effect of memory demands on the performance of visual short-term memory across low- and high-memory groups. Group differences emerged as set size exceeded memory limits, even when task difficulty was controlled. In a change-detection task, the low-memory group performed more poorly when set size exceeded their memory limits. We then predicted that low-memory individuals encoding items beyond measured memory limits would result in the degraded fidelity of memory representations. A continuous report task confirmed that low, but not high, memory individuals demonstrated decreased memory fidelity as set size exceeded measured memory limits. The current study demonstrates that items held in VSTM are stored distinctly across groups and task demands. These results link the ability to maintain high quality representations with overall cognitive ability.

Divergent roles of early visual cortex and inferior frontal junction in visual working memory

BACKGROUND

Recent studies indicate that both prefrontal and visual regions play critical roles in visual working memory (VWM), with prefrontal regions mainly associated with executive functions, and visual cortices linked to representations of memory contents. VWM involves the selective filtering of irrelevant information, yet the specific contributions of the prefrontal regions and visual cortex in this process remain unclear.

OBJECTIVE

To understand the dynamic causal roles of prefrontal and visual regions in VWM.

METHODS

The differentiation of VWM components was achieved using a computational model that incorporated a swap rate for non-target stimuli. Single-pulse magnetic transcranial stimulation (spTMS) was delivered to the early visual cortex (EVC) and the inferior frontal junction (IFJ) across different phases of an orientation recall task that with or without distractors.

RESULTS

Our results indicate that spTMS over the EVC and IFJ influences VWM particularly when distractors are present. VWM precision can be impacted by spTMS applied to either region during the early retention, while spTMS effect is especially prominent when EVC is stimulated during the late retention phase and when directed at the ipsilateral EVC. Conversely, the probability of accurately recalling the target exhibited comparable patterns when spTMS was administered to either the EVC or IFJ.

CONCLUSIONS

We highlight the "sensory recruitment" of VWM characterized by critical involvement of EVC particularly in the information-filtering process within VWM. The maintenance of memory content representations necessitates ongoing communication between the EVC and IFJ throughout the entirety of the VWM process in a dynamic pattern.

Causal functional maps of brain rhythms in working memory

Human working memory is a key cognitive process that engages multiple functional anatomical nodes across the brain. Despite a plethora of correlative neuroimaging evidence regarding the working memory architecture, our understanding of critical hubs causally controlling overall performance is incomplete. Causal interpretation requires cognitive testing following safe, temporal, and controllable neuromodulation of specific functional anatomical nodes. Such experiments became available in healthy humans with the advance of transcranial alternating current stimulation (tACS). Here, we synthesize findings of 28 placebo-controlled studies (in total, 1,057 participants) that applied frequency-specific noninvasive stimulation of neural oscillations and examined working memory performance in neurotypical adults. We use a computational meta-modeling method to simulate each intervention in realistic virtual brains and test reported behavioral outcomes against the stimulation-induced electric fields in different brain nodes. Our results show that stimulating anterior frontal and medial temporal theta oscillations and occipitoparietal gamma rhythms leads to significant dose-dependent improvement in working memory task performance. Conversely, prefrontal gamma modulation is detrimental to performance. Moreover, we found distinct spatial expression of theta subbands, where working memory changes followed orbitofrontal high-theta modulation and medial temporal low-theta modulation. Finally, all these results are driven by changes in working memory accuracy rather than processing time measures. These findings provide a fresh view of the working memory mechanisms, complementary to neuroimaging research, and propose hypothesis-driven targets for the clinical treatment of working memory deficits.

Trichotomy revisited: A monolithic theory of attentional control

The control of attention was long held to reflect the influence of two competing mechanisms of assigning priority, one goal-directed and the other stimulus-driven. Learning-dependent influences on the control of attention that could not be attributed to either of those two established mechanisms of control gave rise to the concept of selection history and a corresponding third mechanism of attentional control. The trichotomy framework that ensued has come to dominate theories of attentional control over the past decade, replacing the historical dichotomy. In this theoretical review, I readily affirm that distinctions between the influence of goals, salience, and selection history are substantive and meaningful, and that abandoning the dichotomy between goal-directed and stimulus-driven mechanisms of control was appropriate. I do, however, question whether a theoretical trichotomy is the right answer to the problem posed by selection history. If we reframe the influence of goals and selection history as different flavors of memory-dependent modulations of attentional priority and if we characterize the influence of salience as a consequence of insufficient competition from such memory-dependent sources of priority, it is possible to account for a wide range of attention-related phenomena with only one mechanism of control. The monolithic framework for the control of attention that I propose offers several concrete advantages over a trichotomy framework, which I explore here.

Brain-wide human oscillatory local field potential activity during visual working memory

Oscillatory activity in the local field potential (LFP) is thought to be a marker of cognitive processes. To understand how it differentiates tasks and brain areas in humans, we recorded LFPs in 15 adults with intracranial depth electrodes, as they performed visual-spatial and shape working memory tasks. Stimulus appearance produced widespread, broad-band activation, including in occipital, parietal, temporal, insular, and prefrontal cortex, and the amygdala and hippocampus. Occipital cortex was characterized by most elevated power in the high-gamma (100-150 Hz) range during the visual stimulus presentation. The most consistent feature of the delay period was a systematic pattern of modulation in the beta frequency (16-40 Hz), which included a decrease in power of variable timing across areas, and rebound during the delay period. These results reveal the widespread nature of oscillatory activity across a broad brain network and region-specific signatures of oscillatory processes associated with visual working memory.

The Reactivation of working memory representations affects attentional guidance

Recent studies have suggested that the neural activity that supported working memory (WM) storage is dynamic over time and this dynamic storage decides memory performance. Does the temporal dynamic of the WM representation also affect visual search, and how does it interact with distractor suppression over time? To address these issues, we tracked the time course of the reactivation of WM representations during visual search by analyzing the electroencephalogram (EEG) and event-related optical signals (EROS) in Experiments 1 and 2, respectively, and investigated the interaction between the representation reactivation and distractor suppression in Experiment 3. Participants had to maintain a color in WM under high- or low-precision requirement and perform a subsequent search task. The reactivation of WM representations was defined by the above-chance decoding accuracy. The EEG results showed that compared with the low-precision requirement, WM-matching distractors captured more attention and the WM representation were reactivated more frequently under high-precision requirement. The EROS results showed that compared with the low-precision requirement, the increased activity in occipital cortex in the WM-matching versus WM-mismatching conditions was observed at 224 ms during visual search under high-precision requirement. Regression analysis showed that the representation reactivation during visual search directly predicted the behavioral WM-based attentional capture effect, while the representation reactivation before visual search impacted the WM-based attentional capture effect through the mediation of distractor suppression during visual search. These results suggest that the reactivation of WM representations and distractor suppression collectively determine WM-based attentional capture.

Visual working memories are abstractions of percepts

Pioneering studies demonstrating that the contents of visual working memory (WM) can be decoded from the patterns of multivoxel activity in early visual cortex transformed not only how we study WM, but theories of how memories are stored. For instance, the ability to decode the orientation of memorized gratings is hypothesized to depend on the recruitment of the same neural encoding machinery used for perceiving orientations. However, decoding evidence cannot be used to test the so-called sensory recruitment hypothesis without understanding the underlying nature of what is being decoded. Although unknown during WM, during perception decoding the orientation of gratings does not simply depend on activities of orientation tuned neurons. Rather, it depends on complex interactions between the orientation of the grating, the aperture edges, and the topographic structure of the visual map. Here, our goals are to 1) test how these aperture biases described during perception may affect WM decoding, and 2) leverage carefully manipulated visual stimulus properties of gratings to test how sensory-like are WM codes. For memoranda, we used gratings multiplied by radial and angular modulators to generate orthogonal aperture biases despite having identical orientations. Therefore, if WM representations are simply maintained sensory representations, they would have similar aperture biases. If they are abstractions of sensory features, they would be unbiased and the modulator would have no effect on orientation decoding. Results indicated that fMRI patterns of delay period activity while maintaining the orientation of a grating with one modulator (eg, radial) were interchangeable with patterns while maintaining a grating with the other modulator (eg, angular). We found significant cross-classification in visual and parietal cortex, suggesting that WM representations are insensitive to aperture biases during perception. Then, we visualized memory abstractions of stimuli using a population receptive field model of the visual field maps. Regardless of aperture biases, WM representations of both modulated gratings were recoded into a single oriented line. These results provide strong evidence that visual WM representations are abstractions of percepts, immune to perceptual aperture biases, and compel revisions of WM theory.

The effect of a cognitive training therapy based on stimulation of brain oscillations in patients with mild cognitive impairment in a Chilean sample: study protocol for a phase IIb, 2 × 3 mixed factorial, double-blind randomised controlled trial

BACKGROUND

The ageing population has increased the prevalence of disabling and high-cost diseases, such as dementia and mild cognitive impairment (MCI). The latter can be considered a prodromal phase of some dementias and a critical stage for interventions to postpone the impairment of functionality. Working memory (WM) is a pivotal cognitive function, representing the fundamental element of executive functions. This project proposes an intervention protocol to enhance WM in these users, combining cognitive training with transcranial electrical stimulation of alternating current (tACS). This technique has been suggested to enhance the neuronal plasticity needed for cognitive processes involving oscillatory patterns. WM stands to benefit significantly from this approach, given its well-defined electrophysiological oscillations. Therefore, tACS could potentially boost WM in patients with neurodegenerative diseases.

METHODS

This study is a phase IIb randomised, double-blind clinical trial with a 3-month follow-up period. The study participants will be 62 participants diagnosed with MCI, aged over 60, from Valparaíso, Chile. Participants will receive an intervention combining twelve cognitive training sessions with tACS. Participants will receive either tACS or placebo stimulation in eight out of twelve training sessions. Sessions will occur twice weekly over 6 weeks. The primary outcomes will be electroencephalographic measurements through the prefrontal theta oscillatory activity, while the secondary effects will be cognitive assessments of WM. The participants will be evaluated before, immediately after, and 3 months after the end of the intervention.

DISCUSSION

The outcomes of this trial will add empirical evidence about the benefits and feasibility of an intervention that combines cognitive training with non-invasive brain stimulation. The objective is to contribute tools for optimal cognitive treatment in patients with MCI. To enhance WM capacity, postpone the impairment of functionality, and obtain a better quality of life.

TRIAL REGISTRATION

ClinicalTrials.gov NCT05291208. Registered on 28 February 2022. ISRCTN87597719 retrospectively registered on 15 September 2023.

Investigating the Roles of the Visual and Parietal Cortex in Representing Content versus Context in Visual Working Memory

Content-to-context binding is crucial for working memory performance. Using a dual-serial retrocueing (DSR) task on oriented gratings, Yu et al. (2020) found that content (orientation) of both prioritized and unprioritized memory items (PMI; UMI) was represented simultaneously in visual cortex, while their context (location) was represented in intraparietal sulcus (IPS), with a priority-based remapping of the representation of content and context of the UMI in each region, respectively. This registered report acquired fMRI of 24 healthy adults while they performed a DSR task with location as the to-be-reported content and orientation as the task-relevant context. We contrasted three accounts: domain-dependent, the engagement of visual and parietal regions depends on the feature domain (orientation vs location); functional, the engagement of these regions depends on their function (content vs context); and hybrid-a combination of the domain-dependent account and the additional stipulation that IPS encodes context regardless of domain. Delay-period activity in early visual cortex conformed most closely with functional predictions: robust priority-sensitive representation of stimulus location (content), but no evidence for the active representation of stimulus orientation (context). Delay-period activity in IPS, in contrast, conformed most closely to predictions of the hybrid account: active representation of content (location) and of prioritized context (orientation). Exploratory analyses further supported the hybrid account of IPS, revealing univariate sensitivity to variation in both content and context load, the latter in a manner that predicted individual differences in behavior. The representation of visual information in working memory is highly dependent on behavioral context.

Finger motor representation supports the autonomy in arithmetic: neuroimaging evidence from abacus training

Researches have reported the close association between fingers and arithmetic. However, it remains unclear whether and how finger training can benefit arithmetic. To address this issue, we used the abacus-based mental calculation (AMC), which combines finger training and mental arithmetic learning, to explore the neural correlates underlying finger-related arithmetic training. A total of 147 Chinese children (75 M/72 F, mean age, 6.89 ± 0.46) were recruited and randomly assigned into AMC and control groups at primary school entry. The AMC group received 5 years of AMC training, and arithmetic abilities and resting-state functional magnetic resonance images data were collected from both groups at year 1/3/5. The connectome-based predictive modeling was used to find the arithmetic-related networks of each group. Compared to controls, the AMC's positively arithmetic-related network was less located in the control module, and the inter-module connections between somatomotor-default and somatomotor-control modules shifted to somatomotor-visual and somatomotor-dorsal attention modules. Furthermore, the positive network of the AMC group exhibited a segregated connectivity pattern, with more intra-module connections than the control group. Overall, our results suggested that finger motor representation with motor module involvement facilitated arithmetic-related network segregation, reflecting increased autonomy of AMC, thus reducing the dependency of arithmetic on higher-order cognitive functions.

GABAergic inhibition shapes behavior and neural dynamics in human visual working memory

Neuronal inhibition, primarily mediated by GABAergic neurotransmission, is crucial for brain development and healthy cognition. Gamma-aminobutyric acid concentration levels in sensory areas have been shown to correlate with hemodynamic and oscillatory neuronal responses. How these measures relate to one another during working memory, a higher-order cognitive process, is still poorly understood. We address this gap by collecting magnetoencephalography, functional magnetic resonance imaging, and Flumazenil positron emission tomography data within the same subject cohort using an n-back working-memory paradigm. By probing the relationship between GABAA receptor distribution, neural oscillations, and Blood Oxygen Level Dependent (BOLD) modulations, we found that GABAA receptor density in higher-order cortical areas predicted the reaction times on the working-memory task and correlated positively with the peak frequency of gamma power modulations and negatively with BOLD amplitude. These findings support and extend theories linking gamma oscillations and hemodynamic responses to gamma-aminobutyric acid neurotransmission and to the excitation-inhibition balance and cognitive performance in humans. Considering the small sample size of the study, future studies should test whether these findings also hold for other, larger cohorts as well as to examine in detail how the GABAergic system and neural fluctuations jointly support working-memory task performance.

Working Memory Maintenance of Visual and Auditory Spatial Information Relies on Supramodal Neural Codes in the Dorsal Frontoparietal Cortex

The frontoparietal attention network plays a pivotal role during working memory (WM) maintenance, especially under high-load conditions. Nevertheless, there is ongoing debate regarding whether this network relies on supramodal or modality-specific neural signatures. In this study, we used multi-voxel pattern analysis (MVPA) to evaluate the neural representation of visual versus auditory information during WM maintenance. During fMRI scanning, participants maintained small or large spatial configurations (low- or high-load trials) of either colour shades or sound pitches in WM for later retrieval. Participants were less accurate in retrieving high- vs. low-load trials, demonstrating an effective manipulation of WM load, irrespective of the sensory modality. The frontoparietal regions involved in maintaining high- vs. low-load spatial maps in either sensory modality were highlighted using a conjunction analysis. Widespread activity was found across the dorsal frontoparietal network, peaking on the frontal eye fields and the superior parietal lobule, bilaterally. Within these regions, MVPAs were performed to quantify the pattern of distinctness of visual vs. auditory neural codes during WM maintenance. These analyses failed to reveal distinguishable patterns in the dorsal frontoparietal regions, thus providing support for a common, supramodal neural code associated with the retention of either visual or auditory spatial configurations.

Prefrontal Cortex Codes Representations of Target Identity and Feature Uncertainty

Many objects in the real world have features that vary over time, creating uncertainty in how they will look in the future. This uncertainty makes statistical knowledge about the likelihood of features critical to attention demanding processes such as visual search. However, little is known about how the uncertainty of visual features is integrated into predictions about search targets in the brain. In the current study, we test the idea that regions prefrontal cortex code statistical knowledge about search targets before the onset of search. Across 20 human participants (13 female; 7 male), we observe target identity in the multivariate pattern and uncertainty in the overall activation of dorsolateral prefrontal cortex (DLPFC) and inferior frontal junction (IFJ) in advance of the search display. This indicates that the target identity (mean) and uncertainty (variance) of the target distribution are coded independently within the same regions. Furthermore, once the search display appears the univariate IFJ signal scaled with the distance of the actual target from the expected mean, but more so when expected variability was low. These results inform neural theories of attention by showing how the prefrontal cortex represents both the identity and expected variability of features in service of top-down attentional control.SIGNIFICANCE STATEMENT Theories of attention and working memory posit that when we engage in complex cognitive tasks our performance is determined by how precisely we remember task-relevant information. However, in the real world the properties of objects change over time, creating uncertainty about many aspects of the task. There is currently a gap in our understanding of how neural systems represent this uncertainty and combine it with target identity information in anticipation of attention demanding cognitive tasks. In this study, we show that the prefrontal cortex represents identity and uncertainty as unique codes before task onset. These results advance theories of attention by showing that the prefrontal cortex codes both target identity and uncertainty to implement top-down attentional control.

Qualitatively Different Delay-Dependent Working Memory Distortions in People With Schizophrenia and Healthy Control Participants

BACKGROUND

Impairments in working memory (WM) have been well documented in people with schizophrenia (PSZ). However, these quantitative WM impairments can often be explained by nonspecific factors, such as impaired goal maintenance. Here, we used a spatial orientation delayed response task to explore a qualitative difference in WM dynamics between PSZ and healthy control participants (HCs). More specifically, we took advantage of the discovery that WM representations may drift either toward or away from previous trial targets (serial dependence). We tested the hypothesis that WM representations would drift toward the previous trial target in HCs but away from the previous trial target in PSZ.

METHODS

We assessed serial dependence in PSZ (n = 31) and HCs (n = 25) using orientation as the to-be-remembered feature and memory delays lasting from 0 to 8 seconds. Participants were asked to remember the orientation of a teardrop-shaped object and reproduce the orientation after a delay period of varying length.

RESULTS

Consistent with prior studies, we found that current trial memory representations were less precise in PSZ than in HCs. We also found that WM for the current trial orientation drifted toward the previous trial orientation in HCs (representational attraction) but drifted away from the previous trial orientation in PSZ (representational repulsion).

CONCLUSIONS

These results demonstrate a qualitative difference in WM dynamics between PSZ and HCs that cannot be easily explained by nuisance factors such as reduced effort. Most computational neuroscience models also fail to explain these results because they maintain information solely by means of sustained neural firing, which does not extend across trials. The results suggest a fundamental difference between PSZ and HCs in longer-term memory mechanisms that persist across trials, such as short-term potentiation and neuronal adaptation.

Spatiotemporal dynamics of self-generated imagery reveal a reverse cortical hierarchy from cue-induced imagery

Visual imagery allows for the construction of rich internal experience in our mental world. However, it has remained poorly understood how imagery experience derives volitionally as opposed to being cue driven. Here, using electroencephalography and functional magnetic resonance imaging, we systematically investigate the spatiotemporal dynamics of self-generated imagery by having participants volitionally imagining one of the orientations from a learned pool. We contrast self-generated imagery with cue-induced imagery, where participants imagined line orientations based on associative cues acquired previously. Our results reveal overlapping neural signatures of cue-induced and self-generated imagery. Yet, these neural signatures display substantially differential sensitivities to the two types of imagery: self-generated imagery is supported by an enhanced involvement of the anterior cortex in representing imagery contents. By contrast, cue-induced imagery is supported by enhanced imagery representations in the posterior visual cortex. These results jointly support a reverse cortical hierarchy in generating and maintaining imagery contents in self-generated versus externally cued imagery.

Computerized Cognitive Training Enhances Episodic Memory by Down-Modulating Posterior Cingulate-Precuneus Connectivity in Older Persons With Mild Cognitive Impairment: A Randomized Controlled Trial

OBJECTIVE

The neural mechanisms underlying the beneficial effects of a computerized cognitive training (CCT) program for improving episodic memory in older persons with mild cognitive impairment (MCI) remain unclear. This study aimed to use both functional and structural brain changes to elucidate the treatment effects of CCT on enhancing episodic memory.

DESIGN, SETTING, AND PARTICIPANTS

Single-blinded, multicenter randomized controlled trial on 60 older adults with MCI in Fuzhou, China.

INTERVENTION

Participants were randomly assigned to either an 8-week 24-hour CCT program or a health education program as the control.

MEASUREMENTS

Clinical outcomes included changes in scores on the immediate and/or delayed recall subtests of the Chinese auditory verbal learning test (CAVLT) and rey complex figure test (CFT), and changes in gray matter volume and the functional connectivity of the posterior cingulate cortex (PCC) and hippocampus in the Papez circuit on magnetic resonance imaging.

RESULTS

Significant group-by-time effects showed greater improvements in both immediate and delayed recall scores of CAVLT and delayed recall scores of Rey CFT in participants receiving the CCT program compared to those in the health education program. Among the CCT participants, seed-based analyses revealed decreases in functional connectivity of the PCC and hippocampus with neural substrates in the parietal and occipital regions. The decreased PCC and precuneus connectivity were found to mediate patients' improvements in immediate recall function.

CONCLUSION

An 8-week CCT program was effective for improving episodic memory in older individuals with MCI. The decrease in connectivity originating from the PCC and hippocampus is suggestive of potential plastic changes in the Papez circuit, which could have alleviated the age-related compensatory mechanism. The findings of this study also shed light on expanding the content and extending the frequency and duration of the CCT program in future studies.

Neural Representations in Visual and Parietal Cortex Differentiate between Imagined, Perceived, and Illusory Experiences

Humans constantly receive massive amounts of information, both perceived from the external environment and imagined from the internal world. To function properly, the brain needs to correctly identify the origin of information being processed. Recent work has suggested common neural substrates for perception and imagery. However, it has remained unclear how the brain differentiates between external and internal experiences with shared neural codes. Here we tested this question in human participants (male and female) by systematically investigating the neural processes underlying the generation and maintenance of visual information from voluntary imagery, veridical perception, and illusion. The inclusion of illusion allowed us to differentiate between objective and subjective internality: while illusion has an objectively internal origin and can be viewed as involuntary imagery, it is also subjectively perceived as having an external origin like perception. Combining fMRI, eye-tracking, multivariate decoding, and encoding approaches, we observed superior orientation representations in parietal cortex during imagery compared with perception, and conversely in early visual cortex. This imagery dominance gradually developed along a posterior-to-anterior cortical hierarchy from early visual to parietal cortex, emerged in the early epoch of imagery and sustained into the delay epoch, and persisted across varied imagined contents. Moreover, representational strength of illusion was more comparable to imagery in early visual cortex, but more comparable to perception in parietal cortex, suggesting content-specific representations in parietal cortex differentiate between subjectively internal and external experiences, as opposed to early visual cortex. These findings together support a domain-general engagement of parietal cortex in internally generated experience.SIGNIFICANCE STATEMENT How does the brain differentiate between imagined and perceived experiences? Combining fMRI, eye-tracking, multivariate decoding, and encoding approaches, the current study revealed enhanced stimulus-specific representations in visual imagery originating from parietal cortex, supporting the subjective experience of imagery. This neural principle was further validated by evidence from visual illusion, wherein illusion resembled perception and imagery at different levels of cortical hierarchy. Our findings provide direct evidence for the critical role of parietal cortex as a domain-general region for content-specific imagery, and offer new insights into the neural mechanisms underlying the differentiation between subjectively internal and external experiences.

Ultra-High Field Imaging of Human Visual Cognition

Functional magnetic resonance imaging (fMRI), the key methodology for mapping the functions of the human brain in a noninvasive manner, is limited by low temporal and spatial resolution. Recent advances in ultra-high field (UHF) fMRI provide a mesoscopic (i.e., submillimeter resolution) tool that allows us to probe laminar and columnar circuits, distinguish bottom-up versus top-down pathways, and map small subcortical areas. We review recent work demonstrating that UHF fMRI provides a robust methodology for imaging the brain across cortical depths and columns that provides insights into the brain's organization and functions at unprecedented spatial resolution, advancing our understanding of the fine-scale computations and interareal communication that support visual cognition.

Neural population dynamics of human working memory

The activity of neurons in macaque prefrontal cortex (PFC) persists during working memory (WM) delays, providing a mechanism for memory.1,2,3,4,5,6,7,8,9,10,11 Although theory,11,12 including formal network models,13,14 assumes that WM codes are stable over time, PFC neurons exhibit dynamics inconsistent with these assumptions.15,16,17,18,19 Recently, multivariate reanalyses revealed the coexistence of both stable and dynamic WM codes in macaque PFC.20,21,22,23 Human EEG studies also suggest that WM might contain dynamics.24,25 Nonetheless, how WM dynamics vary across the cortical hierarchy and which factors drive dynamics remain unknown. To elucidate WM dynamics in humans, we decoded WM content from fMRI responses across multiple cortical visual field maps.26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48 We found coexisting stable and dynamic neural representations of WM during a memory-guided saccade task. Geometric analyses of neural subspaces revealed that early visual cortex exhibited stronger dynamics than high-level visual and frontoparietal cortex. Leveraging models of population receptive fields, we visualized and made the neural dynamics interpretable. We found that during WM delays, V1 population initially encoded a narrowly tuned bump of activation centered on the peripheral memory target. Remarkably, this bump then spread inward toward foveal locations, forming a vector along the trajectory of the forthcoming memory-guided saccade. In other words, the neural code transformed into an abstraction of the stimulus more proximal to memory-guided behavior. Therefore, theories of WM must consider both sensory features and their task-relevant abstractions because changes in the format of memoranda naturally drive neural dynamics.

Neural evidence for advantaged representation of first items in memory

Visual areas activated during perception can retain specific information held in memory without the presence of physical stimuli via distributed activity patterns. Neuroimaging studies have shown that the delay-period representation of information in visual areas is modulated by factors such as memory load and task demands, raising the possibility of serial position as another potential modulator. Specifically, enhanced representation of first items during the post-encoding delay period may serve as a mechanism underlying the well-established but not well-understood primacy effect - the mnemonic advantage of first items. To test this hypothesis, 13 males and 16 females performed a human fMRI task, wherein each trial consisted of the sequential encoding of two stimuli (a famous face and landscape, order counterbalanced), followed by a distracting task, a delay period, and then a cued recall of one of the items. Participants exhibited the expected behavioral primacy effect, manifested as faster recall of the first items. In order to elucidate the still debated neural underpinnings of this effect, using multivariate decoding, a classifier was trained on data collected during encoding to differentiate stimulus categories (i.e., faces vs. landscapes) and tested on data collected during the post-encoding period. Greater reactivation of first versus second items was observed in the ventral occipito-temporal cortex during the entire post-encoding period but not during encoding. Moreover, trial-level analyses revealed that the degree of first-item neural advantage during the post-encoding delay predicted the behavioral primacy effect. These findings highlight the role of item reinstatement in ventral occipito-temporal cortex in the primacy effect and are discussed in the context of the uniqueness of the very first item and event boundaries, illuminating putative neural mechanisms underlying the effect.

Mapping multidimensional content representations to neural and behavioral expressions of episodic memory

Human neuroimaging studies have shown that the contents of episodic memories are represented in distributed patterns of neural activity. However, these studies have mostly been limited to decoding simple, unidimensional properties of stimuli. Semantic encoding models, in contrast, offer a means for characterizing the rich, multidimensional information that comprises episodic memories. Here, we extensively sampled four human fMRI subjects to build semantic encoding models and then applied these models to reconstruct content from natural scene images as they were viewed and recalled from memory. First, we found that multidimensional semantic information was successfully reconstructed from activity patterns across visual and lateral parietal cortices, both when viewing scenes and when recalling them from memory. Second, whereas visual cortical reconstructions were much more accurate when images were viewed versus recalled from memory, lateral parietal reconstructions were comparably accurate across visual perception and memory. Third, by applying natural language processing methods to verbal recall data, we showed that fMRI-based reconstructions reliably matched subjects' verbal descriptions of their memories. In fact, reconstructions from ventral temporal cortex more closely matched subjects' own verbal recall than other subjects' verbal recall of the same images. Fourth, encoding models reliably transferred across subjects: memories were successfully reconstructed using encoding models trained on data from entirely independent subjects. Together, these findings provide evidence for successful reconstructions of multidimensional and idiosyncratic memory representations and highlight the differential sensitivity of visual cortical and lateral parietal regions to information derived from the external visual environment versus internally-generated memories.

Action space restructures visual working memory in prefrontal cortex

Visual working memory enables flexible behavior by decoupling sensory stimuli from behavioral actions. While previous studies have predominantly focused on the storage component of working memory, the role of future actions in shaping working memory remains unknown. To answer this question, we used two working memory tasks that allowed the dissociation of sensory and action components of working memory. We measured behavioral performance and neuronal activity in the macaque prefrontal cortex area, frontal eye fields. We show that the action space reshapes working memory, as evidenced by distinct patterns of memory tuning and attentional orienting between the two tasks. Notably, neuronal activity during the working memory period predicted future behavior and exhibited mixed selectivity in relation to the sensory space but linear selectivity relative to the action space. This linear selectivity was achieved through the rapid transformation from sensory to action space and was subsequently maintained as a stable cross-temporal population activity pattern. Combined, we provide direct physiological evidence of the action-oriented nature of frontal eye field neurons during memory tasks and demonstrate that the anticipation of behavioral outcomes plays a significant role in transforming and maintaining the contents of visual working memory.

Categorical working memory codes in human visual cortex

Working memory contents are represented in neural activity patterns across multiple regions of the cortical hierarchy. A division of labor has been proposed where more anterior regions harbor increasingly abstract and categorical representations while the most detailed representations are held in primary sensory cortices. Here, using fMRI and multivariate encoding modeling, we demonstrate that for color stimuli categorical codes are already present at the level of extrastriate visual cortex (V4 and VO1), even when subjects are neither implicitly nor explicitly encouraged to categorize the stimuli. Importantly, this categorical coding was observed during working memory, but not during perception. Thus, visual working memory is likely to rely at least in part on categorical representations. SIGNIFICANCE STATEMENT: Working memory is the representational basis for human cognition. Recent work has demonstrated that numerous regions across the human brain can represent the contents of working memory. We use fMRI brain scanning and machine learning methods to demonstrate that different regions can represent the same content differently during working memory. Reading out the neural codes used to store working memory contents, we show that already in sensory cortex, areas V4 and VO1 represent color in a categorical format rather than a purely sensory fashion. Thereby, we provide a better understanding of how different regions of the brain might serve working memory and cognition.

Strategic control of location and ordinal context in visual working memory

Working memory (WM) requires encoding stimulus identity and context (e.g. where or when stimuli were encountered). To explore the neural bases of the strategic control of context binding in WM, we acquired fMRI while subjects performed delayed recognition of 3 orientation patches presented serially and at different locations. The recognition probe was an orientation patch with a superimposed digit, and pretrial instructions directed subjects to respond according to its location ("location-relevant"), to the ordinal position corresponding to its digit ("order-relevant"), or to just its orientation (relative to all three samples; "context-irrelevant"). Delay period signal in PPC was greater for context-relevant than for "context-irrelevant" trials, and multivariate decoding revealed strong sensitivity to context binding requirements (relevant vs. "irrelevant") and to context domain ("location-" vs. "order-relevant") in both occipital cortex and PPC. At recognition, multivariate inverted encoding modeling revealed markedly different patterns in these 2 regions, suggesting different context-processing functions. In occipital cortex, an active representation of the location of each of the 3 samples was reinstated regardless of the trial type. The pattern in PPC, by contrast, suggested a trial type-dependent filtering of sample information. These results indicate that PPC exerts strategic control over the representation of stimulus context in visual WM.

Representational Geometries Reveal Differential Effects of Response Correlations on Population Codes in Neurophysiology and Functional Magnetic Resonance Imaging

Two sensory neurons usually display trial-by-trial spike-count correlations given the repeated representations of a stimulus. The effects of such response correlations on population-level sensory coding have been the focal contention in computational neuroscience over the past few years. In the meantime, multivariate pattern analysis (MVPA) has become the leading analysis approach in functional magnetic resonance imaging (fMRI), but the effects of response correlations among voxel populations remain underexplored. Here, instead of conventional MVPA analysis, we calculate linear Fisher information of population responses in human visual cortex (five males, one female) and hypothetically remove response correlations between voxels. We found that voxelwise response correlations generally enhance stimulus information, a result standing in stark contrast to the detrimental effects of response correlations reported in empirical neurophysiological studies. By voxel-encoding modeling, we further show that these two seemingly opposite effects actually can coexist within the primate visual system. Furthermore, we use principal component analysis to decompose stimulus information in population responses onto different principal dimensions in a high-dimensional representational space. Interestingly, response correlations simultaneously reduce and enhance information on higher- and lower-variance principal dimensions, respectively. The relative strength of the two antagonistic effects within the same computational framework produces the apparent discrepancy in the effects of response correlations in neuronal and voxel populations. Our results suggest that multivariate fMRI data contain rich statistical structures that are directly related to sensory information representation, and the general computational framework to analyze neuronal and voxel population responses can be applied in many types of neural measurements.SIGNIFICANCE STATEMENT Despite the vast research interest in the effect of spike-count noise correlations on population codes in neurophysiology, it remains unclear how the response correlations between voxels influence MVPA in human imaging. We used an information-theoretic approach and showed that unlike the detrimental effects of response correlations reported in neurophysiology, voxelwise response correlations generally improve sensory coding. We conducted a series of in-depth analyses and demonstrated that neuronal and voxel response correlations can coexist within the visual system and share some common computational mechanisms. These results shed new light on how the population codes of sensory information can be evaluated via different neural measurements.

Mnemonic representations in human lateral geniculate nucleus

There is a growing appreciation for the role of the thalamus in high-level cognition. Motivated by findings that internal cognitive state drives activity in feedback layers of primary visual cortex (V1) that target the lateral geniculate nucleus (LGN), we investigated the role of LGN in working memory (WM). Specifically, we leveraged model-based neuroimaging approaches to test the hypothesis that human LGN encodes information about spatial locations temporarily encoded in WM. First, we localized and derived a detailed topographic organization in LGN that accords well with previous findings in humans and non-human primates. Next, we used models constructed on the spatial preferences of LGN populations in order to reconstruct spatial locations stored in WM as subjects performed modified memory-guided saccade tasks. We found that population LGN activity faithfully encoded the spatial locations held in memory in all subjects. Importantly, our tasks and models allowed us to dissociate the locations of retinal stimulation and the motor metrics of memory-guided saccades from the maintained spatial locations, thus confirming that human LGN represents true WM information. These findings add LGN to the growing list of subcortical regions involved in WM, and suggest a key pathway by which memories may influence incoming processing at the earliest levels of the visual hierarchy.

Decoding of Working Memory Contents in Auditory Cortex Is Not Distractor-Resistant

Working memory enables the temporary storage of relevant information in the service of behavior. Neuroimaging studies have suggested that sensory cortex is involved in maintaining contents in working memory. This raised the question of how sensory regions maintain memory representations during the exposure to distracting stimuli. Multivariate pattern analysis of fMRI signals in visual cortex has shown that the contents of visual working memory could be decoded concurrently with passively viewed distractors. The present fMRI study tested whether this finding extends to auditory working memory and to active distractor processing. We asked participants to memorize the pitch of a target sound and to compare it with a probe sound presented after a 13 s delay period. In separate conditions, we compared a blank delay phase (no distraction) with either passive listening to, or active processing of, an auditory distractor presented throughout the memory delay. Consistent with previous reports, pitch-specific memory information could be decoded in auditory cortex during the delay in trials without distraction. In contrast, decoding of target sounds in early auditory cortex dropped to chance level during both passive and active distraction. This was paralleled by memory performance decrements under distraction. Extending the analyses beyond sensory cortex yielded some evidence for memory content-specific activity in inferior frontal and superior parietal cortex during active distraction. In summary, while our findings question the involvement of early auditory cortex in the maintenance of distractor-resistant working memory contents, further research should elucidate the role of hierarchically higher regions.SIGNIFICANCE STATEMENT Information about sensory features held in working memory can be read out from hemodynamic activity recorded in human sensory cortices. Moreover, visual cortex can in parallel store visual content and process newly incoming, task-irrelevant visual input. The present study investigated the role of auditory cortex for working memory maintenance under distraction. While memorized sound frequencies could be decoded in auditory cortex in the absence of distraction, auditory distraction during the delay phase impaired memory performance and prevented decoding of information stored in working memory. Apparently, early auditory cortex is not sufficient to represent working memory contents under distraction that impairs performance. However, exploratory analyses indicated that, under distraction, higher-order frontal and parietal regions might contribute to content-specific working memory storage.

Distinct cognitive changes in male patients with obstructive sleep apnoea without co-morbidities

Introduction

Obstructive sleep apnoea (OSA) is a multisystem, debilitating, chronic disorder of breathing during sleep, resulting in a relatively consistent pattern of cognitive deficits. More recently, it has been argued that those cognitive deficits, especially in middle-aged patients, may be driven by cardiovascular and metabolic comorbidities, rather than by distinct OSA-processes, such as are for example ensuing nocturnal intermittent hypoxaemia, oxidative stress, neuroinflammation, and sleep fragmentation.

Methods

Thus, we undertook to define cognitive performance in a group of 27 middle-aged male patients with untreated OSA, who had no concomitant comorbidities, compared with seven matched controls (AHI mean ± S.D.: 1.9 ± 1.4 events/h; mean age 34.0 ± 9.3 years; mean BMI 23.8 ± 2.3 kg/m2). Of the 27 patients, 16 had mild OSA (AHI mean ± S.D.:11.7 ± 4.0 events/h; mean age 42.6 ± 8.2 years; mean BMI 26.7 ± 4.1 kg/m2), and 11 severe OSA (AHI 41.8 ± 20.7 events/h; age: 46.9 ± 10.9 years, BMI: 28.0 ± 3.2 kg/m2).

Results

In our patient cohort, we demonstrate poorer executive-functioning, visuospatial memory, and deficits in vigilance sustained attention, psychomotor and impulse control. Remarkably, we also report, for the first time, effects on social cognition in this group of male, middle-aged OSA patients.

Conclusion

Our findings suggest that distinct, OSA-driven processes may be sufficient for cognitive changes to occur as early as in middle age, in otherwise healthy individuals.

Qualitatively different delay-dependent working memory distortions in people with schizophrenia and healthy control subjects

Background

Impairments in working memory(WM) have been well-documented in people with schizophrenia(PSZ). However, these quantitative WM impairments can often be explained by nonspecific factors, such as impaired goal maintenance. Here, we used a spatial orientation delayed-response task to explore a qualitative difference in WM dynamics between PSZ and healthy control subjects(HCS). Specifically, we took advantage of the discovery that WM representations may drift either toward or away from previous-trial targets(serial dependence). We tested the hypothesis that WM representations drift toward the previous-trial target in HCS but away from the previous-trial target in PSZ.

Methods

We assessed serial dependence in PSZ(N=31) and HCS(N=25), using orientation as the to-be-remembered feature and memory delays from 0 to 8s. Participants were asked to remember the orientation of a teardrop-shaped object and reproduce the orientation after a varying delay period.

Results

Consistent with prior studies, we found that current-trial memory representations were less precise in PSZ than in HCS. We also found that WM for the current-trial orientation drifted toward the previous-trial orientation in HCS(representational attraction) but drifted away from the previous-trial orientation in PSZ(representational repulsion).

Conclusions

These results demonstrate a qualitative difference in WM dynamics between PSZ and HCS that cannot easily be explained by nuisance factors such as reduced effort. Most computational neuroscience models also fail to explain these results, because they maintain information solely by means of sustained neural firing, which does not extend across trials. The results suggest a fundamental difference between PSZ and HCS in longer-term memory mechanisms that persist across trials, such as short-term potentiation and neuronal adaptation.

Causal evidence for the role of the sensory visual cortex in visual short-term memory maintenance

The role of the sensory visual cortex during visual short-term memory (VSTM) remains controversial. This controversy is possibly due to methodological issues in previous attempts to investigate the effects of transcranial magnetic stimulation (TMS) on VSTM. The aim of this study was to use TMS, while covering previous methodological deficits. Sixty-four young adults were recruited to participate in two experiments (Experiment 1: n = 36; Experiment 2: n = 28) using a VSTM orientation change-detection task under TMS. Monocular vision was ensured using red-blue goggles combined with red-blue stimuli. Double-pulse TMS was delivered at different times (Experiment 1: 0, 200 or 1000 ms; Experiment 2: 200, 1000 ms) during a 2 s maintenance phase, on one side of the occipital hemisphere. In Experiment 2, a sham TMS condition was introduced. Decreased detection sensitivity (d') in the ipsilateral occipital hemisphere to visual hemifield, and in the real TMS (compared with sham TMS) condition indicated inhibitory TMS effects, and thus, a causal involvement of the sensory visual cortex during early (200 ms) and late (1000 ms) maintenance in VSTM. These findings are aligned with sensory recruitment, which proposes that both perceptual and memory processes rely upon the same neural substrates in the sensory visual cortex. The methods used in this study were preregistered and had received in-principle acceptance on 6 June 2022 (Stage 1 protocol can be found in: https://doi.org/10.17605/OSF.IO/EMPDT).

The medial temporal lobe supports the quality of visual short-term memory representation

The quality of short-term memory (STM) underlies our ability to recall the exact details of a recent event, yet how the human brain enables this core cognitive function remains poorly understood. Here we use multiple experimental approaches to test the hypothesis that the quality of STM, such as its precision or fidelity, relies on the medial temporal lobe (MTL), a region commonly associated with the ability to distinguish similar information remembered in long-term memory. First, with intracranial recordings, we find that delay-period MTL activity retains item-specific STM content that is predictive of subsequent recall precision. Second, STM recall precision is associated with an increase in the strength of intrinsic MTL-to-neocortical functional connections during a brief retention interval. Finally, perturbing the MTL through electrical stimulation or surgical removal can selectively reduce STM precision. Collectively, these findings provide converging evidence that the MTL is critically involved in the quality of STM representation.