The impact of early visual cortex transcranial magnetic stimulation on visual working memory precision and guess rate

The causal involvement of the visual cortex in visual working memory remains uncertain

The role of the early visual cortex in visual working memory (VWM) is a matter of current debate. Neuroimaging studies have consistently shown that visual areas encode the content of working memory, while transcranial magnetic stimulation (TMS) studies have presented incongruent results. Thus, we lack conclusive evidence supporting the causal role of early visual areas in VWM. In a recent registered report, Phylactou et al. (Phylactou P, Shimi A, Konstantinou N 2023 R. Soc. Open Sci. 10, 230321 (doi:10.1098/rsos.230321)) sought to tackle this controversy via two well-powered TMS experiments, designed to correct possible methodological issues of previous attempts identified in a preceding systematic review and meta-analysis (Phylactou P, Traikapi A, Papadatou-Pastou M, Konstantinou N 2022 Psychon. Bull. Rev. 29, 1594-1624 (doi:10.3758/s13423-022-02107-y)). However, a key part of their critique and experimental design was based on a misunderstanding of the visual system. They disregarded two important anatomical facts, namely that early visual areas of each hemisphere represent the contralateral visual hemifield, and that each hemisphere receives equally strong input from each eye-both leading to confounded conditions and artefactual effects in their studies. Here, we explain the correct anatomy, describe why their experiments failed to address current issues in the literature and perform a thorough reanalysis of their TMS data revealing important null results. We conclude that the causal role of the visual cortex in VWM remains uncertain.

Dynamic fading memory and expectancy effects in the monkey primary visual cortex

In order to investigate the involvement of the primary visual cortex (V1) in working memory (WM), parallel, multisite recordings of multi-unit activity were obtained from monkey V1 while the animals performed a delayed match-to-sample (DMS) task. During the delay period, V1 population firing rate vectors maintained a lingering trace of the sample stimulus that could be reactivated by intervening impulse stimuli that enhanced neuronal firing. This fading trace of the sample did not require active engagement of the monkeys in the DMS task and likely reflects the intrinsic dynamics of recurrent cortical networks in lower visual areas. This renders an active, attention-dependent involvement of V1 in the maintenance of WM contents unlikely. By contrast, population responses to the test stimulus depended on the probabilistic contingencies between sample and test stimuli. Responses to tests that matched expectations were reduced which agrees with concepts of predictive coding.

Assessing the interaction between working memory and perception through time

Content maintained in visual working memory changes concurrent visual processing, suggesting that visual working memory may recruit an overlapping neural representation with visual perception. However, it remains unclear whether visual working memory representations persist as a sensory code through time, or are recoded later into an abstract code. Here, we directly contrasted a temporal decay + visual code account and a temporal decay + abstract code account within the temporal dynamics of the interaction between working memory and perception. By manipulating the ISI (inter-stimulus interval) between working memory encoding and a perceptual discrimination task, we found that task-relevant and therefore actively maintained perceptual information parametrically altered participants' ability to discriminate perceptual stimuli even 4 s after encoding, whereas task-irrelevant information caused only an acutely transient effect. While continuously present, the size of this shift in discrimination thresholds gradually decreased over time. Concomitantly, the size of the bias in working memory reports increased over time. The opposing directions of threshold and bias effects are consistent with the local maintenance of information in perceptual areas, explained by a temporal decay + visual code account. As the maintained representation decays over time, its ability to alter incoming perceptual signals decreases (reduced threshold effects) while its likelihood of being impacted by those same signals increases (increased bias effects). Altogether, these results suggest that the readout of working memory relies on a sensory representation at a cost of increased interference by ongoing perception.

Are irrelevant items actively deleted from visual working memory?: No evidence from repulsion and attraction effects in dual-retrocue tasks

Some theories propose that working memory (WM) involves the active deletion of irrelevant information, including items that were retained in WM, but are no longer relevant for ongoing cognition. Considerable evidence suggests that active-deletion occurs for categorical representations, but whether it also occurs for recall of features that are typically bound together in an object, such as line orientations, is unclear. In two experiments, with or without binding instructions, healthy young adults maintained two orientations, focused attention to recall the orientation cued first, and then switched attention to recall the orientation cued second, at which point the uncued orientation was no longer relevant on the trial. In contrast to the active-deletion hypothesis, the results showed that the no-longer-relevant items exerted the strongest bias on participants' recall, which was either repulsive or attractive depending on both the degree of difference between the target and nontarget orientations and the proximity to cardinal axes. We suggest that visual WM can bind features like line orientations into chunked representations, and an irrelevant feature of a chunked object cannot be actively deleted - it biases recall of the target feature. Models of WM need to be updated to explain this and related dynamic phenomena.

Causal evidence for the role of the sensory visual cortex in visual short-term memory maintenance

The role of the sensory visual cortex during visual short-term memory (VSTM) remains controversial. This controversy is possibly due to methodological issues in previous attempts to investigate the effects of transcranial magnetic stimulation (TMS) on VSTM. The aim of this study was to use TMS, while covering previous methodological deficits. Sixty-four young adults were recruited to participate in two experiments (Experiment 1: n = 36; Experiment 2: n = 28) using a VSTM orientation change-detection task under TMS. Monocular vision was ensured using red-blue goggles combined with red-blue stimuli. Double-pulse TMS was delivered at different times (Experiment 1: 0, 200 or 1000 ms; Experiment 2: 200, 1000 ms) during a 2 s maintenance phase, on one side of the occipital hemisphere. In Experiment 2, a sham TMS condition was introduced. Decreased detection sensitivity (d') in the ipsilateral occipital hemisphere to visual hemifield, and in the real TMS (compared with sham TMS) condition indicated inhibitory TMS effects, and thus, a causal involvement of the sensory visual cortex during early (200 ms) and late (1000 ms) maintenance in VSTM. These findings are aligned with sensory recruitment, which proposes that both perceptual and memory processes rely upon the same neural substrates in the sensory visual cortex. The methods used in this study were preregistered and had received in-principle acceptance on 6 June 2022 (Stage 1 protocol can be found in: https://doi.org/10.17605/OSF.IO/EMPDT).

Recognition and Processing of Visual Information after Neuronavigated Transcranial Magnetic Stimulation Session

Background: Transcranial magnetic stimulation (TMS) is a method of noninvasive and painless stimulation of the nervous system, which is based on Faraday’s law of electromagnetic induction. Over the past twenty years, the TMS technique has been deployed as a tool for the diagnosis and therapy of neurodegenerative diseases, as well as in the treatment of mental disorders (e.g., depression). Methods: We tested the inhibitory effects of repetitive TMS (rTMS) on reaction times to militarily relevant visual stimuli amidst distractors and on accompanying blood oxygenation level dependent (BOLD) signal functional magnetic resonance imaging (fMRI) in 20 healthy people. rTMS was applied over the visual cortices, V1, on both hemispheres with the inhibitory theta burst paradigm with the intensity of 70% of the active motor threshold fMRI in 20 healthy people. Results: Analysis of the reaction time to visual stimuli after using TMS to the V1 visual cortex revealed an increase in the number of incorrect recognitions, and the reaction time was from 843 to 910 ms. In the subgroup of participants (n = 15), after the stimulation, there were significant reductions of BOLD signal in blood flow within V1 cortices. Conclusions: The studies of reaction times after the rTMS revealed the inhibitory effect of rTMS on the reaction times and recognition performance of significant (military) objects in the visual field.

Phonological Working Memory Representations in the Left Inferior Parietal Lobe in the Face of Distraction and Neural Stimulation

The neural basis of phonological working memory (WM) was investigated through an examination of the effects of irrelevant speech distractors and disruptive neural stimulation from transcranial magnetic stimulation (TMS). Embedded processes models argue that the same regions involved in speech perception are used to support phonological WM whereas buffer models assume that a region separate from speech perception regions is used to support WM. Thus, according to the embedded processes approach but not the buffer approach, irrelevant speech and TMS to the speech perception region should disrupt the decoding of phonological WM representations. According to the buffer account, decoding of WM items should be possible in the buffer region despite distraction and should be disrupted with TMS to this region. Experiment 1 used fMRI and representational similarity analyses (RSA) with a delayed recognition memory paradigm using nonword stimuli. Results showed that decoding of memory items in the speech perception regions (superior temporal gyrus, STG) was possible in the absence of distractors. However, the decoding evidence in the left STG was susceptible to interference from distractors presented during the delay period whereas decoding in the proposed buffer region (supramarginal gyrus, SMG) persisted. Experiment 2 examined the causal roles of the speech processing region and the buffer region in phonological WM performance using TMS. TMS to the SMG during the early delay period caused a disruption in recognition performance for the memory nonwords, whereas stimulations at the STG and an occipital control region did not affect WM performance. Taken together, results from the two experiments are consistent with predictions of a buffer model of phonological WM, pointing to a critical role of the left SMG in maintaining phonological representations.

Sensory recruitment in visual short-term memory: A systematic review and meta-analysis of sensory visual cortex interference using transcranial magnetic stimulation

Sensory visual areas are involved in encoding information in visual short-term memory (VSTM). Yet it remains unclear whether sensory visual cortex is a necessary component of the brain network for maintenance of information in VSTM. Here, we aimed to systematically review studies that have investigated the role of the sensory visual cortex in VSTM using transcranial magnetic stimulation (TMS) and to quantitatively explore these effects using meta-analyses. Fourteen studies were identified and reviewed. Eight studies provided sufficient data for meta-analysis. Two meta-analyses, one regarding the VSTM encoding phase (17 effect sizes) and one regarding the VSTM maintenance phase (15 effect sizes), two meta-regressions (32 effect sizes in each), and one exploratory meta-analysis were conducted. Our results indicate that the sensory visual cortex is similarly involved in both the encoding and maintenance VSTM phase. We suggest that some cases where evidence did not show significant TMS effects was due to low memory or perceptual task demands. Overall, these findings support the idea that sensory visual areas are part of the brain network responsible for successfully maintaining information in VSTM.

State-dependent effects of neural stimulation on brain function and cognition

Invasive and non-invasive brain stimulation methods are widely used in neuroscience to establish causal relationships between distinct brain regions and the sensory, cognitive and motor functions they subserve. When combined with concurrent brain imaging, such stimulation methods can reveal patterns of neuronal activity responsible for regulating simple and complex behaviours at the level of local circuits and across widespread networks. Understanding how fluctuations in physiological states and task demands might influence the effects of brain stimulation on neural activity and behaviour is at the heart of how we use these tools to understand cognition. Here we review the concept of such 'state-dependent' changes in brain activity in response to neural stimulation, and consider examples from research on altered states of consciousness (for example, sleep and anaesthesia) and from task-based manipulations of selective attention and working memory. We relate relevant findings from non-invasive methods used in humans to those obtained from direct electrical and optogenetic stimulation of neuronal ensembles in animal models. Given the widespread use of brain stimulation as a research tool in the laboratory and as a means of augmenting or restoring brain function, consideration of the influence of changing physiological and cognitive states is crucial for increasing the reliability of these interventions.

Spontaneous Fluctuations in Oscillatory Brain State Cause Differences in Transcranial Magnetic Stimulation Effects Within and Between Individuals

Transcranial magnetic stimulation (TMS) can cause measurable effects on neural activity and behavioral performance in healthy volunteers. In addition, TMS is increasingly used in clinical practice for treating various neuropsychiatric disorders. Unfortunately, TMS-induced effects show large intra- and inter-subject variability, hindering its reliability, and efficacy. One possible source of this variability may be the spontaneous fluctuations of neuronal oscillations. We present recent studies using multimodal TMS including TMS-EMG (electromyography), TMS-tACS (transcranial alternating current stimulation), and concurrent TMS-EEG-fMRI (electroencephalography, functional magnetic resonance imaging), to evaluate how individual oscillatory brain state affects TMS signal propagation within targeted networks. We demonstrate how the spontaneous oscillatory state at the time of TMS influences both immediate and longer-lasting TMS effects. These findings indicate that at least part of the variability in TMS efficacy may be attributable to the current practice of ignoring (spontaneous) oscillatory fluctuations during TMS. Ignoring this state-dependent spread of activity may cause great individual variability which so far is poorly understood and has proven impossible to control. We therefore also compare two technical solutions to directly account for oscillatory state during TMS, namely, to use (a) tACS to externally control these oscillatory states and then apply TMS at the optimal (controlled) brain state, or (b) oscillatory state-triggered TMS (closed-loop TMS). The described multimodal TMS approaches are paramount for establishing more robust TMS effects, and to allow enhanced control over the individual outcome of TMS interventions aimed at modulating information flow in the brain to achieve desirable changes in cognition, mood, and behavior.

Visual Short-Term Memory for Coherent and Sequential Motion: A rTMS Investigation

We investigated the role of the human medio-temporal complex (hMT+) in the memory encoding and storage of a sequence of four coherently moving random dot kinematograms (RDKs), by applying repetitive transcranial magnetic stimulation (rTMS) during an early or late phase of the retention interval. Moreover, in a second experiment, we also tested whether disrupting the functional integrity of hMT+ during the early phase impaired the precision of the encoded motion directions. Overall, results showed that both recognition accuracy and precision were worse in middle serial positions, suggesting the occurrence of primacy and recency effects. We found that rTMS delivered during the early (but not the late) phase of the retention interval was able to impair not only recognition of RDKs, but also the precision of the retained motion direction. However, such impairment occurred only for RDKs presented in middle positions along the presented sequence, where performance was already closer to chance level. Altogether these findings suggest an involvement of hMT+ in the memory encoding of visual motion direction. Given that both position sequence and rTMS modulated not only recognition but also the precision of the stored information, these findings are in support of a model of visual short-term memory with a variable resolution of each stored item, consistent with the assigned amount of memory resources, and that such item-specific memory resolution is supported by the functional integrity of area hMT+.

Persistent Activity During Working Memory From Front to Back

Working memory (WM) extends the duration over which information is available for processing. Given its importance in supporting a wide-array of high level cognitive abilities, uncovering the neural mechanisms that underlie WM has been a primary goal of neuroscience research over the past century. Here, we critically review what we consider the two major "arcs" of inquiry, with a specific focus on findings that were theoretically transformative. For the first arc, we briefly review classic studies that led to the canonical WM theory that cast the prefrontal cortex (PFC) as a central player utilizing persistent activity of neurons as a mechanism for memory storage. We then consider recent challenges to the theory regarding the role of persistent neural activity. The second arc, which evolved over the last decade, stemmed from sophisticated computational neuroimaging approaches enabling researchers to decode the contents of WM from the patterns of neural activity in many parts of the brain including early visual cortex. We summarize key findings from these studies, their implications for WM theory, and finally the challenges these findings pose. Our goal in doing so is to identify barriers to developing a comprehensive theory of WM that will require a unification of these two "arcs" of research.

Free-recall benefit, inhomogeneity and between-item interference in working memory

We investigated visual working memory (VWM) with a whole-report task, where participants were asked to sequentially recall all the items in an order either chosen by themselves (free recall) or randomly chosen by the computer (forced recall). Comparisons between free and forced recalls helped us understand important but largely neglected aspects of VWM, such as inhomogeneity (different levels of precision) and between-item interference. One unique part of our task was the introduction of a separate item-selection stage before each recall, during which participants located the next item to recall. Their mouse trajectory was recorded and served as a dynamic measure of between-item interference over time. We found a free-recall benefit: the overall precision of all items is higher in free recall than in forced recall. Meanwhile, during item-selection, free recall is associated with faster localization of the target and less interference from the other items in memory. We also found evidence for inhomogeneity and discuss the connection of inhomogeneity and between-item interference to the free-recall benefit.

Dynamic functional connectome predicts individual working memory performance across diagnostic categories

•

We created transdiagnostic predictive working memory models using connectome-based predictive modeling (CPM).

•

Dynamic functional connectivity-based CPM models successfully predicted working memory.

•

Static functional connectivity-based CPM models fell short in prediction.

•

Frontoparietal, somato-motor, default mode and visual networks contributed most to prediction.

Working memory impairment is a common feature of psychiatric disorders. Although its neural mechanisms have been extensively examined in healthy subjects or individuals with a certain clinical condition, studies investigating neural predictors of working memory in a transdiagnostic sample are scarce. The objective of this study was to create a transdiagnostic predictive working memory model from whole-brain functional connectivity using connectome-based predictive modeling (CPM), a recently developed machine learning approach. Resting-state functional MRI data from 242 subjects across 4 diagnostic categories (healthy controls and individuals with schizophrenia, bipolar disorder, and attention deficit/hyperactivity) were used to construct dynamic and static functional connectomes. Spatial working memory was assessed by the spatial capacity task. CPM was conducted to predict individual working memory from dynamic and static functional connectivity patterns. Results showed that dynamic connectivity-based CPM models successfully predicted overall working memory capacity and accuracy as well as mean reaction time, yet their static counterparts fell short in the prediction. At the neural level, we found that dynamic connectivity of the frontoparietal and somato-motor networks were negatively correlated with working memory capacity and accuracy, and those of the default mode and visual networks were positively associated with mean reaction time. Moreover, different feature selection thresholds, parcellation strategies and model validation methods as well as diagnostic categories did not significantly influence the prediction results. Our findings not only are coherent with prior reports that dynamic functional connectivity encodes more behavioral information than static connectivity, but also help advance the translation of cognitive “connectome fingerprinting” into real-world application.

Revisit once more the sensory storage account of visual working memory

Recent work has highlighted the role of early visual areas in visual working memory (VWM) storage and put forward a sensory storage account of VWM. Using a distractor interference paradigm, however, we previolsy showed that the contribution of early visual areas to VWM storage may not be essential. Instead, higher cortical regions such as the posterior parietal cortex may play a more significant role in VWM storage. This is consistent with reviews of other available behavioral, neuroimaging and neurophysiology results. Recently, a number of studies brought forward new evidence regarding this debate. Here I review these new pieces of evidence in detail and show that there is still no strong and definitive evidence supporting an essential role of the early visual areas in VWM storage. Instead, converging evidence suggests that early visual areas may contribute to the decision stage of a VWM task by facilitating target and probe comparison. Aside from further clarifying this debate, it is also important to note that whether or not VWM storage uses a sensory code depends on how it is defined, and that behavioral interactions between VWM and perception tasks do not necessarily support the involvement of sensory regions in VWM storage.

New Cognitive Neurotechnology Facilitates Studies of Cortical-Subcortical Interactions

Most of the studies employing neuroimaging have focused on cortical and subcortical signals individually to obtain neurophysiological signatures of cognitive functions. However, understanding the dynamic communication between the cortex and subcortical structures is essential for unraveling the neural correlates of cognition. In this quest, magnetoencephalography (MEG) and electroencephalography (EEG) are the methods of choice because they are noninvasive electrophysiological recording techniques with high temporal resolution. Sophisticated MEG/EEG source estimation techniques and network analysis methods, developed recently, can provide a more comprehensive understanding of the neurophysiological mechanisms of fundamental cognitive processes. Used together with noninvasive modulation of cortical-subcortical communication, these approaches may open up new possibilities for expanding the repertoire of noninvasive cognitive neurotechnology.

Selection of Visual Objects in Perception and Working Memory One at a Time

How does the content of visual working memory influence the way we process the visual environment? We addressed this question using the steady-state visual evoked potential (SSVEP), which provides a discernible measure of visuocortical activation to multiple stimuli simultaneously. Fifty-six adults were asked to remember a set of two oriented gratings. During the retention interval, two frequency-tagged oriented gratings were presented to probe the visuocortical processing of matching versus mismatching orientations relative to the memory set. Matching probes prompted an increased visuocortical response, whereas mismatching stimuli were suppressed. This suggests that the visual cortex prioritizes attentional selection of memory-relevant features at the expense of non-memory-relevant features. When two memory items were probed simultaneously, visuocortical amplification alternated between the two stimuli at a rate of 3 Hz to 4 Hz, consistent with the rate of attentional sampling of sensory events from the external world. These results suggest a serial, single-item attentional sampling of remembered features.

Visual working memory directly alters perception

Visual working memory (VWM), the ability to temporarily maintain and manipulate information, underlies a variety of critical high-level behaviours from directing attention^1-4 to making complex decisions⁵. Here we show that its impact extends to even the most basic levels of perceptual processing, directly interacting with and even distorting the physical appearance of visual features. This interference results from and can be predicted by the recruitment of posterior perceptual cortices to maintain information in VWM^6-9, which causes an overlap with the neuronal populations supporting perceptual processing^10-15. Across three sets of experiments, we demonstrated bidirectional interference between VWM and low-level perception. Specifically, for both maintained colours and orientations, presenting a distractor created bias in VWM representation depending on the similarity between incoming and maintained information, consistent with the known tuning curves for these features. Moreover, holding an item in mind directly altered the appearance of new stimuli, demonstrated by changes in psychophysical discrimination thresholds. Thus, as a consequence of sharing the early visual cortices, what you see and what you are holding in mind are intertwined at even the most fundamental stages of processing.

Electrical Stimulation Over Human Posterior Parietal Cortex Selectively Enhances the Capacity of Visual Short-Term Memory

Visual short-term memory (VSTM) provides an on-line mental space for incoming sensory information to be temporally maintained to carry out complex behavioral tasks. Despite its essential functions, the capacity at which VSTM could maintain sensory information is limited (i.e., VSTM can hold only about three to four visual items at once). Moreover, the quality of sensory representation (i.e., precision) degrades as more information has to be maintained in VSTM. Correlational evidence suggests that the level and the pattern of neural activity measured in the posterior parietal cortex (PPC) track both VSTM capacity and precision. However, the causal contributions of the PPC to these different VSTM operations are unclear. Here, we tested whether stimulating the PPC with transcranial direct current stimulation (tDCS) could increase VSTM capacity or precision. We found that stimulating the PPC in male and female human participants selectively enhanced VSTM capacity when the number of memory items exceeded capacity limit, without significant effects on VSTM precision. Moreover, this enhancement of VSTM capacity is region specific as stimulating the prefrontal cortex did not change VSTM capacity or precision. Null stimulation effects in the sensory memory condition confirmed that the tDCS-induced enhancement of VSTM capacity was not simply due to changes in sensory or attentional processes. Altogether, these results provide causal evidence suggesting that the PPC has a more dominant role in supporting the storage capacity of VSTM compared with maintaining the quality of sensory representations. Furthermore, tDCS could be used as a promising noninvasive method to enhance this PPC VSTM-related function.SIGNIFICANCE STATEMENT Correlational evidence from neuroimaging and electrophysiology suggests that the posterior parietal cortex (PPC) supports the storage capacity of visual short-term memory (VSTM) and the precision of sensory representations maintained in VSTM. However, the causal contributions of the PPC to these different VSTM functions were unclear. Here, we found that electrical stimulation over the PPC selectively enhanced VSTM capacity without changing VSTM precision. Overall, our findings suggest that the PPC has a dominant and causal role in supporting the storage capacity of VSTM.

Reevaluating the Sensory Account of Visual Working Memory Storage

Recent human fMRI pattern-decoding studies have highlighted the involvement of sensory areas in visual working memory (VWM) tasks and argue for a sensory account of VWM storage. In this review, evidence is examined from human behavior, fMRI decoding, and transcranial magnetic stimulation (TMS) studies, as well as from monkey neurophysiology studies. Contrary to the prevalent view, the available evidence provides little support for the sensory account of VWM storage. Instead, when the ability to resist distraction and the existence of top-down feedback are taken into account, VWM-related activities in sensory areas seem to reflect feedback signals indicative of VWM storage elsewhere in the brain. Collectively, the evidence shows that prefrontal and parietal regions, rather than sensory areas, play more significant roles in VWM storage.