Causal evidence for subliminal percept-to-memory interference in early visual cortex

Divergent roles of early visual cortex and inferior frontal junction in visual working memory

BACKGROUND

Recent studies indicate that both prefrontal and visual regions play critical roles in visual working memory (VWM), with prefrontal regions mainly associated with executive functions, and visual cortices linked to representations of memory contents. VWM involves the selective filtering of irrelevant information, yet the specific contributions of the prefrontal regions and visual cortex in this process remain unclear.

OBJECTIVE

To understand the dynamic causal roles of prefrontal and visual regions in VWM.

METHODS

The differentiation of VWM components was achieved using a computational model that incorporated a swap rate for non-target stimuli. Single-pulse magnetic transcranial stimulation (spTMS) was delivered to the early visual cortex (EVC) and the inferior frontal junction (IFJ) across different phases of an orientation recall task that with or without distractors.

RESULTS

Our results indicate that spTMS over the EVC and IFJ influences VWM particularly when distractors are present. VWM precision can be impacted by spTMS applied to either region during the early retention, while spTMS effect is especially prominent when EVC is stimulated during the late retention phase and when directed at the ipsilateral EVC. Conversely, the probability of accurately recalling the target exhibited comparable patterns when spTMS was administered to either the EVC or IFJ.

CONCLUSIONS

We highlight the "sensory recruitment" of VWM characterized by critical involvement of EVC particularly in the information-filtering process within VWM. The maintenance of memory content representations necessitates ongoing communication between the EVC and IFJ throughout the entirety of the VWM process in a dynamic pattern.

Unconsciously registered items reduce working memory capacity

The assumption that the contents of consciousness correspond to those of working memory (WM) is challenged by evidence that stimuli masked from awareness can be retained for several seconds (Soto et al., 2011; Bergström & Eriksson, 2015). To assess whether conscious and unconscious items compete in a unitary WM store we conducted an experiment in which some of the memory items in an array were masked from conscious sight using continuous flash suppression (CFS) while others remained visible. After a retention interval, participants decided whether the probed item (either masked or visible) had changed its orientation. Behavioral results indicated that change detection for visible items was significantly impaired when masked items were present, suggesting that masked items either displaced or reduced the precision of visible items in WM. However, change detection for masked items was at chance levels, indicating that these items were not stored. The unsuccessful attempt to encode them may have drawn upon a common pool of attentional resources needed to retain or retrieve visible items. Contralateral Delay Activity, an EEG index of net WM load, failed to temporally localize this interference.

Sensory recruitment in visual short-term memory: A systematic review and meta-analysis of sensory visual cortex interference using transcranial magnetic stimulation

Sensory visual areas are involved in encoding information in visual short-term memory (VSTM). Yet it remains unclear whether sensory visual cortex is a necessary component of the brain network for maintenance of information in VSTM. Here, we aimed to systematically review studies that have investigated the role of the sensory visual cortex in VSTM using transcranial magnetic stimulation (TMS) and to quantitatively explore these effects using meta-analyses. Fourteen studies were identified and reviewed. Eight studies provided sufficient data for meta-analysis. Two meta-analyses, one regarding the VSTM encoding phase (17 effect sizes) and one regarding the VSTM maintenance phase (15 effect sizes), two meta-regressions (32 effect sizes in each), and one exploratory meta-analysis were conducted. Our results indicate that the sensory visual cortex is similarly involved in both the encoding and maintenance VSTM phase. We suggest that some cases where evidence did not show significant TMS effects was due to low memory or perceptual task demands. Overall, these findings support the idea that sensory visual areas are part of the brain network responsible for successfully maintaining information in VSTM.

Effects of online repetitive transcranial magnetic stimulation (rTMS) on cognitive processing: A meta-analysis and recommendations for future studies

Online repetitive transcranial magnetic stimulation (rTMS), applied while subjects are performing a task, is widely used to disrupt brain regions underlying cognition. However, online rTMS has also induced "paradoxical enhancement". Given the rapid proliferation of this approach, it is crucial to develop a better understanding of how online stimulation influences cognition, and the optimal parameters to achieve desired effects. To accomplish this goal, a quantitative meta-analysis was performed with random-effects models fitted to reaction time (RT) and accuracy data. The final dataset included 126 studies published between 1998 and 2016, with 244 total effects for reaction times, and 202 for accuracy. Meta-analytically, rTMS at 10 Hz and 20 Hz disrupted accuracy for attention, executive, language, memory, motor, and perception domains, while no effects were found with 1 Hz or 5 Hz. Stimulation applied at and 10 and 20 Hz slowed down RTs in attention and perception tasks. No performance enhancement was found. Meta-regression analysis showed that fMRI-guided targeting and short inter-trial intervals are associated with increased disruptive effects with rTMS.

Right hemisphere occipital rTMS impairs working memory in visualizers but not in verbalizers

Distinguishing between verbal and visual working memory processes is complicated by the fact that the strategy used is hard to control or even assess. Many stimuli used in working memory tasks can be processed via verbal or visual coding, such as the digits in the digit span backwards task (DSB). The present study used repetitive transcranial magnetic stimulation (rTMS) to examine the use of visual processing strategies in the DSB. A total of 47 German university students took part in the study, 23 spontaneously using a verbal processing strategy and 24 using a visual strategy. After rTMS to the right occipital cortex, visualizers showed a significantly stronger mean performance decrease compared to verbalizers. The results indicate that the visual cortex is more critical for visualizers compared to verbalizers in the DSB task. Furthermore, the favored processing modality seems to be determined by the preference for a cognitive strategy rather than the presentation modality, and people are aware of the applied strategy. These findings provide insight into inter-individual differences in working memory processing and yield important implications for laboratory studies as well as clinical practice: the stimulus does not necessarily determine the processing and the participant can be aware of that.

The evolution of lossy compression

In complex environments, there are costs to both ignorance and perception. An organism needs to track fitness-relevant information about its world, but the more information it tracks, the more resources it must devote to perception. As a first step towards a general understanding of this trade-off, we use a tool from information theory, rate-distortion theory, to study large, unstructured environments with fixed, randomly drawn penalties for stimuli confusion ('distortions'). We identify two distinct regimes for organisms in these environments: a high-fidelity regime where perceptual costs grow linearly with environmental complexity, and a low-fidelity regime where perceptual costs are, remarkably, independent of the number of environmental states. This suggests that in environments of rapidly increasing complexity, well-adapted organisms will find themselves able to make, just barely, the most subtle distinctions in their environment.

The impact of early visual cortex transcranial magnetic stimulation on visual working memory precision and guess rate

Neuroimaging studies have demonstrated that activity patterns in early visual areas predict stimulus properties actively maintained in visual working memory. Yet, the mechanisms by which such information is represented remain largely unknown. In this study, observers remembered the orientations of 4 briefly presented gratings, one in each quadrant of the visual field. A 10Hz Transcranial Magnetic Stimulation (TMS) triplet was applied directly at stimulus offset, or midway through a 2-second delay, targeting early visual cortex corresponding retinotopically to a sample item in the lower hemifield. Memory for one of the four gratings was probed at random, and participants reported this orientation via method of adjustment. Recall errors were smaller when the visual field location targeted by TMS overlapped with that of the cued memory item, compared to errors for stimuli probed diagonally to TMS. This implied topographic storage of orientation information, and a memory-enhancing effect at the targeted location. Furthermore, early pulses impaired performance at all four locations, compared to late pulses. Next, response errors were fit empirically using a mixture model to characterize memory precision and guess rates. Memory was more precise for items proximal to the pulse location, irrespective of pulse timing. Guesses were more probable with early TMS pulses, regardless of stimulus location. Thus, while TMS administered at the offset of the stimulus array might disrupt early-phase consolidation in a non-topographic manner, TMS also boosts the precise representation of an item at its targeted retinotopic location, possibly by increasing attentional resources or by injecting a beneficial amount of noise.

Exploring the contributions of the supplementary eye field to subliminal inhibition using double-pulse transcranial magnetic stimulation

It is widely accepted that the supplementary eye fields (SEF) are involved in the control of voluntary eye movements. However, recent evidence suggests that SEF may also be important for unconscious and involuntary motor processes. Indeed, Sumner et al. ([2007]: Neuron 54:697-711) showed that patients with micro-lesions of the SEF demonstrated an absence of subliminal inhibition as evoked by masked-prime stimuli. Here, we used double-pulse transcranial magnetic stimulation (TMS) in healthy volunteers to investigate the role of SEF in subliminal priming. We applied double-pulse TMS at two time windows in a masked-prime task: the first during an early phase, 20-70 ms after the onset of the mask but before target presentation, during which subliminal inhibition is present; and the second during a late phase, 20-70 ms after target onset, during which the saccade is being prepared. We found no effect of TMS with the early time window of stimulation, whereas a reduction in the benefit of an incompatible subliminal prime stimulus was found when SEF TMS was applied at the late time window. These findings suggest that there is a role for SEF related to the effects of subliminal primes on eye movements, but the results do not support a role in inhibiting the primed tendency. Hum Brain Mapp 38:339-351, 2017. © 2016 Wiley Periodicals, Inc.

Is conscious awareness needed for all working memory processes?

Stein and colleagues argue there is no yet conclusive evidence for nonconscious working memory (WM) and that is critical to probe WM while ensuring null sensitivity to memory cues. While this stringent approach reduces the likelihood of nonconscious signaling for WM, we discuss existing work meeting this null sensitivity criteria, and, related work on nonconscious cognition in keeping with WM/awareness dissociations on the basis of a functional operational definition of WM. Further, because it is likely that WM is a nonunitary functional construct and visual awareness a gradual phenomenon, we propose that delineating the neural mechanisms for distinct WM types across different levels of awareness may prove the most fruitful approach for understanding the interplay between WM and consciousness.

How is working memory content consciously experienced? The 'conscious copy' model of WM introspection

We address the issue of how visual information stored in working memory (WM) is introspected. In other words, how do we become aware of WM content in order to consciously examine or manipulate it? Influential models of WM have suggested that WM representations are either conscious by definition, or directly accessible for conscious inspection. We propose that WM introspection does not operate on the actual memory trace but rather requires a new representation to be created for the conscious domain. This conscious representation exists in addition and in parallel to the actual memory representation. The existence of such a separate representation is revealed by and reflected in the qualitatively different functional characteristics between the actual memory trace and its conscious experience, and their distinct interactions within external visual input. Our model differs from state-based models in that WM introspection does not involve a change in the state of WM content, but rather involves the creation of a new, second representation existing in parallel to the original memory trace.

Partial dissociation in the neural bases of VSTM and imagery in the early visual cortex

Visual short-term memory (VSTM) and visual imagery are believed to involve overlapping neuronal representations in the early visual cortex. While a number of studies have provided evidence for this overlap, at the behavioral level VSTM and imagery are dissociable processes; this begs the question of how their neuronal mechanisms differ. Here we used transcranial magnetic stimulation (TMS) to examine whether the neural bases of imagery and VSTM maintenance are dissociable in the early visual cortex (EVC). We intentionally used a similar task for VSTM and imagery in order to equate their assessment. We hypothesized that any differential effect of TMS on VSTM and imagery would indicate that their neuronal bases differ at the level of EVC. In the “alone” condition, participants were asked to engage either in VSTM or imagery, whereas in the “concurrent” condition, each trial required both VSTM maintenance and imagery simultaneously. A dissociation between VSTM and imagery was observed for reaction times: TMS slowed down responses for VSTM but not for imagery. The impact of TMS on sensitivity did not differ between VSTM and imagery, but did depend on whether the tasks were carried concurrently or alone. This study shows that neural processes associated with VSTM and imagery in the early visual cortex can be partially dissociated.

•

Both VSTM and visual imagery are believed to involve early visual cortex (EVC).

•

TMS was used to investigate whether their neural bases can be dissociated.

•

TMS dissociated VSTM and imagery at the level of reaction times.

•

No differences in the effect of TMS on task sensitivity.

Supraliminal but not subliminal distracters bias working memory recall

Information of which observers are not consciously aware can nevertheless influence perceptual processes. Whether subliminal information might exert an influence on working memory (WM) representations is less clear, and relatively few studies have examined the interactions between subliminal and supraliminal information in WM. We present 3 experiments examining this issue. Experiments 1a and b replicated the finding that orientation stimuli can influence behavior subliminally in a visuomotor priming task. Experiments 2 and 3 used the same orientation stimuli, but participants had to remember a target orientation and report it back by adjusting a probe orientation after a memory delay. Before or after presentation of the target orientation, a subliminal or supraliminal distracter orientation was presented that was either irrelevant for task completion and never had to be reported (Experiment 2), or was relevant for task completion because it had to be reported on some trials (Experiment 3). In both experiments, presentation of a supraliminal distracter influenced WM recall of the target orientation. When the distracter was presented subliminally, however, there was no bias in orientation recall. These results suggest that information stored in WM is protected from influences of subliminal stimuli, while online information processing is modulated by subliminal information.

Reappraising the relationship between working memory and conscious awareness

Classically, the operation of working memory (WM) has been strongly coupled with conscious states; it is thought that WM operates on conscious input and that we are conscious of the contents and operations of WM. Here, we re-evaluate the relationship between WM and conscious awareness in light of current data and question the views that awareness is mandatory for the operation of WM and that WM contents are necessarily linked to experiential states that are consciously accessible for perceptual report. We propose a novel framework for the relationship between WM and conscious awareness.

Accuracy and confidence of visual short-term memory do not go hand-in-hand: behavioral and neural dissociations

Currently influential models of working memory posit that memory content is highly accessible to conscious inspection. These models predict that metacognition of memory performance should go hand-in-hand with the accuracy of the underlying memory representation. To test this view, we investigated how visual information presented during the maintenance period affects VSTM accuracy and confidence. We used a delayed cue-target orientation discrimination task in which participants were asked to hold in memory a grating, and during the maintenance period a second memory cue could be presented. VSTM accuracy of the first memory cue was impaired when the orientation of the second memory cue was sufficiently different. However, participants' response confidence was reduced whenever the second memory cue was presented; thus VSTM accuracy and confidence were dissociated. In a second experiment, we applied transcranial direct current stimulation (tDCS) over the right dorsolateral prefrontal cortex (DLPFC) to investigate the causal role of this region in VSTM metacognition. Relative to the sham condition, anodal tDCS induced a general reduction in confidence ratings but did not affect VSTM accuracy. Overall, these results indicate that our metacognition of memory performance is influenced by factors other than the accuracy of the underlying memory representation.

Causal evidence for a privileged working memory state in early visual cortex

Emerging evidence suggests that items held in working memory (WM) might not all be in the same representational state. One item might be privileged over others, making it more accessible and thereby recalled with greater precision. Here, using transcranial magnetic stimulation (TMS), we provide causal evidence in human participants that items in WM are differentially susceptible to disruptive TMS, depending on their state, determined either by task relevance or serial position. Across two experiments, we applied TMS to area MT+ during the WM retention of two motion directions. In Experiment 1, we used an "incidental cue" to bring one of the two targets into a privileged state. In Experiment 2, we presented the targets sequentially so that the last item was in a privileged state by virtue of recency. In both experiments, recall precision of motion direction was differentially affected by TMS, depending on the state of the memory target at the time of disruption. Privileged items were recalled with less precision, whereas nonprivileged items were recalled with higher precision. Thus, only the privileged item was susceptible to disruptive TMS over MT+. By contrast, precision of the nonprivileged item improved either directly because of facilitation by TMS or indirectly through reduced interference from the privileged item. Our results provide a unique line of evidence, as revealed by TMS over a posterior sensory brain region, for at least two different states of item representation in WM.

Revisiting the role of persistent neural activity during working memory

What are the neural mechanisms underlying working memory (WM)? One influential theory posits that neurons in the lateral prefrontal cortex (lPFC) store WM information via persistent activity. In this review, we critically evaluate recent findings that together indicate that this model of WM needs revision. We argue that sensory cortex, not the lPFC, maintains high-fidelity representations of WM content. By contrast, the lPFC simultaneously maintains representations of multiple goal-related variables that serve to bias stimulus-specific activity in sensory regions. This work highlights multiple neural mechanisms supporting WM, including temporally dynamic population coding in addition to persistent activity. These new insights focus the question on understanding how the mechanisms that underlie WM are related, interact, and are coordinated in the lPFC and sensory cortex.

The importance of perceptual experience in the esthetic appreciation of the body

Several studies suggest that sociocultural models conveying extreme thinness as the widespread ideal of beauty exert an important influence on the perceptual and emotional representation of body image. The psychological mechanisms underlying such environmental influences, however, are unclear. Here, we utilized a perceptual adaptation paradigm to investigate how perceptual experience modulates body esthetic appreciation. We found that the liking judgments of round bodies increased or decreased after brief exposure to round or thin bodies, respectively. No change occurred in the liking judgments of thin bodies. The results suggest that perceptual experience may shape our esthetic appreciation to favor more familiar round body figures. Importantly, individuals with more deficits in interoceptive awareness were less prone to increase their liking ratings of round bodies after exposure, suggesting a specific risk factor for the susceptibility to the influence of the extreme thin vs. round body ideals of beauty portrayed by the media.

Visual short-term memory for global motion revealed by directional and speed-tuned masking

Neurobehavioral research with non-human primates has shown that different attributes of motion stimuli, such as direction and speed can be stored in visual short-term memory (VSTM) with a high degree of accuracy. We examined VSTM for global motion with a memory masking paradigm to determine which stimulus attributes are important in the storage process. We presented in two visual quadrants global motion random dot kinematograms (RDKs), whereas in the two remaining visual quadrants we presented random-motion RDKs. This pattern of stimulation was displayed in two distinct temporal intervals, i.e., sample and test stimuli (duration: 200 ms), separated in time by a 3.2-s delay period. During the delay period a random- or directional-motion mask was presented briefly (200 ms) either at the beginning, in the middle or at the end of the delay period. The results showed that the mask mainly interferes with performance when displayed 200 ms after the offset of the sample and when it had a coherent direction rather than random directions. Moreover, the mask is significantly more effective when its direction and speed matched that of the remembered sample. These results support the notion that the memory representation of global motion is selective for direction and speed, being compromised by intervening directional stimuli presented immediately after the encoding phase. Moreover, this selectivity suggests that the same neural mechanisms involved in the processing of global motion may be recruited for its storage.

Metacognition of Visual Short-Term Memory: Dissociation between Objective and Subjective Components of VSTM

The relationship between the objective accuracy of visual short-term memory (VSTM) representations and their subjective conscious experience is unknown. We investigated this issue by assessing how the objective and subjective components of VSTM in a delayed cue-target orientation discrimination task are affected by intervening distracters. On each trial, participants were shown a memory cue (a grating), the orientation of which they were asked to hold in memory. On approximately half of the trials, a distracter grating appeared during the maintenance interval; its orientation was either identical to that of the memory cue, or it differed by 10° or 40°. The distracters were masked and presented briefly, so they were only consciously perceived on a subset of trials. At the end of the delay period, a memory test probe was presented, and participants were asked to indicate whether it was tilted to the left or right relative to the memory cue (VSTM accuracy; objective performance). In order to assess subjective metacognition, participants were asked indicate the vividness of their memory for the original memory cue. Finally, participants were asked rate their awareness of the distracter. Results showed that objective VSTM performance was impaired by distracters only when the distracters were very different from the cue, and that this occurred with both subjectively visible and invisible distracters. Subjective metacognition, however, was impaired by distracters of all orientations, but only when these distracters were subjectively invisible. Our results thus indicate that the objective and subjective components of VSTM are to some extent dissociable.