Modulation of activity in human visual area V1 during memory masking

Visual processing deficits in patients with schizophrenia spectrum and bipolar disorders and associations with psychotic symptoms, and intellectual abilities

Objective

Low-level sensory disruption is hypothesized as a precursor to clinical and cognitive symptoms in severe mental disorders. We compared visual discrimination performance in patients with schizophrenia spectrum disorder or bipolar disorder with healthy controls, and investigated associations with clinical symptoms and IQ.

Methods

Patients with schizophrenia spectrum disorder (n = 32), bipolar disorder (n = 55) and healthy controls (n = 152) completed a computerized visual discrimination task. Participants responded whether the latter of two consecutive grids had higher or lower spatial frequency, and discrimination thresholds were estimated using an adaptive maximum likelihood procedure. Case-control differences in threshold were assessed using linear regression, F-test and post-hoc pair-wise comparisons. Linear models were used to test for associations between visual discrimination threshold and psychotic symptoms derived from the PANSS and IQ assessed using the Matrix Reasoning and Vocabulary subtests from the Wechsler Abbreviated Scale of Intelligence (WASI).

Results

Robust regression revealed a significant main effect of diagnosis on discrimination threshold (robust F = 6.76, p = .001). Post-hoc comparisons revealed that patients with a schizophrenia spectrum disorder (mean = 14%, SD = 0.08) had higher thresholds compared to healthy controls (mean = 10.8%, SD = 0.07, β = 0.35, t = 3.4, p = .002), as did patients with bipolar disorder (12.23%, SD = 0.07, β = 0.21, t = 2.42, p = .04). There was no significant difference between bipolar disorder and schizophrenia (β = -0.14, t = -1.2, p = .45). Linear models revealed negative associations between IQ and threshold across all participants when controlling for diagnostic group (β = -0.3, t = -3.43, p = .0007). This association was found within healthy controls (t = -3.72, p = .0003) and patients with bipolar disorder (t = -2.53, p = .015), and no significant group by IQ interaction on threshold (F = 0.044, p = .97). There were no significant associations between PANSS domain scores and discrimination threshold.

Conclusion

Patients with schizophrenia spectrum or bipolar disorders exhibited higher visual discrimination thresholds than healthy controls, supporting early visual deficits among patients with severe mental illness. Discrimination threshold was negatively associated with IQ among healthy controls and bipolar disorder patients. These findings elucidate perception-related disease mechanisms in severe mental illness, which warrants replication in independent samples.

Long-term memory for spatial frequency: a non-replication

Reports on stability of spatial frequency in short-term memory span have confirmed low-level perceptual memory mechanism in early visual processing. However, some studies have also claimed evidence for high-fidelity perceptual long-term storage of spatial frequency. We report an attempted replication of Magnussen et al. (Psychol Sci 14:74-76, 2003) where participants were asked to discriminate the spatial frequency of a reference grating from a test stimulus after intervals of 5 s or 24 h. Group thresholds after 24 h were significantly higher than after 5 s, therefore failing to support long-term storage of spatial frequency.

Visual Short-Term Memory for Coherent and Sequential Motion: A rTMS Investigation

We investigated the role of the human medio-temporal complex (hMT+) in the memory encoding and storage of a sequence of four coherently moving random dot kinematograms (RDKs), by applying repetitive transcranial magnetic stimulation (rTMS) during an early or late phase of the retention interval. Moreover, in a second experiment, we also tested whether disrupting the functional integrity of hMT+ during the early phase impaired the precision of the encoded motion directions. Overall, results showed that both recognition accuracy and precision were worse in middle serial positions, suggesting the occurrence of primacy and recency effects. We found that rTMS delivered during the early (but not the late) phase of the retention interval was able to impair not only recognition of RDKs, but also the precision of the retained motion direction. However, such impairment occurred only for RDKs presented in middle positions along the presented sequence, where performance was already closer to chance level. Altogether these findings suggest an involvement of hMT+ in the memory encoding of visual motion direction. Given that both position sequence and rTMS modulated not only recognition but also the precision of the stored information, these findings are in support of a model of visual short-term memory with a variable resolution of each stored item, consistent with the assigned amount of memory resources, and that such item-specific memory resolution is supported by the functional integrity of area hMT+.

Visual objects interact differently during encoding and memory maintenance

The storage mechanisms of working memory are the matter of an ongoing debate. The sensory recruitment hypothesis states that memory maintenance and perceptual encoding rely on the same neural substrate. This suggests that the same cortical mechanisms that shape object perception also apply to maintained memory content. We tested this prediction using the Direction Illusion, i.e., the mutual repulsion of two concurrently visible motion directions. Participants memorized the directions of two random dot patterns for later recall. In Experiments 1 and 2, we varied the temporal separation of spatially distinct stimuli to manipulate perceptual concurrency, while keeping concurrency within working memory constant. We observed mutual motion repulsion only under simultaneous stimulus presentation, but proactive repulsion and retroactive attraction under immediate stimulus succession. At inter-stimulus intervals of 0.5 and 2 s, however, proactive repulsion vanished, while the retroactive attraction remained. In Experiment 3, we presented both stimuli at the same spatial position and observed a reappearance of the repulsion effect. Our results indicate that the repulsive mechanisms that shape object perception across space fade during the transition from a perceptual representation to a consolidated memory content. This suggests differences in the underlying structure of perceptual and mnemonic representations. The persistence of local interactions, however, indicates different mechanisms of spatially global and local feature interactions.

Flexible Coding of Visual Working Memory Representations during Distraction

Visual working memory (VWM) recruits a broad network of brain regions, including prefrontal, parietal, and visual cortices. Recent evidence supports a "sensory recruitment" model of VWM, whereby precise visual details are maintained in the same stimulus-selective regions responsible for perception. A key question in evaluating the sensory recruitment model is how VWM representations persist through distracting visual input, given that the early visual areas that putatively represent VWM content are susceptible to interference from visual stimulation.To address this question, we used a functional magnetic resonance imaging inverted encoding model approach to quantitatively assess the effect of distractors on VWM representations in early visual cortex and the intraparietal sulcus (IPS), another region previously implicated in the storage of VWM information. This approach allowed us to reconstruct VWM representations for orientation, both before and after visual interference, and to examine whether oriented distractors systematically biased these representations. In our human participants (both male and female), we found that orientation information was maintained simultaneously in early visual areas and IPS in anticipation of possible distraction, and these representations persisted in the absence of distraction. Importantly, early visual representations were susceptible to interference; VWM orientations reconstructed from visual cortex were significantly biased toward distractors, corresponding to a small attractive bias in behavior. In contrast, IPS representations did not show such a bias. These results provide quantitative insight into the effect of interference on VWM representations, and they suggest a dynamic tradeoff between visual and parietal regions that allows flexible adaptation to task demands in service of VWM.SIGNIFICANCE STATEMENT Despite considerable evidence that stimulus-selective visual regions maintain precise visual information in working memory, it remains unclear how these representations persist through subsequent input. Here, we used quantitative model-based fMRI analyses to reconstruct the contents of working memory and examine the effects of distracting input. Although representations in the early visual areas were systematically biased by distractors, those in the intraparietal sulcus appeared distractor-resistant. In contrast, early visual representations were most reliable in the absence of distraction. These results demonstrate the dynamic, adaptive nature of visual working memory processes, and provide quantitative insight into the ways in which representations can be affected by interference. Further, they suggest that current models of working memory should be revised to incorporate this flexibility.

Visual Memories Bypass Normalization

How distinct are visual memory representations from visual perception? Although evidence suggests that briefly remembered stimuli are represented within early visual cortices, the degree to which these memory traces resemble true visual representations remains something of a mystery. Here, we tested whether both visual memory and perception succumb to a seemingly ubiquitous neural computation: normalization. Observers were asked to remember the contrast of visual stimuli, which were pitted against each other to promote normalization either in perception or in visual memory. Our results revealed robust normalization between visual representations in perception, yet no signature of normalization occurring between working memory stores-neither between representations in memory nor between memory representations and visual inputs. These results provide unique insight into the nature of visual memory representations, illustrating that visual memory representations follow a different set of computational rules, bypassing normalization, a canonical visual computation.

Decoding working memory of stimulus contrast in early visual cortex

Most studies of the early stages of visual analysis (V1-V3) have focused on the properties of neurons that support processing of elemental features of a visual stimulus or scene, such as local contrast, orientation, or direction of motion. Recent evidence from electrophysiology and neuroimaging studies, however, suggests that early visual cortex may also play a role in retaining stimulus representations in memory for short periods. For example, fMRI responses obtained during the delay period between two presentations of an oriented visual stimulus can be used to decode the remembered stimulus orientation with multivariate pattern analysis. Here, we investigated whether orientation is a special case or if this phenomenon generalizes to working memory traces of other visual features. We found that multivariate classification of fMRI signals from human visual cortex could be used to decode the contrast of a perceived stimulus even when the mean response changes were accounted for, suggesting some consistent spatial signal for contrast in these areas. Strikingly, we found that fMRI responses also supported decoding of contrast when the stimulus had to be remembered. Furthermore, classification generalized from perceived to remembered stimuli and vice versa, implying that the corresponding pattern of responses in early visual cortex were highly consistent. In additional analyses, we show that stimulus decoding here is driven by biases depending on stimulus eccentricity. This places important constraints on the interpretation for decoding stimulus properties for which cortical processing is known to vary with eccentricity, such as contrast, color, spatial frequency, and temporal frequency.

Visual short-term memory for global motion revealed by directional and speed-tuned masking

Neurobehavioral research with non-human primates has shown that different attributes of motion stimuli, such as direction and speed can be stored in visual short-term memory (VSTM) with a high degree of accuracy. We examined VSTM for global motion with a memory masking paradigm to determine which stimulus attributes are important in the storage process. We presented in two visual quadrants global motion random dot kinematograms (RDKs), whereas in the two remaining visual quadrants we presented random-motion RDKs. This pattern of stimulation was displayed in two distinct temporal intervals, i.e., sample and test stimuli (duration: 200 ms), separated in time by a 3.2-s delay period. During the delay period a random- or directional-motion mask was presented briefly (200 ms) either at the beginning, in the middle or at the end of the delay period. The results showed that the mask mainly interferes with performance when displayed 200 ms after the offset of the sample and when it had a coherent direction rather than random directions. Moreover, the mask is significantly more effective when its direction and speed matched that of the remembered sample. These results support the notion that the memory representation of global motion is selective for direction and speed, being compromised by intervening directional stimuli presented immediately after the encoding phase. Moreover, this selectivity suggests that the same neural mechanisms involved in the processing of global motion may be recruited for its storage.

Visual short-term memory: activity supporting encoding and maintenance in retinotopic visual cortex

Recent studies have demonstrated that retinotopic cortex maintains information about visual stimuli during retention intervals. However, the process by which transient stimulus-evoked sensory responses are transformed into enduring memory representations is unknown. Here, using fMRI and short-term visual memory tasks optimized for univariate and multivariate analysis approaches, we report differential involvement of human retinotopic areas during memory encoding of the low-level visual feature orientation. All visual areas show weaker responses when memory encoding processes are interrupted, possibly due to effects in orientation-sensitive primary visual cortex (V1) propagating across extrastriate areas. Furthermore, intermediate areas in both dorsal (V3a/b) and ventral (LO1/2) streams are significantly more active during memory encoding compared with non-memory (active and passive) processing of the same stimulus material. These effects in intermediate visual cortex are also observed during memory encoding of a different stimulus feature (spatial frequency), suggesting that these areas are involved in encoding processes on a higher level of representation. Using pattern-classification techniques to probe the representational content in visual cortex during delay periods, we further demonstrate that simply initiating memory encoding is not sufficient to produce long-lasting memory traces. Rather, active maintenance appears to underlie the observed memory-specific patterns of information in retinotopic cortex.

Multiple spatial frequency channels in human visual perceptual memory

Current models of short-term visual perceptual memory invoke mechanisms that are closely allied to low-level perceptual discrimination mechanisms. The purpose of this study was to investigate the extent to which human visual perceptual memory for spatial frequency is based upon multiple, spatially tuned channels similar to those found in the earliest stages of visual processing. To this end we measured how performance on a delayed spatial frequency discrimination paradigm was affected by the introduction of interfering or 'memory masking' stimuli of variable spatial frequency during the delay period. Masking stimuli were shown to induce shifts in the points of subjective equality (PSE) when their spatial frequencies were within a bandwidth of 1.2 octaves of the reference spatial frequency. When mask spatial frequencies differed by more than this value, there was no change in the PSE from baseline levels. This selective pattern of masking was observed for different spatial frequencies and demonstrates the existence of multiple, spatially tuned mechanisms in visual perceptual memory. Memory masking effects were also found to occur for horizontal separations of up to 6 deg between the masking and test stimuli and lacked any orientation selectivity. These findings add further support to the view that low-level sensory processing mechanisms form the basis for the retention of spatial frequency information in perceptual memory. However, the broad range of transfer of memory masking effects across spatial location and other dimensions indicates more long range, long duration interactions between spatial frequency channels that are likely to rely contributions from neural processes located in higher visual areas.