Learning of goal-relevant and -irrelevant complex visual sequences in human V1

Anatomo-functional changes in neural substrates of cognitive memory in developmental amnesia: Insights from automated and manual Magnetic Resonance Imaging examinations

Despite bilateral hippocampal damage dating to the perinatal or early childhood period and severely impaired episodic memory, patients with developmental amnesia continue to exhibit well-developed semantic memory across the developmental trajectory. Detailed information on the extent and focality of brain damage in these patients is needed to hypothesize about the neural substrate that supports their remarkable capacity for encoding and retrieval of semantic memory. In particular, we need to assess whether the residual hippocampal tissue is involved in this preservation, or whether the surrounding cortical areas reorganize to rescue aspects of these critical cognitive memory processes after early injury. We used voxel-based morphometry (VBM) analysis, automatic (FreeSurfer) and manual segmentation to characterize structural changes in the brain of an exceptionally large cohort of 23 patients with developmental amnesia in comparison with 32 control subjects. Both the VBM and the FreeSurfer analyses revealed severe structural alterations in the hippocampus and thalamus of patients with developmental amnesia. Milder damage was found in the amygdala, caudate, and parahippocampal gyrus. Manual segmentation demonstrated differences in the degree of atrophy of the hippocampal subregions in patients. The level of atrophy in CA-DG subregions and subicular complex was more than 40%, while the atrophy of the uncus was moderate (-24%). Anatomo-functional correlations were observed between the volumes of residual hippocampal subregions in patients and selective aspects of their cognitive performance, viz, intelligence, working memory, and verbal and visuospatial recall. Our findings suggest that in patients with developmental amnesia, cognitive processing is compromised as a function of the extent of atrophy in hippocampal subregions. More severe hippocampal damage may be more likely to promote structural and/or functional reorganization in areas connected to the hippocampus. In this hypothesis, different levels of hippocampal function may be rescued following this variable reorganization. Our findings document not only the extent, but also the limits of circuit reorganization occurring in the young brain after early bilateral hippocampal damage.

Anticipating multisensory environments: Evidence for a supra-modal predictive system

Our perceptual experience is generally framed in multisensory environments abundant in predictive information. Previous research on statistical learning has shown that humans can learn regularities in different sensory modalities in parallel, but it has not yet determined whether multisensory predictions are generated through a modality-specific predictive mechanism or instead, rely on a supra-modal predictive system. Here, across two experiments, we tested these hypotheses by presenting participants with concurrent pairs of predictable auditory and visual low-level stimuli (i.e., tones and gratings). In different experimental blocks, participants had to attend the stimuli in one modality while ignoring stimuli from the other sensory modality (distractors), and perform a perceptual discrimination task on the second stimulus of the attended modality (targets). Orthogonal to the task goal, both the attended and unattended pairs followed transitional probabilities, so targets and distractors could be expected or unexpected. We found that participants performed better for expected compared to unexpected targets. This effect generalized to the distractors but only when relevant targets were expected. Such interactive effects suggest that predictions may be gated by a supra-modal system with shared resources across sensory modalities that are distributed according to their respective behavioural relevance.

Finding Pattern in the Noise: Persistent Implicit Statistical Knowledge Impacts the Processing of Unpredictable Stimuli

Humans can extract statistical regularities of the environment to predict upcoming events. Previous research recognized that implicitly acquired statistical knowledge remained persistent and continued to influence behavior even when the regularities were no longer present in the environment. Here, in an fMRI experiment, we investigated how the persistence of statistical knowledge is represented in the brain. Participants (n = 32) completed a visual, four-choice, RT task consisting of statistical regularities. Two types of blocks constantly alternated with one another throughout the task: predictable statistical regularities in one block type and unpredictable ones in the other. Participants were unaware of the statistical regularities and their changing distribution across the blocks. Yet, they acquired the statistical regularities and showed significant statistical knowledge at the behavioral level not only in the predictable blocks but also in the unpredictable ones, albeit to a smaller extent. Brain activity in a range of cortical and subcortical areas, including early visual cortex, the insula, the right inferior frontal gyrus, and the right globus pallidus/putamen contributed to the acquisition of statistical regularities. The right insula, inferior frontal gyrus, and hippocampus as well as the bilateral angular gyrus seemed to play a role in maintaining this statistical knowledge. The results altogether suggest that statistical knowledge could be exploited in a relevant, predictable context as well as transmitted to and retrieved in an irrelevant context without a predictable structure.

Differential effects of bilateral hippocampal CA3 damage on the implicit learning and recognition of complex event sequences

Learning regularities in the environment is a fundament of human cognition, which is supported by a network of brain regions that include the hippocampus. In two experiments, we assessed the effects of selective bilateral damage to human hippocampal subregion CA3, which was associated with autobiographical episodic amnesia extending ~50 years prior to the damage, on the ability to recognize complex, deterministic event sequences presented either in a spatial or a non-spatial configuration. In contrast to findings from related paradigms, modalities, and homologue species, hippocampal damage did not preclude recognition memory for an event sequence studied and tested at four spatial locations, whereas recognition memory for an event sequence presented at a single location was at chance. In two additional experiments, recognition memory for novel single-items was intact, whereas the ability to recognize novel single-items in a different location from that presented at study was at chance. The results are at variance with a general role of the hippocampus in the learning and recognition of complex event sequences based on non-adjacent spatial and temporal dependencies. We discuss the impact of the results on established theoretical accounts of the hippocampal contributions to implicit sequence learning and episodic memory.

Location-specific deviant responses to object sequences in macaque inferior temporal cortex

Many species learn temporal regularities in their visual environment, demonstrating visual statistical learning. In this study, we explored the sensitivity of macaque inferior temporal (IT) cortical neurons to transition probabilities of sequentially presented visual images, presented at different locations in the visual field. We exposed monkeys to sequences of two images, where the first image was presented either foveally or peripherally, and the second image was consistently presented foveally. Following several weeks of exposure, we recorded IT responses to assess differences between the exposed (Fixed) and new, Deviant sequences, where the identity of the first image in a sequence differed from the exposure phase. While enhanced responses to Deviant sequences were observed when both images of a pair were foveally presented during exposure, no such deviant responses were present when the first image was presented peripherally. This finding challenges the notion that mere exposure to image sequences always leads to deviant responses in macaque IT. The results highlight the complexity of the mechanisms underlying statistical learning in primates, particularly in the context of peripheral image presentations, emphasizing the need for further investigation into the origins of these responses in the IT cortex.

Humans predict the forest, not the trees: statistical learning of spatiotemporal structure in visual scenes

The human brain is capable of using statistical regularities to predict future inputs. In the real world, such inputs typically comprise a collection of objects (e.g. a forest constitutes numerous trees). The present study aimed to investigate whether perceptual anticipation relies on lower-level or higher-level information. Specifically, we examined whether the human brain anticipates each object in a scene individually or anticipates the scene as a whole. To explore this issue, we first trained participants to associate co-occurring objects within fixed spatial arrangements. Meanwhile, participants implicitly learned temporal regularities between these displays. We then tested how spatial and temporal violations of the structure modulated behavior and neural activity in the visual system using fMRI. We found that participants only showed a behavioral advantage of temporal regularities when the displays conformed to their previously learned spatial structure, demonstrating that humans form configuration-specific temporal expectations instead of predicting individual objects. Similarly, we found suppression of neural responses for temporally expected compared with temporally unexpected objects in lateral occipital cortex only when the objects were embedded within expected configurations. Overall, our findings indicate that humans form expectations about object configurations, demonstrating the prioritization of higher-level over lower-level information in temporal expectation.

Inferior temporal cortex leads prefrontal cortex in response to a violation of a learned sequence

Primates learn statistical regularities that are embedded in visual sequences, a form of statistical learning. Single-unit recordings in macaques showed that inferior temporal (IT) neurons are sensitive to statistical regularities in visual sequences. Here, we asked whether ventrolateral prefrontal cortex (VLPFC), which is connected to IT, is also sensitive to the transition probabilities in visual sequences and whether the statistical learning signal in IT originates in VLPFC. We recorded simultaneously multiunit activity (MUA) and local field potentials (LFPs) in IT and VLPFC after monkeys were exposed to triplets of images with a fixed presentation order. In both areas, the MUA was stronger to images that violated the learned sequence (deviants) compared to the same images presented in the learned triplets. The high-gamma and beta LFP power showed an enhanced and suppressed response, respectively, to the deviants in both areas. The enhanced response was present also for the image following the deviant, suggesting a sensitivity for temporal adjacent dependencies in IT and VLPFC. The increased response to the deviant occurred later in VLPFC than in IT, suggesting that the deviant response in IT was not inherited from VLPFC. These data support predictive coding theories that propose a feedforward flow of prediction errors.

When the brain, but not the person, remembers: Cortical reinstatement is modulated by retrieval goal in developmental amnesia

Developmental amnesia (DA) is associated with early hippocampal damage and subsequent episodic amnesia emerging in childhood alongside age-appropriate development of semantic knowledge. We employed fMRI to assess whether patients with DA show evidence of 'cortical reinstatement', a neural correlate of episodic memory, despite their amnesia. At study, 23 participants (5 patients) were presented with words overlaid on a scene or a scrambled image for later recognition. Scene reinstatement was indexed by scene memory effects (greater activity for previously presented words paired with a scene rather than scrambled images) that overlapped with scene perception effects. Patients with DA demonstrated scene reinstatement effects in the parahippocampal and retrosplenial cortex that were equivalent to those shown by healthy controls. Behaviourally, however, patients with DA showed markedly impaired scene memory. The data indicate that reinstatement can occur despite hippocampal damage, but that cortical reinstatement is insufficient to support accurate memory performance. Furthermore, scene reinstatement effects were diminished during a retrieval task in which scene information was not relevant for accurate responding, indicating that strategic mnemonic processes operate normally in DA. The data suggest that cortical reinstatement of trial-specific contextual information is decoupled from the experience of recollection in the presence of severe hippocampal atrophy.

Awake suppression after brief exposure to a familiar stimulus

Newly learned information undergoes a process of awake reactivation shortly after the learning offset and we recently demonstrated that this effect can be observed as early as area V1. However, reactivating all experiences can be wasteful and unnecessary, especially for familiar stimuli. Therefore, here we tested whether awake reactivation occurs differentially for new and familiar stimuli. Subjects completed a brief visual task on a stimulus that was either novel or highly familiar due to extensive prior training on it. Replicating our previous results, we found that awake reactivation occurred in V1 for the novel stimulus. On the other hand, brief exposure to the familiar stimulus led to 'awake suppression' such that neural activity patterns immediately after exposure to the familiar stimulus diverged from the patterns associated with that stimulus. Further, awake reactivation was observed selectively in V1, whereas awake suppression had similar strength across areas V1-V3. These results are consistent with the presence of a competition between local awake reactivation and top-down awake suppression, with suppression becoming dominant for familiar stimuli.

Statistical Learning Signals for Complex Visual Images in Macaque Early Visual Cortex

Animals of several species, including primates, learn the statistical regularities of their environment. In particular, they learn the temporal regularities that occur in streams of visual images. Previous human neuroimaging studies reported discrepant effects of such statistical learning, ranging from stronger occipito-temporal activations for sequences in which image order was fixed, compared with sequences of randomly ordered images, to weaker activations for fixed-order sequences compared with sequences that violated the learned order. Several single-unit studies in macaque monkeys reported that after statistical learning of temporal regularities, inferior temporal (IT) neurons show reduced responses to learned fixed-order sequences of visual images compared with random or mispredicted sequences. However, it is unknown how other macaque brain areas respond to such temporal statistical regularities. To address this gap, we exposed rhesus monkeys (Macaca mulatta) to two types of sequences of complex images. The “regular” sequences consisted of a continuous stream of quartets, and within each quartet, the image order was fixed. The quartets themselves were displayed, uninterrupted, in a random order. The same monkeys were exposed to sequences of other images having a pseudorandomized order (“random” sequence). After exposure, both monkeys were scanned with functional MRI (fMRI) using a block design with three conditions: regular sequence, random sequence, and fixation-only blocks. A whole-brain analysis showed a reduced activation in mainly the occipito-temporal cortex for the regular compared to the random sequences. Marked response reductions for the regular sequence were observed in early extrastriate visual cortical areas, including area V2, despite the use of rather complex images of animals. These data suggest that statistical learning signals are already present in early visual areas of monkeys, even for complex visual images. These monkey fMRI data are in line with recent human fMRI studies that showed a reduced activation in early visual areas for predicted compared with mispredicted complex images.

A Novel Framework for Unconscious Processing

Understanding the distinction between conscious and unconscious cognition remains a priority in psychology and neuroscience. A comprehensive neurocognitive account of conscious awareness will not be possible without a sound framework to isolate and understand unconscious information processing. Here, we provide a brain-based framework that allows the identification of unconscious processes, even with null effects on behaviour.

Feature-Specific Awake Reactivation in Human V1 after Visual Training

Brain activity patterns exhibited during task performance have been shown to spontaneously reemerge in the following restful awake state. Such "awake reactivation" has been observed across higher-order cortex for complex images or associations. However, it is still unclear whether the reactivation extends to primary sensory areas that encode simple stimulus features. To address this question, we trained human subjects from both sexes on a particular visual feature (Gabor orientation) and tested whether this feature will be reactivated immediately after training. We found robust reactivation in human V1 that lasted for at least 8 min after training offset. This effect was not present in higher retinotopic areas, such as V2, V3, V3A, or V4v. Further analyses suggested that the amount of awake reactivation was related to the amount of performance improvement on the visual task. These results demonstrate that awake reactivation extends beyond higher-order areas and also occurs in early sensory cortex.SIGNIFICANCE STATEMENT How do we acquire new memories and skills? New information is known to be consolidated during offline periods of rest. Recent studies suggest that a critical process during this period of consolidation is the spontaneous reactivation of previously experienced patterns of neural activity. However, research in humans has mostly examined such reactivation processes in higher-order cortex. Here we show that awake reactivation occurs even in the primary visual cortex V1 and that this reactivation is related to the amount of behavioral learning. These results pinpoint awake reactivation as a process that likely occurs across the entire human brain and could play an integral role in memory consolidation.