Decoding stimulus identity in occipital, parietal and inferotemporal cortices during visual mental imagery

Immersive scene representation in human visual cortex with ultra-wide-angle neuroimaging

While human vision spans 220°, traditional functional MRI setups display images only up to central 10-15°. Thus, it remains unknown how the brain represents a scene perceived across the full visual field. Here, we introduce a method for ultra-wide angle display and probe signatures of immersive scene representation. An unobstructed view of 175° is achieved by bouncing the projected image off angled-mirrors onto a custom-built curved screen. To avoid perceptual distortion, scenes are created with wide field-of-view from custom virtual environments. We find that immersive scene representation drives medial cortex with far-peripheral preferences, but shows minimal modulation in classic scene regions. Further, scene and face-selective regions maintain their content preferences even with extreme far-periphery stimulation, highlighting that not all far-peripheral information is automatically integrated into scene regions computations. This work provides clarifying evidence on content vs. peripheral preferences in scene representation and opens new avenues to research immersive vision.

Uncovering the Role of the Early Visual Cortex in Visual Mental Imagery

The question of whether the early visual cortex (EVC) is involved in visual mental imagery remains a topic of debate. In this paper, I propose that the inconsistency in findings can be explained by the unique challenges associated with investigating EVC activity during imagery. During perception, the EVC processes low-level features, which means that activity is highly sensitive to variation in visual details. If the EVC has the same role during visual mental imagery, any change in the visual details of the mental image would lead to corresponding changes in EVC activity. Within this context, the question should not be whether the EVC is 'active' during imagery but how its activity relates to specific imagery properties. Studies using methods that are sensitive to variation in low-level features reveal that imagery can recruit the EVC in similar ways as perception. However, not all mental images contain a high level of visual details. Therefore, I end by considering a more nuanced view, which states that imagery can recruit the EVC, but that does not mean that it always does so.

Confidence ratings do not distinguish imagination from reality

Perceptual reality monitoring refers to the ability to distinguish internally triggered imagination from externally triggered reality. Such monitoring can take place at perceptual or cognitive levels-for example, in lucid dreaming, perceptual experience feels real but is accompanied by a cognitive insight that it is not real. We recently developed a paradigm to reveal perceptual reality monitoring errors during wakefulness in the general population, showing that imagined signals can be erroneously attributed to perception during a perceptual detection task. In the current study, we set out to investigate whether people have insight into perceptual reality monitoring errors by additionally measuring perceptual confidence. We used hierarchical Bayesian modeling of confidence criteria to characterize metacognitive insight into the effects of imagery on detection. Over two experiments, we found that confidence criteria moved in tandem with the decision criterion shift, indicating a failure of reality monitoring not only at a perceptual but also at a metacognitive level. These results further show that such failures have a perceptual rather than a decisional origin. Interestingly, offline queries at the end of the experiment revealed global, task-level insight, which was uncorrelated with local, trial-level insight as measured with confidence ratings. Taken together, our results demonstrate that confidence ratings do not distinguish imagination from reality during perceptual detection. Future research should further explore the different cognitive dimensions of insight into reality judgments and how they are related.

Immersive scene representation in human visual cortex with ultra-wide angle neuroimaging

While humans experience the visual environment in a panoramic 220° view, traditional functional MRI setups are limited to display images like postcards in the central 10-15° of the visual field. Thus, it remains unknown how a scene is represented in the brain when perceived across the full visual field. Here, we developed a novel method for ultra-wide angle visual presentation and probed for signatures of immersive scene representation. To accomplish this, we bounced the projected image off angled-mirrors directly onto a custom-built curved screen, creating an unobstructed view of 175°. Scene images were created from custom-built virtual environments with a compatible wide field-of-view to avoid perceptual distortion. We found that immersive scene representation drives medial cortex with far-peripheral preferences, but surprisingly had little effect on classic scene regions. That is, scene regions showed relatively minimal modulation over dramatic changes of visual size. Further, we found that scene and face-selective regions maintain their content preferences even under conditions of central scotoma, when only the extreme far-peripheral visual field is stimulated. These results highlight that not all far-peripheral information is automatically integrated into the computations of scene regions, and that there are routes to high-level visual areas that do not require direct stimulation of the central visual field. Broadly, this work provides new clarifying evidence on content vs. peripheral preferences in scene representation, and opens new neuroimaging research avenues to understand immersive visual representation.

Working memory signals in early visual cortex are present in weak and strong imagers

It has been suggested that visual images are memorized across brief periods of time by vividly imagining them as if they were still there. In line with this, the contents of both working memory and visual imagery are known to be encoded already in early visual cortex. If these signals in early visual areas were indeed to reflect a combined imagery and memory code, one would predict them to be weaker for individuals with reduced visual imagery vividness. Here, we systematically investigated this question in two groups of participants. Strong and weak imagers were asked to remember images across brief delay periods. We were able to reliably reconstruct the memorized stimuli from early visual cortex during the delay. Importantly, in contrast to the prediction, the quality of reconstruction was equally accurate for both strong and weak imagers. The decodable information also closely reflected behavioral precision in both groups, suggesting it could contribute to behavioral performance, even in the extreme case of completely aphantasic individuals. Our data thus suggest that working memory signals in early visual cortex can be present even in the (near) absence of phenomenal imagery.

Spatiotemporal dynamics of self-generated imagery reveal a reverse cortical hierarchy from cue-induced imagery

Visual imagery allows for the construction of rich internal experience in our mental world. However, it has remained poorly understood how imagery experience derives volitionally as opposed to being cue driven. Here, using electroencephalography and functional magnetic resonance imaging, we systematically investigate the spatiotemporal dynamics of self-generated imagery by having participants volitionally imagining one of the orientations from a learned pool. We contrast self-generated imagery with cue-induced imagery, where participants imagined line orientations based on associative cues acquired previously. Our results reveal overlapping neural signatures of cue-induced and self-generated imagery. Yet, these neural signatures display substantially differential sensitivities to the two types of imagery: self-generated imagery is supported by an enhanced involvement of the anterior cortex in representing imagery contents. By contrast, cue-induced imagery is supported by enhanced imagery representations in the posterior visual cortex. These results jointly support a reverse cortical hierarchy in generating and maintaining imagery contents in self-generated versus externally cued imagery.

Visual imagery deficits in posterior cortical atrophy

Visual imagery has a close overlapping relationship with visual perception. Posterior cortical atrophy (PCA) is a neurodegenerative syndrome marked by early impairments in visuospatial processing and visual object recognition. We asked whether PCA would therefore also be marked by deficits in visual imagery, tested using objective forced-choice questionnaires, and whether imagery deficits would be selective for certain properties. We recruited four patients with PCA and a patient with integrative visual agnosia due to bilateral occipitotemporal strokes for comparison. We administered a test battery probing imagery for object shape, size, colour lightness, hue, upper-case letters, lower-case letters, word shape, letter construction, and faces. All subjects showed significant impairments in visual imagery, with imagery for lower-case letters most likely to be spared. We conclude that PCA subjects can show severe deficits in visual imagery. Further work is needed to establish how frequently this occurs and how early it can be found.

Neural Representations in Visual and Parietal Cortex Differentiate between Imagined, Perceived, and Illusory Experiences

Humans constantly receive massive amounts of information, both perceived from the external environment and imagined from the internal world. To function properly, the brain needs to correctly identify the origin of information being processed. Recent work has suggested common neural substrates for perception and imagery. However, it has remained unclear how the brain differentiates between external and internal experiences with shared neural codes. Here we tested this question in human participants (male and female) by systematically investigating the neural processes underlying the generation and maintenance of visual information from voluntary imagery, veridical perception, and illusion. The inclusion of illusion allowed us to differentiate between objective and subjective internality: while illusion has an objectively internal origin and can be viewed as involuntary imagery, it is also subjectively perceived as having an external origin like perception. Combining fMRI, eye-tracking, multivariate decoding, and encoding approaches, we observed superior orientation representations in parietal cortex during imagery compared with perception, and conversely in early visual cortex. This imagery dominance gradually developed along a posterior-to-anterior cortical hierarchy from early visual to parietal cortex, emerged in the early epoch of imagery and sustained into the delay epoch, and persisted across varied imagined contents. Moreover, representational strength of illusion was more comparable to imagery in early visual cortex, but more comparable to perception in parietal cortex, suggesting content-specific representations in parietal cortex differentiate between subjectively internal and external experiences, as opposed to early visual cortex. These findings together support a domain-general engagement of parietal cortex in internally generated experience.SIGNIFICANCE STATEMENT How does the brain differentiate between imagined and perceived experiences? Combining fMRI, eye-tracking, multivariate decoding, and encoding approaches, the current study revealed enhanced stimulus-specific representations in visual imagery originating from parietal cortex, supporting the subjective experience of imagery. This neural principle was further validated by evidence from visual illusion, wherein illusion resembled perception and imagery at different levels of cortical hierarchy. Our findings provide direct evidence for the critical role of parietal cortex as a domain-general region for content-specific imagery, and offer new insights into the neural mechanisms underlying the differentiation between subjectively internal and external experiences.

Visual Representations: Insights from Neural Decoding

Patterns of brain activity contain meaningful information about the perceived world. Recent decades have welcomed a new era in neural analyses, with computational techniques from machine learning applied to neural data to decode information represented in the brain. In this article, we review how decoding approaches have advanced our understanding of visual representations and discuss efforts to characterize both the complexity and the behavioral relevance of these representations. We outline the current consensus regarding the spatiotemporal structure of visual representations and review recent findings that suggest that visual representations are at once robust to perturbations, yet sensitive to different mental states. Beyond representations of the physical world, recent decoding work has shone a light on how the brain instantiates internally generated states, for example, during imagery and prediction. Going forward, decoding has remarkable potential to assess the functional relevance of visual representations for human behavior, reveal how representations change across development and during aging, and uncover their presentation in various mental disorders.

Connectivity reveals homology between the visual systems of the human and macaque brains

The visual systems of humans and nonhuman primates share many similarities in both anatomical and functional organization. Understanding the homology and differences between the two systems can provide important insights into the neural basis of visual perception and cognition. This research aims to investigate the homology between human and macaque visual systems based on connectivity, using diffusion tensor imaging and resting-state functional magnetic resonance imaging to construct structural and functional connectivity fingerprints of the visual systems in humans and macaques, and quantitatively analyze the connectivity patterns. By integrating multimodal magnetic resonance imaging, this research explored the homology and differences between the two systems. The results showed that 9 brain regions in the macaque visual system formed highly homologous mapping relationships with 11 brain regions in the human visual system, and the related brain regions between the two species showed highly structure homologous, with their functional organization being essentially conserved across species. Finally, this research generated a homology information map of the visual system for humans and macaques, providing a new perspective for subsequent cross-species analysis.

Neuropsychological Activations and Networks While Performing Visual and Kinesthetic Motor Imagery

This study aimed to answer the questions 'What are the neural networks and mechanisms involved in visual and kinesthetic motor imagery?', and 'Is part of cognitive processing included during visual and kinesthetic motor imagery?' by investigating the neurophysiological networks and activations during visual and kinesthetic motor imagery using motor imagery tasks (golf putting). The experiment was conducted with 19 healthy adults. Functional magnetic resonance imaging (fMRI) was used to examine neural activations and networks during visual and kinesthetic motor imagery using golf putting tasks. The findings of the analysis on cerebral activation patterns based on the two distinct types of motor imagery indicate that the posterior lobe, occipital lobe, and limbic lobe exhibited activation, and the right hemisphere was activated during the process of visual motor imagery. The activation of the temporal lobe and the parietal lobe were observed during the process of kinesthetic motor imagery. This study revealed that visual motor imagery elicited stronger activation in the right frontal lobe, whereas kinesthetic motor imagery resulted in greater activation in the left frontal lobe. It seems that kinesthetic motor imagery activates the primary somatosensory cortex (BA 2), the secondary somatosensory cortex (BA 5 and 7), and the temporal lobe areas and induces human sensibility. The present investigation evinced that the neural network and the regions of the brain that are activated exhibit variability contingent on the category of motor imagery.

Default mode network mechanisms of repeated transcranial magnetic stimulation in heroin addiction

Repetitive transcranial magnetic stimulation (rTMS) over the left dorsolateral prefrontal cortex (DLPFC) has been shown to reduce cravings in heroin-dependent (HD) individuals, but the mechanisms underlying the anti-craving effects of rTMS are unknown. Abnormalities in the default mode network (DMN) are known to be consistent findings in HD individuals and are involved in cravings. We assessed the effect of rTMS on DMN activity and its relationship to the treatment response. Thirty HD individuals were included in this self-controlled study, and all HD participants received 10-Hz rTMS 7-session during a week. Data for cravings and withdrawal symptoms and resting-state functional magnetic resonance imaging data were collected before and after rTMS treatment. Thirty demographically matched healthy individuals who did not receive rTMS were included as controls. We focused on changes in coupling seeded from the medial prefrontal cortex (MPFC), posterior cingulate cortex (PCC), and bilateral inferior parietal lobe (IPL), which are the core regions of the DMN. The craving and withdrawal symptom score of HD individuals decreased significantly after rTMS treatment. The left IPL-left middle frontal gyrus coupling and the left IPL-right inferior occipital gyrus coupling decreased significantly, and the changes in the left IPL-left middle frontal gyrus coupling were positively correlated with changes in drug-cue induced cravings. rTMS could modulate the coupling between the DMN and executive control network (ECN). Alterations of the left IPL-left middle frontal gyrus coupling may play an important mechanistic role in reducing drug cue-induced cravings.

Semantic fMRI neurofeedback: A Multi-Subject Study at 3 Tesla

OBJECTIVE

Real-time fMRI neurofeedback is a non-invasive procedure allowing the self-regulation of brain functions via enhanced self-control of fMRI based neural activation. In semantic real-time fMRI neurofeedback, an estimated relation between multivariate fMRI activation patterns and abstract mental states is exploited for a multi-dimensional feedback stimulus via real-time representational similarity analysis (rt-RSA). Here, we assessed the performances of this framework in a multi-subject multi-session study on a 3T MRI clinical scanner.

APPROACH

Eighteen healthy volunteers underwent two semantic real-time fMRI neurofeedback sessions on two different days. In each session, participants were first requested to engage in specific mental states while local fMRI patterns of brain activity were recorded during stimulated mental imagery of concrete objects (pattern generation). The obtained neural representations were to be replicated and modulated by the participants in subsequent runs of the same session under the guidance of a rt-RSA generated visual feedback (pattern modulation). Performance indicators were derived from the rt-RSA output to assess individual abilities in replicating (and maintaining over time) a target pattern. Simulations were carried out to assess the impact of the geometric distortions implied by the low-dimensional representation of patterns' dissimilarities in the visual feedback.

MAIN RESULTS

Sixteen subjects successfully completed both semantic real-time fMRI neurofeedback sessions. Considering some performance indicators, a significant improvement between the first and the second runs, and within run increasing modulation performances were observed, whereas no improvements were found between sessions. Simulations confirmed that in a small percentage of cases visual feedback could be affected by metric distortions due to dimensionality reduction implicit to the rt-RSA approach.

SIGNIFICANCE

Our results proved the feasibility of the semantic real-time fMRI neurofeedback at 3T, showing that subjects can successfully modulate and maintain a target mental state, guided by rt-RSA derived feedback. Further development is needed to encourage future clinical applications.

Confidence modulates the decodability of scene prediction during partially-observable maze exploration in humans

Prediction ability often involves some degree of uncertainty-a key determinant of confidence. Here, we sought to assess whether predictions are decodable in partially-observable environments where one's state is uncertain, and whether this information is sensitive to confidence produced by such uncertainty. We used functional magnetic resonance imaging-based, partially-observable maze navigation tasks in which subjects predicted upcoming scenes and reported their confidence regarding these predictions. Using a multi-voxel pattern analysis, we successfully decoded both scene predictions and subjective confidence from activities in the localized parietal and prefrontal regions. We also assessed confidence in their beliefs about where they were in the maze. Importantly, prediction decodability varied according to subjective scene confidence in the superior parietal lobule and state confidence estimated by the behavioral model in the inferior parietal lobule. These results demonstrate that prediction in uncertain environments depends on the prefrontal-parietal network within which prediction and confidence interact.

Schema representations in distinct brain networks support narrative memory during encoding and retrieval

Schematic prior knowledge can scaffold the construction of event memories during perception and also provide structured cues to guide memory search during retrieval. We measured the activation of story-specific and schematic representations using fMRI while participants were presented with 16 stories and then recalled each of the narratives, and related these activations to memory for specific story details. We predicted that schema representations in medial prefrontal cortex (mPFC) would be correlated with successful recall of story details. In keeping with this prediction, an anterior mPFC region showed a significant correlation between activation of schema representations at encoding and subsequent behavioral recall performance; however, this mPFC region was not implicated in schema representation during retrieval. More generally, our analyses revealed largely distinct brain networks at encoding and retrieval in which schema activation was related to successful recall. These results provide new insight into when and where event knowledge can support narrative memory.

Flights and Perchings of the BrainMind: A Temporospatial Approach to Psychotherapy

This article introduces a process-oriented approach for improving present moment conceptualization in psychotherapy that is in alignment with neuroscience: the Temporospatial movements of mind (TSMM) model. We elaborate on seven temporal movements that describe the moment-to-moment morphogenesis of emotional feelings and thoughts from inception to maturity. Temporal refers to the passage of time through which feelings and thoughts develop, and electromagnetic activity, that among other responsibilities, bind information across time. Spatial dynamics extend from an undifferentiated to three dimensional experiences of emotional and cognitive processes. Neurophysiologically, spatial refers to structures within the brain and their varying interactions with one another. This article culminates in the development of an atheoretical temporospatial grid that may help clinicians conceptualize where patients are in their cognitive and emotional development to further guide technique.

Imagery adds stimulus-specific sensory evidence to perceptual detection

Internally generated imagery and externally triggered perception rely on overlapping sensory processes. This overlap poses a challenge for perceptual reality monitoring: determining whether sensory signals reflect reality or imagination. In this study, we used psychophysics to investigate how imagery and perception interact to determine visual experience. Participants were instructed to detect oriented gratings that gradually appeared in noise while simultaneously either imagining the same grating, a grating perpendicular to the to-be-detected grating, or nothing. We found that, compared to both incongruent imagery and no imagery, congruent imagery caused a leftward shift of the psychometric function relating stimulus contrast to perceptual threshold. We discuss how this effect can best be explained by a model in which imagery adds sensory signal to the perceptual input, thereby increasing the visibility of perceived stimuli. These results suggest that, in contrast to changes in sensory signals caused by self-generated movement, the brain does not discount the influence of self-generated sensory signals on perception.

Decoding category and familiarity information during visual imagery

Visual imagery relies on a widespread network of brain regions, partly engaged during the perception of external stimuli. Beyond the recruitment of category-selective areas (FFA, PPA), perception of familiar faces and places has been reported to engage brain areas associated with semantic information, comprising the precuneus, temporo-parietal junction (TPJ), medial prefrontal cortex (mPFC) and posterior cingulate cortex (PCC). Here we used multivariate pattern analyzes (MVPA) to examine to which degree areas of the visual imagery network, category-selective and semantic areas contain information regarding the category and familiarity of imagined stimuli. Participants were instructed via auditory cues to imagine personally familiar and unfamiliar stimuli (i.e. faces and places). Using region-of-interest (ROI)-based MVPA, we were able to distinguish between imagined faces and places within nodes of the visual imagery network (V1, SPL, aIPS), within category-selective inferotemporal regions (FFA, PPA) and across all brain regions of the extended semantic network (i.e. precuneus, mPFC, IFG and TPJ). Moreover, we were able to decode familiarity of imagined stimuli in the SPL and aIPS, and in some regions of the extended semantic network (in particular, right precuneus, right TPJ), but not in V1. Our results suggest that posterior visual areas - including V1 - host categorical representations about imagined stimuli, and that stimulus familiarity might be an additional aspect that is shared between perception and visual imagery.

A Study of the Brain Network Connectivity in Visual-Word Pairing Associative Learning and Episodic Memory Reactivating Task

Episodic memory allows a person to recall and mentally reexperience specific episodes from one's personal past. Studies of episodic memory are of great significance for the diagnosis and the exploration of the mechanism of memory generation. Most of the current studies focus on certain brain regions and pay less attention to the interrelationship between multiple brain regions. To explore the interrelationship in the brain network, we use an open fMRI dataset to construct the brain functional connectivity and effective connectivity network. We establish a binary directed network of the memory when it is reactivated. The binary directed network shows that the occipital lobe and parietal lobe have the most causal connections. The number of edges starting from the superior parietal lobule is the highest, with 49 edges, and 31 of which are connected to the occipital cortex. This means that the interaction between the superior parietal lobule and the occipital lobe plays the most important role in episodic memory, and the superior parietal lobule plays a more causal role in causality. In addition, memory regions such as the precuneus and fusiform also have some edges. The results show that the posterior parietal cortex plays an important role of hub node in the episodic memory network. From the brain network model, more information can be obtained, which is conducive to exploring the brain's changing pattern in the whole memory process.