Semantic and emotional content of imagined representations in human occipitotemporal cortex

Occipital-temporal cortical tuning to semantic and affective features of natural images predicts associated behavioral responses

In everyday life, people need to respond appropriately to many types of emotional stimuli. Here, we investigate whether human occipital-temporal cortex (OTC) shows co-representation of the semantic category and affective content of visual stimuli. We also explore whether OTC transformation of semantic and affective features extracts information of value for guiding behavior. Participants viewed 1620 emotional natural images while functional magnetic resonance imaging data were acquired. Using voxel-wise modeling we show widespread tuning to semantic and affective image features across OTC. The top three principal components underlying OTC voxel-wise responses to image features encoded stimulus animacy, stimulus arousal and interactions of animacy with stimulus valence and arousal. At low to moderate dimensionality, OTC tuning patterns predicted behavioral responses linked to each image better than regressors directly based on image features. This is consistent with OTC representing stimulus semantic category and affective content in a manner suited to guiding behavior.

The Action Cycle Theory of Perception and Mental Imagery

The Action Cycle Theory (ACT) is an enactive theory of the perception and a mental imagery system that is comprised of six modules: Schemata, Objects, Actions, Affect, Goals and Others' Behavior. The evidence supporting these six connected modules is reviewed in light of research on mental imagery vividness. The six modules and their interconnections receive empirical support from a wide range of studies. All six modules of perception and mental imagery are influenced by individual differences in vividness. Real-world applications of ACT show interesting potential to improve human wellbeing in both healthy people and patients. Mental imagery can be applied in creative ways to make new collective goals and actions for change that are necessary to maximize the future prospects of the planet.

Voluntary control of semantic neural representations by imagery with conflicting visual stimulation

Neural representations of visual perception are affected by mental imagery and attention. Although attention is known to modulate neural representations, it is unknown how imagery changes neural representations when imagined and perceived images semantically conflict. We hypothesized that imagining an image would activate a neural representation during its perception even while watching a conflicting image. To test this hypothesis, we developed a closed-loop system to show images inferred from electrocorticograms using a visual semantic space. The successful control of the feedback images demonstrated that the semantic vector inferred from electrocorticograms became closer to the vector of the imagined category, even while watching images from different categories. Moreover, modulation of the inferred vectors by mental imagery depended asymmetrically on the perceived and imagined categories. Shared neural representation between mental imagery and perception was still activated by the imagery under semantically conflicting perceptions depending on the semantic category.

In this study, intracranial EEG recordings show that neural representations of imagined images can still be present in humans even when they are shown conflicting images.

Fear of the known: semantic generalisation of fear conditioning across languages in bilinguals

While modern theories of emotion emphasize the role of higher-order cognitive processes such as semantics in human emotion, much research into emotional learning has ignored the potential contributions of such processes. This study aimed to determine whether emotional learning affects semantic representations of words independent of perceptual features by assessing whether fear conditioning to a neutral word generalises across languages in bilingual participants. Two sessions differing according to the reinforced language were performed by English-Spanish bilinguals. In each session, a neutral word was reinforced by an electrical shock whereas its equivalent in the other language was never paired with shock. Across two sessions within our sample, we found replicable evidence that fear conditioning consistently transferred to the non-reinforced language as measured by both self-reported fear and electrodermal activity, irrespective of the conditioned language. Our findings extend knowledge about the role of semantic similarity in fear generalisation and highlight the importance of higher-order cognitive processes in human emotions.

Memories Fade: The Relationship Between Memory Vividness and Remembered Visual Salience

Past events, particularly emotional experiences, are often vividly recollected. However, it remains unclear how qualitative information, such as low-level visual salience, is reconstructed and how the precision and bias of this information relate to subjective memory vividness. Here, we tested whether remembered visual salience contributes to vivid recollection. In three experiments, participants studied emotionally negative and neutral images that varied in luminance and color saturation, and they reconstructed the visual salience of each image in a subsequent test. Results revealed, unexpectedly, that memories were recollected as less visually salient than they were encoded, demonstrating a novel memory-fading effect, whereas negative emotion increased subjective memory vividness and the precision with which visual features were encoded. Finally, memory vividness tracked both the precision and remembered salience (bias) of visual information. These findings provide evidence that low-level visual information fades in memory and contributes to the experience of vivid recollection.

Visual imagery of faces and cars in face-selective visual areas

Neuroimaging provides a unique tool to investigate otherwise difficult-to-access mental processes like visual imagery. Prior studies support the idea that visual imagery is a top-down reinstatement of visual perception, and it is likely that this extends to object processing. Here we use functional MRI and multi-voxel pattern analysis to ask if mental imagery of cars engages the fusiform face area, similar to what is found during perception. We test only individuals who we assumed could imagine individual car models based on their above-average perceptual abilities with cars. Our results provide evidence that cars are represented differently from common objects in face-selective visual areas, at least in those with above-average car recognition ability. Moreover, pattern classifiers trained on data acquired during imagery can decode the neural response pattern acquired during perception, suggesting that the tested object categories are represented similarly during perception and visual imagery. The results suggest that, even at high-levels of visual processing, visual imagery mirrors perception to some extent, and that face-selective areas may in part support non-face object imagery.

Animacy and real-world size shape object representations in the human medial temporal lobes

Identifying what an object is, and whether an object has been encountered before, is a crucial aspect of human behavior. Despite this importance, we do not yet have a complete understanding of the neural basis of these abilities. Investigations into the neural organization of human object representations have revealed category specific organization in the ventral visual stream in perceptual tasks. Interestingly, these categories fall within broader domains of organization, with reported distinctions between animate, inanimate large, and inanimate small objects. While there is some evidence for category specific effects in the medial temporal lobe (MTL), in particular in perirhinal and parahippocampal cortex, it is currently unclear whether domain level organization is also present across these structures. To this end, we used fMRI with a continuous recognition memory task. Stimuli were images of objects from several different categories, which were either animate or inanimate, or large or small within the inanimate domain. We employed representational similarity analysis (RSA) to test the hypothesis that object-evoked responses in MTL structures during recognition-memory judgments also show evidence for domain-level organization along both dimensions. Our data support this hypothesis. Specifically, object representations were shaped by either animacy, real-world size, or both, in perirhinal and parahippocampal cortex, and the hippocampus. While sensitivity to these dimensions differed across structures when probed individually, hinting at interesting links to functional differentiation, similarities in organization across MTL structures were more prominent overall. These results argue for continuity in the organization of object representations in the ventral visual stream and the MTL.

The Involvement of Endogenous Neural Oscillations in the Processing of Rhythmic Input: More Than a Regular Repetition of Evoked Neural Responses

It is undisputed that presenting a rhythmic stimulus leads to a measurable brain response that follows the rhythmic structure of this stimulus. What is still debated, however, is the question whether this brain response exclusively reflects a regular repetition of evoked responses, or whether it also includes entrained oscillatory activity. Here we systematically present evidence in favor of an involvement of entrained neural oscillations in the processing of rhythmic input while critically pointing out which questions still need to be addressed before this evidence could be considered conclusive. In this context, we also explicitly discuss the potential functional role of such entrained oscillations, suggesting that these stimulus-aligned oscillations reflect, and serve as, predictive processes, an idea often only implicitly assumed in the literature.

The mental simulation of state/psychological verbs in the adolescent brain: An fMRI study

This fMRI study investigated mental simulation of state/psychological and action verbs during adolescence. Sixteen healthy subjects silently read verbs describing a motor scene or not (STIMULUS: motor, state/psychological verbs) and they were explicitly asked to imagine the situation or they performed letter detection preventing them from using simulation (TASK: imagery vs. letter detection). A significant task by stimuli interaction showed that imagery of state/psychological verbs, as compared to action stimuli (controlled by the letter detection) selectively increased activation in the right supramarginal gyrus/rolandic operculum and in the right insula, and decreased activation in the right intraparietal sulcus. We compared these data to those from a group of older participants (Tomasino et al. 2014a). Activation in the left supramarginal gyrus decreased for the latter group (as compared to the present group) for imagery of state/psychological verbs. By contrast, activation in the right superior frontal gyrus decreased for the former group (as compared to the older group) for imagery of state/psychological verbs.

Individual differences shape the content of visual representations

Visually perceiving a stimulus activates a pictorial representation of that item in the brain, but how pictorial is the representation of a stimulus in the absence of visual stimulation? Here I address this question with a review of the literatures on visual imagery (VI), visual working memory (VWM), and visual preparatory templates, all of which require activating visual information in the absence of sensory stimulation. These processes have historically been studied separately, but I propose that they can provide complimentary evidence for the pictorial nature of their contents. One major challenge in studying the contents of visual representations is the discrepant findings concerning the extent of overlap (both cortical and behavioral) between externally and internally sourced visual representations. I argue that these discrepancies may in large part be due to individual differences in VI vividness and precision, the specific representative abilities required to perform a task, appropriateness of visual preparatory strategies, visual cortex anatomy, and level of expertise with a particular object category. Individual differences in visual representative abilities greatly impact task performance and may influence the likelihood of experiences such as intrusive VI and hallucinations, but research still predominantly focuses on uniformities in visual experience across individuals. In this paper I review the evidence for the pictorial content of visual representations activated for VI, VWM, and preparatory templates, and highlight the importance of accounting for various individual differences in conducting research on this topic.