From Architecture to Evolution: Multisensory Evidence of Decentralized Emotion

An extremely fast neural mechanism to detect emotional visual stimuli: A two-experiment study

Defining the brain mechanisms underlying initial emotional evaluation is a key but unexplored clue to understanding affective processing. Event-related potentials (ERPs), especially suited for investigating this issue, were recorded in two experiments (n = 36 and n = 35). We presented emotionally negative (spiders) and neutral (wheels) silhouettes homogenized regarding their visual parameters. In Experiment 1, stimuli appeared at fixation or in the periphery (200 trials per condition and location), the former eliciting a N40 (39 milliseconds) and a P80 (or C1: 80 milliseconds) component, and the latter only a P80. In Experiment 2, stimuli were presented only at fixation (500 trials per condition). Again, an N40 (45 milliseconds) was observed, followed by a P100 (or P1: 105 milliseconds). Analyses revealed significantly greater N40-C1P1 peak-to-peak amplitudes for spiders in both experiments, and ANCOVAs showed that these effects were not explained by C1P1 alone, but that processes underlying N40 significantly contributed. Source analyses pointed to V1 as an N40 focus (more clearly in Experiment 2). Sources for C1P1 included V1 (P80) and V2/LOC (P80 and P100). These results and their timing point to low-order structures (such as visual thalamic nuclei or superior colliculi) or the visual cortex itself, as candidates for initial evaluation structures.

Characterizing affiliative touch in humans and its role in advancing haptic design

An emerging view in cognitive neuroscience holds that the extraction of emotional relevance from sensory experience extends beyond the centralized appraisal of sensation in associative brain regions, including frontal and medial-temporal cortices. This view holds that sensory information can be emotionally valenced from the point of contact with the world. This view is supported by recent research characterizing the human affiliative touch system, which carries signals of soft, stroking touch to the central nervous system and is mediated by dedicated C-tactile afferent receptors. This basic scientific research on the human affiliative touch system is informed by, and informs, technology design for communicating and regulating emotion through touch. Here, we review recent research on the basic biology and cognitive neuroscience of affiliative touch, its regulatory effects across the lifespan, and the factors that modulate it. We further review recent work on the design of haptic technologies, devices that stimulate the affiliative touch system, such as wearable technologies that apply the sensation of soft stroking or other skin-to-skin contact, to promote physiological regulation. We then point to future directions in interdisciplinary research aimed at both furthering scientific understanding and application of haptic technology for health and wellbeing.

Dissociating representations of affect and motion in visual cortices

While a delicious dessert being presented to us may elicit strong feelings of happiness and excitement, the same treat falling slowly away can lead to sadness and disappointment. Our emotional response to the item depends on its visual motion direction. Despite this importance, it remains unclear whether (and how) cortical areas devoted to decoding motion direction represents or integrates emotion with perceived motion direction. Motion-selective visual area V5/MT+ sits, both functionally and anatomically, at the nexus of dorsal and ventral visual streams. These pathways, however, differ in how they are modulated by emotional cues. The current study was designed to disentangle how emotion and motion perception interact, as well as use emotion-dependent modulation of visual cortices to understand the relation of V5/MT+ to canonical processing streams. During functional magnetic resonance imaging (fMRI), approaching, receding, or static motion after-effects (MAEs) were induced on stationary positive, negative, and neutral stimuli. An independent localizer scan was conducted to identify the visual-motion area V5/MT+. Through univariate and multivariate analyses, we demonstrated that emotion representations in V5/MT+ share a more similar response profile to that observed in ventral visual than dorsal, visual structures. Specifically, V5/MT+ and ventral structures were sensitive to the emotional content of visual stimuli, whereas dorsal visual structures were not. Overall, this work highlights the critical role of V5/MT+ in the representation and processing of visually acquired emotional content. It further suggests a role for this region in utilizing affectively salient visual information to augment motion perception of biologically relevant stimuli.

Sensing fear: fast and precise threat evaluation in human sensory cortex

Animal models of threat processing have evolved beyond the amygdala to incorporate a distributed neural network. In human research, evidence has intensified in recent years to challenge the canonical threat circuitry centered on the amygdala, urging revision of threat conceptualization. A strong surge of research into threat processing in the sensory cortex in the past decade has generated particularly useful insights to inform the reconceptualization. Here, synthesizing findings from both animal and human research, we highlight sensitive, specific, and adaptable threat representations in the sensory cortex, arising from experience-based sculpting of sensory coding networks. We thus propose that the human sensory cortex can drive smart (fast and precise) threat evaluation, producing threat-imbued sensory afferents to elicit network-wide threat responses.

Holistic processing and visual characteristics of regulated and spontaneous expressions

The rapid and efficient recognition of facial expressions is crucial for adaptive behaviors, and holistic processing is one of the critical processing methods to achieve this adaptation. Therefore, this study integrated the effects and attentional characteristics of the authenticity of facial expressions on holistic processing. The results show that both regulated and spontaneous expressions were processed holistically. However, the spontaneous expression details did not indicate typical holistic processing, with the congruency effect observed equally for aligned and misaligned conditions. No significant difference between the two expressions was observed in terms of reaction times and eye movement characteristics (i.e., total fixation duration, fixation counts, and first fixation duration). These findings suggest that holistic processing strategies differ between the two expressions. Nevertheless, the difference was not reflected in attentional engagement.

Decomposing Neural Representational Patterns of Discriminatory and Hedonic Information during Somatosensory Stimulation

The ability to interrogate specific representations in the brain, determining how, and where, difference sources of information are instantiated can provide invaluable insight into neural functioning. Pattern component modeling (PCM) is a recent analytic technique for human neuroimaging that allows the decomposition of representational patterns in brain into contributing subcomponents. In the current study, we present a novel PCM variant that tracks the contribution of prespecified representational patterns to brain representation across areas, thus allowing hypothesis-guided employment of the technique. We apply this technique to investigate the contributions of hedonic and nonhedonic information to the neural representation of tactile experience. We applied aversive pressure (AP) and appetitive brush (AB) to stimulate distinct peripheral nerve pathways for tactile information (C-/CT-fibers, respectively) while patients underwent functional magnetic resonance imaging (fMRI) scanning. We performed representational similarity analyses (RSAs) with pattern component modeling to dissociate how discriminatory versus hedonic tactile information contributes to population code representations in the human brain. Results demonstrated that information about appetitive and aversive tactile sensation is represented separately from nonhedonic tactile information across cortical structures. This also demonstrates the potential of new hypothesis-guided PCM variants to help delineate how information is instantiated in the brain.

A deep neural network model of the primate superior colliculus for emotion recognition

Although sensory processing is pivotal to nearly every theory of emotion, the evaluation of the visual input as ‘emotional’ (e.g. a smile as signalling happiness) has been traditionally assumed to take place in supramodal ‘limbic’ brain regions. Accordingly, subcortical structures of ancient evolutionary origin that receive direct input from the retina, such as the superior colliculus (SC), are traditionally conceptualized as passive relay centres. However, mounting evidence suggests that the SC is endowed with the necessary infrastructure and computational capabilities for the innate recognition and initial categorization of emotionally salient features from retinal information. Here, we built a neurobiologically inspired convolutional deep neural network (DNN) model that approximates physiological, anatomical and connectional properties of the retino-collicular circuit. This enabled us to characterize and isolate the initial computations and discriminations that the DNN model of the SC can perform on facial expressions, based uniquely on the information it directly receives from the virtual retina. Trained to discriminate facial expressions of basic emotions, our model matches human error patterns and above chance, yet suboptimal, classification accuracy analogous to that reported in patients with V1 damage, who rely on retino-collicular pathways for non-conscious vision of emotional attributes. When presented with gratings of different spatial frequencies and orientations never ‘seen’ before, the SC model exhibits spontaneous tuning to low spatial frequencies and reduced orientation discrimination, as can be expected from the prevalence of the magnocellular (M) over parvocellular (P) projections. Likewise, face manipulation that biases processing towards the M or P pathway affects expression recognition in the SC model accordingly, an effect that dovetails with variations of activity in the human SC purposely measured with ultra-high field functional magnetic resonance imaging. Lastly, the DNN generates saliency maps and extracts visual features, demonstrating that certain face parts, like the mouth or the eyes, provide higher discriminative information than other parts as a function of emotional expressions like happiness and sadness. The present findings support the contention that the SC possesses the necessary infrastructure to analyse the visual features that define facial emotional stimuli also without additional processing stages in the visual cortex or in ‘limbic’ areas.

This article is part of the theme issue ‘Cracking the laugh code: laughter through the lens of biology, psychology and neuroscience’.

Lateralization of autonomic output in response to limb-specific threat

In times of stress or danger, the autonomic nervous system (ANS) signals the fight or flight response. A canonical function of ANS activity is to globally mobilize metabolic resources, preparing the organism to respond to threat. Yet a body of research has demonstrated that, rather than displaying a homogenous pattern across the body, autonomic responses to arousing events - as measured through changes in electrodermal activity (EDA) - can differ between right and left body locations. Surprisingly, an attempt to identify a function of ANS asymmetry consistent with its metabolic role has not been investigated. In the current study, we investigated whether asymmetric autonomic responses could be induced through limb-specific aversive stimulation. Participants were given mild electric stimulation to either the left or right arm while EDA was monitored bilaterally. In a group-level analyses, an ipsilateral EDA response bias was observed, with increased EDA response in the hand adjacent to the stimulation. This effect was observable in ∼50% of individual particpants. These results demonstrate that autonomic output is more complex than canonical interpretations suggest. We suggest that, in stressful situations, autonomic outputs can prepare either the whole-body fight or flight response, or a simply a limb-localized flick, which can effectively neutralize the threat while minimizing global resource consumption. These findings are consistent with recent theories proposing evolutionary leveraging of neural structures organized to mediate sensory responses for processing of cognitive emotional cues.Significance statementThe present study constitutes novel evidence for an autonomic nervous response specific to the side of the body exposed to direct threat. We identify a robust pattern of electrodermal response at the body location that directly receives aversive tactile stimulation. Thus, we demonstrate for the first time in contemporary research that the ANS is capable of location-specific outputs within single effector organs in response to small scale threat. This extends the canonical view of the role of ANS responses in stressful or dangerous stresses - that of provoking a 'fight or flight' response - suggesting a further role of this system: preparation of targeted limb-specific action, i.e., a flick.

Spinal ascending pathways for somatosensory information processing

The somatosensory system processes diverse types of information including mechanical, thermal, and chemical signals. It has an essential role in sensory perception and body movement and, thus, is crucial for organism survival. The neural network for processing somatosensory information comprises multiple key nodes. Spinal projection neurons represent the key node for transmitting somatosensory information from the periphery to the brain. Although the anatomy of spinal ascending pathways has been characterized, the mechanisms underlying somatosensory information processing by spinal ascending pathways are incompletely understood. Recent studies have begun to reveal the diversity of spinal ascending pathways and their functional roles in somatosensory information processing. Here, we review the anatomic, molecular, and functional characteristics of spinal ascending pathways.

The Missing Link in Early Emotional Processing

Initial evaluation structures (IESs) currently proposed as the earliest detectors of affective stimuli (e.g., amygdala, orbitofrontal cortex, or insula) are high-order structures (a) whose response latency cannot account for the first visual cortex emotion-related response (~80 ms), and (b) lack the necessary infrastructure to locally analyze the visual features that define emotional stimuli. Several thalamic structures accomplish both criteria. The lateral geniculate nucleus (LGN), a first-order thalamic nucleus that actively processes visual information, with the complement of the thalamic reticular nucleus (TRN) are proposed as core IESs. This LGN–TRN tandem could be supported by the pulvinar, a second-order thalamic structure, and by other extrathalamic nuclei. The visual thalamus, scarcely explored in affective neurosciences, seems crucial in early emotional evaluation.

Modality-general and modality-specific audiovisual valence processing

A fundamental question in affective neuroscience is whether there is a common hedonic system for valence processing independent of modality, or there are distinct neural systems for different modalities. To address this question, we used both region of interest and whole-brain representational similarity analyses on functional magnetic resonance imaging data to identify modality-general and modality-specific brain areas involved in valence processing across visual and auditory modalities. First, region of interest analyses showed that the superior temporal cortex was associated with both modality-general and auditory-specific models, while the primary visual cortex was associated with the visual-specific model. Second, the whole-brain searchlight analyses also identified both modality-general and modality-specific representations. The modality-general regions included the superior temporal, medial superior frontal, inferior frontal, precuneus, precentral, postcentral, supramarginal, paracentral lobule and middle cingulate cortices. The modality-specific regions included both perceptual cortices and higher-order brain areas. The valence representations derived from individualized behavioral valence ratings were consistent with these results. Together, these findings suggest both modality-general and modality-specific representations of valence.

Wearable Sensors for Assessing the Role of Olfactory Training on the Autonomic Response to Olfactory Stimulation

Wearable sensors are nowadays largely employed to assess physiological signals derived from the human body without representing a burden in terms of obtrusiveness. One of the most intriguing fields of application for such systems include the assessment of physiological responses to sensory stimuli. In this specific regard, it is not yet known which are the main psychophysiological drivers of olfactory-related pleasantness, as the current literature has demonstrated the relationship between odor familiarity and odor valence, but has not clarified the consequentiality between the two domains. Here, we enrolled a group of university students to whom olfactory training lasting 3 months was administered. Thanks to the analysis of electrocardiogram (ECG) and galvanic skin response (GSR) signals at the beginning and at the end of the training period, we observed different autonomic responses, with higher parasympathetically-mediated response at the end of the period with respect to the first evaluation. This possibly suggests that an increased familiarity to the proposed stimuli would lead to a higher tendency towards relaxation. Such results could suggest potential applications to other domains, including personalized treatments based on odors and foods in neuropsychiatric and eating disorders.

Pathogens, odors, and disgust in rodents

All animals are under the constant threat of attack by parasites. The mere presence of parasite threat can alter behavior before infection takes place. These effects involve pathogen disgust, an evolutionarily conserved affective/emotional system that functions to detect cues associated with parasites and infection and facilitate avoidance behaviors. Animals gauge the infection status of conspecific and the salience of the threat they represent on the basis of various sensory cues. Odors in particular are a major source of social information about conspecifics and the infection threat they present. Here we briefly consider the origins, expression, and regulation of the fundamental features of odor mediated pathogen disgust in rodents. We briefly review aspects of: (1) the expression of affective states and emotions and in particular, disgust, in rodents; (2) olfactory mediated recognition and avoidance of potentially infected conspecifics and the impact of pathogen disgust and its' fundamental features on behavior; (3) pathogen disgust associated trade-offs; (4) the neurobiological mechanisms, and in particular the roles of the nonapeptide, oxytocin, and steroidal hormones, in the expression of pathogen disgust and the regulation of avoidance behaviors and concomitant trade-offs. Understanding the roles of pathogen disgust in rodents can provide insights into the regulation and expression of responses to pathogens and infection in humans.