1
|
Gao C, Uchitomi H, Miyake Y. Influence of Multimodal Emotional Stimulations on Brain Activity: An Electroencephalographic Study. SENSORS (BASEL, SWITZERLAND) 2023; 23:4801. [PMID: 37430714 PMCID: PMC10221168 DOI: 10.3390/s23104801] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/20/2023] [Revised: 05/05/2023] [Accepted: 05/12/2023] [Indexed: 07/12/2023]
Abstract
This study aimed to reveal the influence of emotional valence and sensory modality on neural activity in response to multimodal emotional stimuli using scalp EEG. In this study, 20 healthy participants completed the emotional multimodal stimulation experiment for three stimulus modalities (audio, visual, and audio-visual), all of which are from the same video source with two emotional components (pleasure or unpleasure), and EEG data were collected using six experimental conditions and one resting state. We analyzed power spectral density (PSD) and event-related potential (ERP) components in response to multimodal emotional stimuli, for spectral and temporal analysis. PSD results showed that the single modality (audio only/visual only) emotional stimulation PSD differed from multi-modality (audio-visual) in a wide brain and band range due to the changes in modality and not from the changes in emotional degree. The most pronounced N200-to-P300 potential shifts occurred in monomodal rather than multimodal emotional stimulations. This study suggests that emotional saliency and sensory processing efficiency perform a significant role in shaping neural activity during multimodal emotional stimulation, with the sensory modality being more influential in PSD. These findings contribute to our understanding of the neural mechanisms involved in multimodal emotional stimulation.
Collapse
Affiliation(s)
- Chenguang Gao
- Department of Computer Science, Tokyo Institute of Technology, Yokohama 226-8502, Japan; (H.U.); (Y.M.)
| | | | | |
Collapse
|
2
|
Neuroaesthetics: a narrative review of neuroimaging techniques. JOURNAL OF BIO-X RESEARCH 2021. [DOI: 10.1097/jbr.0000000000000095] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022] Open
|
3
|
Kanayama N, Mio S, Yaita R, Ohashi T, Yamawaki S. The Shape of Water Stream Induces Differences in P300 and Alpha Oscillation. Front Hum Neurosci 2020; 13:460. [PMID: 32038197 PMCID: PMC6984336 DOI: 10.3389/fnhum.2019.00460] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/07/2019] [Accepted: 12/16/2019] [Indexed: 11/13/2022] Open
Abstract
Touching is a fundamental human behavior used to evaluate objects in the external world. Many previous studies have used tactile stimulation to conduct psychological and psychophysiological experiments. However, most of these studies used solid material, not water stream, as an experimental stimulus. To investigate water perception, or to easily control the temperature of an experimental stimulus, it is important to be able to control the water stimulus. In this study, we investigated the usability of water as an experimental stimulus for electroencephalography (EEG) experiments and report the basic EEG response to water stimulus. We developed a tactile stimulation device using a water stream to study EEG responses, with the ability to control the stimulus onset timing. As stimuli, we selected two types of water stream, normal and soft, based on a psychological experiment to confirm a difference of subjective feeling induced by these water streams. We conducted a typical oddball task using the two different water streams and recorded EEG waveforms from 64 electrodes while participants touched the water streams. We calculated P300 at the Pz electrode, alpha asymmetry at the frontal electrodes, and alpha suppression at the parietal area. As a result, we observed typical P300 differentiation based on the stimulus proportion (target 20% and standard 80%). We observed a weaker alpha suppression when participants touched the soft water stream compared to the normal shower. These results demonstrate the usability of water stream in psychophysiological studies and suggested that alpha suppression could be a candidate to evaluate comfort of water stream.
Collapse
Affiliation(s)
- Noriaki Kanayama
- Human Informatics Research Institute, National Institute of Advanced Industrial Science and Technology (AIST), Tsukuba, Japan
- Center for Brain, Mind and KANSEI Sciences Research, Hiroshima University, Hiroshima, Japan
| | - Shumpei Mio
- TOTO Limited, Research Institute, Chigasaki, Japan
| | - Ryohei Yaita
- TOTO Limited, Research Institute, Chigasaki, Japan
| | | | - Shigeto Yamawaki
- Center for Brain, Mind and KANSEI Sciences Research, Hiroshima University, Hiroshima, Japan
| |
Collapse
|
4
|
Abstract
At any given moment, we receive input through our different sensory systems, and this information needs to be processed and integrated. Multisensory processing requires the coordinated activity of distinct cortical areas. Key mechanisms implicated in these processes include local neural oscillations and functional connectivity between distant cortical areas. Evidence is now emerging that neural oscillations in distinct frequency bands reflect different mechanisms of multisensory processing. Moreover, studies suggest that aberrant neural oscillations contribute to multisensory processing deficits in clinical populations, such as schizophrenia. In this article, we review recent literature on the neural mechanisms underlying multisensory processing, focusing on neural oscillations. We derive a framework that summarizes findings on (1) stimulus-driven multisensory processing, (2) the influence of top-down information on multisensory processing, and (3) the role of predictions for the formation of multisensory perception. We propose that different frequency band oscillations subserve complementary mechanisms of multisensory processing. These processes can act in parallel and are essential for multisensory processing.
Collapse
Affiliation(s)
- Julian Keil
- 1 Biological Psychology, Christian-Albrechts-University Kiel, Kiel, Germany
- 2 Department of Psychiatry and Psychotherapy, St. Hedwig Hospital, Charité-Universitätsmedizin Berlin, Berlin, Germany
| | - Daniel Senkowski
- 2 Department of Psychiatry and Psychotherapy, St. Hedwig Hospital, Charité-Universitätsmedizin Berlin, Berlin, Germany
| |
Collapse
|
5
|
Pan Z, Liu X, Luo Y, Chen X. Emotional Intensity Modulates the Integration of Bimodal Angry Expressions: ERP Evidence. Front Neurosci 2017; 11:349. [PMID: 28680388 PMCID: PMC5478688 DOI: 10.3389/fnins.2017.00349] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/22/2016] [Accepted: 06/06/2017] [Indexed: 11/18/2022] Open
Abstract
Integration of information from face and voice plays a central role in social interactions. The present study investigated the modulation of emotional intensity on the integration of facial-vocal emotional cues by recording EEG for participants while they were performing emotion identification task on facial, vocal, and bimodal angry expressions varying in emotional intensity. Behavioral results showed the rates of anger and reaction speed increased as emotional intensity across modalities. Critically, the P2 amplitudes were larger for bimodal expressions than for the sum of facial and vocal expressions for low emotional intensity stimuli, but not for middle and high emotional intensity stimuli. These findings suggested that emotional intensity modulates the integration of facial-vocal angry expressions, following the principle of Inverse Effectiveness (IE) in multimodal sensory integration.
Collapse
Affiliation(s)
- Zhihui Pan
- Key Laboratory of Behavior and Cognitive Psychology in Shaanxi Province, School of Psychology, Shaanxi Normal UniversityXi'an, China
| | - Xi Liu
- State Key Laboratory of Cognitive Neuroscience and Learning and IDG/McGovern Institute for Brain Research, School of Brain Cognitive Science, Beijing Normal UniversityBeijing, China
| | - Yangmei Luo
- Key Laboratory of Behavior and Cognitive Psychology in Shaanxi Province, School of Psychology, Shaanxi Normal UniversityXi'an, China
| | - Xuhai Chen
- Key Laboratory of Behavior and Cognitive Psychology in Shaanxi Province, School of Psychology, Shaanxi Normal UniversityXi'an, China
| |
Collapse
|
6
|
Symons AE, El-Deredy W, Schwartze M, Kotz SA. The Functional Role of Neural Oscillations in Non-Verbal Emotional Communication. Front Hum Neurosci 2016; 10:239. [PMID: 27252638 PMCID: PMC4879141 DOI: 10.3389/fnhum.2016.00239] [Citation(s) in RCA: 40] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/11/2016] [Accepted: 05/09/2016] [Indexed: 12/18/2022] Open
Abstract
Effective interpersonal communication depends on the ability to perceive and interpret nonverbal emotional expressions from multiple sensory modalities. Current theoretical models propose that visual and auditory emotion perception involves a network of brain regions including the primary sensory cortices, the superior temporal sulcus (STS), and orbitofrontal cortex (OFC). However, relatively little is known about how the dynamic interplay between these regions gives rise to the perception of emotions. In recent years, there has been increasing recognition of the importance of neural oscillations in mediating neural communication within and between functional neural networks. Here we review studies investigating changes in oscillatory activity during the perception of visual, auditory, and audiovisual emotional expressions, and aim to characterize the functional role of neural oscillations in nonverbal emotion perception. Findings from the reviewed literature suggest that theta band oscillations most consistently differentiate between emotional and neutral expressions. While early theta synchronization appears to reflect the initial encoding of emotionally salient sensory information, later fronto-central theta synchronization may reflect the further integration of sensory information with internal representations. Additionally, gamma synchronization reflects facilitated sensory binding of emotional expressions within regions such as the OFC, STS, and, potentially, the amygdala. However, the evidence is more ambiguous when it comes to the role of oscillations within the alpha and beta frequencies, which vary as a function of modality (or modalities), presence or absence of predictive information, and attentional or task demands. Thus, the synchronization of neural oscillations within specific frequency bands mediates the rapid detection, integration, and evaluation of emotional expressions. Moreover, the functional coupling of oscillatory activity across multiples frequency bands supports a predictive coding model of multisensory emotion perception in which emotional facial and body expressions facilitate the processing of emotional vocalizations.
Collapse
Affiliation(s)
- Ashley E. Symons
- School of Psychological Sciences, University of ManchesterManchester, UK
| | - Wael El-Deredy
- School of Psychological Sciences, University of ManchesterManchester, UK
- School of Biomedical Engineering, Universidad de ValparaisoValparaiso, Chile
| | - Michael Schwartze
- Department of Neuropsychology, Max Planck Institute for Human Cognitive and Brain SciencesLeipzig, Germany
- Faculty of Psychology and Neuroscience, Department of Neuropsychology and Psychopharmacology, Maastricht UniversityMaastricht, Netherlands
| | - Sonja A. Kotz
- School of Psychological Sciences, University of ManchesterManchester, UK
- Department of Neuropsychology, Max Planck Institute for Human Cognitive and Brain SciencesLeipzig, Germany
- Faculty of Psychology and Neuroscience, Department of Neuropsychology and Psychopharmacology, Maastricht UniversityMaastricht, Netherlands
| |
Collapse
|
7
|
Abstract
Over the past 15 years, the emotion regulation perspective has been widely integrated into theoretical and applied contexts in clinical psychology and beyond. Recent refinements to behavioral, subjective, psychophysiological and neuroimaging methods allow emotion regulation to be captured and assessed in the laboratory with greater precision. Technological advances enabling investigators to leverage information from multiple modalities are increasingly accessible, and as such, will further efforts to generate testable hypotheses about specific mechanisms implicated in emotion regulation and difficulties therein. In combination with theory-driven design, progressively sophisticated methods for laboratory assessment have potential to further emotion regulation as both a valid scientific construct and a useful paradigm for human emotion and behavior that has applicability to both clinical and non-clinical contexts.
Collapse
|
8
|
Klasen M, Kreifelts B, Chen YH, Seubert J, Mathiak K. Neural processing of emotion in multimodal settings. Front Hum Neurosci 2014; 8:822. [PMID: 25374523 PMCID: PMC4204532 DOI: 10.3389/fnhum.2014.00822] [Citation(s) in RCA: 16] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/15/2014] [Accepted: 09/26/2014] [Indexed: 02/04/2023] Open
Affiliation(s)
- Martin Klasen
- Department of Psychiatry, Psychotherapy, and Psychosomatics, Medical School, RWTH Aachen University Aachen, Germany ; Jülich Aachen Research Alliance-Translational Brain Medicine, RWTH Aachen University Aachen, Germany
| | - Benjamin Kreifelts
- Department of Psychiatry and Psychotherapy, University of Tuebingen Tuebingen, Germany
| | - Yu-Han Chen
- Department of Psychiatry, The University of New Mexico School of Medicine Albuquerque, NM, USA
| | - Janina Seubert
- Psychology Division, Department of Clinical Neuroscience, Karolinska Institutet Stockholm, Sweden
| | - Klaus Mathiak
- Department of Psychiatry, Psychotherapy, and Psychosomatics, Medical School, RWTH Aachen University Aachen, Germany ; Jülich Aachen Research Alliance-Translational Brain Medicine, RWTH Aachen University Aachen, Germany
| |
Collapse
|
9
|
Sarwate AD, Plis SM, Turner JA, Arbabshirani MR, Calhoun VD. Sharing privacy-sensitive access to neuroimaging and genetics data: a review and preliminary validation. Front Neuroinform 2014; 8:35. [PMID: 24778614 PMCID: PMC3985022 DOI: 10.3389/fninf.2014.00035] [Citation(s) in RCA: 39] [Impact Index Per Article: 3.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/18/2013] [Accepted: 03/19/2014] [Indexed: 11/16/2022] Open
Abstract
The growth of data sharing initiatives for neuroimaging and genomics represents an exciting opportunity to confront the “small N” problem that plagues contemporary neuroimaging studies while further understanding the role genetic markers play in the function of the brain. When it is possible, open data sharing provides the most benefits. However, some data cannot be shared at all due to privacy concerns and/or risk of re-identification. Sharing other data sets is hampered by the proliferation of complex data use agreements (DUAs) which preclude truly automated data mining. These DUAs arise because of concerns about the privacy and confidentiality for subjects; though many do permit direct access to data, they often require a cumbersome approval process that can take months. An alternative approach is to only share data derivatives such as statistical summaries—the challenges here are to reformulate computational methods to quantify the privacy risks associated with sharing the results of those computations. For example, a derived map of gray matter is often as identifiable as a fingerprint. Thus alternative approaches to accessing data are needed. This paper reviews the relevant literature on differential privacy, a framework for measuring and tracking privacy loss in these settings, and demonstrates the feasibility of using this framework to calculate statistics on data distributed at many sites while still providing privacy.
Collapse
Affiliation(s)
- Anand D Sarwate
- Department of Electrical and Computer Engineering, Rutgers, The State University of New Jersey Piscataway, NJ, USA
| | | | - Jessica A Turner
- Mind Research Network Albuquerque, NM, USA ; Department of Psychology and Neuroscience Institute, Georgia State University Atlanta, GA, USA
| | - Mohammad R Arbabshirani
- Mind Research Network Albuquerque, NM, USA ; Department of Electrical and Computer Engineering, University of New Mexico Albuquerque, NM, USA
| | - Vince D Calhoun
- Mind Research Network Albuquerque, NM, USA ; Department of Electrical and Computer Engineering, University of New Mexico Albuquerque, NM, USA
| |
Collapse
|