1
|
Frumento S, Preatoni G, Chee L, Gemignani A, Ciotti F, Menicucci D, Raspopovic S. Unconscious multisensory integration: behavioral and neural evidence from subliminal stimuli. Front Psychol 2024; 15:1396946. [PMID: 39091706 PMCID: PMC11291458 DOI: 10.3389/fpsyg.2024.1396946] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/06/2024] [Accepted: 07/04/2024] [Indexed: 08/04/2024] Open
Abstract
Introduction The prevailing theories of consciousness consider the integration of different sensory stimuli as a key component for this phenomenon to rise on the brain level. Despite many theories and models have been proposed for multisensory integration between supraliminal stimuli (e.g., the optimal integration model), we do not know if multisensory integration occurs also for subliminal stimuli and what psychophysical mechanisms it follows. Methods To investigate this, subjects were exposed to visual (Virtual Reality) and/or haptic stimuli (Electro-Cutaneous Stimulation) above or below their perceptual threshold. They had to discriminate, in a two-Alternative Forced Choice Task, the intensity of unimodal and/or bimodal stimuli. They were then asked to discriminate the sensory modality while recording their EEG responses. Results We found evidence of multisensory integration for supraliminal condition, following the classical optimal model. Importantly, even for subliminal trials participant's performances in the bimodal condition were significantly more accurate when discriminating the intensity of the stimulation. Moreover, significant differences emerged between unimodal and bimodal activity templates in parieto-temporal areas known for their integrative role. Discussion These converging evidences - even if preliminary and needing confirmation from the collection of further data - suggest that subliminal multimodal stimuli can be integrated, thus filling a meaningful gap in the debate about the relationship between consciousness and multisensory integration.
Collapse
Affiliation(s)
- Sergio Frumento
- Department of Surgical, Medical, Molecular and Critical Area Pathology, University of Pisa, Pisa, Italy
| | - Greta Preatoni
- Laboratory for Neuroengineering, Department of Health Sciences and Technology, Institute of Robotics and Intelligent Systems, ETH Zürich, Zürich, Switzerland
| | - Lauren Chee
- Laboratory for Neuroengineering, Department of Health Sciences and Technology, Institute of Robotics and Intelligent Systems, ETH Zürich, Zürich, Switzerland
| | - Angelo Gemignani
- Department of Surgical, Medical, Molecular and Critical Area Pathology, University of Pisa, Pisa, Italy
- Clinical Psychology Branch, Azienda Ospedaliero-Universitaria Pisana, Pisa, Italy
| | - Federico Ciotti
- Laboratory for Neuroengineering, Department of Health Sciences and Technology, Institute of Robotics and Intelligent Systems, ETH Zürich, Zürich, Switzerland
| | - Danilo Menicucci
- Department of Surgical, Medical, Molecular and Critical Area Pathology, University of Pisa, Pisa, Italy
| | - Stanisa Raspopovic
- Laboratory for Neuroengineering, Department of Health Sciences and Technology, Institute of Robotics and Intelligent Systems, ETH Zürich, Zürich, Switzerland
| |
Collapse
|
2
|
Understanding the domain development through a word status observation model. J Informetr 2023. [DOI: 10.1016/j.joi.2023.101395] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/07/2023]
|
3
|
Agrawal T, Schachner A. Hearing water temperature: characterizing the development of nuanced perception of sound sources. Dev Sci 2022; 26:e13321. [PMID: 36068928 DOI: 10.1111/desc.13321] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/19/2022] [Revised: 08/18/2022] [Accepted: 08/22/2022] [Indexed: 11/30/2022]
Abstract
Without conscious thought, listeners link events in the world to sounds they hear. We study one surprising example: Adults can judge the temperature of water simply from hearing it being poured. We test development of the ability to hear water temperature, with the goal of informing developmental theories regarding the origins and cognitive bases of nuanced sound source judgments. We first confirmed that adults accurately distinguished the sounds of hot and cold water (pre-registered Exps. 1, 2; total N = 384), even though many were unaware or uncertain of this ability. By contrast, children showed protracted development of this skill over the course of middle childhood (Exps. 2, 3; total N = 178). In spite of accurately identifying other sounds and hot/cold images, older children (7-11 years) but not younger children (3-6 years) reliably distinguished the sounds of hot and cold water. Accuracy increased with age; 11 year old's performance was similar to adults'. Adults also showed individual differences in accuracy that were predicted by their amount of prior relevant experience (Exp. 1). Experience may similarly play a role in children's performance; differences in auditory sensitivity and multimodal integration may also contribute to young children's failures. The ability to hear water temperature develops slowly over childhood, such that nuanced auditory information that is easily and quickly accessible to adults is not available to guide young children's behavior. Adults can make nuanced judgments from sound, including accurately judging the temperature of water from the sound of it being poured. Children showed protracted development of this skill over the course of middle childhood, such that 7-11 year-olds reliably succeeded while 3-6 year-olds performed at chance. Developmental changes may be due to experience (adults with greater relevant experience showed higher accuracy) and development of multimodal integration and auditory sensitivity. Young children may not detect subtle auditory information that adults easily perceive. This article is protected by copyright. All rights reserved.
Collapse
Affiliation(s)
| | - Adena Schachner
- Department of Psychology, University of California, San Diego, USA
| |
Collapse
|
4
|
Beyond motion extrapolation: vestibular contribution to head-rotation-induced flash-lag effects. PSYCHOLOGICAL RESEARCH 2022; 86:2083-2098. [DOI: 10.1007/s00426-021-01638-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/19/2021] [Accepted: 12/19/2021] [Indexed: 11/30/2022]
|
5
|
Zhao HC, Lv R, Zhang GY, He LM, Cai XT, Sun Q, Yan CY, Bao XY, Lv XY, Fu B. Alterations of Prefrontal-Posterior Information Processing Patterns in Autism Spectrum Disorders. Front Neurosci 2022; 15:768219. [PMID: 35173572 PMCID: PMC8841879 DOI: 10.3389/fnins.2021.768219] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/31/2021] [Accepted: 12/27/2021] [Indexed: 11/22/2022] Open
Abstract
Autism spectrum disorder (ASD) is a heterogeneous disorder characterized by different levels of repetitive and stereotypic behavior as well as deficits in social interaction and communication. In this current study, we explored the changes in cerebral neural activities in ASD. The purpose of this study is to investigate whether there exists a dysfunction of interactive information processing between the prefrontal cortex and posterior brain regions in ASD. We investigated the atypical connectivity and information flow between the prefrontal cortex and posterior brain regions in ASD utilizing the entropy connectivity (a kind of directional connectivity) method. Eighty-nine patients with ASD and 94 typical developing (TD) teenagers participated in this study. Two-sample t-tests revealed weakened interactive entropy connectivity between the prefrontal cortex and posterior brain regions. This result indicates that there exists interactive prefrontal-posterior underconnectivity in ASD, and this disorder might lead to less prior knowledge being used and updated. Our proposals highlighted that aforementioned atypical change might accelerate the deoptimization of brain networks in ASD.
Collapse
|
6
|
Xiao K, Gao Y, Imran SA, Chowdhury S, Commuri S, Jiang F. Cross-modal motion aftereffects transfer between vision and touch in early deaf adults. Sci Rep 2021; 11:4395. [PMID: 33623083 PMCID: PMC7902672 DOI: 10.1038/s41598-021-83960-0] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/27/2019] [Accepted: 12/29/2020] [Indexed: 11/23/2022] Open
Abstract
Previous research on early deafness has primarily focused on the behavioral and neural changes in the intact visual and tactile modalities. However, how early deafness changes the interplay of these two modalities is not well understood. In the current study, we investigated the effect of auditory deprivation on visuo-tactile interaction by measuring the cross-modal motion aftereffect. Consistent with previous findings, motion aftereffect transferred between vision and touch in a bidirectional manner in hearing participants. However, for deaf participants, the cross-modal transfer occurred only in the tactile-to-visual direction but not in the visual-to-tactile direction. This unidirectional cross-modal motion aftereffect found in the deaf participants could not be explained by unisensory motion aftereffect or discrimination threshold. The results suggest a reduced visual influence on tactile motion perception in early deaf individuals.
Collapse
Affiliation(s)
- Kunchen Xiao
- Institute of Brain and Psychological Sciences, Sichuan Normal University, Chengdu, 610066, Sichuan Province, China.
- Department of Psychology, University of Nevada, Reno, NV, 89557-0296, USA.
| | - Yi Gao
- Department of Psychology, University of Nevada, Reno, NV, 89557-0296, USA
| | - Syed Asif Imran
- Department of Electrical and Biomedical Engineering, University of Nevada, Reno, NV, 89557-0260, USA
| | - Shahida Chowdhury
- Department of Psychology, University of Nevada, Reno, NV, 89557-0296, USA
| | - Sesh Commuri
- Department of Electrical and Biomedical Engineering, University of Nevada, Reno, NV, 89557-0260, USA
| | - Fang Jiang
- Department of Psychology, University of Nevada, Reno, NV, 89557-0296, USA.
| |
Collapse
|
7
|
Cortical processes underlying the effects of static sound timing on perceived visual speed. Neuroimage 2019; 199:194-205. [DOI: 10.1016/j.neuroimage.2019.05.062] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/25/2019] [Revised: 04/09/2019] [Accepted: 05/24/2019] [Indexed: 01/10/2023] Open
|
8
|
Chotsrisuparat C, Koning A, Jacobs R, van Lier R. Effects of Auditory Patterns on Judged Displacements of an Occluded Moving Object. Multisens Res 2018; 31:623-643. [PMID: 31264610 DOI: 10.1163/22134808-18001294] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/24/2017] [Accepted: 12/22/2017] [Indexed: 11/19/2022]
Abstract
Using displays in which a moving disk disappeared behind an occluder, we examined whether an accompanying auditory rhythm influenced the perceived displacement of the disk during occlusion. We manipulated a baseline rhythm, comprising a relatively fast alternation of equal sound and pause durations. We had two different manipulations to create auditory sequences with a slower rhythm: either the pause durations or the sound durations were increased. In the trial, a disk moved at a constant speed, and at a certain point moved behind an occluder during which an auditory rhythm was played. Participants were instructed to track the occluded disk, and judge the expected position of the disk at the moment that the auditory rhythm ended by touching the judged position on a touch screen. We investigated the influence of the auditory rhythm, i.e., ratio of sound to pause duration, and the influence of auditory density, i.e., the number of sound onsets per time unit, on the judged distance. The results showed that the temporal characteristics affected the spatial judgments. Overall, we found that in the current paradigm relatively slow rhythms led to shorter judged distance as compared to relatively fast rhythms for both pause and sound variations. There was no main effect of auditory density on the judged distance of an expected visual event. That is, whereas the speed of the auditory rhythm appears crucial, the number of sound onsets per time unit as such, i.e., the auditory density, appears a much weaker factor.
Collapse
Affiliation(s)
- Chayada Chotsrisuparat
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, The Netherlands
| | - Arno Koning
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, The Netherlands
| | - Richard Jacobs
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, The Netherlands
| | - Rob van Lier
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, The Netherlands
| |
Collapse
|
9
|
Ogulmus C, Karacaoglu M, Kafaligonul H. Temporal ventriloquism along the path of apparent motion: speed perception under different spatial grouping principles. Exp Brain Res 2017; 236:629-643. [DOI: 10.1007/s00221-017-5159-1] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/29/2017] [Accepted: 12/18/2017] [Indexed: 10/18/2022]
|
10
|
Kaya U, Yildirim FZ, Kafaligonul H. The involvement of centralized and distributed processes in sub-second time interval adaptation: an ERP investigation of apparent motion. Eur J Neurosci 2017; 46:2325-2338. [DOI: 10.1111/ejn.13691] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/21/2017] [Revised: 08/22/2017] [Accepted: 08/29/2017] [Indexed: 11/30/2022]
Affiliation(s)
- Utku Kaya
- National Magnetic Resonance Research Center (UMRAM); Bilkent University; Ankara Turkey
- Informatics Institute; Middle East Technical University; Ankara Turkey
| | - Fazilet Zeynep Yildirim
- Interdisciplinary Neuroscience Program; Bilkent University; Ankara Turkey
- Institute of Psychology; University of Bern; Bern Switzerland
| | - Hulusi Kafaligonul
- National Magnetic Resonance Research Center (UMRAM); Bilkent University; Ankara Turkey
- Interdisciplinary Neuroscience Program; Bilkent University; Ankara Turkey
| |
Collapse
|
11
|
Stock AK, Gohil K, Huster RJ, Beste C. On the effects of multimodal information integration in multitasking. Sci Rep 2017; 7:4927. [PMID: 28687804 PMCID: PMC5501795 DOI: 10.1038/s41598-017-04828-w] [Citation(s) in RCA: 39] [Impact Index Per Article: 5.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/16/2016] [Accepted: 05/04/2017] [Indexed: 11/26/2022] Open
Abstract
There have recently been considerable advances in our understanding of the neuronal mechanisms underlying multitasking, but the role of multimodal integration for this faculty has remained rather unclear. We examined this issue by comparing different modality combinations in a multitasking (stop-change) paradigm. In-depth neurophysiological analyses of event-related potentials (ERPs) were conducted to complement the obtained behavioral data. Specifically, we applied signal decomposition using second order blind identification (SOBI) to the multi-subject ERP data and source localization. We found that both general multimodal information integration and modality-specific aspects (potentially related to task difficulty) modulate behavioral performance and associated neurophysiological correlates. Simultaneous multimodal input generally increased early attentional processing of visual stimuli (i.e. P1 and N1 amplitudes) as well as measures of cognitive effort and conflict (i.e. central P3 amplitudes). Yet, tactile-visual input caused larger impairments in multitasking than audio-visual input. General aspects of multimodal information integration modulated the activity in the premotor cortex (BA 6) as well as different visual association areas concerned with the integration of visual information with input from other modalities (BA 19, BA 21, BA 37). On top of this, differences in the specific combination of modalities also affected performance and measures of conflict/effort originating in prefrontal regions (BA 6).
Collapse
Affiliation(s)
- Ann-Kathrin Stock
- Cognitive Neurophysiology, Department of Child and Adolescent Psychiatry, Carl Gustav Carus Faculty of Medicine, TU Dresden, Germany.
| | - Krutika Gohil
- Cognitive Neurophysiology, Department of Child and Adolescent Psychiatry, Carl Gustav Carus Faculty of Medicine, TU Dresden, Germany
| | - René J Huster
- Department of Psychology, University of Oslo, Oslo, Norway
| | - Christian Beste
- Cognitive Neurophysiology, Department of Child and Adolescent Psychiatry, Carl Gustav Carus Faculty of Medicine, TU Dresden, Germany.,Experimental Neurobiology, National Institute of Mental Health, Klecany, Czech Republic
| |
Collapse
|
12
|
Hidaka S, Higuchi S, Teramoto W, Sugita Y. Neural mechanisms underlying sound-induced visual motion perception: An fMRI study. Acta Psychol (Amst) 2017; 178:66-72. [PMID: 28600968 DOI: 10.1016/j.actpsy.2017.05.013] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/16/2016] [Revised: 05/17/2017] [Accepted: 05/25/2017] [Indexed: 10/19/2022] Open
Abstract
Studies of crossmodal interactions in motion perception have reported activation in several brain areas, including those related to motion processing and/or sensory association, in response to multimodal (e.g., visual and auditory) stimuli that were both in motion. Recent studies have demonstrated that sounds can trigger illusory visual apparent motion to static visual stimuli (sound-induced visual motion: SIVM): A visual stimulus blinking at a fixed location is perceived to be moving laterally when an alternating left-right sound is also present. Here, we investigated brain activity related to the perception of SIVM using a 7T functional magnetic resonance imaging technique. Specifically, we focused on the patterns of neural activities in SIVM and visually induced visual apparent motion (VIVM). We observed shared activations in the middle occipital area (V5/hMT), which is thought to be involved in visual motion processing, for SIVM and VIVM. Moreover, as compared to VIVM, SIVM resulted in greater activation in the superior temporal area and dominant functional connectivity between the V5/hMT area and the areas related to auditory and crossmodal motion processing. These findings indicate that similar but partially different neural mechanisms could be involved in auditory-induced and visually-induced motion perception, and neural signals in auditory, visual, and, crossmodal motion processing areas closely and directly interact in the perception of SIVM.
Collapse
|
13
|
Chotsrisuparat C, Koning A, Jacobs R, van Lier R. Auditory Rhythms Influence Judged Time to Contact of an Occluded Moving Object. Multisens Res 2017. [DOI: 10.1163/22134808-00002592] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/19/2022]
Abstract
We studied the expected moment of reappearance of a moving object after it disappeared from sight. In particular, we investigated whether auditory rhythms influence time to contact (TTC) judgments. Using displays in which a moving disk disappears behind an occluder, we examined whether an accompanying auditory rhythm influences the expected TTC of an occluded moving object. We manipulated a baseline auditory rhythm — consisting of equal sound and pause durations — in two ways: either the pause durations or the sound durations were increased to create slower rhythms. Participants had to press a button at the moment they expected the disk to reappear. Variations in pause duration (Experiments 1 and 2) affected expected TTC, in contrast to variations in sound duration (Experiment 3). These results show that auditory rhythms affect expected reappearance of an occluded moving object. Second, these results suggest that temporal auditory grouping is an important factor in TTC.
Collapse
Affiliation(s)
- Chayada Chotsrisuparat
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, The Netherlands
| | - Arno Koning
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, The Netherlands
| | - Richard Jacobs
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, The Netherlands
| | - Rob van Lier
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, The Netherlands
| |
Collapse
|