1
|
Jertberg RM, Wienicke FJ, Andruszkiewicz K, Begeer S, Chakrabarti B, Geurts HM, Vries RD, der Burg EV. Differences Between Autistic and Non-Autistic Individuals in Audiovisual Speech Integration: A Systematic Review and Meta-analysis. Neurosci Biobehav Rev 2024:105787. [PMID: 38945419 DOI: 10.1016/j.neubiorev.2024.105787] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/22/2023] [Revised: 05/15/2024] [Accepted: 06/24/2024] [Indexed: 07/02/2024]
Abstract
Research has indicated unique challenges in audiovisual integration of speech among autistic individuals, although methodological differences have led to divergent findings. We conducted a systematic literature search to identify studies that measured audiovisual speech integration among both autistic and non-autistic individuals. Across the 18 identified studies (combined N = 952), autistic individuals showed impaired audiovisual integration compared to their non-autistic peers (g = 0.69, 95% CI [0.53, 0.85], p <.001). This difference was not found to be influenced by participants' mean ages, studies' sample sizes, risk-of-bias scores, or paradigms investigated. However, a subgroup analysis suggested that child studies may show larger between-group differences than adult ones. The prevailing pattern of impaired audiovisual speech integration in autism may have cascading effects on communicative and social behavior. However, small samples and inconsistency in design/analysis translated into considerable heterogeneity in findings and opacity regarding the influence of underlying unisensory and attentional factors. We recommend three key directions for future research: larger samples, more research with adults, and standardization of methodology and analytical approaches.
Collapse
Affiliation(s)
- Robert M Jertberg
- Department of Clinical and Developmental Psychology, Vrije Universiteit Amsterdam, The Netherlands and Amsterdam Public Health Research Institute, Amsterdam, Netherlands.
| | - Frederik J Wienicke
- Department of Clinical Psychology, Behavioural Science Institute, Radboud University, Nijmegen, Netherlands
| | - Krystian Andruszkiewicz
- Department of Clinical and Developmental Psychology, Vrije Universiteit Amsterdam, The Netherlands and Amsterdam Public Health Research Institute, Amsterdam, Netherlands
| | - Sander Begeer
- Department of Clinical and Developmental Psychology, Vrije Universiteit Amsterdam, The Netherlands and Amsterdam Public Health Research Institute, Amsterdam, Netherlands
| | - Bhismadev Chakrabarti
- Centre for Autism, School of Psychology and Clinical Language Sciences, University of Reading, UK; India Autism Center, Kolkata, India; Department of Psychology, Ashoka University, India
| | - Hilde M Geurts
- Department of Psychology, Universiteit van Amsterdam, the Netherlands; Leo Kannerhuis (Youz/Parnassiagroup), the Netherlands
| | - Ralph de Vries
- Medical Library, Vrije Universiteit, Amsterdam, the Netherlands
| | - Erik Van der Burg
- Department of Psychology, Universiteit van Amsterdam, the Netherlands
| |
Collapse
|
2
|
Ross LA, Molholm S, Butler JS, Del Bene VA, Brima T, Foxe JJ. Neural correlates of audiovisual narrative speech perception in children and adults on the autism spectrum: A functional magnetic resonance imaging study. Autism Res 2024; 17:280-310. [PMID: 38334251 DOI: 10.1002/aur.3104] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/03/2023] [Accepted: 01/19/2024] [Indexed: 02/10/2024]
Abstract
Autistic individuals show substantially reduced benefit from observing visual articulations during audiovisual speech perception, a multisensory integration deficit that is particularly relevant to social communication. This has mostly been studied using simple syllabic or word-level stimuli and it remains unclear how altered lower-level multisensory integration translates to the processing of more complex natural multisensory stimulus environments in autism. Here, functional neuroimaging was used to examine neural correlates of audiovisual gain (AV-gain) in 41 autistic individuals to those of 41 age-matched non-autistic controls when presented with a complex audiovisual narrative. Participants were presented with continuous narration of a story in auditory-alone, visual-alone, and both synchronous and asynchronous audiovisual speech conditions. We hypothesized that previously identified differences in audiovisual speech processing in autism would be characterized by activation differences in brain regions well known to be associated with audiovisual enhancement in neurotypicals. However, our results did not provide evidence for altered processing of auditory alone, visual alone, audiovisual conditions or AV- gain in regions associated with the respective task when comparing activation patterns between groups. Instead, we found that autistic individuals responded with higher activations in mostly frontal regions where the activation to the experimental conditions was below baseline (de-activations) in the control group. These frontal effects were observed in both unisensory and audiovisual conditions, suggesting that these altered activations were not specific to multisensory processing but reflective of more general mechanisms such as an altered disengagement of Default Mode Network processes during the observation of the language stimulus across conditions.
Collapse
Affiliation(s)
- Lars A Ross
- The Frederick J. and Marion A. Schindler Cognitive Neurophysiology Laboratory, The Ernest J. Del Monte Institute for Neuroscience, Department of Neuroscience, University of Rochester School of Medicine and Dentistry, Rochester, New York, USA
- Department of Imaging Sciences, University of Rochester Medical Center, University of Rochester School of Medicine and Dentistry, Rochester, New York, USA
- The Cognitive Neurophysiology Laboratory, Departments of Pediatrics and Neuroscience, Albert Einstein College of Medicine & Montefiore Medical Center, Bronx, New York, USA
| | - Sophie Molholm
- The Frederick J. and Marion A. Schindler Cognitive Neurophysiology Laboratory, The Ernest J. Del Monte Institute for Neuroscience, Department of Neuroscience, University of Rochester School of Medicine and Dentistry, Rochester, New York, USA
- The Cognitive Neurophysiology Laboratory, Departments of Pediatrics and Neuroscience, Albert Einstein College of Medicine & Montefiore Medical Center, Bronx, New York, USA
| | - John S Butler
- The Cognitive Neurophysiology Laboratory, Departments of Pediatrics and Neuroscience, Albert Einstein College of Medicine & Montefiore Medical Center, Bronx, New York, USA
- School of Mathematics and Statistics, Technological University Dublin, City Campus, Dublin, Ireland
| | - Victor A Del Bene
- The Cognitive Neurophysiology Laboratory, Departments of Pediatrics and Neuroscience, Albert Einstein College of Medicine & Montefiore Medical Center, Bronx, New York, USA
- Heersink School of Medicine, Department of Neurology, University of Alabama at Birmingham, Birmingham, Alabama, USA
| | - Tufikameni Brima
- The Frederick J. and Marion A. Schindler Cognitive Neurophysiology Laboratory, The Ernest J. Del Monte Institute for Neuroscience, Department of Neuroscience, University of Rochester School of Medicine and Dentistry, Rochester, New York, USA
| | - John J Foxe
- The Frederick J. and Marion A. Schindler Cognitive Neurophysiology Laboratory, The Ernest J. Del Monte Institute for Neuroscience, Department of Neuroscience, University of Rochester School of Medicine and Dentistry, Rochester, New York, USA
- The Cognitive Neurophysiology Laboratory, Departments of Pediatrics and Neuroscience, Albert Einstein College of Medicine & Montefiore Medical Center, Bronx, New York, USA
| |
Collapse
|
3
|
Jones SA, Noppeney U. Multisensory Integration and Causal Inference in Typical and Atypical Populations. ADVANCES IN EXPERIMENTAL MEDICINE AND BIOLOGY 2024; 1437:59-76. [PMID: 38270853 DOI: 10.1007/978-981-99-7611-9_4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/26/2024]
Abstract
Multisensory perception is critical for effective interaction with the environment, but human responses to multisensory stimuli vary across the lifespan and appear changed in some atypical populations. In this review chapter, we consider multisensory integration within a normative Bayesian framework. We begin by outlining the complex computational challenges of multisensory causal inference and reliability-weighted cue integration, and discuss whether healthy young adults behave in accordance with normative Bayesian models. We then compare their behaviour with various other human populations (children, older adults, and those with neurological or neuropsychiatric disorders). In particular, we consider whether the differences seen in these groups are due only to changes in their computational parameters (such as sensory noise or perceptual priors), or whether the fundamental computational principles (such as reliability weighting) underlying multisensory perception may also be altered. We conclude by arguing that future research should aim explicitly to differentiate between these possibilities.
Collapse
Affiliation(s)
- Samuel A Jones
- Department of Psychology, Nottingham Trent University, Nottingham, UK.
| | - Uta Noppeney
- Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, The Netherlands
| |
Collapse
|
4
|
Monti M, Molholm S, Cuppini C. Atypical development of causal inference in autism inferred through a neurocomputational model. Front Comput Neurosci 2023; 17:1258590. [PMID: 37927544 PMCID: PMC10620690 DOI: 10.3389/fncom.2023.1258590] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/14/2023] [Accepted: 10/05/2023] [Indexed: 11/07/2023] Open
Abstract
In everyday life, the brain processes a multitude of stimuli from the surrounding environment, requiring the integration of information from different sensory modalities to form a coherent perception. This process, known as multisensory integration, enhances the brain's response to redundant congruent sensory cues. However, it is equally important for the brain to segregate sensory inputs from distinct events, to interact with and correctly perceive the multisensory environment. This problem the brain must face, known as the causal inference problem, is strictly related to multisensory integration. It is widely recognized that the ability to integrate information from different senses emerges during the developmental period, as a function of our experience with multisensory stimuli. Consequently, multisensory integrative abilities are altered in individuals who have atypical experiences with cross-modal cues, such as those on the autistic spectrum. However, no research has been conducted on the developmental trajectories of causal inference and its relationship with experience thus far. Here, we used a neuro-computational model to simulate and investigate the development of causal inference in both typically developing children and those in the autistic spectrum. Our results indicate that higher exposure to cross-modal cues accelerates the acquisition of causal inference abilities, and a minimum level of experience with multisensory stimuli is required to develop fully mature behavior. We then simulated the altered developmental trajectory of causal inference in individuals with autism by assuming reduced multisensory experience during training. The results suggest that causal inference reaches complete maturity much later in these individuals compared to neurotypical individuals. Furthermore, we discuss the underlying neural mechanisms and network architecture involved in these processes, highlighting that the development of causal inference follows the evolution of the mechanisms subserving multisensory integration. Overall, this study provides a computational framework, unifying causal inference and multisensory integration, which allows us to suggest neural mechanisms and provide testable predictions about the development of such abilities in typically developed and autistic children.
Collapse
Affiliation(s)
- Melissa Monti
- Department of Electrical, Electronic, and Information Engineering Guglielmo Marconi, University of Bologna, Bologna, Italy
| | - Sophie Molholm
- Departments of Pediatrics and Neuroscience, Albert Einstein College of Medicine, Bronx, NY, United States
| | - Cristiano Cuppini
- Department of Electrical, Electronic, and Information Engineering Guglielmo Marconi, University of Bologna, Bologna, Italy
| |
Collapse
|
5
|
Hughes L, Kargas N, Wilhelm M, Meyerhoff HS, Föcker J. The Impact of Audio-Visual, Visual and Auditory Cues on Multiple Object Tracking Performance in Children with Autism. Percept Mot Skills 2023; 130:2047-2068. [PMID: 37452765 PMCID: PMC10552336 DOI: 10.1177/00315125231187984] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/18/2023]
Abstract
Previous studies have documented differences in processing multisensory information by children with autism compared to typically developing children. Furthermore, children with autism have been found to track fewer multiple objects on a screen than those without autism, suggesting reduced attentional control. In the present study, we investigated whether children with autism (n = 33) and children without autism (n = 33) were able to track four target objects moving amongst four indistinguishable distractor objects while sensory cues were presented. During tracking, we presented various types of cues - auditory, visual, or audio-visual or no cues while target objects bounced off the inner boundary of a centralized circle. We found that children with autism tracked fewer targets than children without autism. Furthermore, children without autism showed improved tracking performance in the presence of visual cues, whereas children with autism did not benefit from sensory cues. Whereas multiple object tracking performance improved with increasing age in children without autism, especially when using audio-visual cues, children with autism did not show age-related improvement in tracking. These results are in line with the hypothesis that attention and the ability to integrate sensory cues during tracking are reduced in children with autism. Our findings could contribute valuable insights for designing interventions that incorporate multisensory information.
Collapse
Affiliation(s)
- Lily Hughes
- School of Psychology, College of Social Science, University of Lincoln, Lincoln, UK
| | - Niko Kargas
- School of Psychology, College of Social Science, University of Lincoln, Lincoln, UK
| | - Maximilian Wilhelm
- Center for Psychotherapy Research, University Hospital Heidelberg, Heidelberg, Germany
| | | | - Julia Föcker
- School of Psychology, College of Social Science, University of Lincoln, Lincoln, UK
| |
Collapse
|
6
|
Choi I, Demir I, Oh S, Lee SH. Multisensory integration in the mammalian brain: diversity and flexibility in health and disease. Philos Trans R Soc Lond B Biol Sci 2023; 378:20220338. [PMID: 37545309 PMCID: PMC10404930 DOI: 10.1098/rstb.2022.0338] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/03/2023] [Accepted: 04/30/2023] [Indexed: 08/08/2023] Open
Abstract
Multisensory integration (MSI) occurs in a variety of brain areas, spanning cortical and subcortical regions. In traditional studies on sensory processing, the sensory cortices have been considered for processing sensory information in a modality-specific manner. The sensory cortices, however, send the information to other cortical and subcortical areas, including the higher association cortices and the other sensory cortices, where the multiple modality inputs converge and integrate to generate a meaningful percept. This integration process is neither simple nor fixed because these brain areas interact with each other via complicated circuits, which can be modulated by numerous internal and external conditions. As a result, dynamic MSI makes multisensory decisions flexible and adaptive in behaving animals. Impairments in MSI occur in many psychiatric disorders, which may result in an altered perception of the multisensory stimuli and an abnormal reaction to them. This review discusses the diversity and flexibility of MSI in mammals, including humans, primates and rodents, as well as the brain areas involved. It further explains how such flexibility influences perceptual experiences in behaving animals in both health and disease. This article is part of the theme issue 'Decision and control processes in multisensory perception'.
Collapse
Affiliation(s)
- Ilsong Choi
- Center for Synaptic Brain Dysfunctions, Institute for Basic Science (IBS), Daejeon 34141, Republic of Korea
| | - Ilayda Demir
- Department of biological sciences, KAIST, Daejeon 34141, Republic of Korea
| | - Seungmi Oh
- Department of biological sciences, KAIST, Daejeon 34141, Republic of Korea
| | - Seung-Hee Lee
- Center for Synaptic Brain Dysfunctions, Institute for Basic Science (IBS), Daejeon 34141, Republic of Korea
- Department of biological sciences, KAIST, Daejeon 34141, Republic of Korea
| |
Collapse
|
7
|
Naigles LR, Yi L, Zhao J. Editorial for the Special Issue - Language and Reading in Autism: Perspectives from Chinese. READING AND WRITING 2023; 36:1359-1368. [PMID: 37216044 PMCID: PMC10170030 DOI: 10.1007/s11145-023-10445-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 03/03/2023] [Indexed: 05/24/2023]
Affiliation(s)
| | - Li Yi
- Peking University, Beijing, China
| | - Jing Zhao
- Department of English, Sun Yat-sen University, 135 Xingangxi Road, Guangzhou, 510275 China
| |
Collapse
|
8
|
Dorsi J, Ostrand R, Rosenblum LD. Semantic priming from McGurk words: Priming depends on perception. Atten Percept Psychophys 2023; 85:1219-1237. [PMID: 37155085 DOI: 10.3758/s13414-023-02689-2] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 02/23/2023] [Indexed: 05/10/2023]
Abstract
The McGurk effect is an illusion in which visible articulations alter the perception of auditory speech (e.g., video 'da' dubbed with audio 'ba' may be heard as 'da'). To test the timing of the multisensory processes that underlie the McGurk effect, Ostrand et al. Cognition 151, 96-107, 2016 used incongruent stimuli, such as auditory 'bait' + visual 'date' as primes in a lexical decision task. These authors reported that the auditory word, but not the perceived (visual) word, induced semantic priming, suggesting that the auditory signal alone can provide the input for lexical access, before multisensory integration is complete. Here, we conceptually replicate the design of Ostrand et al. (2016), using different stimuli chosen to optimize the success of the McGurk illusion. In contrast to the results of Ostrand et al. (2016), we find that the perceived (i.e., visual) word of the incongruent stimulus usually induced semantic priming. We further find that the strength of this priming corresponded to the magnitude of the McGurk effect for each word combination. These findings suggest, in contrast to the findings of Ostrand et al. (2016), that lexical access makes use of integrated multisensory information which is perceived by the listener. These findings further suggest that which unimodal signal of a multisensory stimulus is used in lexical access is dependent on the perception of that stimulus.
Collapse
Affiliation(s)
- Josh Dorsi
- Department of Psychology, University of California, Riverside, 900 University Ave, Riverside, CA, 92521, USA.
- Penn State University, College of Medicine, State College, PA, USA.
| | | | - Lawrence D Rosenblum
- Department of Psychology, University of California, Riverside, 900 University Ave, Riverside, CA, 92521, USA
| |
Collapse
|
9
|
Suri KN, Whedon M, Lewis M. Perception of audio-visual synchrony in infants at elevated likelihood of developing autism spectrum disorder. Eur J Pediatr 2023; 182:2105-2117. [PMID: 36820895 DOI: 10.1007/s00431-023-04871-y] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/06/2022] [Revised: 02/05/2023] [Accepted: 02/08/2023] [Indexed: 02/24/2023]
Abstract
UNLABELLED The inability to perceive audio-visual speech as a unified event may contribute to social impairments and language deficits in children with autism spectrum disorder (ASD). In this study, we examined and compared two groups of infants on their sensitivity to audio-visual asynchrony for a social (speaking face) and non-social event (bouncing ball) and assessed the relations between multisensory integration and language production. Infants at elevated likelihood of developing ASD were less sensitive to audio-visual synchrony for the social event than infants without elevated likelihood. Among infants without elevated likelihood, greater sensitivity to audio-visual synchrony for the social event was associated with a larger productive vocabulary. CONCLUSION Findings suggest that early deficits in multisensory integration may impair language development among infants with elevated likelihood of developing ASD. WHAT IS KNOWN •Perceptual integration of auditory and visual cues within speech is important for language development. •Prior work suggests that children with ASD are less sensitive to the temporal synchrony within audio-visual speech. WHAT IS NEW •In this study, infants at elevated likelihood of developing ASD showed a larger temporal binding window for adynamic social event (Speaking Face) than TD infants, suggesting less efficient multisensory integration.
Collapse
Affiliation(s)
- Kirin N Suri
- Institute for the Study of Child Development, Rutgers Robert Wood Johnson Medical School, 89 French Street, New Brunswick, NJ, 08901, USA.,Children's Health at Hackensack Meridian, Hackensack, NJ, 07601, USA
| | - Margaret Whedon
- Institute for the Study of Child Development, Rutgers Robert Wood Johnson Medical School, 89 French Street, New Brunswick, NJ, 08901, USA.
| | - Michael Lewis
- Institute for the Study of Child Development, Rutgers Robert Wood Johnson Medical School, 89 French Street, New Brunswick, NJ, 08901, USA
| |
Collapse
|
10
|
Feldman JI, Tu A, Conrad JG, Kuang W, Santapuram P, Woynaroski TG. The Impact of Singing on Visual and Multisensory Speech Perception in Children on the Autism Spectrum. Multisens Res 2022; 36:57-74. [PMID: 36731528 PMCID: PMC9924934 DOI: 10.1163/22134808-bja10087] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/10/2022] [Accepted: 11/22/2022] [Indexed: 12/31/2022]
Abstract
Autistic children show reduced multisensory integration of audiovisual speech stimuli in response to the McGurk illusion. Previously, it has been shown that adults can integrate sung McGurk tokens. These sung speech tokens offer more salient visual and auditory cues, in comparison to the spoken tokens, which may increase the identification and integration of visual speech cues in autistic children. Forty participants (20 autism, 20 non-autistic peers) aged 7-14 completed the study. Participants were presented with speech tokens in four modalities: auditory-only, visual-only, congruent audiovisual, and incongruent audiovisual (i.e., McGurk; auditory 'ba' and visual 'ga'). Tokens were also presented in two formats: spoken and sung. Participants indicated what they perceived via a four-button response box (i.e., 'ba', 'ga', 'da', or 'tha'). Accuracies and perception of the McGurk illusion were calculated for each modality and format. Analysis of visual-only identification indicated a significant main effect of format, whereby participants were more accurate in sung versus spoken trials, but no significant main effect of group or interaction effect. Analysis of the McGurk trials indicated no significant main effect of format or group and no significant interaction effect. Sung speech tokens improved identification of visual speech cues, but did not boost the integration of visual cues with heard speech across groups. Additional work is needed to determine what properties of spoken speech contributed to the observed improvement in visual accuracy and to evaluate whether more prolonged exposure to sung speech may yield effects on multisensory integration.
Collapse
Affiliation(s)
- Jacob I. Feldman
- Department of Hearing and Speech Sciences, Vanderbilt
University Medical Center, Nashville, TN, USA
- Frist Center for Autism and Innovation, Vanderbilt
University, Nashville, TN, USA
| | - Alexander Tu
- Neuroscience Undergraduate Program, Vanderbilt University,
Nashville, TN, USA
- Present Address: Department of Otolaryngology and
Communication Sciences, Medical College of Wisconsin, Milwaukee, WI, USA
| | - Julie G. Conrad
- Neuroscience Undergraduate Program, Vanderbilt University,
Nashville, TN, USA
- Present Address: Department of Pediatrics, University of
Illinois, Chicago, IL, USA
| | - Wayne Kuang
- Neuroscience Undergraduate Program, Vanderbilt University,
Nashville, TN, USA
- Present Address: Department of Pediatrics, Los Angeles
County and University of Southern California (LAC+USC) Medical Center, University of
Southern California, Los Angeles, CA, USA
| | - Pooja Santapuram
- Neuroscience Undergraduate Program, Vanderbilt University,
Nashville, TN, USA
- Present Address: Department of Anesthesiology, Columbia
University Irving Medical Center, New York, NY, USA
| | - Tiffany G. Woynaroski
- Department of Hearing and Speech Sciences, Vanderbilt
University Medical Center, Nashville, TN, USA
- Frist Center for Autism and Innovation, Vanderbilt
University, Nashville, TN, USA
- Vanderbilt Kennedy Center, Vanderbilt University Medical
Center, Nashville, TN, USA
- Vanderbilt Brain Institute, Vanderbilt University,
Nashville, TN, USA
| |
Collapse
|
11
|
Crosse MJ, Foxe JJ, Tarrit K, Freedman EG, Molholm S. Resolution of impaired multisensory processing in autism and the cost of switching sensory modality. Commun Biol 2022; 5:601. [PMID: 35773473 PMCID: PMC9246932 DOI: 10.1038/s42003-022-03519-1] [Citation(s) in RCA: 13] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/17/2021] [Accepted: 05/23/2022] [Indexed: 11/09/2022] Open
Abstract
Children with autism spectrum disorders (ASD) exhibit alterations in multisensory processing, which may contribute to the prevalence of social and communicative deficits in this population. Resolution of multisensory deficits has been observed in teenagers with ASD for complex, social speech stimuli; however, whether this resolution extends to more basic multisensory processing deficits remains unclear. Here, in a cohort of 364 participants we show using simple, non-social audiovisual stimuli that deficits in multisensory processing observed in high-functioning children and teenagers with ASD are not evident in adults with the disorder. Computational modelling indicated that multisensory processing transitions from a default state of competition to one of facilitation, and that this transition is delayed in ASD. Further analysis revealed group differences in how sensory channels are weighted, and how this is impacted by preceding cross-sensory inputs. Our findings indicate that there is a complex and dynamic interplay among the sensory systems that differs considerably in individuals with ASD. Crosse et al. study a cohort of 364 participants with autism spectrum disorders (ASD) and matched controls, and show that deficits in multisensory processing observed in high-functioning children and teenagers with ASD are not evident in adults with the disorder. Using computational modelling they go on to demonstrate that there is a delayed transition of multisensory processing from a default state of competition to one of facilitation in ASD, as well as differences in sensory weighting and the ability to switch between sensory modalities, which sheds light on the interplay among sensory systems that differ in ASD individuals.
Collapse
Affiliation(s)
- Michael J Crosse
- The Cognitive Neurophysiology Laboratory, Department of Pediatrics, Albert Einstein College of Medicine, Bronx, NY, USA. .,The Dominick P. Purpura Department of Neuroscience, Rose F. Kennedy Intellectual and Developmental Disabilities Research Center, Albert Einstein College of Medicine, Bronx, NY, USA. .,Trinity Centre for Biomedical Engineering, Department of Mechanical, Manufacturing & Biomedical Engineering, Trinity College Dublin, Dublin, Ireland.
| | - John J Foxe
- The Cognitive Neurophysiology Laboratory, Department of Pediatrics, Albert Einstein College of Medicine, Bronx, NY, USA.,The Dominick P. Purpura Department of Neuroscience, Rose F. Kennedy Intellectual and Developmental Disabilities Research Center, Albert Einstein College of Medicine, Bronx, NY, USA.,The Cognitive Neurophysiology Laboratory, Del Monte Institute for Neuroscience, Department of Neuroscience, University of Rochester School of Medicine and Dentistry, Rochester, NY, USA
| | - Katy Tarrit
- The Cognitive Neurophysiology Laboratory, Del Monte Institute for Neuroscience, Department of Neuroscience, University of Rochester School of Medicine and Dentistry, Rochester, NY, USA
| | - Edward G Freedman
- The Cognitive Neurophysiology Laboratory, Del Monte Institute for Neuroscience, Department of Neuroscience, University of Rochester School of Medicine and Dentistry, Rochester, NY, USA
| | - Sophie Molholm
- The Cognitive Neurophysiology Laboratory, Department of Pediatrics, Albert Einstein College of Medicine, Bronx, NY, USA. .,The Dominick P. Purpura Department of Neuroscience, Rose F. Kennedy Intellectual and Developmental Disabilities Research Center, Albert Einstein College of Medicine, Bronx, NY, USA. .,The Cognitive Neurophysiology Laboratory, Del Monte Institute for Neuroscience, Department of Neuroscience, University of Rochester School of Medicine and Dentistry, Rochester, NY, USA.
| |
Collapse
|
12
|
Chen N, Watanabe K, Wada M. People With High Autistic Traits Show Fewer Consensual Crossmodal Correspondences Between Visual Features and Tastes. Front Psychol 2021; 12:714277. [PMID: 34566793 PMCID: PMC8457010 DOI: 10.3389/fpsyg.2021.714277] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/25/2021] [Accepted: 08/11/2021] [Indexed: 11/19/2022] Open
Abstract
Crossmodal correspondences between visual features (e.g., color/shape) and tastes have been extensively documented in recent years. Visual colors and shapes have been shown to consensually match to specific tastes. Meanwhile, individuals with autism spectrum disorder are reported to have atypical sensory processing and deficits in multisensory integration. However, the influence of autistic traits on the formation of such correspondences is relatively unknown. Here, we examined whether autistic traits could influence visual–taste associations using an online questionnaire survey among Japanese participants. The results showed that the participants exhibited strong color–taste, shape–taste, and shape–color associations, and the proportions of choosing the consensual color–taste/shape–color associations were significantly associated with autistic traits. The participants with higher autistic quotient scores chose fewer of the consensual color–taste/shape–color associations while there was no difference in choosing shape–taste associations. We interpreted the results as statistical learning with a reduced prior knowledge effect in participants with higher autistic quotient scores.
Collapse
Affiliation(s)
- Na Chen
- Department of Rehabilitation for Brain Functions, Research Institute of National Rehabilitation Center for Persons With Disabilities, Tokorozawa, Japan
| | - Katsumi Watanabe
- Faculty of Science and Engineering, Waseda University, Tokyo, Japan.,Faculty of Arts, Design, and Architecture, University of New South Wales, Sydney, NSW, Australia
| | - Makoto Wada
- Department of Rehabilitation for Brain Functions, Research Institute of National Rehabilitation Center for Persons With Disabilities, Tokorozawa, Japan
| |
Collapse
|
13
|
Feng S, Lu H, Wang Q, Li T, Fang J, Chen L, Yi L. Face-viewing patterns predict audiovisual speech integration in autistic children. Autism Res 2021; 14:2592-2602. [PMID: 34415113 DOI: 10.1002/aur.2598] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/09/2021] [Revised: 07/06/2021] [Accepted: 07/30/2021] [Indexed: 11/10/2022]
Abstract
Autistic children show audiovisual speech integration deficits, though the underlying mechanisms remain unclear. The present study examined how audiovisual speech integration deficits in autistic children could be affected by their looking patterns. We measured audiovisual speech integration in 26 autistic children and 26 typically developing (TD) children (4- to 7-year-old) employing the McGurk task (a videotaped speaker uttering phonemes with her eyes open or closed) and tracked their eye movements. We found that, compared with TD children, autistic children showed weaker audiovisual speech integration (i.e., the McGurk effect) in the open-eyes condition and similar audiovisual speech integration in the closed-eyes condition. Autistic children viewed the speaker's mouth less in non-McGurk trials than in McGurk trials in both conditions. Importantly, autistic children's weaker audiovisual speech integration could be predicted by their reduced mouth-looking time. The present study indicated that atypical face-viewing patterns could serve as one of the cognitive mechanisms of audiovisual speech integration deficits in autistic children. LAY SUMMARY: McGurk effect occurs when the visual part of a phoneme (e.g., "ga") and the auditory part of another phoneme (e.g., "ba") uttered by a speaker were integrated into a fused perception (e.g., "da"). The present study examined how McGurk effect in autistic children could be affected by their looking patterns for the speaker's face. We found that less looking time for the speaker's mouth in autistic children could predict weaker McGurk effect. As McGurk effect manifests audiovisual speech integration, our findings imply that we could improve audiovisual speech integration in autistic children by directing them to look at the speaker's mouth in future intervention.
Collapse
Affiliation(s)
- Shuyuan Feng
- Institute for Applied Linguistics, School of Foreign Languages, Central South University, Changsha, Hunan, China.,School of Psychological and Cognitive Sciences and Beijing Key Laboratory of Behavior and Mental Health, Peking University, Beijing, China
| | - Haoyang Lu
- Academy for Advanced Interdisciplinary Studies, Peking University, Beijing, China.,Peking-Tsinghua Center for Life Sciences, Peking University, Beijing, China
| | - Qiandong Wang
- Beijing Key Laboratory of Applied Experimental Psychology, National Demonstration Center for Experimental Psychology Education, Faculty of Psychology, Beijing Normal University, Beijing, China
| | - Tianbi Li
- School of Psychological and Cognitive Sciences and Beijing Key Laboratory of Behavior and Mental Health, Peking University, Beijing, China
| | - Jing Fang
- Qingdao Autism Research Institute, Qingdao, China
| | - Lihan Chen
- School of Psychological and Cognitive Sciences and Beijing Key Laboratory of Behavior and Mental Health, Peking University, Beijing, China
| | - Li Yi
- School of Psychological and Cognitive Sciences and Beijing Key Laboratory of Behavior and Mental Health, Peking University, Beijing, China.,IDG/McGovern Institute for Brain Research at PKU, Peking University, Beijing, China
| |
Collapse
|
14
|
Feldman JI, Conrad JG, Kuang W, Tu A, Liu Y, Simon DM, Wallace MT, Woynaroski TG. Relations Between the McGurk Effect, Social and Communication Skill, and Autistic Features in Children with and without Autism. J Autism Dev Disord 2021; 52:1920-1928. [PMID: 34101080 PMCID: PMC8842559 DOI: 10.1007/s10803-021-05074-w] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 05/09/2021] [Indexed: 12/20/2022]
Abstract
Children with autism show alterations in multisensory integration that have been theoretically and empirically linked with the core and related features of autism. It is unclear, however, to what extent multisensory integration maps onto features of autism within children with and without autism. This study, thus, evaluates relations between audiovisual integration and core and related autism features across children with and without autism. Thirty-six children reported perceptions of the McGurk illusion during a psychophysical task. Parents reported on participants' autistic features. Increased report of illusory percepts tended to covary with reduced autistic features and greater communication skill. Some relations, though, were moderated by group. This work suggests that associations between multisensory integration and higher-order skills are present, but in some instances vary according to diagnostic group.
Collapse
Affiliation(s)
- Jacob I Feldman
- Department of Hearing and Speech Sciences, Vanderbilt University, MCE 8310 South Tower, 1215 21st Avenue South, Nashville, TN, 37232, USA.
- Frist Center for Autism & Innovation, Vanderbilt University, Nashville, TN, USA.
- Department of Hearing and Speech Sciences, Vanderbilt University Medical Center, Nashville, TN, USA.
| | - Julie G Conrad
- Neuroscience Undergraduate Program, Vanderbilt University, Nashville, TN, USA
- College of Medicine, University of Illinois, Chicago, IL, USA
| | - Wayne Kuang
- Neuroscience Undergraduate Program, Vanderbilt University, Nashville, TN, USA
- College of Osteopathic Medicine of the Pacific, Western University of Health Sciences, Pomona, CA, USA
| | - Alexander Tu
- Neuroscience Undergraduate Program, Vanderbilt University, Nashville, TN, USA
- College of Medicine, University of Nebraska Medical Center, Omaha, NE, USA
| | - Yupeng Liu
- Neuroscience Undergraduate Program, Vanderbilt University, Nashville, TN, USA
- Washington University School of Medicine, Washington University in St. Louis, St. Louis, MO, USA
| | - David M Simon
- Vanderbilt Brain Institute, Vanderbilt University, Nashville, TN, USA
| | - Mark T Wallace
- Department of Hearing and Speech Sciences, Vanderbilt University, MCE 8310 South Tower, 1215 21st Avenue South, Nashville, TN, 37232, USA
- Frist Center for Autism & Innovation, Vanderbilt University, Nashville, TN, USA
- Vanderbilt Brain Institute, Vanderbilt University, Nashville, TN, USA
- Vanderbilt Kennedy Center, Vanderbilt University Medical Center, Nashville, TN, USA
- Department of Psychology, Vanderbilt University, Nashville, TN, USA
- Department of Psychiatry and Behavioral Sciences, Vanderbilt University Medical Center, Nashville, TN, USA
- Department of Pharmacology, Vanderbilt University, Nashville, TN, USA
| | - Tiffany G Woynaroski
- Frist Center for Autism & Innovation, Vanderbilt University, Nashville, TN, USA
- Department of Hearing and Speech Sciences, Vanderbilt University Medical Center, Nashville, TN, USA
- Vanderbilt Brain Institute, Vanderbilt University, Nashville, TN, USA
- Vanderbilt Kennedy Center, Vanderbilt University Medical Center, Nashville, TN, USA
| |
Collapse
|
15
|
Abstract
Visual speech cues play an important role in speech recognition, and the McGurk effect is a classic demonstration of this. In the original McGurk & Macdonald (Nature 264, 746-748 1976) experiment, 98% of participants reported an illusory "fusion" percept of /d/ when listening to the spoken syllable /b/ and watching the visual speech movements for /g/. However, more recent work shows that subject and task differences influence the proportion of fusion responses. In the current study, we varied task (forced-choice vs. open-ended), stimulus set (including /d/ exemplars vs. not), and data collection environment (lab vs. Mechanical Turk) to investigate the robustness of the McGurk effect. Across experiments, using the same stimuli to elicit the McGurk effect, we found fusion responses ranging from 10% to 60%, thus showing large variability in the likelihood of experiencing the McGurk effect across factors that are unrelated to the perceptual information provided by the stimuli. Rather than a robust perceptual illusion, we therefore argue that the McGurk effect exists only for some individuals under specific task situations.Significance: This series of studies re-evaluates the classic McGurk effect, which shows the relevance of visual cues on speech perception. We highlight the importance of taking into account subject variables and task differences, and challenge future researchers to think carefully about the perceptual basis of the McGurk effect, how it is defined, and what it can tell us about audiovisual integration in speech.
Collapse
|
16
|
Ujiie Y, Takahashi K. Weaker McGurk Effect for Rubin's Vase-Type Speech in People With High Autistic Traits. Multisens Res 2021; 34:1-17. [PMID: 33873157 DOI: 10.1163/22134808-bja10047] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/18/2020] [Accepted: 04/05/2021] [Indexed: 11/19/2022]
Abstract
While visual information from facial speech modulates auditory speech perception, it is less influential on audiovisual speech perception among autistic individuals than among typically developed individuals. In this study, we investigated the relationship between autistic traits (Autism-Spectrum Quotient; AQ) and the influence of visual speech on the recognition of Rubin's vase-type speech stimuli with degraded facial speech information. Participants were 31 university students (13 males and 18 females; mean age: 19.2, SD: 1.13 years) who reported normal (or corrected-to-normal) hearing and vision. All participants completed three speech recognition tasks (visual, auditory, and audiovisual stimuli) and the AQ-Japanese version. The results showed that accuracies of speech recognition for visual (i.e., lip-reading) and auditory stimuli were not significantly related to participants' AQ. In contrast, audiovisual speech perception was less susceptible to facial speech perception among individuals with high rather than low autistic traits. The weaker influence of visual information on audiovisual speech perception in autism spectrum disorder (ASD) was robust regardless of the clarity of the visual information, suggesting a difficulty in the process of audiovisual integration rather than in the visual processing of facial speech.
Collapse
Affiliation(s)
- Yuta Ujiie
- Graduate School of Psychology, Chukyo University, 101-2 Yagoto Honmachi, Showa-ku, Nagoya-shi, Aichi, 466-8666, Japan
- Japan Society for the Promotion of Science, Kojimachi Business Center Building, 5-3-1 Kojimachi, Chiyoda-ku, Tokyo 102-0083, Japan
- Research and Development Initiative, Chuo University, 1-13-27, Kasuga, Bunkyo-ku, Tokyo, 112-8551, Japan
| | - Kohske Takahashi
- School of Psychology, Chukyo University, 101-2 Yagoto Honmachi, Showa-ku, Nagoya-shi, Aichi, 466-8666, Japan
| |
Collapse
|
17
|
Mercado E, Chow K, Church BA, Lopata C. Perceptual category learning in autism spectrum disorder: Truth and consequences. Neurosci Biobehav Rev 2020; 118:689-703. [PMID: 32910926 PMCID: PMC7744437 DOI: 10.1016/j.neubiorev.2020.08.016] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/29/2020] [Revised: 08/01/2020] [Accepted: 08/29/2020] [Indexed: 02/01/2023]
Abstract
The ability to categorize is fundamental to cognitive development. Some categories emerge effortlessly and rapidly while others can take years of experience to acquire. Children with autism spectrum disorder (ASD) are often able to name and sort objects, suggesting that their categorization abilities are largely intact. However, recent experimental work shows that the categories formed by individuals with ASD may diverge substantially from those that most people learn. This review considers how atypical perceptual category learning can affect cognitive development in children with ASD and how atypical categorization may contribute to many of the socially problematic symptoms associated with this disorder. Theoretical approaches to understanding perceptual processing and category learning at both the behavioral and neural levels are assessed in relation to known alterations in perceptual category learning associated with ASD. Mismatches between the ways in which children learn to organize perceived events relative to their peers and adults can accumulate over time, leading to difficulties in communication, social interactions, academic performance, and behavioral flexibility.
Collapse
Affiliation(s)
- Eduardo Mercado
- University at Buffalo, The State University of New York, Dept. of Psychology, Buffalo, NY, 14260, USA.
| | - Karen Chow
- University at Buffalo, The State University of New York, Dept. of Psychology, Buffalo, NY, 14260, USA
| | - Barbara A Church
- Georgia State University, Language Research Center, 3401 Panthersville Rd., Decatur, GA, 30034, USA
| | - Christopher Lopata
- Canisius College, Institute for Autism Research, Science Hall, 2001 Main St., Buffalo, NY, 14208, USA
| |
Collapse
|
18
|
Ujiie Y, Wakabayashi A. Intact lip-reading but weaker McGurk effect in individuals with high autistic traits. INTERNATIONAL JOURNAL OF DEVELOPMENTAL DISABILITIES 2019; 68:47-55. [PMID: 35173963 PMCID: PMC8843195 DOI: 10.1080/20473869.2019.1699350] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/12/2019] [Revised: 11/05/2019] [Accepted: 11/26/2019] [Indexed: 06/14/2023]
Abstract
A weaker McGurk effect is observed in individuals with autism spectrum disorder (ASD); weaker integration is considered to be the key to understanding how low-order atypical processing leads to their maladaptive social behaviors. However, the mechanism for this weaker McGurk effect has not been fully understood. Here, we investigated (1) whether the weaker McGurk effect in individuals with high autistic traits is caused by poor lip-reading ability and (2) whether the hearing environment modifies the weaker McGurk effect in individuals with high autistic traits. To confirm them, we conducted two analogue studies among university students, based on the dimensional model of ASD. Results showed that individuals with high autistic traits have intact lip-reading ability as well as abilities to listen and recognize audiovisual congruent speech (Experiment 1). Furthermore, a weaker McGurk effect in individuals with high autistic traits, which appear under the without-noise condition, would disappear under the high noise condition (Experiments 1 and 2). Our findings suggest that high background noise might shift weight on the visual cue, thereby increasing the strength of the McGurk effect among individuals with high autistic traits.
Collapse
Affiliation(s)
- Yuta Ujiie
- Graduate School of Psychology, Chukyo University, Nagoya-shi, Aichi, Japan
- Japan Society for the Promotion of Science, Tokyo, Japan
- Research and Development Initiative, Chuo University, Tokyo, Japan
| | | |
Collapse
|
19
|
Tietze FA, Hundertmark L, Roy M, Zerr M, Sinke C, Wiswede D, Walter M, Münte TF, Szycik GR. Auditory Deficits in Audiovisual Speech Perception in Adult Asperger's Syndrome: fMRI Study. Front Psychol 2019; 10:2286. [PMID: 31649597 PMCID: PMC6795762 DOI: 10.3389/fpsyg.2019.02286] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/11/2019] [Accepted: 09/24/2019] [Indexed: 01/23/2023] Open
Abstract
Audiovisual (AV) integration deficits have been proposed to underlie difficulties in speech perception in Asperger’s syndrome (AS). It is not known, if the AV deficits are related to alterations in sensory processing at the level of unisensory processing or at levels of conjoint multisensory processing. Functional Magnetic-resonance images (MRI) was performed in 16 adult subjects with AS and 16 healthy controls (HC) matched for age, gender, and verbal IQ as they were exposed to disyllabic AV congruent and AV incongruent nouns. A simple semantic categorization task was used to ensure subjects’ attention to the stimuli. The left auditory cortex (BA41) showed stronger activation in HC than in subjects with AS with no interaction regarding AV congruency. This suggests that alterations in auditory processing in unimodal low-level areas underlie AV speech perception deficits in AS. Whether this is signaling a difficulty in the deployment of attention remains to be demonstrated.
Collapse
Affiliation(s)
- Fabian-Alexander Tietze
- Department of Psychiatry, Social Psychiatry and Psychotherapy, Hannover Medical School, Hanover, Germany
| | - Laura Hundertmark
- Department of Psychiatry, Social Psychiatry and Psychotherapy, Hannover Medical School, Hanover, Germany
| | - Mandy Roy
- Department of Psychiatry, Social Psychiatry and Psychotherapy, Hannover Medical School, Hanover, Germany.,Asklepios Clinic North - Ochsenzoll, Hamburg, Germany
| | - Michael Zerr
- Department of Psychosomatic Medicine and Psychotherapy, Hannover Medical School, Hanover, Germany
| | - Christopher Sinke
- Department of Psychiatry, Social Psychiatry and Psychotherapy, Hannover Medical School, Hanover, Germany
| | - Daniel Wiswede
- Institute of Psychology II, University of Lübeck, Lübeck, Germany.,Department of Neurology, University of Lübeck, Lübeck, Germany
| | - Martin Walter
- Department of Psychiatry and Psychotherapy, University of Tübingen, Tübingen, Germany
| | - Thomas F Münte
- Institute of Psychology II, University of Lübeck, Lübeck, Germany.,Department of Neurology, University of Lübeck, Lübeck, Germany
| | - Gregor R Szycik
- Department of Psychiatry, Social Psychiatry and Psychotherapy, Hannover Medical School, Hanover, Germany
| |
Collapse
|
20
|
Ostrolenk A, Bao VA, Mottron L, Collignon O, Bertone A. Reduced multisensory facilitation in adolescents and adults on the Autism Spectrum. Sci Rep 2019; 9:11965. [PMID: 31427634 PMCID: PMC6700191 DOI: 10.1038/s41598-019-48413-9] [Citation(s) in RCA: 13] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/23/2018] [Accepted: 07/29/2019] [Indexed: 01/01/2023] Open
Abstract
Individuals with autism are reported to integrate information from visual and auditory channels in an idiosyncratic way. Multisensory integration (MSI) of simple, non-social stimuli (i.e., flashes and beeps) was evaluated in adolescents and adults with (n = 20) and without autism (n = 19) using a reaction time (RT) paradigm using audio, visual, and audiovisual stimuli. For each participant, the race model analysis compares the RTs on the audiovisual condition to a bound value computed from the unimodal RTs that reflects the effect of redundancy. If the actual audiovisual RTs are significantly faster than this bound, the race model is violated, indicating evidence of MSI. Our results show that the race model violation occurred only for the typically-developing (TD) group. While the TD group shows evidence of MSI, the autism group does not. These results suggest that multisensory integration of simple information, void of social content or complexity, is altered in autism. Individuals with autism may not benefit from the advantage conferred by multisensory stimulation to the same extent as TD individuals. Altered MSI for simple, non-social information may have cascading effects on more complex perceptual processes related to language and behaviour in autism.
Collapse
Affiliation(s)
- Alexia Ostrolenk
- Perceptual Neuroscience Lab for Autism and Development (PNLab), McGill University, Montreal, Canada.,University of Montreal Center of Excellence for Pervasive Developmental Disorders (CETEDUM), CIUSSS du Nord-de-l'Île de Montréal, Montreal, Canada
| | - Vanessa A Bao
- Perceptual Neuroscience Lab for Autism and Development (PNLab), McGill University, Montreal, Canada.,School/Applied Child Psychology, Department of Education and Counselling Psychology, McGill University, Montreal, Canada
| | - Laurent Mottron
- University of Montreal Center of Excellence for Pervasive Developmental Disorders (CETEDUM), CIUSSS du Nord-de-l'Île de Montréal, Montreal, Canada
| | - Olivier Collignon
- Centre for Mind/Brain Science (CIMeC), University of Trento, Trento, Italy.,Institut de recherche en Psychologie (IPSY) et en Neuroscience (IoNS), Université de Louvain-la-Neuve, Ottignies-Louvain-la-Neuve, Belgium
| | - Armando Bertone
- Perceptual Neuroscience Lab for Autism and Development (PNLab), McGill University, Montreal, Canada. .,School/Applied Child Psychology, Department of Education and Counselling Psychology, McGill University, Montreal, Canada. .,University of Montreal Center of Excellence for Pervasive Developmental Disorders (CETEDUM), CIUSSS du Nord-de-l'Île de Montréal, Montreal, Canada.
| |
Collapse
|
21
|
van Laarhoven T, Stekelenburg JJ, Vroomen J. Increased sub-clinical levels of autistic traits are associated with reduced multisensory integration of audiovisual speech. Sci Rep 2019; 9:9535. [PMID: 31267024 PMCID: PMC6606565 DOI: 10.1038/s41598-019-46084-0] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/04/2019] [Accepted: 06/20/2019] [Indexed: 12/21/2022] Open
Abstract
Recent studies suggest that sub-clinical levels of autistic symptoms may be related to reduced processing of artificial audiovisual stimuli. It is unclear whether these findings extent to more natural stimuli such as audiovisual speech. The current study examined the relationship between autistic traits measured by the Autism spectrum Quotient and audiovisual speech processing in a large non-clinical population using a battery of experimental tasks assessing audiovisual perceptual binding, visual enhancement of speech embedded in noise and audiovisual temporal processing. Several associations were found between autistic traits and audiovisual speech processing. Increased autistic-like imagination was related to reduced perceptual binding measured by the McGurk illusion. Increased overall autistic symptomatology was associated with reduced visual enhancement of speech intelligibility in noise. Participants reporting increased levels of rigid and restricted behaviour were more likely to bind audiovisual speech stimuli over longer temporal intervals, while an increased tendency to focus on local aspects of sensory inputs was related to a more narrow temporal binding window. These findings demonstrate that increased levels of autistic traits may be related to alterations in audiovisual speech processing, and are consistent with the notion of a spectrum of autistic traits that extends to the general population.
Collapse
Affiliation(s)
- Thijs van Laarhoven
- Department of Cognitive Neuropsychology, Tilburg University, P.O. Box 90153, 5000 LE, Tilburg, The Netherlands.
| | - Jeroen J Stekelenburg
- Department of Cognitive Neuropsychology, Tilburg University, P.O. Box 90153, 5000 LE, Tilburg, The Netherlands
| | - Jean Vroomen
- Department of Cognitive Neuropsychology, Tilburg University, P.O. Box 90153, 5000 LE, Tilburg, The Netherlands
| |
Collapse
|
22
|
Feldman JI, Dunham K, Cassidy M, Wallace MT, Liu Y, Woynaroski TG. Audiovisual multisensory integration in individuals with autism spectrum disorder: A systematic review and meta-analysis. Neurosci Biobehav Rev 2018; 95:220-234. [PMID: 30287245 PMCID: PMC6291229 DOI: 10.1016/j.neubiorev.2018.09.020] [Citation(s) in RCA: 87] [Impact Index Per Article: 14.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/18/2018] [Revised: 09/10/2018] [Accepted: 09/25/2018] [Indexed: 02/04/2023]
Abstract
An ever-growing literature has aimed to determine how individuals with autism spectrum disorder (ASD) differ from their typically developing (TD) peers on measures of multisensory integration (MSI) and to ascertain the degree to which differences in MSI are associated with the broad range of symptoms associated with ASD. Findings, however, have been highly variable across the studies carried out to date. The present work systematically reviews and quantitatively synthesizes the large literature on audiovisual MSI in individuals with ASD to evaluate the cumulative evidence for (a) group differences between individuals with ASD and TD peers, (b) correlations between MSI and autism symptoms in individuals with ASD and (c) study level factors that may moderate findings (i.e., explain differential effects) observed across studies. To identify eligible studies, a comprehensive search strategy was employed using the ProQuest search engine, PubMed database, forwards and backwards citation searches, direct author contact, and hand-searching of select conference proceedings. A significant between-group difference in MSI was evident in the literature, with individuals with ASD demonstrating worse audiovisual integration on average across studies compared to TD controls. This effect was moderated by mean participant age, such that between-group differences were more pronounced in younger samples. The mean correlation between MSI and autism and related symptomatology was also significant, indicating that increased audiovisual integration in individuals with ASD is associated with better language/communication abilities and/or reduced autism symptom severity in the extant literature. This effect was moderated by whether the stimuli were linguistic versus non-linguistic in nature, such that correlation magnitudes tended to be significantly greater when linguistic stimuli were utilized in the measure of MSI. Limitations and future directions for primary and meta-analytic research are discussed.
Collapse
Affiliation(s)
- Jacob I Feldman
- Department of Hearing and Speech Sciences, Vanderbilt University, 1215 21st Ave S, MCE South Tower 8310, Nashville, TN, 37232, USA.
| | - Kacie Dunham
- Neuroscience Undergraduate Program, Vanderbilt University, Nashville, TN, USA
| | - Margaret Cassidy
- Neuroscience Undergraduate Program, Vanderbilt University, Nashville, TN, USA
| | - Mark T Wallace
- Department of Psychology, Vanderbilt University, Nashville, TN, USA; Department of Psychiatry and Behavioral Sciences, Vanderbilt University Medical Center, Nashville, TN, USA; Department of Pharmacology, Vanderbilt University, Nashville, TN, USA; Vanderbilt Kennedy Center, Vanderbilt University Medical Center, 110 Magnolia Cir, Nashville, TN, 37203, USA; Vanderbilt Brain Institute, Vanderbilt University, 465 21st Avenue South, Nashville, TN, 37232, USA; Department of Hearing and Speech Sciences, Vanderbilt University Medical Center, 1215 21st Ave S, MCE South Tower 8310, Nashville, TN, 27323, USA.
| | - Yupeng Liu
- Neuroscience Undergraduate Program, Vanderbilt University, Nashville, TN, USA
| | - Tiffany G Woynaroski
- Vanderbilt Kennedy Center, Vanderbilt University Medical Center, 110 Magnolia Cir, Nashville, TN, 37203, USA; Vanderbilt Brain Institute, Vanderbilt University, 465 21st Avenue South, Nashville, TN, 37232, USA; Department of Hearing and Speech Sciences, Vanderbilt University Medical Center, 1215 21st Ave S, MCE South Tower 8310, Nashville, TN, 27323, USA.
| |
Collapse
|
23
|
Brown VA, Hedayati M, Zanger A, Mayn S, Ray L, Dillman-Hasso N, Strand JF. What accounts for individual differences in susceptibility to the McGurk effect? PLoS One 2018; 13:e0207160. [PMID: 30418995 PMCID: PMC6231656 DOI: 10.1371/journal.pone.0207160] [Citation(s) in RCA: 25] [Impact Index Per Article: 4.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/08/2018] [Accepted: 10/25/2018] [Indexed: 11/29/2022] Open
Abstract
The McGurk effect is a classic audiovisual speech illusion in which discrepant auditory and visual syllables can lead to a fused percept (e.g., an auditory /bɑ/ paired with a visual /gɑ/ often leads to the perception of /dɑ/). The McGurk effect is robust and easily replicated in pooled group data, but there is tremendous variability in the extent to which individual participants are susceptible to it. In some studies, the rate at which individuals report fusion responses ranges from 0% to 100%. Despite its widespread use in the audiovisual speech perception literature, the roots of the wide variability in McGurk susceptibility are largely unknown. This study evaluated whether several perceptual and cognitive traits are related to McGurk susceptibility through correlational analyses and mixed effects modeling. We found that an individual's susceptibility to the McGurk effect was related to their ability to extract place of articulation information from the visual signal (i.e., a more fine-grained analysis of lipreading ability), but not to scores on tasks measuring attentional control, processing speed, working memory capacity, or auditory perceptual gradiency. These results provide support for the claim that a small amount of the variability in susceptibility to the McGurk effect is attributable to lipreading skill. In contrast, cognitive and perceptual abilities that are commonly used predictors in individual differences studies do not appear to underlie susceptibility to the McGurk effect.
Collapse
Affiliation(s)
- Violet A. Brown
- Department of Psychology, Carleton College, Northfield, Minnesota, United States of America
| | - Maryam Hedayati
- Department of Psychology, Carleton College, Northfield, Minnesota, United States of America
| | - Annie Zanger
- Department of Psychology, Carleton College, Northfield, Minnesota, United States of America
| | - Sasha Mayn
- Department of Psychology, Carleton College, Northfield, Minnesota, United States of America
| | - Lucia Ray
- Department of Psychology, Carleton College, Northfield, Minnesota, United States of America
| | - Naseem Dillman-Hasso
- Department of Psychology, Carleton College, Northfield, Minnesota, United States of America
| | - Julia F. Strand
- Department of Psychology, Carleton College, Northfield, Minnesota, United States of America
| |
Collapse
|
24
|
Published estimates of group differences in multisensory integration are inflated. PLoS One 2018; 13:e0202908. [PMID: 30231054 PMCID: PMC6145544 DOI: 10.1371/journal.pone.0202908] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/22/2018] [Accepted: 08/10/2018] [Indexed: 11/19/2022] Open
Abstract
A common measure of multisensory integration is the McGurk effect, an illusion in which incongruent auditory and visual speech are integrated to produce an entirely different percept. Published studies report that participants who differ in age, gender, culture, native language, or traits related to neurological or psychiatric disorders also differ in their susceptibility to the McGurk effect. These group-level differences are used as evidence for fundamental alterations in sensory processing between populations. Using empirical data and statistical simulations tested under a range of conditions, we show that published estimates of group differences in the McGurk effect are inflated when only statistically significant (p < 0.05) results are published. With a sample size typical of published studies, a group difference of 10% would be reported as 31%. As a consequence of this inflation, follow-up studies often fail to replicate published reports of large between-group differences. Inaccurate estimates of effect sizes and replication failures are especially problematic in studies of clinical populations involving expensive and time-consuming interventions, such as training paradigms to improve sensory processing. Reducing effect size inflation and increasing replicability requires increasing the number of participants by an order of magnitude compared with current practice.
Collapse
|
25
|
Stevenson RA, Sheffield SW, Butera IM, Gifford RH, Wallace MT. Multisensory Integration in Cochlear Implant Recipients. Ear Hear 2018; 38:521-538. [PMID: 28399064 DOI: 10.1097/aud.0000000000000435] [Citation(s) in RCA: 47] [Impact Index Per Article: 7.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/25/2022]
Abstract
Speech perception is inherently a multisensory process involving integration of auditory and visual cues. Multisensory integration in cochlear implant (CI) recipients is a unique circumstance in that the integration occurs after auditory deprivation and the provision of hearing via the CI. Despite the clear importance of multisensory cues for perception, in general, and for speech intelligibility, specifically, the topic of multisensory perceptual benefits in CI users has only recently begun to emerge as an area of inquiry. We review the research that has been conducted on multisensory integration in CI users to date and suggest a number of areas needing further research. The overall pattern of results indicates that many CI recipients show at least some perceptual gain that can be attributable to multisensory integration. The extent of this gain, however, varies based on a number of factors, including age of implantation and specific task being assessed (e.g., stimulus detection, phoneme perception, word recognition). Although both children and adults with CIs obtain audiovisual benefits for phoneme, word, and sentence stimuli, neither group shows demonstrable gain for suprasegmental feature perception. Additionally, only early-implanted children and the highest performing adults obtain audiovisual integration benefits similar to individuals with normal hearing. Increasing age of implantation in children is associated with poorer gains resultant from audiovisual integration, suggesting a sensitive period in development for the brain networks that subserve these integrative functions, as well as length of auditory experience. This finding highlights the need for early detection of and intervention for hearing loss, not only in terms of auditory perception, but also in terms of the behavioral and perceptual benefits of audiovisual processing. Importantly, patterns of auditory, visual, and audiovisual responses suggest that underlying integrative processes may be fundamentally different between CI users and typical-hearing listeners. Future research, particularly in low-level processing tasks such as signal detection will help to further assess mechanisms of multisensory integration for individuals with hearing loss, both with and without CIs.
Collapse
Affiliation(s)
- Ryan A Stevenson
- 1Department of Psychology, University of Western Ontario, London, Ontario, Canada; 2Brain and Mind Institute, University of Western Ontario, London, Ontario, Canada; 3Walter Reed National Military Medical Center, Audiology and Speech Pathology Center, London, Ontario, Canada; 4Vanderbilt Brain Institute, Nashville, Tennesse; 5Vanderbilt Kennedy Center, Nashville, Tennesse; 6Department of Psychology, Vanderbilt University, Nashville, Tennesse; 7Department of Psychiatry, Vanderbilt University Medical Center, Nashville, Tennesse; and 8Department of Hearing and Speech Sciences, Vanderbilt University Medical Center, Nashville, Tennesse
| | | | | | | | | |
Collapse
|
26
|
Moro SS, Steeves JKE. Audiovisual plasticity following early abnormal visual experience: Reduced McGurk effect in people with one eye. Neurosci Lett 2018; 672:103-107. [PMID: 29474874 DOI: 10.1016/j.neulet.2018.02.031] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/20/2017] [Revised: 02/13/2018] [Accepted: 02/15/2018] [Indexed: 11/29/2022]
Abstract
Previously, we have shown that people who have had one eye surgically removed early in life during visual development have enhanced sound localization [1] and lack visual dominance, commonly observed in binocular and monocular (eye-patched) viewing controls [2]. Despite these changes, people with one eye integrate auditory and visual components of multisensory events optimally [3]. The current study investigates how people with one eye perceive the McGurk effect, an audiovisual illusion where a new syllable is perceived when visual lip movements do not match the corresponding sound [4]. We compared individuals with one eye to binocular and monocular viewing controls and found that they have a significantly smaller McGurk effect compared to binocular controls. Additionally, monocular controls tended to perceive the McGurk effect less often than binocular controls suggesting a small transient modulation of the McGurk effect. These results suggest altered weighting of the auditory and visual modalities with both short and long-term monocular viewing. These results indicate the presence of permanent adaptive perceptual accommodations in people who have lost one eye early in life that may serve to mitigate the loss of binocularity during early brain development.
Collapse
Affiliation(s)
- Stefania S Moro
- Department of Psychology and Centre for Vision Research, York University, Toronto, Canada; The Hospital for Sick Children, Toronto, Canada
| | - Jennifer K E Steeves
- Department of Psychology and Centre for Vision Research, York University, Toronto, Canada; The Hospital for Sick Children, Toronto, Canada.
| |
Collapse
|
27
|
Evers K, Van der Hallen R, Noens I, Wagemans J. Perceptual Organization in Individuals With Autism Spectrum Disorder. CHILD DEVELOPMENT PERSPECTIVES 2018. [DOI: 10.1111/cdep.12280] [Citation(s) in RCA: 13] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/27/2022]
|
28
|
Individual differences and the effect of face configuration information in the McGurk effect. Exp Brain Res 2018; 236:973-984. [PMID: 29383400 DOI: 10.1007/s00221-018-5188-4] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/08/2017] [Accepted: 01/23/2018] [Indexed: 10/18/2022]
Abstract
The McGurk effect, which denotes the influence of visual information on audiovisual speech perception, is less frequently observed in individuals with autism spectrum disorder (ASD) compared to those without it; the reason for this remains unclear. Several studies have suggested that facial configuration context might play a role in this difference. More specifically, people with ASD show a local processing bias for faces-that is, they process global face information to a lesser extent. This study examined the role of facial configuration context in the McGurk effect in 46 healthy students. Adopting an analogue approach using the Autism-Spectrum Quotient (AQ), we sought to determine whether this facial configuration context is crucial to previously observed reductions in the McGurk effect in people with ASD. Lip-reading and audiovisual syllable identification tasks were assessed via presentation of upright normal, inverted normal, upright Thatcher-type, and inverted Thatcher-type faces. When the Thatcher-type face was presented, perceivers were found to be sensitive to the misoriented facial characteristics, causing them to perceive a weaker McGurk effect than when the normal face was presented (this is known as the McThatcher effect). Additionally, the McGurk effect was weaker in individuals with high AQ scores than in those with low AQ scores in the incongruent audiovisual condition, regardless of their ability to read lips or process facial configuration contexts. Our findings, therefore, do not support the assumption that individuals with ASD show a weaker McGurk effect due to a difficulty in processing facial configuration context.
Collapse
|
29
|
Beker S, Foxe JJ, Molholm S. Ripe for solution: Delayed development of multisensory processing in autism and its remediation. Neurosci Biobehav Rev 2018; 84:182-192. [PMID: 29162518 PMCID: PMC6389331 DOI: 10.1016/j.neubiorev.2017.11.008] [Citation(s) in RCA: 66] [Impact Index Per Article: 11.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/25/2017] [Revised: 11/09/2017] [Accepted: 11/13/2017] [Indexed: 12/24/2022]
Abstract
Difficulty integrating inputs from different sensory sources is commonly reported in individuals with Autism Spectrum Disorder (ASD). Accumulating evidence consistently points to altered patterns of behavioral reactions and neural activity when individuals with ASD observe or act upon information arriving through multiple sensory systems. For example, impairments in the integration of seen and heard speech appear to be particularly acute, with obvious implications for interpersonal communication. Here, we explore the literature on multisensory processing in autism with a focus on developmental trajectories. While much remains to be understood, some consistent observations emerge. Broadly, sensory integration deficits are found in children with an ASD whereas these appear to be much ameliorated, or even fully recovered, in older teenagers and adults on the spectrum. This protracted delay in the development of multisensory processing raises the possibility of applying early intervention strategies focused on multisensory integration, to accelerate resolution of these functions. We also consider how dysfunctional cross-sensory oscillatory neural communication may be one key pathway to impaired multisensory processing in ASD.
Collapse
Affiliation(s)
- Shlomit Beker
- The Sheryl and Daniel R. Tishman Cognitive Neurophysiology Laboratory, Department of Pediatrics, Albert Einstein College of Medicine, Bronx, NY, United States; Rose F. Kennedy Intellectual and Developmental Disabilities Research Center (IDDRC), Department of Neuroscience, Albert Einstein College of Medicine, Bronx, NY, United States
| | - John J Foxe
- The Sheryl and Daniel R. Tishman Cognitive Neurophysiology Laboratory, Department of Pediatrics, Albert Einstein College of Medicine, Bronx, NY, United States; Rose F. Kennedy Intellectual and Developmental Disabilities Research Center (IDDRC), Department of Neuroscience, Albert Einstein College of Medicine, Bronx, NY, United States; The Ernest J. Del Monte Institute for Neuroscience, Department of Neuroscience, University of Rochester Medical Center, Rochester, NY, United States
| | - Sophie Molholm
- The Sheryl and Daniel R. Tishman Cognitive Neurophysiology Laboratory, Department of Pediatrics, Albert Einstein College of Medicine, Bronx, NY, United States; Rose F. Kennedy Intellectual and Developmental Disabilities Research Center (IDDRC), Department of Neuroscience, Albert Einstein College of Medicine, Bronx, NY, United States; The Ernest J. Del Monte Institute for Neuroscience, Department of Neuroscience, University of Rochester Medical Center, Rochester, NY, United States.
| |
Collapse
|
30
|
Thye MD, Bednarz HM, Herringshaw AJ, Sartin EB, Kana RK. The impact of atypical sensory processing on social impairments in autism spectrum disorder. Dev Cogn Neurosci 2018; 29:151-167. [PMID: 28545994 PMCID: PMC6987885 DOI: 10.1016/j.dcn.2017.04.010] [Citation(s) in RCA: 224] [Impact Index Per Article: 37.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/06/2016] [Revised: 02/25/2017] [Accepted: 04/18/2017] [Indexed: 02/03/2023] Open
Abstract
Altered sensory processing has been an important feature of the clinical descriptions of autism spectrum disorder (ASD). There is evidence that sensory dysregulation arises early in the progression of ASD and impacts social functioning. This paper reviews behavioral and neurobiological evidence that describes how sensory deficits across multiple modalities (vision, hearing, touch, olfaction, gustation, and multisensory integration) could impact social functions in ASD. Theoretical models of ASD and their implications for the relationship between sensory and social functioning are discussed. Furthermore, neural differences in anatomy, function, and connectivity of different regions underlying sensory and social processing are also discussed. We conclude that there are multiple mechanisms through which early sensory dysregulation in ASD could cascade into social deficits across development. Future research is needed to clarify these mechanisms, and specific focus should be given to distinguish between deficits in primary sensory processing and altered top-down attentional and cognitive processes.
Collapse
Affiliation(s)
- Melissa D Thye
- Department of Psychology, University of Alabama at Birmingham, Birmingham, AL 35233, United States
| | - Haley M Bednarz
- Department of Psychology, University of Alabama at Birmingham, Birmingham, AL 35233, United States
| | - Abbey J Herringshaw
- Department of Psychology, University of Alabama at Birmingham, Birmingham, AL 35233, United States
| | - Emma B Sartin
- Department of Psychology, University of Alabama at Birmingham, Birmingham, AL 35233, United States
| | - Rajesh K Kana
- Department of Psychology, University of Alabama at Birmingham, Birmingham, AL 35233, United States.
| |
Collapse
|
31
|
Alsius A, Paré M, Munhall KG. Forty Years After Hearing Lips and Seeing Voices: the McGurk Effect Revisited. Multisens Res 2018; 31:111-144. [PMID: 31264597 DOI: 10.1163/22134808-00002565] [Citation(s) in RCA: 52] [Impact Index Per Article: 8.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/04/2016] [Accepted: 03/09/2017] [Indexed: 11/19/2022]
Abstract
Since its discovery 40 years ago, the McGurk illusion has been usually cited as a prototypical paradigmatic case of multisensory binding in humans, and has been extensively used in speech perception studies as a proxy measure for audiovisual integration mechanisms. Despite the well-established practice of using the McGurk illusion as a tool for studying the mechanisms underlying audiovisual speech integration, the magnitude of the illusion varies enormously across studies. Furthermore, the processing of McGurk stimuli differs from congruent audiovisual processing at both phenomenological and neural levels. This questions the suitability of this illusion as a tool to quantify the necessary and sufficient conditions under which audiovisual integration occurs in natural conditions. In this paper, we review some of the practical and theoretical issues related to the use of the McGurk illusion as an experimental paradigm. We believe that, without a richer understanding of the mechanisms involved in the processing of the McGurk effect, experimenters should be really cautious when generalizing data generated by McGurk stimuli to matching audiovisual speech events.
Collapse
Affiliation(s)
- Agnès Alsius
- Psychology Department, Queen's University, Humphrey Hall, 62 Arch St., Kingston, Ontario, K7L 3N6 Canada
| | - Martin Paré
- Psychology Department, Queen's University, Humphrey Hall, 62 Arch St., Kingston, Ontario, K7L 3N6 Canada
| | - Kevin G Munhall
- Psychology Department, Queen's University, Humphrey Hall, 62 Arch St., Kingston, Ontario, K7L 3N6 Canada
| |
Collapse
|
32
|
Feczko E, Balba NM, Miranda-Dominguez O, Cordova M, Karalunas SL, Irwin L, Demeter DV, Hill AP, Langhorst BH, Grieser Painter J, Van Santen J, Fombonne EJ, Nigg JT, Fair DA. Subtyping cognitive profiles in Autism Spectrum Disorder using a Functional Random Forest algorithm. Neuroimage 2017; 172:674-688. [PMID: 29274502 DOI: 10.1016/j.neuroimage.2017.12.044] [Citation(s) in RCA: 75] [Impact Index Per Article: 10.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/07/2017] [Revised: 12/11/2017] [Accepted: 12/14/2017] [Indexed: 12/21/2022] Open
Abstract
DSM-5 Autism Spectrum Disorder (ASD) comprises a set of neurodevelopmental disorders characterized by deficits in social communication and interaction and repetitive behaviors or restricted interests, and may both affect and be affected by multiple cognitive mechanisms. This study attempts to identify and characterize cognitive subtypes within the ASD population using our Functional Random Forest (FRF) machine learning classification model. This model trained a traditional random forest model on measures from seven tasks that reflect multiple levels of information processing. 47 ASD diagnosed and 58 typically developing (TD) children between the ages of 9 and 13 participated in this study. Our RF model was 72.7% accurate, with 80.7% specificity and 63.1% sensitivity. Using the random forest model, the FRF then measures the proximity of each subject to every other subject, generating a distance matrix between participants. This matrix is then used in a community detection algorithm to identify subgroups within the ASD and TD groups, and revealed 3 ASD and 4 TD putative subgroups with unique behavioral profiles. We then examined differences in functional brain systems between diagnostic groups and putative subgroups using resting-state functional connectivity magnetic resonance imaging (rsfcMRI). Chi-square tests revealed a significantly greater number of between group differences (p < .05) within the cingulo-opercular, visual, and default systems as well as differences in inter-system connections in the somato-motor, dorsal attention, and subcortical systems. Many of these differences were primarily driven by specific subgroups suggesting that our method could potentially parse the variation in brain mechanisms affected by ASD.
Collapse
Affiliation(s)
- E Feczko
- Department of Behavioral Neuroscience, Oregon Health & Science University, Portland, OR 97239, USA; Department of Medical Informatics and Clinical Epidemiology, Oregon Health & Science University, Portland OR, 97239, USA.
| | - N M Balba
- Department of Behavioral Neuroscience, Oregon Health & Science University, Portland, OR 97239, USA
| | - O Miranda-Dominguez
- Department of Behavioral Neuroscience, Oregon Health & Science University, Portland, OR 97239, USA
| | - M Cordova
- Department of Behavioral Neuroscience, Oregon Health & Science University, Portland, OR 97239, USA
| | - S L Karalunas
- Department of Psychiatry, Oregon Health & Science University, Portland, OR 97239, USA
| | - L Irwin
- Department of Behavioral Neuroscience, Oregon Health & Science University, Portland, OR 97239, USA
| | - D V Demeter
- The University of Texas at Austin, Department of Psychology, Austin, TX 78713, USA
| | - A P Hill
- Department of Pediatrics, Oregon Health & Science University, Portland, OR 97239, USA; Center for Spoken Language Understanding, Institute on Development & Disability, Oregon Health & Science University, Portland, OR 97239, USA
| | - B H Langhorst
- Department of Behavioral Neuroscience, Oregon Health & Science University, Portland, OR 97239, USA
| | - J Grieser Painter
- Department of Behavioral Neuroscience, Oregon Health & Science University, Portland, OR 97239, USA
| | - J Van Santen
- Department of Pediatrics, Oregon Health & Science University, Portland, OR 97239, USA; Center for Spoken Language Understanding, Institute on Development & Disability, Oregon Health & Science University, Portland, OR 97239, USA
| | - E J Fombonne
- Department of Pediatrics, Oregon Health & Science University, Portland, OR 97239, USA; Department of Psychiatry, Oregon Health & Science University, Portland, OR 97239, USA
| | - J T Nigg
- Department of Behavioral Neuroscience, Oregon Health & Science University, Portland, OR 97239, USA; Department of Psychiatry, Oregon Health & Science University, Portland, OR 97239, USA; Advanced Imaging Research Center, Oregon Health & Science University, Portland, OR 97239, USA
| | - D A Fair
- Department of Behavioral Neuroscience, Oregon Health & Science University, Portland, OR 97239, USA; Department of Psychiatry, Oregon Health & Science University, Portland, OR 97239, USA; Advanced Imaging Research Center, Oregon Health & Science University, Portland, OR 97239, USA
| |
Collapse
|
33
|
Cardon GJ, Hepburn S, Rojas DC. Structural Covariance of Sensory Networks, the Cerebellum, and Amygdala in Autism Spectrum Disorder. Front Neurol 2017; 8:615. [PMID: 29230189 PMCID: PMC5712069 DOI: 10.3389/fneur.2017.00615] [Citation(s) in RCA: 17] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/07/2017] [Accepted: 11/03/2017] [Indexed: 11/13/2022] Open
Abstract
Sensory dysfunction is a core symptom of autism spectrum disorder (ASD), and abnormalities with sensory responsivity and processing can be extremely debilitating to ASD patients and their families. However, relatively little is known about the underlying neuroanatomical and neurophysiological factors that lead to sensory abnormalities in ASD. Investigation into these aspects of ASD could lead to significant advancements in our general knowledge about ASD, as well as provide targets for treatment and inform diagnostic procedures. Thus, the current study aimed to measure the covariation of volumes of brain structures (i.e., structural magnetic resonance imaging) that may be involved in abnormal sensory processing, in order to infer connectivity of these brain regions. Specifically, we quantified the structural covariation of sensory-related cerebral cortical structures, in addition to the cerebellum and amygdala by computing partial correlations between the structural volumes of these structures. These analyses were performed in participants with ASD (n = 36), as well as typically developing peers (n = 32). Results showed decreased structural covariation between sensory-related cortical structures, especially between the left and right cerebral hemispheres, in participants with ASD. In contrast, these same participants presented with increased structural covariation of structures in the right cerebral hemisphere. Additionally, sensory-related cerebral structures exhibited decreased structural covariation with functionally identified cerebellar networks. Also, the left amygdala showed significantly increased structural covariation with cerebral structures related to visual processing. Taken together, these results may suggest several patterns of altered connectivity both within and between cerebral cortices and other brain structures that may be related to sensory processing.
Collapse
Affiliation(s)
- Garrett J Cardon
- Department of Psychology, Colorado State University, Fort Collins, CO, United States
| | - Susan Hepburn
- Department of Human Development and Family Studies, Colorado State University, Fort Collins, CO, United States
| | - Donald C Rojas
- Department of Psychology, Colorado State University, Fort Collins, CO, United States
| |
Collapse
|
34
|
Cuppini C, Ursino M, Magosso E, Ross LA, Foxe JJ, Molholm S. A Computational Analysis of Neural Mechanisms Underlying the Maturation of Multisensory Speech Integration in Neurotypical Children and Those on the Autism Spectrum. Front Hum Neurosci 2017; 11:518. [PMID: 29163099 PMCID: PMC5670153 DOI: 10.3389/fnhum.2017.00518] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/12/2017] [Accepted: 10/11/2017] [Indexed: 11/13/2022] Open
Abstract
Failure to appropriately develop multisensory integration (MSI) of audiovisual speech may affect a child's ability to attain optimal communication. Studies have shown protracted development of MSI into late-childhood and identified deficits in MSI in children with an autism spectrum disorder (ASD). Currently, the neural basis of acquisition of this ability is not well understood. Here, we developed a computational model informed by neurophysiology to analyze possible mechanisms underlying MSI maturation, and its delayed development in ASD. The model posits that strengthening of feedforward and cross-sensory connections, responsible for the alignment of auditory and visual speech sound representations in posterior superior temporal gyrus/sulcus, can explain behavioral data on the acquisition of MSI. This was simulated by a training phase during which the network was exposed to unisensory and multisensory stimuli, and projections were crafted by Hebbian rules of potentiation and depression. In its mature architecture, the network also reproduced the well-known multisensory McGurk speech effect. Deficits in audiovisual speech perception in ASD were well accounted for by fewer multisensory exposures, compatible with a lack of attention, but not by reduced synaptic connectivity or synaptic plasticity.
Collapse
Affiliation(s)
- Cristiano Cuppini
- Department of Electric, Electronic and Information Engineering, University of Bologna, Bologna, Italy
| | - Mauro Ursino
- Department of Electric, Electronic and Information Engineering, University of Bologna, Bologna, Italy
| | - Elisa Magosso
- Department of Electric, Electronic and Information Engineering, University of Bologna, Bologna, Italy
| | - Lars A. Ross
- Departments of Pediatrics and Neuroscience, Albert Einstein College of Medicine, Bronx, NY, United States
| | - John J. Foxe
- Departments of Pediatrics and Neuroscience, Albert Einstein College of Medicine, Bronx, NY, United States
- Department of Neuroscience and The Del Monte Institute for Neuroscience, University of Rochester School of Medicine, Rochester, NY, United States
| | - Sophie Molholm
- Departments of Pediatrics and Neuroscience, Albert Einstein College of Medicine, Bronx, NY, United States
| |
Collapse
|
35
|
Stevenson RA, Toulmin JK, Youm A, Besney RMA, Schulz SE, Barense MD, Ferber S. Increases in the autistic trait of attention to detail are associated with decreased multisensory temporal adaptation. Sci Rep 2017; 7:14354. [PMID: 29085016 PMCID: PMC5662613 DOI: 10.1038/s41598-017-14632-1] [Citation(s) in RCA: 28] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/15/2017] [Accepted: 10/12/2017] [Indexed: 11/09/2022] Open
Abstract
Recent empirical evidence suggests that autistic individuals perceive the world differently than their typically-developed peers. One theoretical account, the predictive coding hypothesis, posits that autistic individuals show a decreased reliance on previous perceptual experiences, which may relate to autism symptomatology. We tested this through a well-characterized, audiovisual statistical-learning paradigm in which typically-developed participants were first adapted to consistent temporal relationships between audiovisual stimulus pairs (audio-leading, synchronous, visual-leading) and then performed a simultaneity judgement task with audiovisual stimulus pairs varying in temporal offset from auditory-leading to visual-leading. Following exposure to the visual-leading adaptation phase, participants' perception of synchrony was biased towards visual-leading presentations, reflecting the statistical regularities of their previously experienced environment. Importantly, the strength of adaptation was significantly related to the level of autistic traits that the participant exhibited, measured by the Autism Quotient (AQ). This was specific to the Attention to Detail subscale of the AQ that assesses the perceptual propensity to focus on fine-grain aspects of sensory input at the expense of more integrative perceptions. More severe Attention to Detail was related to weaker adaptation. These results support the predictive coding framework, and suggest that changes in sensory perception commonly reported in autism may contribute to autistic symptomatology.
Collapse
Affiliation(s)
- Ryan A Stevenson
- Western University, Department of Psychology, London, ON, Canada.
- Western University, Brain and Mind Institute, London, ON, Canada.
- Western University, Program in Neuroscience, London, ON, Canada.
- Western University, Department of Psychiatry, London, ON, Canada.
- York University, Centre for Vision Research, Toronto, ON, Canada.
| | - Jennifer K Toulmin
- The University of Toronto, Department of Psychology, Toronto, ON, Canada
| | - Ariana Youm
- The University of Toronto, Department of Psychology, Toronto, ON, Canada
| | | | - Samantha E Schulz
- Western University, Department of Psychology, London, ON, Canada
- Western University, Brain and Mind Institute, London, ON, Canada
| | - Morgan D Barense
- The University of Toronto, Department of Psychology, Toronto, ON, Canada
- The Rotman Research Institute, Toronto, ON, Canada
| | - Susanne Ferber
- The University of Toronto, Department of Psychology, Toronto, ON, Canada
- The Rotman Research Institute, Toronto, ON, Canada
| |
Collapse
|
36
|
Multisensory Integration of Low-level Information in Autism Spectrum Disorder: Measuring Susceptibility to the Flash-Beep Illusion. J Autism Dev Disord 2017; 47:2535-2543. [DOI: 10.1007/s10803-017-3172-7] [Citation(s) in RCA: 12] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/19/2022]
|
37
|
Stevenson RA, Segers M, Ncube BL, Black KR, Bebko JM, Ferber S, Barense MD. The cascading influence of multisensory processing on speech perception in autism. AUTISM : THE INTERNATIONAL JOURNAL OF RESEARCH AND PRACTICE 2017; 22:609-624. [PMID: 28506185 DOI: 10.1177/1362361317704413] [Citation(s) in RCA: 76] [Impact Index Per Article: 10.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/18/2022]
Abstract
It has been recently theorized that atypical sensory processing in autism relates to difficulties in social communication. Through a series of tasks concurrently assessing multisensory temporal processes, multisensory integration and speech perception in 76 children with and without autism, we provide the first behavioral evidence of such a link. Temporal processing abilities in children with autism contributed to impairments in speech perception. This relationship was significantly mediated by their abilities to integrate social information across auditory and visual modalities. These data describe the cascading impact of sensory abilities in autism, whereby temporal processing impacts multisensory information of social information, which, in turn, contributes to deficits in speech perception. These relationships were found to be specific to autism, specific to multisensory but not unisensory integration, and specific to the processing of social information.
Collapse
Affiliation(s)
| | | | | | | | | | - Susanne Ferber
- 3 University of Toronto, Canada.,4 Rotman Research Institute at Baycrest, Canada
| | - Morgan D Barense
- 3 University of Toronto, Canada.,4 Rotman Research Institute at Baycrest, Canada
| |
Collapse
|
38
|
Haß K, Sinke C, Reese T, Roy M, Wiswede D, Dillo W, Oranje B, Szycik GR. Enlarged temporal integration window in schizophrenia indicated by the double-flash illusion. Cogn Neuropsychiatry 2017; 22:145-158. [PMID: 28253091 DOI: 10.1080/13546805.2017.1287693] [Citation(s) in RCA: 30] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 10/20/2022]
Abstract
INTRODUCTION In the present study we were interested in the processing of audio-visual integration in schizophrenia compared to healthy controls. The amount of sound-induced double-flash illusions served as an indicator of audio-visual integration. We expected an altered integration as well as a different window of temporal integration for patients. METHODS Fifteen schizophrenia patients and 15 healthy volunteers matched for age and gender were included in this study. We used stimuli with eight different temporal delays (stimulus onset asynchronys (SOAs) 25, 50, 75, 100, 125, 150, 200 and 300 ms) to induce a double-flash illusion. Group differences and the widths of temporal integration windows were calculated on percentages of reported double-flash illusions. RESULTS Patients showed significantly more illusions (ca. 36-44% vs. 9-16% in control subjects) for SOAs 150-300. The temporal integration window for control participants went from SOAs 25 to 200 whereas for patients integration was found across all included temporal delays. We found no significant relationship between the amount of illusions and either illness severity, chlorpromazine equivalent doses or duration of illness in patients. CONCLUSIONS Our results are interpreted in favour of an enlarged temporal integration window for audio-visual stimuli in schizophrenia patients, which is consistent with previous research.
Collapse
Affiliation(s)
- Katharina Haß
- a Clinic for Psychiatry, Social Psychiatry and Psychotherapy , Hannover Medical School , Hanover , Germany
| | - Christopher Sinke
- a Clinic for Psychiatry, Social Psychiatry and Psychotherapy , Hannover Medical School , Hanover , Germany
| | - Tanya Reese
- a Clinic for Psychiatry, Social Psychiatry and Psychotherapy , Hannover Medical School , Hanover , Germany
| | - Mandy Roy
- a Clinic for Psychiatry, Social Psychiatry and Psychotherapy , Hannover Medical School , Hanover , Germany
| | - Daniel Wiswede
- b Department of Neurology , University of Lübeck , Lübeck , Germany
| | - Wolfgang Dillo
- a Clinic for Psychiatry, Social Psychiatry and Psychotherapy , Hannover Medical School , Hanover , Germany
| | - Bob Oranje
- c Department of Psychiatry , University Medical Centre , Utrecht , The Netherlands
| | - Gregor R Szycik
- a Clinic for Psychiatry, Social Psychiatry and Psychotherapy , Hannover Medical School , Hanover , Germany
| |
Collapse
|
39
|
Cassidy S, Stenger B, Van Dongen L, Yanagisawa K, Anderson R, Wan V, Baron-Cohen S, Cipolla R. Expressive visual text-to-speech as an assistive technology for individuals with autism spectrum conditions. COMPUTER VISION AND IMAGE UNDERSTANDING : CVIU 2016; 148:193-200. [PMID: 27375348 PMCID: PMC4913554 DOI: 10.1016/j.cviu.2015.08.011] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 03/29/2015] [Revised: 08/07/2015] [Accepted: 08/31/2015] [Indexed: 06/06/2023]
Abstract
Adults with Autism Spectrum Conditions (ASC) experience marked difficulties in recognising the emotions of others and responding appropriately. The clinical characteristics of ASC mean that face to face or group interventions may not be appropriate for this clinical group. This article explores the potential of a new interactive technology, converting text to emotionally expressive speech, to improve emotion processing ability and attention to faces in adults with ASC. We demonstrate a method for generating a near-videorealistic avatar (XpressiveTalk), which can produce a video of a face uttering inputted text, in a large variety of emotional tones. We then demonstrate that general population adults can correctly recognize the emotions portrayed by XpressiveTalk. Adults with ASC are significantly less accurate than controls, but still above chance levels for inferring emotions from XpressiveTalk. Both groups are significantly more accurate when inferring sad emotions from XpressiveTalk compared to the original actress, and rate these expressions as significantly more preferred and realistic. The potential applications for XpressiveTalk as an assistive technology for adults with ASC is discussed.
Collapse
Affiliation(s)
- S.A. Cassidy
- Centre for Psychology, Behaviour and Achievement, Coventry University, Coventry CV1 5FB, UK
- Autism Research Centre, Department of Psychiatry, University of Cambridge, Douglas House, 18B Trumpington Road, Cambridge CB2 8AH, UK
| | - B. Stenger
- Toshiba Research Europe Ltd., 208 Science Park, Cambridge CB4 0GZ, UK
| | - L. Van Dongen
- Maastricht University, Faculty of Psychology, Maastricht 6200 MD, The Netherlands
| | - K. Yanagisawa
- Toshiba Research Europe Ltd., 208 Science Park, Cambridge CB4 0GZ, UK
| | - R. Anderson
- Autism Research Centre, Department of Psychiatry, University of Cambridge, Douglas House, 18B Trumpington Road, Cambridge CB2 8AH, UK
- Cambridgeshire and Peterborough Foundation NHS Trust, CLASS Clinic, UK
| | - V. Wan
- Autism Research Centre, Department of Psychiatry, University of Cambridge, Douglas House, 18B Trumpington Road, Cambridge CB2 8AH, UK
- Cambridgeshire and Peterborough Foundation NHS Trust, CLASS Clinic, UK
| | - S. Baron-Cohen
- Autism Research Centre, Department of Psychiatry, University of Cambridge, Douglas House, 18B Trumpington Road, Cambridge CB2 8AH, UK
- Cambridgeshire and Peterborough Foundation NHS Trust, CLASS Clinic, UK
| | - R. Cipolla
- Toshiba Research Europe Ltd., 208 Science Park, Cambridge CB4 0GZ, UK
- Engineering Department, University of Cambridge, Cambridge CB2 1PZ, UK
| |
Collapse
|
40
|
Abstract
In the McGurk effect, incongruent auditory and visual syllables are perceived as a third, completely different syllable. This striking illusion has become a popular assay of multisensory integration for individuals and clinical populations. However, there is enormous variability in how often the illusion is evoked by different stimuli and how often the illusion is perceived by different individuals. Most studies of the McGurk effect have used only one stimulus, making it impossible to separate stimulus and individual differences. We created a probabilistic model to separately estimate stimulus and individual differences in behavioral data from 165 individuals viewing up to 14 different McGurk stimuli. The noisy encoding of disparity (NED) model characterizes stimuli by their audiovisual disparity and characterizes individuals by how noisily they encode the stimulus disparity and by their disparity threshold for perceiving the illusion. The model accurately described perception of the McGurk effect in our sample, suggesting that differences between individuals are stable across stimulus differences. The most important benefit of the NED model is that it provides a method to compare multisensory integration across individuals and groups without the confound of stimulus differences. An added benefit is the ability to predict frequency of the McGurk effect for stimuli never before seen by an individual.
Collapse
|
41
|
Zmigrod L, Zmigrod S. On the Temporal Precision of Thought: Individual Differences in the Multisensory Temporal Binding Window Predict Performance on Verbal and Nonverbal Problem Solving Tasks. Multisens Res 2016. [DOI: 10.1163/22134808-00002532] [Citation(s) in RCA: 19] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/06/2023]
Abstract
Although psychology is greatly preoccupied by the tight link between the way that individuals perceive the world and their intelligent, creative behavior, there is little experimental work on the relationship between individual differences in perception and cognitive ability in healthy populations. Here, individual differences in problem solving ability were examined in relation to multisensory perception as measured by tolerance for temporal asynchrony between auditory and visual inputs, i.e., the multisensory temporal binding window. The results demonstrated that enhanced performance in both verbal and nonverbal problem solving tasks (the Remote Associates Test and Raven’s Advanced Progressive Matrices Task) is predicted by a narrower audio-visual temporal binding window, which reflects greater sensitivity to subtle discrepancies in sensory inputs. This suggests that the precision of individuals’ temporal window of multisensory integration might mirror their capacities for complex reasoning and thus the precision of their thoughts.
Collapse
Affiliation(s)
- Leor Zmigrod
- Department of Psychology, University of Cambridge, Cambridge, UK
| | - Sharon Zmigrod
- Institute for Psychological Research & Leiden Institute for Brain and Cognition, Leiden University, Leiden, The Netherlands
| |
Collapse
|
42
|
Stevenson RA, Segers M, Ferber S, Barense MD, Camarata S, Wallace MT. Keeping time in the brain: Autism spectrum disorder and audiovisual temporal processing. Autism Res 2015; 9:720-38. [PMID: 26402725 DOI: 10.1002/aur.1566] [Citation(s) in RCA: 58] [Impact Index Per Article: 6.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/14/2015] [Revised: 08/22/2015] [Accepted: 08/29/2015] [Indexed: 12/21/2022]
Abstract
A growing area of interest and relevance in the study of autism spectrum disorder (ASD) focuses on the relationship between multisensory temporal function and the behavioral, perceptual, and cognitive impairments observed in ASD. Atypical sensory processing is becoming increasingly recognized as a core component of autism, with evidence of atypical processing across a number of sensory modalities. These deviations from typical processing underscore the value of interpreting ASD within a multisensory framework. Furthermore, converging evidence illustrates that these differences in audiovisual processing may be specifically related to temporal processing. This review seeks to bridge the connection between temporal processing and audiovisual perception, and to elaborate on emerging data showing differences in audiovisual temporal function in autism. We also discuss the consequence of such changes, the specific impact on the processing of different classes of audiovisual stimuli (e.g. speech vs. nonspeech, etc.), and the presumptive brain processes and networks underlying audiovisual temporal integration. Finally, possible downstream behavioral implications, and possible remediation strategies are outlined. Autism Res 2016, 9: 720-738. © 2015 International Society for Autism Research, Wiley Periodicals, Inc.
Collapse
Affiliation(s)
- Ryan A Stevenson
- Department of Psychology, University of Toronto, Toronto, Ontario, Canada
| | - Magali Segers
- Department of Psychology, York University, Toronto, Ontario, Canada
| | - Susanne Ferber
- Department of Psychology, University of Toronto, Toronto, Ontario, Canada.,Rotman Research Institute, Toronto, Ontario, Canada
| | - Morgan D Barense
- Department of Psychology, University of Toronto, Toronto, Ontario, Canada.,Rotman Research Institute, Toronto, Ontario, Canada
| | - Stephen Camarata
- Department of Hearing and Speech Sciences, Vanderbilt University Medical Center, Nashville, Tennessee.,Vanderbilt Kennedy Center, Vanderbilt University Medical Center, Nashville, Tennessee
| | - Mark T Wallace
- Department of Hearing and Speech Sciences, Vanderbilt University Medical Center, Nashville, Tennessee.,Vanderbilt Kennedy Center, Vanderbilt University Medical Center, Nashville, Tennessee.,Vanderbilt Brain Institute, Vanderbilt University Medical Center, Nashville, Tennessee.,Department of Psychology, Vanderbilt University, Nashville, Tennessee.,Department of Psychiatry, Vanderbilt University Medical Center, Nashville, Tennessee
| |
Collapse
|
43
|
Cassidy S, Mitchell P, Chapman P, Ropar D. Processing of Spontaneous Emotional Responses in Adolescents and Adults with Autism Spectrum Disorders: Effect of Stimulus Type. Autism Res 2015; 8:534-44. [PMID: 25735657 PMCID: PMC4964927 DOI: 10.1002/aur.1468] [Citation(s) in RCA: 33] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/06/2014] [Accepted: 02/04/2015] [Indexed: 11/08/2022]
Abstract
Recent research has shown that adults with autism spectrum disorders (ASD) have difficulty interpreting others' emotional responses, in order to work out what actually happened to them. It is unclear what underlies this difficulty; important cues may be missed from fast paced dynamic stimuli, or spontaneous emotional responses may be too complex for those with ASD to successfully recognise. To explore these possibilities, 17 adolescents and adults with ASD and 17 neurotypical controls viewed 21 videos and pictures of peoples' emotional responses to gifts (chocolate, a handmade novelty or Monopoly money), then inferred what gift the person received and the emotion expressed by the person while eye movements were measured. Participants with ASD were significantly more accurate at distinguishing who received a chocolate or homemade gift from static (compared to dynamic) stimuli, but significantly less accurate when inferring who received Monopoly money from static (compared to dynamic) stimuli. Both groups made similar emotion attributions to each gift in both conditions (positive for chocolate, feigned positive for homemade and confused for Monopoly money). Participants with ASD only made marginally significantly fewer fixations to the eyes of the face, and face of the person than typical controls in both conditions. Results suggest adolescents and adults with ASD can distinguish subtle emotion cues for certain emotions (genuine from feigned positive) when given sufficient processing time, however, dynamic cues are informative for recognising emotion blends (e.g. smiling in confusion). This indicates difficulties processing complex emotion responses in ASD. Autism Res2015, 8: 534–544. © 2015 International Society for Autism Research, Wiley Periodicals, Inc.
Collapse
Affiliation(s)
- Sarah Cassidy
- School of Psychology, University of Nottingham, Science Park, Nottingham, NG7 2RD.,Centre for Research in Psychology, Behaviour and Achievement, Coventry University, Priory Street, Coventry, CV1 5FB
| | - Peter Mitchell
- School of Psychology, University of Nottingham, Science Park, Nottingham, NG7 2RD
| | - Peter Chapman
- School of Psychology, University of Nottingham, Science Park, Nottingham, NG7 2RD
| | - Danielle Ropar
- School of Psychology, University of Nottingham, Science Park, Nottingham, NG7 2RD
| |
Collapse
|
44
|
Wallace MT, Stevenson RA. The construct of the multisensory temporal binding window and its dysregulation in developmental disabilities. Neuropsychologia 2014; 64:105-23. [PMID: 25128432 PMCID: PMC4326640 DOI: 10.1016/j.neuropsychologia.2014.08.005] [Citation(s) in RCA: 195] [Impact Index Per Article: 19.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/26/2014] [Revised: 08/04/2014] [Accepted: 08/05/2014] [Indexed: 01/18/2023]
Abstract
Behavior, perception and cognition are strongly shaped by the synthesis of information across the different sensory modalities. Such multisensory integration often results in performance and perceptual benefits that reflect the additional information conferred by having cues from multiple senses providing redundant or complementary information. The spatial and temporal relationships of these cues provide powerful statistical information about how these cues should be integrated or "bound" in order to create a unified perceptual representation. Much recent work has examined the temporal factors that are integral in multisensory processing, with many focused on the construct of the multisensory temporal binding window - the epoch of time within which stimuli from different modalities is likely to be integrated and perceptually bound. Emerging evidence suggests that this temporal window is altered in a series of neurodevelopmental disorders, including autism, dyslexia and schizophrenia. In addition to their role in sensory processing, these deficits in multisensory temporal function may play an important role in the perceptual and cognitive weaknesses that characterize these clinical disorders. Within this context, focus on improving the acuity of multisensory temporal function may have important implications for the amelioration of the "higher-order" deficits that serve as the defining features of these disorders.
Collapse
Affiliation(s)
- Mark T Wallace
- Vanderbilt Brain Institute, Vanderbilt University, 465 21st Avenue South, Nashville, TN 37232, USA; Department of Hearing & Speech Sciences, Vanderbilt University, Nashville, TN, USA; Department of Psychology, Vanderbilt University, Nashville, TN, USA; Department of Psychiatry, Vanderbilt University, Nashville, TN, USA.
| | - Ryan A Stevenson
- Department of Psychology, University of Toronto, Toronto, ON, Canada
| |
Collapse
|
45
|
Stevenson RA, Segers M, Ferber S, Barense MD, Wallace MT. The impact of multisensory integration deficits on speech perception in children with autism spectrum disorders. Front Psychol 2014; 5:379. [PMID: 24904448 PMCID: PMC4033130 DOI: 10.3389/fpsyg.2014.00379] [Citation(s) in RCA: 66] [Impact Index Per Article: 6.6] [Reference Citation Analysis] [Key Words] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/05/2014] [Accepted: 04/10/2014] [Indexed: 11/16/2022] Open
Affiliation(s)
| | - Magali Segers
- Department of Psychology, York UniversityToronto, ON, Canada
| | - Susanne Ferber
- Department of Psychology, University of TorontoToronto, ON, Canada
| | - Morgan D. Barense
- Department of Psychology, University of TorontoToronto, ON, Canada
- Rotman Research InstituteToronto, ON, Canada
| | - Mark T. Wallace
- Department of Hearing and Speech Sciences, Vanderbilt University Medical CenterNashville, TN, USA
- Vanderbilt University Medical Center, Vanderbilt Brain InstituteNashville, TN, USA
- Vanderbilt Kennedy CenterNashville, TN, USA
- Department of Psychology, Vanderbilt UniversityNashville, TN, USA
- Department of Psychiatry, Vanderbilt University Medical CenterNashville, TN, USA
| |
Collapse
|