Crossmodal synesthetic congruency improves visual timing in dyslexic children

Differences in audiovisual temporal processing in autistic adults are specific to simultaneity judgments

Research has shown that children on the autism spectrum and adults with high levels of autistic traits are less sensitive to audiovisual asynchrony compared to their neurotypical peers. However, this evidence has been limited to simultaneity judgments (SJ) which require participants to consider the timing of two cues together. Given evidence of partly divergent perceptual and neural mechanisms involved in making temporal order judgments (TOJ) and SJ, and given that SJ require a more global type of processing which may be impaired in autistic individuals, here we ask whether the observed differences in audiovisual temporal processing are task and stimulus specific. We examined the ability to detect audiovisual asynchrony in a group of 26 autistic adult males and a group of age and IQ-matched neurotypical males. Participants were presented with beep-flash, point-light drumming, and face-voice displays with varying degrees of asynchrony and asked to make SJ and TOJ. The results indicated that autistic participants were less able to detect audiovisual asynchrony compared to the control group, but this effect was specific to SJ and more complex social stimuli (e.g., face-voice) with stronger semantic correspondence between the cues, requiring a more global type of processing. This indicates that audiovisual temporal processing is not generally different in autistic individuals and that a similar level of performance could be achieved by using a more local type of processing, thus informing multisensory integration theory as well as multisensory training aimed to aid perceptual abilities in this population.

Synesthetic Correspondence: An Overview

Intramodal and cross-modal perceptual grouping based on the spatial proximity and temporal closeness between multiple sensory stimuli, as an operational principle has built a coherent and meaningful representation of the multisensory event/object. To implement and investigate the cross-modal perceptual grouping, researchers have employed excellent paradigms of spatial/temporal ventriloquism and cross-modal dynamic capture and have revealed the conditional constraints as well as the functional facilitations among various correspondence of sensory properties, with featured behavioral evidence, computational framework as well as brain oscillation patterns. Typically, synesthetic correspondence as a special type of cross-modal correspondence can shape the efficiency and effect-size of cross-modal interaction. For example, factors such as pitch/loudness in the auditory dimension with size/brightness in the visual dimension could modulate the strength of the cross-modal temporal capture. The empirical behavioral findings, as well as psychophysical and neurophysiological evidence to address the cross-modal perceptual grouping and synesthetic correspondence, were summarized in this review. Finally, the potential applications (such as artificial synesthesia device) and how synesthetic correspondence interface with semantics (sensory linguistics), as well as the promising research questions in this field have been discussed.

Audiovisual multisensory integration in individuals with reading and language impairments: A systematic review and meta-analysis

Differences in sensory function have been documented for a number of neurodevelopmental conditions, including reading and language impairments. Prior studies have measured audiovisual multisensory integration (i.e., the ability to combine inputs from the auditory and visual modalities) in these populations. The present study sought to systematically review and quantitatively synthesize the extant literature on audiovisual multisensory integration in individuals with reading and language impairments. A comprehensive search strategy yielded 56 reports, of which 38 were used to extract 109 group difference and 68 correlational effect sizes. There was an overall difference between individuals with reading and language impairments and comparisons on audiovisual integration. There was a nonsignificant trend towards moderation according to sample type (i.e., reading versus language) and publication/small study bias for this model. Overall, there was a small but non-significant correlation between metrics of audiovisual integration and reading or language ability; this model was not moderated by sample or study characteristics, nor was there evidence of publication/small study bias. Limitations and future directions for primary and meta-analytic research are discussed.

Reduced audiovisual temporal sensitivity in Chinese children with dyslexia

Background

Temporal processing deficits regarding audiovisual cross-modal stimuli could affect children's speed and accuracy of decoding.

Aim

To investigate the characteristics of audiovisual temporal sensitivity (ATS) in Chinese children, with and without developmental dyslexia and its impact on reading ability.

Methods

The audiovisual simultaneity judgment and temporal order judgment tasks were performed to investigate the ATS of 106 Chinese children (53 with dyslexia) aged 8 to 12 and 37 adults without a history of dyslexia. The predictive effect of children's audiovisual time binding window on their reading ability and the effects of extra cognitive processing in the temporal order judgment task on participants' ATS were also investigated.

Outcomes and results

With increasing inter-stimulus intervals, the percentage of synchronous responses in adults declined more rapidly than in children. Adults and typically developing children had significantly narrower time binding windows than children with dyslexia. The size of visual stimuli preceding auditory stimuli time binding window had a marginally significant predictive effect on children's reading fluency. Compared with the simultaneity judgment task, the extra cognitive processing of the temporal order judgment task affected children's ATS.

Conclusion and implications

The ATS of 8-12-year-old Chinese children is immature. Chinese children with dyslexia have lower ATS than their peers.

Unisensory and multisensory temporal processing in autism and dyslexia: A systematic review and meta-analysis

This study presents a comprehensive systematic review and meta-analysis of temporal processing in autism spectrum disorder (ASD) and developmental dyslexia (DD), two neurodevelopmental disorders in which temporal processing deficits have been highly researched. The results provide strong evidence for impairments in temporal processing in both ASD (g = 0.48) and DD (g = 0.82), as measured by judgments of temporal order and simultaneity. In individual analyses, multisensory temporal processing was impaired for both ASD and DD, and unisensory auditory, visual and tactile processing were all impaired in DD. In ASD, speech stimuli showed moderate impairment effect sizes, whereas nonspeech stimuli showed small effects. Greater reading and spelling skills in DD were associated with greater temporal precision. Temporal deficits did not show changes with age in either disorder. In addition to more clearly defining temporal impairments in ASD and DD, the results highlight common and distinct patterns of temporal processing between these disorders. Deficits are discussed in relation to existing theoretical models, and recommendations are made for future research.

Audiovisual temporal integration: Cognitive processing, neural mechanisms, developmental trajectory and potential interventions

To integrate auditory and visual signals into a unified percept, the paired stimuli must co-occur within a limited time window known as the Temporal Binding Window (TBW). The width of the TBW, a proxy of audiovisual temporal integration ability, has been found to be correlated with higher-order cognitive and social functions. A comprehensive review of studies investigating audiovisual TBW reveals several findings: (1) a wide range of top-down processes and bottom-up features can modulate the width of the TBW, facilitating adaptation to the changing and multisensory external environment; (2) a large-scale brain network works in coordination to ensure successful detection of audiovisual (a)synchrony; (3) developmentally, audiovisual TBW follows a U-shaped pattern across the lifespan, with a protracted developmental course into late adolescence and rebounding in size again in late life; (4) an enlarged TBW is characteristic of a number of neurodevelopmental disorders; and (5) the TBW is highly flexible via perceptual and musical training. Interventions targeting the TBW may be able to improve multisensory function and ameliorate social communicative symptoms in clinical populations.

Temporal Reference, Attentional Modulation, and Crossmodal Assimilation

Crossmodal assimilation effect refers to the prominent phenomenon by which ensemble mean extracted from a sequence of task-irrelevant distractor events, such as auditory intervals, assimilates/biases the perception (such as visual interval) of the subsequent task-relevant target events in another sensory modality. In current experiments, using visual Ternus display, we examined the roles of temporal reference, materialized as the time information accumulated before the onset of target event, as well as the attentional modulation in crossmodal temporal interaction. Specifically, we examined how the global time interval, the mean auditory inter-intervals and the last interval in the auditory sequence assimilate and bias the subsequent percept of visual Ternus motion (element motion vs. group motion). We demonstrated that both the ensemble (geometric) mean and the last interval in the auditory sequence contribute to bias the percept of visual motion. Longer mean (or last) interval elicited more reports of group motion, whereas the shorter mean (or last) auditory intervals gave rise to more dominant percept of element motion. Importantly, observers have shown dynamic adaptation to the temporal reference of crossmodal assimilation: when the target visual Ternus stimuli were separated by a long gap interval after the preceding sound sequence, the assimilation effect by ensemble mean was reduced. Our findings suggested that crossmodal assimilation relies on a suitable temporal reference on adaptation level, and revealed a general temporal perceptual grouping principle underlying complex audio-visual interactions in everyday dynamic situations.

Cognitive Styles Differentiate Crossmodal Correspondences Between Pitch Glide and Visual Apparent Motion

Crossmodal correspondences are the automatic associations that most people have between different basic sensory stimulus attributes, dimensions, or features. For instance, people often show a systematic tendency to associate moving objects with changing pitches. Cognitive styles are defined as an individual's consistent approach to think, perceive, and remember information, and they reflect qualitative rather than quantitative differences between individuals in their thinking processes. Here we asked whether cognitive styles played a role in modulating the crossmodal interaction. We used the visual Ternus display in our study, since it elicits two distinct apparent motion percepts: element motion (with a shorter interval between the two Ternus frames) and group motion (with a longer interval between the two frames). We examined the audiovisual correspondences between the visual Ternus movement directions (upward or downward) and the changes of pitches of concurrent glides (ascending frequency or descending frequency). Moreover, we measured the cognitive styles (with the Embedded Figure Test) for each participant. The results showed that congruent correspondence between pitch-ascending (decreasing) glides and moving upward (downward) visual directions led to a more dominant percept of 'element motion', and such an effect was typically observed in the field-independent group. Importantly, field-independent participants demonstrated a high efficiency for identifying the properties of audiovisual events and applying the crossmodal correspondence in crossmodal interaction. The results suggest cognitive styles could differentiate crossmodal correspondences in crossmodal interaction.