Dorsal-movement and ventral-form regions are functionally connected during visual-speech recognition

A multimodal interface for speech perception: the role of the left superior temporal sulcus in social cognition and autism

Multimodal integration is crucial for human interaction, in particular for social communication, which relies on integrating information from various sensory modalities. Recently a third visual pathway specialized in social perception was proposed, which includes the right superior temporal sulcus (STS) playing a key role in processing socially relevant cues and high-level social perception. Importantly, it has also recently been proposed that the left STS contributes to audiovisual integration of speech processing. In this article, we propose that brain areas along the right STS that support multimodal integration for social perception and cognition can be considered homologs to those in the left, language-dominant hemisphere, sustaining multimodal integration of speech and semantic concepts fundamental for social communication. Emphasizing the significance of the left STS in multimodal integration and associated processes such as multimodal attention to socially relevant stimuli, we underscore its potential relevance in comprehending neurodevelopmental conditions characterized by challenges in social communication such as autism spectrum disorder (ASD). Further research into this left lateral processing stream holds the promise of enhancing our understanding of social communication in both typical development and ASD, which may lead to more effective interventions that could improve the quality of life for individuals with atypical neurodevelopment.

Intelligibility of audiovisual sentences drives multivoxel response patterns in human superior temporal cortex

Regions of the human posterior superior temporal gyrus and sulcus (pSTG/S) respond to the visual mouth movements that constitute visual speech and the auditory vocalizations that constitute auditory speech, and neural responses in pSTG/S may underlie the perceptual benefit of visual speech for the comprehension of noisy auditory speech. We examined this possibility through the lens of multivoxel pattern responses in pSTG/S. BOLD fMRI data was collected from 22 participants presented with speech consisting of English sentences presented in five different formats: visual-only; auditory with and without added auditory noise; and audiovisual with and without auditory noise. Participants reported the intelligibility of each sentence with a button press and trials were sorted post-hoc into those that were more or less intelligible. Response patterns were measured in regions of the pSTG/S identified with an independent localizer. Noisy audiovisual sentences with very similar physical properties evoked very different response patterns depending on their intelligibility. When a noisy audiovisual sentence was reported as intelligible, the pattern was nearly identical to that elicited by clear audiovisual sentences. In contrast, an unintelligible noisy audiovisual sentence evoked a pattern like that of visual-only sentences. This effect was less pronounced for noisy auditory-only sentences, which evoked similar response patterns regardless of intelligibility. The successful integration of visual and auditory speech produces a characteristic neural signature in pSTG/S, highlighting the importance of this region in generating the perceptual benefit of visual speech.

Genetic Architecture Underlying Differential Resting-state Functional Connectivity of Subregions Within the Human Visual Cortex

The human visual cortex is a heterogeneous entity that has multiple subregions showing substantial variability in their functions and connections. We aimed to identify genes associated with resting-state functional connectivity (rsFC) of visual subregions using transcriptome-neuroimaging spatial correlations in discovery and validation datasets. Results showed that rsFC of eight visual subregions were associated with expression measures of eight gene sets, which were specifically expressed in brain tissue and showed the strongest correlations with visual behavioral processes. Moreover, there was a significant divergence in these gene sets and their functional features between medial and lateral visual subregions. Relative to those associated with lateral subregions, more genes associated with medial subregions were found to be enriched for neuropsychiatric diseases and more diverse biological functions and pathways, and to be specifically expressed in multiple types of neurons and immune cells and during the middle and late stages of cortical development. In addition to shared behavioral processes, lateral subregion associated genes were uniquely correlated with high-order cognition. These findings of commonalities and differences in the identified rsFC-related genes and their functional features across visual subregions may improve our understanding of the functional heterogeneity of the visual cortex from the perspective of underlying genetic architecture.

Visual mechanisms for voice-identity recognition flexibly adjust to auditory noise level

Recognising the identity of voices is a key ingredient of communication. Visual mechanisms support this ability: recognition is better for voices previously learned with their corresponding face (compared to a control condition). This so‐called ‘face‐benefit’ is supported by the fusiform face area (FFA), a region sensitive to facial form and identity. Behavioural findings indicate that the face‐benefit increases in noisy listening conditions. The neural mechanisms for this increase are unknown. Here, using functional magnetic resonance imaging, we examined responses in face‐sensitive regions while participants recognised the identity of auditory‐only speakers (previously learned by face) in high (SNR −4 dB) and low (SNR +4 dB) levels of auditory noise. We observed a face‐benefit in both noise levels, for most participants (16 of 21). In high‐noise, the recognition of face‐learned speakers engaged the right posterior superior temporal sulcus motion‐sensitive face area (pSTS‐mFA), a region implicated in the processing of dynamic facial cues. The face‐benefit in high‐noise also correlated positively with increased functional connectivity between this region and voice‐sensitive regions in the temporal lobe in the group of 16 participants with a behavioural face‐benefit. In low‐noise, the face‐benefit was robustly associated with increased responses in the FFA and to a lesser extent the right pSTS‐mFA. The findings highlight the remarkably adaptive nature of the visual network supporting voice‐identity recognition in auditory‐only listening conditions.

Visual Perception in Autism Spectrum Disorder: A Review of Neuroimaging Studies

Although autism spectrum disorder (ASD) is a neurodevelopmental disorder characterized by social impairments, patients with ASD frequently manifest atypical sensory behaviors. Recently, atypical sensory perception in ASD has received much attention, yet little is known about its cause or neurobiology. Herein, we review the findings from neuroimaging studies related to visual perception in ASD. Specifically, we examined the neural underpinnings of visual detection, motion perception, and face processing in ASD. Results from neuroimaging studies indicate that atypical visual perception in ASD may be influenced by attention or higher order cognitive mechanisms, and atypical face perception may be affected by disrupted social brain network. However, there is considerable evidence for atypical early visual processing in ASD. It is likely that visual perceptual abnormalities are independent of deficits of social functions or cognition. Importantly, atypical visual perception in ASD may enhance difficulties in dealing with complex and subtle social stimuli, or improve outstanding abilities in certain fields in individuals with Savant syndrome. Thus, future research is required to elucidate the characteristics and neurobiology of autistic visual perception to effectively apply these findings in the interventions of ASD.

Dorsal-movement and ventral-form regions are functionally connected during visual-speech recognition

Faces convey social information such as emotion and speech. Facial emotion processing is supported via interactions between dorsal-movement and ventral-form visual cortex regions. Here, we explored, for the first time, whether similar dorsal-ventral interactions (assessed via functional connectivity), might also exist for visual-speech processing. We then examined whether altered dorsal-ventral connectivity is observed in adults with high-functioning autism spectrum disorder (ASD), a disorder associated with impaired visual-speech recognition. We acquired functional magnetic resonance imaging (fMRI) data with concurrent eye tracking in pairwise matched control and ASD participants. In both groups, dorsal-movement regions in the visual motion area 5 (V5/MT) and the temporal visual speech area (TVSA) were functionally connected to ventral-form regions (i.e., the occipital face area [OFA] and the fusiform face area [FFA]) during the recognition of visual speech, in contrast to the recognition of face identity. Notably, parts of this functional connectivity were decreased in the ASD group compared to the controls (i.e., right V5/MT-right OFA, left TVSA-left FFA). The results confirmed our hypothesis that functional connectivity between dorsal-movement and ventral-form regions exists during visual-speech processing. Its partial dysfunction in ASD might contribute to difficulties in the recognition of dynamic face information relevant for successful face-to-face communication.