Neural development of networks for audiovisual speech comprehension

Interhemispheric functional connectivity following prenatal or perinatal brain injury predicts receptive language outcome

Early brain injury alters both structural and functional connectivity between the cerebral hemispheres. Despite increasing knowledge on the individual hemispheric contributions to recovery from such injury, we know very little about how their interactions affect this process. In the present study, we related interhemispheric structural and functional connectivity to receptive language outcome following early left hemisphere stroke. We used functional magnetic resonance imaging to study 14 people with neonatal brain injury, and 25 age-matched controls during passive story comprehension. With respect to structural connectivity, we found that increased volume of the corpus callosum predicted good receptive language outcome, but that this is not specific to people with injury. In contrast, we found that increased posterior superior temporal gyrus interhemispheric functional connectivity during story comprehension predicted better receptive language performance in people with early brain injury, but worse performance in typical controls. This suggests that interhemispheric functional connectivity is one potential compensatory mechanism following early injury. Further, this pattern of results suggests refinement of the prevailing notion that better language outcome following early left hemisphere injury relies on the contribution of the contralesional hemisphere (i.e., the "right-hemisphere-take-over" theory). This pattern of results was also regionally specific; connectivity of the angular gyrus predicted poorer performance in both groups, independent of brain injury. These results present a complex picture of recovery, and in some cases, such recovery relies on increased cooperation between the injured hemisphere and homologous regions in the contralesional hemisphere, but in other cases, the opposite appears to hold.

Brain function overlaps when people observe emblems, speech, and grasping

A hand grasping a cup or gesturing "thumbs-up", while both manual actions, have different purposes and effects. Grasping directly affects the cup, whereas gesturing "thumbs-up" has an effect through an implied verbal (symbolic) meaning. Because grasping and emblematic gestures ("emblems") are both goal-oriented hand actions, we pursued the hypothesis that observing each should evoke similar activity in neural regions implicated in processing goal-oriented hand actions. However, because emblems express symbolic meaning, observing them should also evoke activity in regions implicated in interpreting meaning, which is most commonly expressed in language. Using fMRI to test this hypothesis, we had participants watch videos of an actor performing emblems, speaking utterances matched in meaning to the emblems, and grasping objects. Our results show that lateral temporal and inferior frontal regions respond to symbolic meaning, even when it is expressed by a single hand action. In particular, we found that left inferior frontal and right lateral temporal regions are strongly engaged when people observe either emblems or speech. In contrast, we also replicate and extend previous work that implicates parietal and premotor responses in observing goal-oriented hand actions. For hand actions, we found that bilateral parietal and premotor regions are strongly engaged when people observe either emblems or grasping. These findings thus characterize converging brain responses to shared features (e.g., symbolic or manual), despite their encoding and presentation in different stimulus modalities.

Functional and structural aging of the speech sensorimotor neural system: functional magnetic resonance imaging evidence

The ability to perceive and produce speech undergoes important changes in late adulthood. The goal of the present study was to characterize functional and structural age-related differences in the cortical network that support speech perception and production, using magnetic resonance imaging, as well as the relationship between functional and structural age-related changes occurring in this network. We asked young and older adults to observe videos of a speaker producing single words (perception), and to observe and repeat the words produced (production). Results show a widespread bilateral network of brain activation for Perception and Production that was not correlated with age. In addition, several regions did show age-related change (auditory cortex, planum temporale, superior temporal sulcus, premotor cortices, SMA-proper). Examination of the relationship between brain signal and regional and global gray matter volume and cortical thickness revealed a complex set of relationships between structure and function, with some regions showing a relationship between structure and function and some not. The present results provide novel findings about the neurobiology of aging and verbal communication.

Sensitive periods for language and recovery from stroke: conceptual and practical parallels

In this review, we consider the literature on sensitive periods for language acquisition from the perspective of the stroke recovery literature treated in this Special Issue. Conceptually, the two areas of study are linked in a number of ways. For example, the fact that learning itself can set the stage for future failures to learn (in second language learning) or to remediate (as described in constraint therapy) is an important insight in both areas, as is the increasing awareness that limits on learning can be overcome by creating the appropriate environmental context. Similar practical issues, such as distinguishing native-like language acquisition or recovery of function from compensatory mechanisms, arise in both areas as well.

Multisensory stimulation in stroke rehabilitation

The brain has a large capacity for automatic simultaneous processing and integration of sensory information. Combining information from different sensory modalities facilitates our ability to detect, discriminate, and recognize sensory stimuli, and learning is often optimal in a multisensory environment. Currently used multisensory stimulation methods in stroke rehabilitation include motor imagery, action observation, training with a mirror or in a virtual environment, and various kinds of music therapy. Non-invasive brain stimulation has showed promising preliminary results in aphasia and neglect. Patient heterogeneity and the interaction of age, gender, genes, and environment are discussed. Randomized controlled longitudinal trials starting earlier post-stroke are needed. The advance in brain network science and neuroimaging enabling longitudinal studies of structural and functional networks are likely to have an important impact on patient selection for specific interventions in future stroke rehabilitation. It is proposed that we should pay more attention to age, gender, and laterality in clinical studies.

Speech comprehension aided by multiple modalities: behavioural and neural interactions

Speech comprehension is a complex human skill, the performance of which requires the perceiver to combine information from several sources - e.g. voice, face, gesture, linguistic context - to achieve an intelligible and interpretable percept. We describe a functional imaging investigation of how auditory, visual and linguistic information interact to facilitate comprehension. Our specific aims were to investigate the neural responses to these different information sources, alone and in interaction, and further to use behavioural speech comprehension scores to address sites of intelligibility-related activation in multifactorial speech comprehension. In fMRI, participants passively watched videos of spoken sentences, in which we varied Auditory Clarity (with noise-vocoding), Visual Clarity (with Gaussian blurring) and Linguistic Predictability. Main effects of enhanced signal with increased auditory and visual clarity were observed in overlapping regions of posterior STS. Two-way interactions of the factors (auditory × visual, auditory × predictability) in the neural data were observed outside temporal cortex, where positive signal change in response to clearer facial information and greater semantic predictability was greatest at intermediate levels of auditory clarity. Overall changes in stimulus intelligibility by condition (as determined using an independent behavioural experiment) were reflected in the neural data by increased activation predominantly in bilateral dorsolateral temporal cortex, as well as inferior frontal cortex and left fusiform gyrus. Specific investigation of intelligibility changes at intermediate auditory clarity revealed a set of regions, including posterior STS and fusiform gyrus, showing enhanced responses to both visual and linguistic information. Finally, an individual differences analysis showed that greater comprehension performance in the scanning participants (measured in a post-scan behavioural test) were associated with increased activation in left inferior frontal gyrus and left posterior STS. The current multimodal speech comprehension paradigm demonstrates recruitment of a wide comprehension network in the brain, in which posterior STS and fusiform gyrus form sites for convergence of auditory, visual and linguistic information, while left-dominant sites in temporal and frontal cortex support successful comprehension.

Neural correlates of interindividual differences in children's audiovisual speech perception

Children use information from both the auditory and visual modalities to aid in understanding speech. A dramatic illustration of this multisensory integration is the McGurk effect, an illusion in which an auditory syllable is perceived differently when it is paired with an incongruent mouth movement. However, there are significant interindividual differences in McGurk perception: some children never perceive the illusion, while others always do. Because converging evidence suggests that the posterior superior temporal sulcus (STS) is a critical site for multisensory integration, we hypothesized that activity within the STS would predict susceptibility to the McGurk effect. To test this idea, we used BOLD fMRI in 17 children aged 6-12 years to measure brain responses to the following three audiovisual stimulus categories: McGurk incongruent, non-McGurk incongruent, and congruent syllables. Two separate analysis approaches, one using independent functional localizers and another using whole-brain voxel-based regression, showed differences in the left STS between perceivers and nonperceivers. The STS of McGurk perceivers responded significantly more than that of nonperceivers to McGurk syllables, but not to other stimuli, and perceivers' hemodynamic responses in the STS were significantly prolonged. In addition to the STS, weaker differences between perceivers and nonperceivers were observed in the fusiform face area and extrastriate visual cortex. These results suggest that the STS is an important source of interindividual variability in children's audiovisual speech perception.

Gesture in the developing brain

Speakers convey meaning not only through words, but also through gestures. Although children are exposed to co-speech gestures from birth, we do not know how the developing brain comes to connect meaning conveyed in gesture with speech. We used functional magnetic resonance imaging (fMRI) to address this question and scanned 8- to 11-year-old children and adults listening to stories accompanied by hand movements, either meaningful co-speech gestures or meaningless self-adaptors. When listening to stories accompanied by both types of hand movement, both children and adults recruited inferior frontal, inferior parietal, and posterior temporal brain regions known to be involved in processing language not accompanied by hand movements. There were, however, age-related differences in activity in posterior superior temporal sulcus (STSp), inferior frontal gyrus, pars triangularis (IFGTr), and posterior middle temporal gyrus (MTGp) regions previously implicated in processing gesture. Both children and adults showed sensitivity to the meaning of hand movements in IFGTr and MTGp, but in different ways. Finally, we found that hand movement meaning modulates interactions between STSp and other posterior temporal and inferior parietal regions for adults, but not for children. These results shed light on the developing neural substrate for understanding meaning contributed by co-speech gesture.

A linear structural equation model for covert verb generation based on independent component analysis of FMRI data from children and adolescents

Human language is a complex and protean cognitive ability. Young children, following well defined developmental patterns learn language rapidly and effortlessly producing full sentences by the age of 3 years. However, the language circuitry continues to undergo significant neuroplastic changes extending well into teenage years. Evidence suggests that the developing brain adheres to two rudimentary principles of functional organization: functional integration and functional specialization. At a neurobiological level, this distinction can be identified with progressive specialization or focalization reflecting consolidation and synaptic reinforcement of a network (Lenneberg, 1967; Muller et al., 1998; Berl et al., 2006). In this paper, we used group independent component analysis and linear structural equation modeling (McIntosh and Gonzalez-Lima, 1994; Karunanayaka et al., 2007) to tease out the developmental trajectories of the language circuitry based on fMRI data from 336 children ages 5–18 years performing a blocked, covert verb generation task. The results are analyzed and presented in the framework of theoretical models for neurocognitive brain development. This study highlights the advantages of combining both modular and connectionist approaches to cognitive functions; from a methodological perspective, it demonstrates the feasibility of combining data-driven and hypothesis driven techniques to investigate the developmental shifts in the semantic network.

The development of multisensory speech perception continues into the late childhood years

Observing a speaker's articulations substantially improves the intelligibility of spoken speech, especially under noisy listening conditions. This multisensory integration of speech inputs is crucial to effective communication. Appropriate development of this ability has major implications for children in classroom and social settings, and deficits in it have been linked to a number of neurodevelopmental disorders, especially autism. It is clear from structural imaging studies that there is a prolonged maturational course within regions of the perisylvian cortex that persists into late childhood, and these regions have been firmly established as being crucial to speech and language functions. Given this protracted maturational timeframe, we reasoned that multisensory speech processing might well show a similarly protracted developmental course. Previous work in adults has shown that audiovisual enhancement in word recognition is most apparent within a restricted range of signal-to-noise ratios (SNRs). Here, we investigated when these properties emerge during childhood by testing multisensory speech recognition abilities in typically developing children aged between 5 and 14 years, and comparing them with those of adults. By parametrically varying SNRs, we found that children benefited significantly less from observing visual articulations, displaying considerably less audiovisual enhancement. The findings suggest that improvement in the ability to recognize speech-in-noise and in audiovisual integration during speech perception continues quite late into the childhood years. The implication is that a considerable amount of multisensory learning remains to be achieved during the later schooling years, and that explicit efforts to accommodate this learning may well be warranted.

The development of multisensory speech perception continues into the late childhood years

Observing a speaker's articulations substantially improves the intelligibility of spoken speech, especially under noisy listening conditions. This multisensory integration of speech inputs is crucial to effective communication. Appropriate development of this ability has major implications for children in classroom and social settings, and deficits in it have been linked to a number of neurodevelopmental disorders, especially autism. It is clear from structural imaging studies that there is a prolonged maturational course within regions of the perisylvian cortex that persists into late childhood, and these regions have been firmly established as being crucial to speech and language functions. Given this protracted maturational timeframe, we reasoned that multisensory speech processing might well show a similarly protracted developmental course. Previous work in adults has shown that audiovisual enhancement in word recognition is most apparent within a restricted range of signal-to-noise ratios (SNRs). Here, we investigated when these properties emerge during childhood by testing multisensory speech recognition abilities in typically developing children aged between 5 and 14 years, and comparing them with those of adults. By parametrically varying SNRs, we found that children benefited significantly less from observing visual articulations, displaying considerably less audiovisual enhancement. The findings suggest that improvement in the ability to recognize speech-in-noise and in audiovisual integration during speech perception continues quite late into the childhood years. The implication is that a considerable amount of multisensory learning remains to be achieved during the later schooling years, and that explicit efforts to accommodate this learning may well be warranted.