Neural evidence for the interplay between language, gesture, and action: a review

Effect of gender matching and attentional focus on the link between action observation and action verb processing

Several studies have demonstrated the existence of a link between action observation (AO) and language. However, the optimizing parameters for this link have not been explored until now. To answer this question, the present study proposed two experiments for assessing the role of motor repertory and attentional focus. Sixty participants performed a priming task in which they had to decide if a verb was or was not an action verb after they saw a point-light display (PLD) representing an action. Only one difference distinguished the experiments. In experiment one, the PLD was either in accordance or not with the gender kinematics of the observer, whereas in the second experiment, the PLD either focused on or was unfocused on the main limbs implied in the action. The results show that motor repertory affects the link between action observation and action verb processing, whereas attentional focus does not. Implications of these results are highlighted in the discussion, notably to better understand the mechanisms that explain the link between action observation and action verb processing.

Developmental screening of neurodevelopmental disorders before age 6: a nationwide health screening program

BACKGROUND

We aimed to investigate the association between developmental screening before 24 months of age and neurodevelopmental disorders (NDDs) at 4-6 years of age.

METHODS

We included 922,899 newborn born between 2014 and 2016 registered in National Health Insurance Service (NHIS). Developmental screening was administered at 9-12 and 18-24 months old with the Korean Developmental Screening Test for Infants & Children (K-DST). Diagnoses of NDDs was based on the World Health Organization's International Classification of Diseases, Tenth Revision (ICD-10), provided by the NHIS database.

RESULTS

Among 637,277 individuals who underwent screening at 9-12 and 18-24 months, Screen-positivity (defined as summed score < -2 standard deviation) for gross motor domain at 9-12 months was significantly associated with the incidence of autism spectrum disorder (aHR, 2.24; 95% CI, 1.80-2.80) and cerebral palsy (aHR, 4.81; 95% CI, 3.62-6.38). Screening positive at language domain at 18-24 months old was associated with autism spectrum disorder (aHR 5.50; 95% CI, 4.31- 7.02) and developmental language disorder (aHR 8.67; 95% CI, 7.27-10.33) at 4-6 years of age.

CONCLUSION

Widespread nationwide implementation of screening programs before 24 months was effective in identifying NDDs at 4-6 years of age. Further strategies integrating with referral and intervention systems should be established.

IMPACT

We investigated the screening effect of nationwide developmental screening program on neurodevelopmental disorders using nationwide data. Gross motor delay during infancy was significant predictor of later neurodevelopmental disorders. Language, cognitive, and social delay before 24 months of age was associated with later autism spectrum disorders and developmental language disorders. Widespread nationwide implementation of screening programs before 24 months was effective in identifying NDDs at 4-6 years of age and should be encouraged.

Between bodily action and conventionalized structure: The neural mechanisms of constructed action in sign language comprehension

Sign languages (SLs) are expressed through different bodily actions, ranging from re-enactment of physical events (constructed action, CA) to sequences of lexical signs with internal structure (plain telling, PT). Despite the prevalence of CA in signed interactions and its significance for SL comprehension, its neural dynamics remain unexplored. We examined the processing of different types of CA (subtle, reduced, and overt) and PT in 35 adult deaf or hearing native signers. The electroencephalographic-based processing of signed sentences with incongruent targets was recorded. Attenuated N300 and early N400 were observed for CA in deaf but not in hearing signers. No differences were found between sentences with CA types in all signers, suggesting a continuum from PT to overt CA. Deaf signers focused more on body movements; hearing signers on faces. We conclude that CA is processed less effortlessly than PT, arguably because of its strong focus on bodily actions.

Brain Signatures of Embodied Semantics and Language: A Consensus Paper

According to embodied theories (including embodied, embedded, extended, enacted, situated, and grounded approaches to cognition), language representation is intrinsically linked to our interactions with the world around us, which is reflected in specific brain signatures during language processing and learning. Moving on from the original rivalry of embodied vs. amodal theories, this consensus paper addresses a series of carefully selected questions that aim at determining when and how rather than whether motor and perceptual processes are involved in language processes. We cover a wide range of research areas, from the neurophysiological signatures of embodied semantics, e.g., event-related potentials and fields as well as neural oscillations, to semantic processing and semantic priming effects on concrete and abstract words, to first and second language learning and, finally, the use of virtual reality for examining embodied semantics. Our common aim is to better understand the role of motor and perceptual processes in language representation as indexed by language comprehension and learning. We come to the consensus that, based on seminal research conducted in the field, future directions now call for enhancing the external validity of findings by acknowledging the multimodality, multidimensionality, flexibility and idiosyncrasy of embodied and situated language and semantic processes.

Some historical notes orienting towards brain mechanisms that could underlie hemispheric asymmetries

The first minor aim of this synthetical historical survey consisted in showing that the discovery of the internal organization of language within the left hemisphere has been mainly determined by theoretical models and cultural factors, whereas the discovery of the left lateralisation of language and of the right lateralization of emotions and of other cognitive and perceptual functions has been mainly determined by empirical observations. A second more relevant aim of the survey consisted in discussing historical and more recent data suggesting that the different lateralisation of language and emotions has influenced not only the asymmetrical representation of other cognitive, affective and perceptual functions, but also (thank to the shaping influence of language on human cognition) of asymmetries regarding more general aspects of thought (such as the distinctions between 'propositional vs automatic' and 'conscious vs unconscious' ways of functioning). In the last part of the review, these data will be included in a more general discussion, concerning the brain functions that could be subsumed by the right hemisphere for three main reasons: (a) to avoid conflicts with the language mediated activities of the left hemisphere; (b) because of unconscious and automatic aspects of its non-verbal organisation or (c) due to the competition for cortical space determined by the development of language within the left hemisphere.

Gestures and pauses to help thought: hands, voice, and silence in the tourist guide's speech

In the body of research on the relationship between gesture and speech, some models propose they form an integrated system while others attribute gestures a compensatory role in communication. This study addresses the gesture-speech relationship by taking disfluency phenomena as a case study. Since it is part of a project aimed at designing virtual agents to be employed in museums, an analysis was performed on the communicative behavior of tourist guides. Results reveal that gesturing is more frequent during speech than pauses. Moreover, when comparing the types of gestures and types of pauses they co-occur with, non-communicative gestures (idles and manipulators) turn out to be more frequent than communicatively-meaningful gestures, which instead more often co-occur with speech. We discuss these findings as relevant for a theoretical model viewing speech and gesture as an integrated system.

Language-related motor facilitation in Italian Sign Language signers

Linguistic tasks facilitate corticospinal excitability as revealed by increased motor evoked potential (MEP) induced by transcranial magnetic stimulation (TMS) in the dominant hand. This modulation of the primary motor cortex (M1) excitability may reflect the relationship between speech and gestures. It is conceivable that in healthy individuals who use a sign language this cortical excitability modulation could be rearranged. The aim of this study was to evaluate the effect of spoken language tasks on M1 excitability in a group of hearing signers. Ten hearing Italian Sign Language (LIS) signers and 16 non-signer healthy controls participated. Single-pulse TMS was applied to either M1 hand area at the baseline and during different tasks: (i) reading aloud, (ii) silent reading, (iii) oral movements, (iv) syllabic phonation and (v) looking at meaningless non-letter strings. Overall, M1 excitability during the linguistic and non-linguistic tasks was higher in LIS group compared to the control group. In LIS group, MEPs were significantly larger during reading aloud, silent reading and non-verbal oral movements, regardless the hemisphere. These results suggest that in hearing signers there is a different modulation of the functional connectivity between the speech-related brain network and the motor system.

Multimodal imitative learning and synchrony in cetaceans: A model for speech and singing evolution

Multimodal imitation of actions, gestures and vocal production is a hallmark of the evolution of human communication, as both, vocal learning and visual-gestural imitation, were crucial factors that facilitated the evolution of speech and singing. Comparative evidence has revealed that humans are an odd case in this respect, as the case for multimodal imitation is barely documented in non-human animals. While there is evidence of vocal learning in birds and in mammals like bats, elephants and marine mammals, evidence in both domains, vocal and gestural, exists for two Psittacine birds (budgerigars and grey parrots) and cetaceans only. Moreover, it draws attention to the apparent absence of vocal imitation (with just a few cases reported for vocal fold control in an orangutan and a gorilla and a prolonged development of vocal plasticity in marmosets) and even for imitation of intransitive actions (not object related) in monkeys and apes in the wild. Even after training, the evidence for productive or "true imitation" (copy of a novel behavior, i.e., not pre-existent in the observer's behavioral repertoire) in both domains is scarce. Here we review the evidence of multimodal imitation in cetaceans, one of the few living mammalian species that have been reported to display multimodal imitative learning besides humans, and their role in sociality, communication and group cultures. We propose that cetacean multimodal imitation was acquired in parallel with the evolution and development of behavioral synchrony and multimodal organization of sensorimotor information, supporting volitional motor control of their vocal system and audio-echoic-visual voices, body posture and movement integration.

Up right, not right up: Primacy of verticality in both language and movement

When describing motion along both the horizontal and vertical axes, languages from different families express the elements encoding verticality before those coding for horizontality (e.g., going up right instead of right up). In light of the motor grounding of language, the present study investigated whether the prevalence of verticality in Path expression also governs the trajectory of arm biological movements. Using a 3D virtual-reality setting, we tracked the kinematics of hand pointing movements in five spatial directions, two of which implied the vertical and horizontal vectors equally (i.e., up right +45° and bottom right −45°). Movement onset could be prompted by visual or auditory verbal cues, the latter being canonical in French (“en haut à droite”/up right) or not (“à droite en haut”/right up). In two experiments, analyses of the index finger kinematics revealed a significant effect of gravity, with earlier acceleration, velocity, and deceleration peaks for upward (+45°) than downward (−45°) movements, irrespective of the instructions. Remarkably, confirming the linguistic observations, we found that vertical kinematic parameters occurred earlier than horizontal ones for upward movements, both for visual and congruent verbal cues. Non-canonical verbal instructions significantly affected this temporal dynamic: for upward movements, the horizontal and vertical components temporally aligned, while they reversed for downward movements where the kinematics of the vertical axis was delayed with respect to that of the horizontal one. This temporal dynamic is so deeply anchored that non-canonical verbal instructions allowed for horizontality to precede verticality only for movements that do not fight against gravity. Altogether, our findings provide new insights into the embodiment of language by revealing that linguistic path may reflect the organization of biological movements, giving priority to the vertical axis.

Speaker-listener neural coupling reveals a right-lateralized mechanism for non-native speech-in-noise comprehension

While the increasingly globalized world has brought more and more demands for non-native language communication, the prevalence of background noise in everyday life poses a great challenge to non-native speech comprehension. The present study employed an interbrain approach based on functional near-infrared spectroscopy (fNIRS) to explore how people adapt to comprehend non-native speech information in noise. A group of Korean participants who acquired Chinese as their non-native language was invited to listen to Chinese narratives at 4 noise levels (no noise, 2 dB, -6 dB, and - 9 dB). These narratives were real-life stories spoken by native Chinese speakers. Processing of the non-native speech was associated with significant fNIRS-based listener-speaker neural couplings mainly over the right hemisphere at both the listener's and the speaker's sides. More importantly, the neural couplings from the listener's right superior temporal gyrus, the right middle temporal gyrus, as well as the right postcentral gyrus were found to be positively correlated with their individual comprehension performance at the strongest noise level (-9 dB). These results provide interbrain evidence in support of the right-lateralized mechanism for non-native speech processing and suggest that both an auditory-based and a sensorimotor-based mechanism contributed to the non-native speech-in-noise comprehension.

Prelinguistic gesture and developmental abilities: A multi-ethnic comparative study

The present study examined the frequency of gesture use and the relationship between frequency of gesture use and developmental abilities in typically developing 9- to 15-month-old, prelinguistic Hispanic and non-Hispanic White children. Data was collected through parent questionnaires, the Mullen Scales of Early Learning (MSEL), and two, 15-min video samples for each participant (semi-structured and structured settings). All video samples were coded for the frequency of the following gestures: total frequency, behavior regulation, social interaction, and joint attention. Results showed that children from both ethnicities used fewer gestures in a semi-structured setting in comparison to a structured setting and non-Hispanic White children produced higher frequencies of behavior regulation gestures and joint attention gestures, but lower frequencies of social interaction gestures. When controlling for ethnicity, gender, and age total frequency of gesture and frequencies of behavior regulation and social interaction were predictive of various developmental abilities. Furthermore, participant gender, age, and ethnicity were significantly related to various developmental abilities explored. These relationships were dependent upon setting. An understanding of the use of gesture and the relationship between gesture use and developmental abilities in prelinguistic children from different ethnic backgrounds has implication for early identification of delays and differences and is important to consider when exploring the connection between gesture and language and whether there are gesture-language, gesture-motor, and/or gesture-cognition integrated systems.

ALS/FTD: Evolution, Aging, and Cellular Metabolic Exhaustion

Amyotrophic lateral sclerosis and frontotemporal dementia (ALS/FTD) are neurodegenerations with evolutionary underpinnings, expansive clinical presentations, and multiple genetic risk factors involving a complex network of pathways. This perspective considers the complex cellular pathology of aging motoneuronal and frontal/prefrontal cortical networks in the context of evolutionary, clinical, and biochemical features of the disease. We emphasize the importance of evolution in the development of the higher cortical function, within the influence of increasing lifespan. Particularly, the role of aging on the metabolic competence of delicately optimized neurons, age-related increased proteostatic costs, and specific genetic risk factors that gradually reduce the energy available for neuronal function leading to neuronal failure and disease.

Gesture-Speech Integration in Typical and Atypical Adolescent Readers

This study investigated gesture-speech integration (GSI) among adolescents who are deaf or hard of hearing (DHH) and those with typical hearing. Thirty-eight adolescents (19 with hearing loss) performed a Stroop-like task in which they watched 120 short video clips of gestures and actions twice at random. Participants were asked to press one button if the visual content of the speaker's movements was related to a written word and to press another button if it was unrelated to a written word while accuracy rates and response times were recorded. We found stronger GSI effects among DHH participants than hearing participants. The semantic congruency effect was significantly larger in DHH participants than in hearing participants, and results of our experiments indicated a significantly larger gender congruency effect in DHH participants as compared to hearing participants. Results of this study shed light on GSI among DHH individuals and suggest future avenues for research examining the impact of gesture on language processing and communication in this population.

Noninvasive stimulation of the unlesioned hemisphere and phonological treatment in a case of chronic anomia post-stroke

Chronic lexical anomia after left hemisphere (LH) stroke improves under personalized phonological treatment (PT). Cortical linking between language and hand motor areas (hand_M1) questioned whether PT-related improvement relies on the unlesioned hemisphere (UH) plasticity when LH is dysfunctional. Our 70-yo-woman case study showed that 10 sessions of excitatory stimulation of UH_hand-M1 combined with PT hastened oral picture naming improvement as compared to sham+PT and changes were maintained together with changes of untrained items andcorticomotor excitability increase. This supports a role of stimulation-induced plasticity of UH_hand M1 in language recovery, at least in the improvement of lexical anomia in chronic stroke.

Autonomic system tuning during gesture observation and reproduction

Gestural communication allows providing information about thoughts and feelings, characterizing face-to-face interactions, also during non-verbal exchanges. In the present study, the autonomic responses and peripheral synchronization mechanisms of two individuals (encoder and decoder) were recorded simultaneously, through the use of biofeedback in hyperscanning, during two different experimental phases consisting in the observation (watching videos of gestures) and reproduction of positive and negative different types of gestures (affective, social and informative) supported by linguistic contexts. Therefore, the main aim of this study was focused on the analysis of simultaneous individuals' peripheral mechanisms during the performing of complex joint action, consisting of the observation (watching videos) and the reproduction of positive and negative social, affective, and informative gestures each supported by a linguistic script. Single-subject and inter-subject correlation analyses were conducted to observe individuals' autonomic responses and physiological synchronization. Single-subject results revealed an increase in emotional arousal, indicated by an increase in electrodermal activity (skin conductance level - SCL and response - SCR), during both the observation (watching videos) and reproduction of negative social and affective gestures contextualized by a linguistic context. Moreover, an increase of emotional engagement, expressed by an increase in heart rate (HR) activity, emerged in the encoder compare to the decoder during gestures reproduction (simulation of gestures). Inter-subject correlation results showed the presence of mirroring mechanisms, indicated by an increase in SCL, SCR, and HR synchronization, during the linguistic contexts and gesture observation (watching videos). Furthermore, an increase in SCL and SCR synchronization emerged during the observation (watching videos) and reproduction of negative social and affective gestures. Therefore, the present study allowed to obtain information on the mirroring mechanisms and physiological synchronization underlying the linguistic and gesture system during non-verbal interaction.

What is Functional Communication? A Theoretical Framework for Real-World Communication Applied to Aphasia Rehabilitation

Aphasia is an impairment of language caused by acquired brain damage such as stroke or traumatic brain injury, that affects a person’s ability to communicate effectively. The aim of rehabilitation in aphasia is to improve everyday communication, improving an individual’s ability to function in their day-to-day life. For that reason, a thorough understanding of naturalistic communication and its underlying mechanisms is imperative. The field of aphasiology currently lacks an agreed, comprehensive, theoretically founded definition of communication. Instead, multiple disparate interpretations of functional communication are used. We argue that this makes it nearly impossible to validly and reliably assess a person’s communicative performance, to target this behaviour through therapy, and to measure improvements post-therapy. In this article we propose a structured, theoretical approach to defining the concept of functional communication. We argue for a view of communication as “situated language use”, borrowed from empirical psycholinguistic studies with non-brain damaged adults. This framework defines language use as: (1) interactive, (2) multimodal, and (3) contextual. Existing research on each component of the framework from non-brain damaged adults and people with aphasia is reviewed. The consequences of adopting this approach to assessment and therapy for aphasia rehabilitation are discussed. The aim of this article is to encourage a more systematic, comprehensive approach to the study and treatment of situated language use in aphasia.

Is There a Causal Link between the Left Lateralization of Language and Other Brain Asymmetries? A Review of Data Gathered in Patients with Focal Brain Lesions

This review evaluated if the hypothesis of a causal link between the left lateralization of language and other brain asymmetries could be supported by a careful review of data gathered in patients with unilateral brain lesions. In a short introduction a distinction was made between brain activities that could: (a) benefit from the shaping influences of language (such as the capacity to solve non-verbal cognitive tasks and the increased levels of consciousness and of intentionality); (b) be incompatible with the properties and the shaping activities of language (e.g., the relations between language and the automatic orienting of visual-spatial attention or between cognition and emotion) and (c) be more represented on the right hemisphere due to competition for cortical space. The correspondence between predictions based on the theoretical impact of language on other brain functions and data obtained in patients with lesions of the right and left hemisphere was then assessed. The reviewed data suggest that different kinds of hemispheric asymmetries observed in patients with unilateral brain lesions could be subsumed by common mechanisms, more or less directly linked to the left lateralization of language.

Enhancing memory using enactment: does meaning matter in action production?

ABSTRACTEnactment is an encoding strategy in which performing an action related to a target item enhances memory for that word, relative to verbal encoding. Precisely how this motor activity aids recall is unclear. We examined whether the action created during encoding needed to be semantically relevant to the to-be-remembered word, to enhance memory. In Experiment 1, participants were asked to either (a) enact, (b) perform unrelated motoric gestures, or (c) read forty-five action verbs. On a subsequent free-recall test, memory for enacted words was significantly higher relative to words read, or encoded with unrelated gestures. In Experiment 2, to reduce the ambiguity associated with initiating an unrelated gesture, participants were instructed to write target words in the air. Results were similar to Experiment 1. In Experiment 3, we replicated the results of Experiment 2 using video conferencing to record the onset time of action initiation for enacted, unrelated gesture, and read trials. Results showed that planning of meaningful actions may also contribute to the memory performance as evidenced by a longer onset time to initiate an action on enactment relative to gesturing and reading trials. These findings suggest that planning and executing meaningful actions drive the enactment benefit.

Suggestions for Improving the Investigation of Gesture in Aphasia

Purpose When we speak, we gesture, and indeed, persons with aphasia gesture more frequently. The reason(s) for this is still being investigated, spurring an increase in the number of studies of gesture in persons with aphasia. As the number of studies increases, so too does the need for a shared set of best practices for gesture research in aphasia. After briefly reviewing the importance and use of gesture in persons with aphasia, this viewpoint puts forth methodological and design considerations when evaluating gesture in persons with aphasia. Method & Results We explore several different design and methodological considerations for gesture research specific to persons with aphasia, such as video angle specifications, data collection techniques, and analysis considerations. The goal of these suggestions is to develop transparent and reproducible methods for evaluating gesture in aphasia to build a solid foundation for continued work in this area. Conclusions We have proposed that it is critical to evaluate multimodal communication in a methodologically robust way to facilitate increased knowledge about the relationship of gesture to spoken language, cognition, and to other aspects of living with aphasia and recovery from aphasia. We conclude by postulating future directions for gesture research in aphasia.

Effects of Verb Network Strengthening Treatment on Sound-Level and Speech Production Errors in Individuals With Aphasia and Acquired Apraxia of Speech

Purpose This retrospective pilot study investigated whether sound-level and speech production errors decreased in confrontation naming following Verb Network Strengthening Treatment (VNeST) for four participants with acquired apraxia of speech (A-AOS) and aphasia for whom lexical retrieval was previously reported. Specifically, we investigated a potential increase in correct number of syllables per word and posttreatment changes across three domains of speech: segmental production, fluency, and prosody. It was hypothesized that treatment shown to increase lexical retrieval in persons with aphasia and A-AOS could potentially facilitate a reduction in sound-level and speech production errors consistent with dual diagnoses of A-AOS and aphasia. Method Naming responses from four participants with aphasia and A-AOS who previously participated in VNeST studies were investigated for correct number of syllables per word and measures of segmental speech, fluency, and prosody. Results Significant gains in at least one measure of speech production were reported for three of the participants. One participant demonstrated decreased segmental speech errors, two showed significant reduction on syllable segmentation, and two demonstrated a significant reduction on false starts and pauses. Significant gains in production of correct number of syllables were limited to one participant, and one participant did not demonstrate increased accuracy on any measure of speech production. Conclusions While speech production errors consistent with motor speech impairment cannot always be definitively distinguished from the confound of aphasia, two participants produced significantly decreased segmentation of syllables, a characteristic unique to A-AOS. The sound-level and speech production changes recorded may be attributed to a combination of interacting motor and language processes and resource allocation. In addition, specific components of VNeST may have contributed to speech production changes. Future work will focus on a prospective study of effects of language therapy (e.g., VNeST) on measures of speech production with investigation beyond the single-word level.

Abnormal Gesture Perception and Clinical High-Risk for Psychosis

Individuals diagnosed with psychotic disorders exhibit abnormalities in the perception of expressive behaviors, which are linked to symptoms and visual information processing domains. Specifically, literature suggests these groups have difficulties perceiving gestures that accompany speech. While our understanding of gesture perception in psychotic disorders is growing, gesture perception abnormalities and clues about potential causes and consequences among individuals meeting criteria for a clinical high-risk (CHR) syndrome is limited. Presently, 29 individuals with a CHR syndrome and 32 healthy controls completed an eye-tracking gesture perception paradigm. In this task, participants viewed an actor using abstract and literal gestures while presenting a story and eye gaze data (eg, fixation counts and total fixation time) was collected. Furthermore, relationships between fixation variables and both symptoms (positive, negative, anxiety, and depression) and measures of visual information processing (working memory and attention) were examined. Findings revealed that the CHR group gazed at abstract gestures fewer times than the control group. When individuals in the CHR group did gaze at abstract gestures, on average, they spent significantly less time fixating compared to controls. Furthermore, reduced fixation (ie, count and time) was related to depression and slower response time on an attentional task. While a similar pattern of group differences in the same direction appeared for literal gestures, the effect was not significant. These data highlight the importance of integrating gesture perception abnormalities into vulnerability models of psychosis and inform the development of targeted treatments for social communicative deficits.

Event related spectral perturbations of gesture congruity: Visuospatial resources are recruited for multimodal discourse comprehension

Here we examine the role of visuospatial working memory (WM) during the comprehension of multimodal discourse with co-speech iconic gestures. EEG was recorded as healthy adults encoded either a sequence of one (low load) or four (high load) dot locations on a grid and rehearsed them until a free recall response was collected later in the trial. During the rehearsal period of the WM task, participants observed videos of a speaker describing objects in which half of the trials included semantically related co-speech gestures (congruent), and the other half included semantically unrelated gestures (incongruent). Discourse processing was indexed by oscillatory EEG activity in the alpha and beta bands during the videos. Across all participants, effects of speech and gesture incongruity were more evident in low load trials than in high load trials. Effects were also modulated by individual differences in visuospatial WM capacity. These data suggest visuospatial WM resources are recruited in the comprehension of multimodal discourse.

Distinct neural networks for the volitional control of vocal and manual actions in the monkey homologue of Broca's area

The ventrolateral frontal lobe (Broca's area) of the human brain is crucial in speech production. In macaques, neurons in the ventrolateral prefrontal cortex, the suggested monkey homologue of Broca's area, signal the volitional initiation of vocalizations. We explored whether this brain area became specialized for vocal initiation during primate evolution and trained macaques to alternate between a vocal and manual action in response to arbitrary cues. During task performance, single neurons recorded from the ventrolateral prefrontal cortex and the rostroventral premotor cortex of the inferior frontal cortex predominantly signaled the impending vocal or, to a lesser extent, manual action, but not both. Neuronal activity was specific for volitional action plans and differed during spontaneous movement preparations. This implies that the primate inferior frontal cortex controls the initiation of volitional utterances via a dedicated network of vocal selective neurons that might have been exploited during the evolution of Broca's area.

Increased links between language and motor areas: A proof-of-concept study on resting-state functional connectivity following Personalized Observation, Execution and Mental imagery therapy in chronic aphasia

A tight coupling of language and motor processes has been established, which is consistent with embodied cognition theory. However, very few therapies have been designed to exploit the synergy between motor and language processes to help rehabilitate people with aphasia (PWA). Moreover, the underlying mechanisms supporting the efficacy of such approaches remain unknown. Previous work in our laboratory has demonstrated that personalized observation, execution, and mental imagery therapy (POEM)-a new therapy using three sensorimotor strategies to trigger action verb naming-leads to significant improvements in verb retrieval in PWA. Moreover, these improvements were supported by significant activations in language and sensorimotor processing areas, which further reinforce the role of both processes in language recovery (Durand et al., 2018). The present study investigates resting state functional connectivity (rsFC) changes following POEM in a pre-/post-POEM therapy design. A whole brain network functional connectivity approach was used to assess and describe changes in rsFC in a group of four PWA, who were matched and compared with four healthy controls (HC). Results showed increased rsFC in PWA within and between visuo-motor and language areas (right cuneal cortex-left supracalcarin (SCC) cortex/right precentral gyrus (PreCG)-left lingual gyrus (LG)) and between areas involved in action processing (right anterior parahippocampal gyrus (aPaHC)-left superior parietal lobule (SPL). In comparison to HC, the PWA group showed increased rsFC between the right inferior frontal gyrus (IFG) and left thalamus, which are areas involved in lexico-semantic processing. This proof-of-concept study suggests that the sensorimotor and language strategies used in POEM may induce modifications in large-scale networks, probably derived from the integration of visual and sensorimotor systems to sustain action naming, which is consistent with the embodied cognition theory.

Action verbs drive motor activity in adolescents but not in children

In adults, grip force has reliably been used to investigate motor simulation evoked by linguistic action, suggesting that motor phenomena are linked to semantic action. The parietal and frontal lobes and their connexions are essential neural structures for pragmatic aspects of hand semantic action. In this perspective, the aim of the study was to determine the extent to which two groups of children and adolescents, classically characterized by degree of axonal myelination in fronto-parietal circuits, monitored the occurrence of nouns and manual action verbs presented auditorily while holding a grip force sensor. Differential effects of grip force were seen only in the adolescents when monitoring action verbs. Interestingly, weaker effects of grip force were modulated by noun targets only in the younger children, revealing that the ability to profit from a full semantic representation of verbs is not clearly established in the younger children. Grip force modulation was observed as early as 300 ms post target onset and peaked at the 500-750 ms window of observation for both groups. These group differences are in line with the motor simulation difficulties seen in younger children. The results may also indicate that degree of grip force in response to specific linguistic categories parallels the maturation of the parietal-frontal circuits, including the anterior intra-parietal area which plays a determining role in semantic aspects of hand action.

Verbal working memory and co-speech gesture processing

Multimodal discourse requires an assembly of cognitive processes that are uniquely recruited for language comprehension in social contexts. In this study, we investigated the role of verbal working memory for the online integration of speech and iconic gestures. Participants memorized and rehearsed a series of auditorily presented digits in low (one digit) or high (four digits) memory load conditions. To observe how verbal working memory load impacts online discourse comprehension, ERPs were recorded while participants watched discourse videos containing either congruent or incongruent speech-gesture combinations during the maintenance portion of the memory task. While expected speech-gesture congruity effects were found in the low memory load condition, high memory load trials elicited enhanced frontal positivities that indicated a unique interaction between online speech-gesture integration and the availability of verbal working memory resources. This work contributes to an understanding of discourse comprehension by demonstrating that language processing in a multimodal context is subject to the relationship between cognitive resource availability and the degree of controlled processing required for task performance. We suggest that verbal working memory is less important for speech-gesture integration than it is for mediating speech processing under high task demands.

Why Language Processing Recruits Modality Specific Brain Regions: It Is Not About Understanding Words, but About Modelling Situations

Whether language comprehension requires the participation of brain structures that evolved for perception and action has been a subject of intense debate. While brain-imaging evidence for the involvement of such modality-specific regions has grown, the fact that lesions to these structures do not necessarily erase word knowledge has invited the conclusion that language-induced activity in these structures might not be essential for word recognition. Why language processing recruits these structures remains unanswered, however. Here, we examine the original findings from a slightly different perspective. We first consider the 'original' function of structures in modality-specific brain regions that are recruited by language activity. We propose that these structures help elaborate 'internal forward models' in motor control (c.f. emulators). Emulators are brain systems that capture the relationship between an action and its sensory consequences. During language processing emulators could thus allow accessing associative memories. We further postulate the existence of a linguistic system that exploits, in a rule-based manner, emulators and other nonlinguistic brain systems, to gain complementary (and redundant) information during language processing. Emulators are therefore just one of several sources of information. We emphasize that whether a given word-form triggers activity in modality-specific brain regions depends on the linguistic context and not on the word-form as such. The role of modality-specific systems in language processing is thus not to help understanding words but to model the verbally depicted situation by supplying memorized context information. We present a model derived from these assumptions and provide predictions and perspectives for future research.

Positive effects of grasping virtual objects on memory for novel words in a second language

Theories of embodied cognition describe language processing and representation as inherently connected to the sensorimotor experiences collected during acquisition. While children grasp their world, collect bodily experiences and name them, in second language (L2), students learn bilingual word lists. Experimental evidence shows that embodiment by mean of gestures enhances memory for words in L2. However, no study has been conducted on the effects of grasping in L2. In a virtual scenario, we trained 46 participants on 18 two- and three-syllabic words of Vimmi, an artificial corpus created for experimental purposes. The words were assigned concrete meanings of graspable objects. Six words were learned audio-visually, by reading the words projected on the wall and by hearing them. Another 6 words were trained by observation of virtual objects. Another 6 words were learned by observation and additional grasping the virtual objects. Thereafter participants were subministered free, cued recall, and reaction time tests in order to assess the word retention and the word recognition. After 30 days, the recall tests were repeated remotely to assess the memory in the long term. The results show that grasping of virtual objects can lead to superior memory performance and to lower reaction times during recognition.

The effect of interbrain synchronization in gesture observation: A fNIRS study

INTRODUCTION

Gestures characterize individuals' nonverbal communicative exchanges, taking on different functions. Several types of research in the neuroscientific field have been interested in the investigation of the neural correlates underlying the observation and implementation of different gestures categories. In particular, different studies have focused on the neural correlates underlying gestures observation, emphasizing the presence of mirroring mechanisms in specific brain areas, which appear to be involved in gesture observation and planning mechanisms.

MATERIALS AND METHODS

Specifically, the present study aimed to investigate the neural mechanisms, through the use of functional Near-Infrared Spectroscopy (fNIRS), underlying the observation of affective, social, and informative gestures with positive and negative valence in individuals' dyads composed by encoder and decoder. The variations of oxygenated (O2Hb) and deoxygenated (HHb) hemoglobin concentrations of both individuals were collected simultaneously through the use of hyperscanning paradigm, allowing the recording of brain responsiveness and interbrain connectivity.

RESULTS

The results showed a different brain activation and an increase of interbrain connectivity according to the type of gestures observed, with a significant increase of O2Hb brain responsiveness and interbrain connectivity and a decrease of HHb brain responsiveness for affective gestures in the dorsolateral prefrontal cortex (DLPFC) and for social gestures in the superior frontal gyrus (SFG). Furthermore, concerning the valence of the observed gestures, an increase of O2Hb brain activity and interbrain connectivity was observed in the left DLPFC for positive affective gestures compared to negative ones.

CONCLUSION

In conclusion, the present study showed different brain responses underlying the observation of different types of positive and negative gestures. Moreover, interbrain connectivity calculation allowed us to underline the presence of mirroring mechanisms involved in gesture-specific frontal regions during gestures observation and action planning.

The link between language and action in aging

OBJECTIVE

Many studies have demonstrated the existence of a link between action verb processing and action. However, little is known about the changes in this relationship with aging.

METHOD

To assess this point, we compare the performances of younger and older people during a priming task consisting of judging whether an image contains a human after listening to an action verb.

RESULTS

In accordance with previous literature, the results showed that younger people were faster to detect the presence of a human in the image in congruent conditions, namely, when the action verb and the image refer to the same action. However, this effect was not present in older adults' participants.

CONCLUSION

These findings suggest that the link between action and language decreases with age. We discuss these findings in the context of the embodied view of cognition.

Effects of Music Reading on Motor Cortex Excitability in Pianists: A Transcranial Magnetic Stimulation Study

Neurophysiological studies suggest that music reading facilitates sensorimotor cortex. The aim of this study was to evaluate (1) whether in pianists, reading notes in bass and treble clef selectively enhances right and left primary motor cortex (M1) excitability; and (2) whether reading notes played with the thumb or little finger selectively modulates the excitability of specific muscles. Twenty musicians (11 pianists, 9 non-pianists) participated. Transcranial magnetic stimulation (TMS) was applied while subjects read the bass or the treble clef of sheets music and during the observation of a blank staff (baseline). When pianists read the treble clef, the excitability of the left M1 was higher compared to that recorded in the right M1. Moreover, in the treble clef condition motor evoked potentials (MEPs) induced by TMS of the left M1 were higher when pianists read notes to be played with the 5° finger (little finger) with respect to 1° finger (thumb) notes, whereas in the bass clef condition TMS of the right M1 induced higher MEPs for 1° finger note compared to 5° finger notes. No significant modulation was observed in non-pianists. These data support the view that music reading may induce specific inter- and intra-hemispheric modulation of the motor cortex excitability.

Motor atypicalities in infancy are associated with general developmental level at 2 years, but not autistic symptoms

LAY ABSTRACT

Atypicalities in motor functioning are often observed in later born infant siblings of children with autism spectrum disorder. The goal of our study was to investigate motor functioning in infants with and without familial history of autism spectrum disorder. Specifically, we investigated how infants catch a ball that is rolling toward them following a non-straight path, a task that requires both efficient planning and execution. Their performance was measured using detailed three-dimensional motion capture technology. We found that several early motor functioning measures were different in infants with an older autistic sibling compared to controls. However, these early motor measures were not related to autistic symptoms at the age of 2 years. Instead, we found that some of the early motor measures were related to their subsequent non-social, general development. The findings of our study help us understand motor functioning early in life and how motor functioning is related to other aspects of development.

A Computational Theory for the Emergence of Grammatical Categories in Cortical Dynamics

A general agreement in psycholinguistics claims that syntax and meaning are unified precisely and very quickly during online sentence processing. Although several theories have advanced arguments regarding the neurocomputational bases of this phenomenon, we argue that these theories could potentially benefit by including neurophysiological data concerning cortical dynamics constraints in brain tissue. In addition, some theories promote the integration of complex optimization methods in neural tissue. In this paper we attempt to fill these gaps introducing a computational model inspired in the dynamics of cortical tissue. In our modeling approach, proximal afferent dendrites produce stochastic cellular activations, while distal dendritic branches–on the other hand–contribute independently to somatic depolarization by means of dendritic spikes, and finally, prediction failures produce massive firing events preventing formation of sparse distributed representations. The model presented in this paper combines semantic and coarse-grained syntactic constraints for each word in a sentence context until grammatically related word function discrimination emerges spontaneously by the sole correlation of lexical information from different sources without applying complex optimization methods. By means of support vector machine techniques, we show that the sparse activation features returned by our approach are well suited—bootstrapping from the features returned by Word Embedding mechanisms—to accomplish grammatical function classification of individual words in a sentence. In this way we develop a biologically guided computational explanation for linguistically relevant unification processes in cortex which connects psycholinguistics to neurobiological accounts of language. We also claim that the computational hypotheses established in this research could foster future work on biologically-inspired learning algorithms for natural language processing applications.

Effects of reader's facial expression on syntactic processing: A brain potential study

Embodied views of language support that facial sensorimotor information can modulate language comprehension. The aim of this study is to test whether the syntactic processing of simple sentences, as measured with event-related brain potentials (ERP), could be affected by reader's facial expressions. Participants performed a correctness decision task using sentences that could be either correct (50%) or contain a morphosyntactic disagreement (either in gender or number), while making one of four facial expressions: participants either (a) posed no facial expression ("control" condition) (b) brought their eyebrows together, making the ends of two golf tees touch ("frown" condition), (c) held a pencil with their teeth ("smile" condition), or (d) held the pencil using their lips ("non-smile" condition). In all conditions the customary left anterior negativities did not appear. In contrast, an N400-like component emerged, which was larger for the "frown" condition and reduced in the "smile" and "non-smile" conditions. These results can be interpreted as the consequence of either an unconscious emotion induction or an interplay between the motor and the language systems subsequent to the effort needed to hold the facial expression.

Looking at ancillary systems for verb recovery: Evidence from non-invasive brain stimulation

Several behavioural and neuroimaging studies have suggested that the language function is not restricted into the left areas but it involves regions not predicted by the classical language model. Accordingly, the Embodied Cognition theory postulates a close interaction between the language and the motor system. Indeed, it has been shown that non-invasive brain stimulation (NIBS) is effective for language recovery also when applied over sensorimotor regions, such as the motor cortex, the cerebellum and the spinal cord. We will review a series of NIBS studies in post-stroke aphasic people aimed to assess the impact of NIBS on verb recovery. We first present results which, following the classical assumption of the Broca's area as the key region for verb processing, have shown that the modulation over this area is efficacious for verb improvement. Then, we will present experiments which, according to Embodied Cognition, have directly investigated through NIBS the role of different sensorimotor regions in enhancing verb production. Since verbs play a crucial role for sentence construction which are most often impaired in the aphasic population, we believe that these results have important clinical implications. Indeed, they address the possibility that different structures might support verb processing.

Action verb processing specifically modulates motor behaviour and sensorimotor neuronal oscillations

Understanding action-related language recruits the brain’s motor system and can interact with motor behaviour. The current study shows MEG oscillatory patterns during verb-motor priming. Hand and foot verbs were followed by hand or foot responses, with faster reaction times for congruent conditions. In ROIs placed in the hand/arm and foot/leg portions of the sensorimotor cortex, this behavioural priming effect was accompanied by modulations in MEG oscillatory patterns preceding the responses. Power suppression in the alpha/beta frequency bands was reduced in congruent conditions in the body-part-specific ROIs. These results imply that the verb-motor priming effect may be a direct consequence of motor cortex contributions to action word processing.

Language, Gesture, and Emotional Communication: An Embodied View of Social Interaction

Spoken language is an innate ability of the human being and represents the most widespread mode of social communication. The ability to share concepts, intentions and feelings, and also to respond to what others are feeling/saying is crucial during social interactions. A growing body of evidence suggests that language evolved from manual gestures, gradually incorporating motor acts with vocal elements. In this evolutionary context, the human mirror mechanism (MM) would permit the passage from “doing something” to “communicating it to someone else.” In this perspective, the MM would mediate semantic processes being involved in both the execution and in the understanding of messages expressed by words or gestures. Thus, the recognition of action related words would activate somatosensory regions, reflecting the semantic grounding of these symbols in action information. Here, the role of the sensorimotor cortex and in general of the human MM on both language perception and understanding is addressed, focusing on recent studies on the integration between symbolic gestures and speech. We conclude documenting some evidence about MM in coding also the emotional aspects conveyed by manual, facial and body signals during communication, and how they act in concert with language to modulate other’s message comprehension and behavior, in line with an “embodied” and integrated view of social interaction.

Speech-accompanying gestures are not processed by the language-processing mechanisms

Speech-accompanying gestures constitute one information channel during communication. Some have argued that processing gestures engages the brain regions that support language comprehension. However, studies that have been used as evidence for shared mechanisms suffer from one or more of the following limitations: they (a) have not directly compared activations for gesture and language processing in the same study and relied on the fallacious reverse inference (Poldrack, 2006) for interpretation, (b) relied on traditional group analyses, which are bound to overestimate overlap (e.g., Nieto-Castañon and Fedorenko, 2012), (c) failed to directly compare the magnitudes of response (e.g., Chen et al., 2017), and (d) focused on gestures that may have activated the corresponding linguistic representations (e.g., "emblems"). To circumvent these limitations, we used fMRI to examine responses to gesture processing in language regions defined functionally in individual participants (e.g., Fedorenko et al., 2010), including directly comparing effect sizes, and covering a broad range of spontaneously generated co-speech gestures. Whenever speech was present, language regions responded robustly (and to a similar degree regardless of whether the video contained gestures or grooming movements). In contrast, and critically, responses in the language regions were low - at or slightly above the fixation baseline - when silent videos were processed (again, regardless of whether they contained gestures or grooming movements). Brain regions outside of the language network, including some in close proximity to its regions, differentiated between gestures and grooming movements, ruling out the possibility that the gesture/grooming manipulation was too subtle. Behavioral studies on the critical video materials further showed robust differentiation between the gesture and grooming conditions. In summary, contra prior claims, language-processing regions do not respond to co-speech gestures in the absence of speech, suggesting that these regions are selectively driven by linguistic input (e.g., Fedorenko et al., 2011). Although co-speech gestures are uncontroversially important in communication, they appear to be processed in brain regions distinct from those that support language comprehension, similar to other extra-linguistic communicative signals, like facial expressions and prosody.

Multimodal communication and language origins: integrating gestures and vocalizations

The presence of divergent and independent research traditions in the gestural and vocal domains of primate communication has resulted in major discrepancies in the definition and operationalization of cognitive concepts. However, in recent years, accumulating evidence from behavioural and neurobiological research has shown that both human and non-human primate communication is inherently multimodal. It is therefore timely to integrate the study of gestural and vocal communication. Herein, we review evidence demonstrating that there is no clear difference between primate gestures and vocalizations in the extent to which they show evidence for the presence of key language properties: intentionality, reference, iconicity and turn-taking. We also find high overlap in the neurobiological mechanisms producing primate gestures and vocalizations, as well as in ontogenetic flexibility. These findings confirm that human language had multimodal origins. Nonetheless, we note that in great apes, gestures seem to fulfil a carrying (i.e. predominantly informative) role in close-range communication, whereas the opposite holds for face-to-face interactions of humans. This suggests an evolutionary shift in the carrying role from the gestural to the vocal stream, and we explore this transition in the carrying modality. Finally, we suggest that future studies should focus on the links between complex communication, sociality and cooperative tendency to strengthen the study of language origins.

Holding a real object during encoding helps the learning of foreign vocabulary

This study aims at assessing and comparing two different methods for learning new vocabulary words in a foreign language. Learning vocabulary with images as non-verbal aids was compared to learning vocabulary with real objects. The Rwandan children who participated in this study learnt French as a third language. They took part in training sessions to learn different French words either seeing the corresponding image or holding the corresponding object. The training program was implemented in a Rwandan primary school with children of different ages (from five to 10 years old). The results showed that the words associated to objects that were held by the children during learning were better memorized than the words associated with images. The global memory performance was lower for the youngest children; however, learning with objects proved to be superior over learning with images for all ages. Taken together, the findings underscore that learning vocabulary with real objects is particularly efficient and support the idea that the embodied theory of language is a key element to effectively master a foreign language.

Identifying patterns of motor performance, executive functioning, and verbal ability in preschool children: A latent profile analysis

BACKGROUND

A relationship between motor performance and cognitive functioning is increasingly being recognized. Yet, little is known about the precise nature of the relationship between both domains, especially in early childhood.

AIMS

To identify distinct constellations of motor performance, executive functioning (EF), and verbal ability in preschool aged children; and to explore how individual and contextual variables are related to profile membership.

METHODS AND PROCEDURES

The sample consisted of 119 3- to 4-year old children (62 boys; 52%). The home based assessments consisted of a standardized motor test (Movement Assessment Battery for Children - 2), five performance-based EF tasks measuring inhibition and working memory, and the Receptive Vocabulary subtest from the Wechsler Preschool and Primary Scale of Intelligence Third Edition. Parents filled out the Behavior Rating Inventory of Executive Function - Preschool version. Latent profile analysis (LPA) was used to delineate profiles of motor performance, EF, and verbal ability. Chi-square statistics and multinomial logistic regression analysis were used to examine whether profile membership was predicted by age, gender, risk of motor coordination difficulties, ADHD symptomatology, language problems, and socioeconomic status (SES).

OUTCOMES AND RESULTS

LPA yielded three profiles with qualitatively distinct response patterns of motor performance, EF, and verbal ability. Quantitatively, the profiles showed most pronounced differences with regard to parent ratings and performance-based tests of EF, as well as verbal ability. Risk of motor coordination difficulties and ADHD symptomatology were associated with profile membership, whereas age, gender, language problems, and SES were not.

CONCLUSIONS AND IMPLICATIONS

Our results indicate that there are distinct subpopulations of children who show differential relations with regard to motor performance, EF, and verbal ability. The fact that we found both quantitative as well as qualitative differences between the three patterns of profiles underscores the need for a person-centered approach with a focus on patterns of individual characteristics.

Role of Sensorimotor Cortex in Gestural-Verbal Integration

Action comprehension that is related to language or gestural integration has been shown to engage the motor system in the brain, thus providing preliminary evidence for the gestural-verbal embodiment concept. Based on the involvement of the sensorimotor cortex (M1) in language processing, we aimed to further explore its role in the cognitive embodiment necessary for gestural-verbal integration. As such, we applied anodal (excitatory) and sham transcranial direct current stimulation (tDCS) over the left M1 (with reference electrode over the contralateral supraorbital region) during a gestural-verbal integration task where subjects had to make a decision about the semantic congruency of the gesture (prime) and the word (target). We used a cross-over within-subject design in young subjects. Attentional load and simple reaction time (RT) tasks served as control conditions, applied during stimulation (order of three tasks was counterbalanced). Our results showed that anodal (atDCS) compared to sham tDCS (stDCS) reduced RTs in the gestural-verbal integration task, specifically for incongruent pairs of gestures and verbal expressions, with no effect on control task performance. Our findings provide evidence for the involvement of the sensorimotor system in gestural-verbal integration performance. Further, our results suggest that functional modulation induced by sensorimotor tDCS may be specific to gestural-verbal integration. Future studies should now evaluate the modulatory effect of tDCS on semantic congruency by using tDCS over additional brain regions and include assessments of neural connectivity.

From Wide Cognition to Mechanisms: A Silent Revolution

In this paper, we argue that several recent ‘wide’ perspectives on cognition (embodied, embedded, extended, enactive, and distributed) are only partially relevant to the study of cognition. While these wide accounts override traditional methodological individualism, the study of cognition has already progressed beyond these proposed perspectives toward building integrated explanations of the mechanisms involved, including not only internal submechanisms but also interactions with others, groups, cognitive artifacts, and their environment. Wide perspectives are essentially research heuristics for building mechanistic explanations. The claim is substantiated with reference to recent developments in the study of “mindreading” and debates on emotions. We argue that the current practice in cognitive (neuro)science has undergone, in effect, a silent mechanistic revolution, and has turned from initial binary oppositions and abstract proposals toward the integration of wide perspectives with the rest of the cognitive (neuro)sciences.

Shared neural and cognitive mechanisms in action and language: The multiscale information transfer framework

This review compares how humans process action and language sequences produced by other humans. On the one hand, we identify commonalities between action and language processing in terms of cognitive mechanisms (e.g., perceptual segmentation, predictive processing, integration across multiple temporal scales), neural resources (e.g., the left inferior frontal cortex), and processing algorithms (e.g., comprehension based on changes in signal entropy). On the other hand, drawing on sign language with its particularly strong motor component, we also highlight what differentiates (both oral and signed) linguistic communication from nonlinguistic action sequences. We propose the multiscale information transfer framework (MSIT) as a way of integrating these insights and highlight directions into which future empirical research inspired by the MSIT framework might fruitfully evolve. This article is categorized under: Psychology > Language Linguistics > Language in Mind and Brain Psychology > Motor Skill and Performance Psychology > Prediction.