Neural reuse of action perception circuits for language, concepts and communication

Motor Circuit and Superior Temporal Sulcus Activities Linked to Individual Differences in Multisensory Speech Perception

Integrating multimodal information into a unified perception is a fundamental human capacity. McGurk effect is a remarkable multisensory illusion that demonstrates a percept different from incongruent auditory and visual syllables. However, not all listeners perceive the McGurk illusion to the same degree. The neural basis for individual differences in modulation of multisensory integration and syllabic perception remains largely unclear. To probe the possible involvement of specific neural circuits in individual differences in multisensory speech perception, we first implemented a behavioral experiment to examine the McGurk susceptibility. Then, functional magnetic resonance imaging was performed in 63 participants to measure the brain activity in response to non-McGurk audiovisual syllables. We revealed significant individual variability in McGurk illusion perception. Moreover, we found significant differential activations of the auditory and visual regions and the left Superior temporal sulcus (STS), as well as multiple motor areas between strong and weak McGurk perceivers. Importantly, the individual engagement of the STS and motor areas could specifically predict the behavioral McGurk susceptibility, contrary to the sensory regions. These findings suggest that the distinct multimodal integration in STS as well as coordinated phonemic modulatory processes in motor circuits may serve as a neural substrate for interindividual differences in multisensory speech perception.

Biological constraints on neural network models of cognitive function

Neural network models are potential tools for improving our understanding of complex brain functions. To address this goal, these models need to be neurobiologically realistic. However, although neural networks have advanced dramatically in recent years and even achieve human-like performance on complex perceptual and cognitive tasks, their similarity to aspects of brain anatomy and physiology is imperfect. Here, we discuss different types of neural models, including localist, auto-associative, hetero-associative, deep and whole-brain networks, and identify aspects under which their biological plausibility can be improved. These aspects range from the choice of model neurons and of mechanisms of synaptic plasticity and learning to implementation of inhibition and control, along with neuroanatomical properties including areal structure and local and long-range connectivity. We highlight recent advances in developing biologically grounded cognitive theories and in mechanistically explaining, on the basis of these brain-constrained neural models, hitherto unaddressed issues regarding the nature, localization and ontogenetic and phylogenetic development of higher brain functions. In closing, we point to possible future clinical applications of brain-constrained modelling.

Multimodal neurocognitive markers of frontal lobe epilepsy: Insights from ecological text processing

The pressing call to detect sensitive cognitive markers of frontal lobe epilepsy (FLE) remains poorly addressed. Standard frameworks prove nosologically unspecific (as they reveal deficits that also emerge across other epilepsy subtypes), possess low ecological validity, and are rarely supported by multimodal neuroimaging assessments. To bridge these gaps, we examined naturalistic action and non-action text comprehension, combined with structural and functional connectivity measures, in 19 FLE patients, 19 healthy controls, and 20 posterior cortex epilepsy (PCE) patients. Our analyses integrated inferential statistics and data-driven machine-learning classifiers. FLE patients were selectively and specifically impaired in action comprehension, irrespective of their neuropsychological profile. These deficits selectively and specifically correlated with (a) reduced integrity of the anterior thalamic radiation, a subcortical structure underlying motoric and action-language processing as well as epileptic seizure spread in this subtype; and (b) hypoconnectivity between the primary motor cortex and the left-parietal/supramarginal regions, two putative substrates of action-language comprehension. Moreover, machine-learning classifiers based on the above neurocognitive measures yielded 75% accuracy rates in discriminating individual FLE patients from both controls and PCE patients. Briefly, action-text assessments, combined with structural and functional connectivity measures, seem to capture ecological cognitive deficits that are specific to FLE, opening new avenues for discriminatory characterizations among epilepsy types.

Rapid microstructural plasticity in the cortical semantic network following a short language learning session

Despite the clear importance of language in our life, our vital ability to quickly and effectively learn new words and meanings is neurobiologically poorly understood. Conventional knowledge maintains that language learning—especially in adulthood—is slow and laborious. Furthermore, its structural basis remains unclear. Even though behavioural manifestations of learning are evident near instantly, previous neuroimaging work across a range of semantic categories has largely studied neural changes associated with months or years of practice. Here, we address rapid neuroanatomical plasticity accompanying new lexicon acquisition, specifically focussing on the learning of action-related language, which has been linked to the brain’s motor systems. Our results show that it is possible to measure and to externally modulate (using transcranial magnetic stimulation (TMS) of motor cortex) cortical microanatomic reorganisation after mere minutes of new word learning. Learning-induced microstructural changes, as measured by diffusion kurtosis imaging (DKI) and machine learning-based analysis, were evident in prefrontal, temporal, and parietal neocortical sites, likely reflecting integrative lexico-semantic processing and formation of new memory circuits immediately during the learning tasks. These results suggest a structural basis for the rapid neocortical word encoding mechanism and reveal the causally interactive relationship of modal and associative brain regions in supporting learning and word acquisition.

This combined neuroimaging and brain stimulation study reveals rapid and distributed microstructural plasticity after a single immersive language learning session, demonstrating the causal relevance of the motor cortex in encoding the meaning of novel action words.

The Same Ultra-Rapid Parallel Brain Dynamics Underpin the Production and Perception of Speech

The temporal dynamics by which linguistic information becomes available is one of the key properties to understand how language is organized in the brain. An unresolved debate between different brain language models is whether words, the building blocks of language, are activated in a sequential or parallel manner. In this study, we approached this issue from a novel perspective by directly comparing the time course of word component activation in speech production versus perception. In an overt object naming task and a passive listening task, we analyzed with mixed linear models at the single-trial level the event-related brain potentials elicited by the same lexico-semantic and phonological word knowledge in the two language modalities. Results revealed that both word components manifested simultaneously as early as 75 ms after stimulus onset in production and perception; differences between the language modalities only became apparent after 300 ms of processing. The data provide evidence for ultra-rapid parallel dynamics of language processing and are interpreted within a neural assembly framework where words recruit the same integrated cell assemblies across production and perception. These word assemblies ignite early on in parallel and only later on reverberate in a behavior-specific manner.

The Role of the Dorsolateral Prefrontal Cortex for Speech and Language Processing

This review article summarizes various functions of the dorsolateral prefrontal cortex (DLPFC) that are related to language processing. To this end, its connectivity with the left-dominant perisylvian language network was considered, as well as its interaction with other functional networks that, directly or indirectly, contribute to language processing. Language-related functions of the DLPFC comprise various aspects of pragmatic processing such as discourse management, integration of prosody, interpretation of nonliteral meanings, inference making, ambiguity resolution, and error repair. Neurophysiologically, the DLPFC seems to be a key region for implementing functional connectivity between the language network and other functional networks, including cortico-cortical as well as subcortical circuits. Considering clinical aspects, damage to the DLPFC causes psychiatric communication deficits rather than typical aphasic language syndromes. Although the number of well-controlled studies on DLPFC language functions is still limited, the DLPFC might be an important target region for the treatment of pragmatic language disorders.

Intangible features extraction in the processing of abstract concepts: Evidence from picture-word priming

Over the last decade, hypotheses ranging from linguistic symbol processing to embodiment have been formulated to account for the content and mechanisms responsible for the representation of abstract concepts. Results of recent studies have suggested that abstract concepts, just like concrete ones, can benefit from knowledge of real-world situational context, but that they can also be processed based on abstract pictures devoid of such situational features. This paper presents two semantic priming experiments to explore such mechanisms further. The first experiment replicates Kuipers, Jones, and Thierry (2018) in a cross-linguistic setting which shows that abstract concepts can be processed from abstract pictures devoid of tangible features. In the second experiment, we studied extraction mechanisms that come into play when participants are presented with abstract and concrete pictures that provide situational information to illustrate target abstract concepts. We expected this facilitatory effect to be limited to concrete picture primes. Our data analysed with both Bayesian and Frequentist tests showed however that even when presented with tangible situational information, the extraction of features still occurred for abstract pictures. We discuss the implications of this with respect to future avenues for studying the processing of abstract concepts.

Event related spectral perturbations of gesture congruity: Visuospatial resources are recruited for multimodal discourse comprehension

Here we examine the role of visuospatial working memory (WM) during the comprehension of multimodal discourse with co-speech iconic gestures. EEG was recorded as healthy adults encoded either a sequence of one (low load) or four (high load) dot locations on a grid and rehearsed them until a free recall response was collected later in the trial. During the rehearsal period of the WM task, participants observed videos of a speaker describing objects in which half of the trials included semantically related co-speech gestures (congruent), and the other half included semantically unrelated gestures (incongruent). Discourse processing was indexed by oscillatory EEG activity in the alpha and beta bands during the videos. Across all participants, effects of speech and gesture incongruity were more evident in low load trials than in high load trials. Effects were also modulated by individual differences in visuospatial WM capacity. These data suggest visuospatial WM resources are recruited in the comprehension of multimodal discourse.

Motor processing modulates word comprehension

We evaluated whether movement modulates the semantic processing of words. To this end, we used homograph words with two meanings, one associated with hand movements (e.g., 'abanico', 'fan' in Spanish) or foot movements ('bota', 'boot' in Spanish), and the other not associated with movement ('abanico', 'range' in Spanish; 'bota', 'wineskin' in Spanish). After the homograph, three words were presented, and participants were asked to choose the word related to one of the two homograph meanings. The words could be either related to the motor meaning of the homograph ('fan-heat'), to the non-motor meaning of the homograph ('range-possibility') or unrelated ('fan-phone'). The task was performed without movement (simple condition) or by performing hand (Experiment 1) and foot (Experiment 2) movements. Compared with the simple condition, the performance of movement oriented the preference towards the motor meaning of the homograph. This pattern of results confirms that movement modulates word comprehension.

Evidence-Based Guidelines and Secondary Meta-Analysis for the Use of Transcranial Direct Current Stimulation in Neurological and Psychiatric Disorders

BACKGROUND

Transcranial direct current stimulation has shown promising clinical results, leading to increased demand for an evidence-based review on its clinical effects.

OBJECTIVE

We convened a team of transcranial direct current stimulation experts to conduct a systematic review of clinical trials with more than 1 session of stimulation testing: pain, Parkinson's disease motor function and cognition, stroke motor function and language, epilepsy, major depressive disorder, obsessive compulsive disorder, Tourette syndrome, schizophrenia, and drug addiction.

METHODS

Experts were asked to conduct this systematic review according to the search methodology from PRISMA guidelines. Recommendations on efficacy were categorized into Levels A (definitely effective), B (probably effective), C (possibly effective), or no recommendation. We assessed risk of bias for all included studies to confirm whether results were driven by potentially biased studies.

RESULTS

Although most of the clinical trials have been designed as proof-of-concept trials, some of the indications analyzed in this review can be considered as definitely effective (Level A), such as depression, and probably effective (Level B), such as neuropathic pain, fibromyalgia, migraine, post-operative patient-controlled analgesia and pain, Parkinson's disease (motor and cognition), stroke (motor), epilepsy, schizophrenia, and alcohol addiction. Assessment of bias showed that most of the studies had low risk of biases, and sensitivity analysis for bias did not change these results. Effect sizes vary from 0.01 to 0.70 and were significant in about 8 conditions, with the largest effect size being in postoperative acute pain and smaller in stroke motor recovery (nonsignificant when combined with robotic therapy).

CONCLUSION

All recommendations listed here are based on current published PubMed-indexed data. Despite high levels of evidence in some conditions, it must be underscored that effect sizes and duration of effects are often limited; thus, real clinical impact needs to be further determined with different study designs.

Is justice grounded? How expertise shapes conceptual representation of institutional concepts

Using abstract concepts is a hallmark of human cognition. While multiple kinds of abstract concepts exist, they so far have been conceived as a unitary kind in opposition to concrete ones. Here, we focus on Institutional concepts, like justice or norm, investigating their fine-grained differences with respect to other kinds of abstract and concrete concepts, and exploring whether their representation varies according to individual proficiency. Specifically, we asked experts and non-experts in the legal field to evaluate four kinds of concepts (i.e., institutional, theoretical, food, artefact) on 16 dimensions: abstractness-concreteness; imageability; contextual availability; familiarity; age of acquisition; modality of acquisition; social valence; social metacognition; arousal; valence; interoception; metacognition; perceptual modality strength; body-object interaction; mouth and hand involvement. Results showed that Institutional concepts rely more than other categories on linguistic/social and inner experiences and are primarily characterized by positive valence. In addition, a more subtle characterization of the institutional domain emerged: Pure-institutional concepts (e.g., parliament) were perceived as more similar to technical tools, while Meta-institutional concepts (e.g., validity) were characterized mainly by abstract components. Importantly, for what concerns individual proficiency, we found that the level of expertise affects conceptual representation. Only law-experts associated Institutional concepts with exteroceptive and emotional experiences, showing also a more grounded and situated representation of the two types of institutional concepts. Overall, our finding highlights the richness and flexibility of abstract concepts and suggests that they differ in the degree of embodiment and grounding. Implications of the results for current theories of conceptual representation and social institutions are discussed.

White-Matter Neuroanatomical Predictors of Aphasic Verb Retrieval

Background: Current neurocognitive models of language function have been primarily built from evidence regarding object naming, and their hypothesized white-matter circuit mechanisms tend to be coarse grained. Methods: In this cross-sectional, observational study, we used novel correlational tractography to assess the white-matter circuit mechanism behind verb retrieval, measured through action picture-naming performance in adults with chronic aphasia. Results: The analysis identified tracts implicated in current neurocognitive dual-stream models of language function, including the left inferior fronto-occipital fasciculus, inferior longitudinal fasciculus, and arcuate fasciculus. However, the majority of tracts associated with verb retrieval were not ones included in dual-stream models of language function. Instead, they were projection pathways that connect frontal and parietal cortices to subcortical regions associated with motor functions, including the left corticothalamic pathway, frontopontine tract, parietopontine tract, corticostriatal pathway, and corticospinal tract. Conclusions: These results highlight that corticosubcortical projection pathways implicated in motor functions may be importantly related to language function. This finding is consistent with grounded accounts of cognition and may furthermore inform neurocognitive models. Impact statement This study suggests that in addition to traditional dual-stream language fiber tracts, the integrity of projection pathways that connect frontal and parietal cortices to subcortical motor regions may be critically associated with verb-retrieval impairments in adults with aphasia. This finding challenges neurological models of language function.

Quantum semantics of text perception

The paper presents quantum model of subjective text perception based on binary cognitive distinctions corresponding to words of natural language. The result of perception is quantum cognitive state represented by vector in the qubit Hilbert space. Complex-valued structure of the quantum state space extends the standard vector-based approach to semantics, allowing to account for subjective dimension of human perception in which the result is constrained, but not fully predetermined by input information. In the case of two distinctions, the perception model generates a two-qubit state, entanglement of which quantifies semantic connection between the corresponding words. This two-distinction perception case is realized in the algorithm for detection and measurement of semantic connectivity between pairs of words. The algorithm is experimentally tested with positive results. The developed approach to cognitive modeling unifies neurophysiological, linguistic, and psychological descriptions in a mathematical and conceptual structure of quantum theory, extending horizons of machine intelligence.

Chinese-English bilinguals show linguistic-perceptual links in the brain associating short spoken phrases with corresponding real-world natural action sounds by semantic category

Higher cognitive functions such as linguistic comprehension must ultimately relate to perceptual systems in the brain, though how and why this forms remains unclear. Different brain networks that mediate perception when hearing real-world natural sounds has recently been proposed to respect a taxonomic model of acoustic-semantic categories. Using functional magnetic resonance imaging (fMRI) with Chinese/English bilingual listeners, the present study explored whether reception of short spoken phrases, in both Chinese (Mandarin) and English, describing corresponding sound-producing events would engage overlapping brain regions at a semantic category level. The results revealed a double-dissociation of cortical regions that were preferential for representing knowledge of human versus environmental action events, whether conveyed through natural sounds or the corresponding spoken phrases depicted by either language. These findings of cortical hubs exhibiting linguistic-perceptual knowledge links at a semantic category level should help to advance neurocomputational models of the neurodevelopment of language systems.

Primary Progressive Aphasia: Toward a Pathophysiological Synthesis

PURPOSE OF REVIEW

The term primary progressive aphasia (PPA) refers to a diverse group of dementias that present with prominent and early problems with speech and language. They present considerable challenges to clinicians and researchers.

RECENT FINDINGS

Here, we review critical issues around diagnosis of the three major PPA variants (semantic variant PPA, nonfluent/agrammatic variant PPA, logopenic variant PPA), as well as considering 'fragmentary' syndromes. We next consider issues around assessing disease stage, before discussing physiological phenotyping of proteinopathies across the PPA spectrum. We also review evidence for core central auditory impairments in PPA, outline critical challenges associated with treatment, discuss pathophysiological features of each major PPA variant, and conclude with thoughts on key challenges that remain to be addressed. New findings elucidating the pathophysiology of PPA represent a major step forward in our understanding of these diseases, with implications for diagnosis, care, management, and therapies.

Meta-Analyses Support a Taxonomic Model for Representations of Different Categories of Audio-Visual Interaction Events in the Human Brain

Our ability to perceive meaningful action events involving objects, people, and other animate agents is characterized in part by an interplay of visual and auditory sensory processing and their cross-modal interactions. However, this multisensory ability can be altered or dysfunctional in some hearing and sighted individuals, and in some clinical populations. The present meta-analysis sought to test current hypotheses regarding neurobiological architectures that may mediate audio-visual multisensory processing. Reported coordinates from 82 neuroimaging studies (137 experiments) that revealed some form of audio-visual interaction in discrete brain regions were compiled, converted to a common coordinate space, and then organized along specific categorical dimensions to generate activation likelihood estimate (ALE) brain maps and various contrasts of those derived maps. The results revealed brain regions (cortical "hubs") preferentially involved in multisensory processing along different stimulus category dimensions, including 1) living versus nonliving audio-visual events, 2) audio-visual events involving vocalizations versus actions by living sources, 3) emotionally valent events, and 4) dynamic-visual versus static-visual audio-visual stimuli. These meta-analysis results are discussed in the context of neurocomputational theories of semantic knowledge representations and perception, and the brain volumes of interest are available for download to facilitate data interpretation for future neuroimaging studies.

Multilayer Connector Hub Mapping Reveals Key Brain Regions Supporting Expressive Language

Introduction: How components of the distributed brain networks that support cognition participate in typical functioning remains a largely unanswered question. An important subgroup of regions in the larger network are connector hubs, which are areas that are highly connected to several other functionally specialized sets of regions, and are likely important for sensorimotor integration. The present study attempts to characterize connector hubs involved in typical expressive language functioning using a data-driven, multimodal, full multilayer magnetoencephalography (MEG) connectivity-based pipeline. Methods: Twelve adolescents, 16-18 years of age (five males), participated in this study. Participants underwent MEG scanning during a verb generation task. MEG and structural connectivity were calculated at the whole-brain level. Amplitude/amplitude coupling (AAC) was used to compute functional connections both within and between discrete frequency bins. AAC values were then multiplied by a binary structural connectivity matrix, and then entered into full multilayer network analysis. Initially, hubs were defined based on multilayer versatility and subsequently reranked by a novel measure called delta centrality on interconnectedness (DCI). DCI is defined as the percent change in interfrequency interconnectedness after removal of a hub. Results: We resolved regions that are important for between-frequency communication among other areas during expressive language, with several potential theoretical and clinical applications that can be generalized to other cognitive domains. Conclusion: Our multilayer, data-driven framework captures nonlinear connections that span across scales that are often missed in conventional analyses. The present study suggests that crucial hubs may be conduits for interfrequency communication between action and perception systems that are crucial for typical functioning.

Brain signatures predict communicative function of speech production in interaction

People normally know what they want to communicate before they start speaking. However, brain indicators of communication are typically observed only after speech act onset, and it is unclear when any anticipatory brain activity prior to speaking might first emerge, along with the communicative intentions it possibly reflects. Here, we investigated brain activity prior to the production of different speech act types, request and naming actions performed by uttering single words embedded into language games with a partner, similar to natural communication. Starting ca. 600 msec before speech onset, an event-related potential maximal at fronto-central electrodes, which resembled the Readiness Potential, was larger when preparing requests compared to naming actions. Analysis of the cortical sources of this anticipatory brain potential suggests a relatively stronger involvement of fronto-central motor regions for requests, which may reflect the speaker's expectation of the partner actions typically following requests, e.g., the handing over of a requested object. Our results indicate that different neuronal circuits underlying the processing of different speech act types activate already before speaking. Results are discussed in light of previous work addressing the neural basis of speech act understanding and predictive brain indexes of language comprehension.

Electrifying discourse: Anodal tDCS of the primary motor cortex selectively reduces action appraisal in naturalistic narratives

Non-invasive stimulation of the primary motor cortex (M1) modulates processing of decontextualized action words and sentences (i.e., verbal units denoting bodily motion). This suggests that language comprehension hinges on brain circuits mediating the bodily experiences evoked by verbal material. Yet, despite its relevance to constrain mechanistic language models, such a finding fails to reveal whether and how relevant circuits operate in the face of full-blown, everyday texts. Using a novel naturalistic discourse paradigm, we examined whether direct modulation of M1 excitability influences the grasping of narrated actions. Following random group assignment, participants received anodal transcranial direct current stimulation over the left M1, or sham stimulation of the same area, or anodal stimulation of the left ventrolateral prefrontal cortex. Immediately afterwards, they listened to action-laden and neutral stories and answered questions on information realized by verbs (denoting action and non-action processes) and circumstances (conveying locative or temporal details). Anodal stimulation of the left M1 selectively decreased outcomes on action-relative to non-action information -a pattern that discriminated between stimulated and sham participants with 74% accuracy. This result was particular to M1 and held irrespective of the subjects' working memory and vocabulary skills, further attesting to its specificity. Our findings suggest that offline modulation of motor-network excitability might lead to transient unavailability of putative resources needed to evoke actions in naturalistic texts, opening promising avenues for the language embodiment framework.

Correlated Brain Indexes of Semantic Prediction and Prediction Error: Brain Localization and Category Specificity

With strong and valid predictions, grasping a message is easy, whereas more demanding processing is required in the absence of robust expectations. We here demonstrate that brain correlates of the interplay between prediction and perception mechanisms in the understanding of meaningful sentences. Sentence fragments that strongly predict subsequent words induced anticipatory brain activity preceding the expected words; this potential was absent if context did not strongly predict subsequent words. Subjective reports of certainty about upcoming words and objective corpus-based measures correlated with the size of the anticipatory signal, thus establishing its status as a semantic prediction potential (SPP). Crucially, there was an inverse correlation between the SPP and the N400 brain response. The main cortical generators of SPP and N400 were found in inferior prefrontal cortex and posterior temporal cortex, respectively. Interestingly, sentence meaning was reflected by both measures, with additional category-specific sources of SPPs and N400s falling into parieto-temporo-occipital (visual) and frontocentral (sensorimotor) areas for animal- and tool-related words, respectively. These results show that the well-known brain index of semantic comprehension, N400, has an antecedent with different brain localization but similar semantic discriminatory function. We discuss whether N400 dynamics may causally depend on mechanisms underlying SPP size and sources.

A Sound Explanation for Motor Cortex Engagement during Action Word Comprehension

Comprehending action words often engages similar brain regions to those involved in perceiving and executing actions. This finding has been interpreted as support for grounding of conceptual processing in motor representations or that conceptual processing involves motor simulation. However, such demonstrations cannot confirm the nature of the mechanism(s) responsible, as word comprehension involves multiple processes (e.g., lexical, semantic, morphological, phonological). In this study, we tested whether this motor cortex engagement instead reflects processing of statistical regularities in sublexical phonological features. Specifically, we measured brain activity in healthy participants using functional magnetic resonance imaging while they performed an auditory lexical decision paradigm involving monosyllabic action words associated with specific effectors (face, arm, and leg). We show that nonwords matched to the action words in terms of their phonotactic probability elicit common patterns of activation. In addition, we show that a measure of the action words' phonological typicality, the extent to which a word's phonology is typical of other words in the grammatical category to which it belongs (i.e., more or less verb-like), is responsible for their activating a significant portion of primary and premotor cortices. These results indicate motor cortex engagement during action word comprehension is more likely to reflect processing of statistical regularities in sublexical phonological features than conceptual processing. We discuss the implications for current neurobiological models of language, all of which implicitly or explicitly assume that the relationship between the sound of a word and its meaning is arbitrary.

Crossmodal Language Grounding in an Embodied Neurocognitive Model

Human infants are able to acquire natural language seemingly easily at an early age. Their language learning seems to occur simultaneously with learning other cognitive functions as well as with playful interactions with the environment and caregivers. From a neuroscientific perspective, natural language is embodied, grounded in most, if not all, sensory and sensorimotor modalities, and acquired by means of crossmodal integration. However, characterizing the underlying mechanisms in the brain is difficult and explaining the grounding of language in crossmodal perception and action remains challenging. In this paper, we present a neurocognitive model for language grounding which reflects bio-inspired mechanisms such as an implicit adaptation of timescales as well as end-to-end multimodal abstraction. It addresses developmental robotic interaction and extends its learning capabilities using larger-scale knowledge-based data. In our scenario, we utilize the humanoid robot NICO in obtaining the EMIL data collection, in which the cognitive robot interacts with objects in a children's playground environment while receiving linguistic labels from a caregiver. The model analysis shows that crossmodally integrated representations are sufficient for acquiring language merely from sensory input through interaction with objects in an environment. The representations self-organize hierarchically and embed temporal and spatial information through composition and decomposition. This model can also provide the basis for further crossmodal integration of perceptually grounded cognitive representations.

Distinct fronto-temporal substrates of distributional and taxonomic similarity among words: evidence from RSA of BOLD signals

A class of semantic theories defines concepts in terms of statistical distributions of lexical items, basing meaning on vectors of word co-occurrence frequencies. A different approach emphasizes abstract hierarchical taxonomic relationships among concepts. However, the functional relevance of these different accounts and how they capture information-encoding of lexical meaning in the brain still remains elusive. We investigated to what extent distributional and taxonomic models explained word-elicited neural responses using cross-validated representational similarity analysis (RSA) of functional magnetic resonance imaging (fMRI) and model comparisons. Our findings show that the brain encodes both types of semantic information, but in distinct cortical regions. Posterior middle temporal regions reflected lexical-semantic similarity based on hierarchical taxonomies, in coherence with the action-relatedness of specific semantic word categories. In contrast, distributional semantics best predicted the representational patterns in left inferior frontal gyrus (LIFG, BA 47). Both representations coexisted in the angular gyrus supporting semantic binding and integration. These results reveal that neuronal networks with distinct cortical distributions across higher-order association cortex encode different representational properties of word meanings. Taxonomy may shape long-term lexical-semantic representations in memory consistently with the sensorimotor details of semantic categories, whilst distributional knowledge in the LIFG (BA 47) may enable semantic combinatorics in the context of language use. Our approach helps to elucidate the nature of semantic representations essential for understanding human language.

The Embodied Penman: Effector-Specific Motor-Language Integration During Handwriting

Several studies have illuminated how processing manual action verbs (MaVs) affects the programming or execution of concurrent hand movements. Here, to circumvent key confounds in extant designs, we conducted the first assessment of motor-language integration during handwriting-a task in which linguistic and motoric processes are co-substantiated. Participants copied MaVs, non-manual action verbs, and non-action verbs as we collected measures of motor programming and motor execution. Programming latencies were similar across conditions, but execution was faster for MaVs than for the other categories, regardless of whether word meanings were accessed implicitly or explicitly. In line with the Hand-Action-Network Dynamic Language Embodiment (HANDLE) model, such findings suggest that effector-congruent verbs can prime manual movements even during highly automatized tasks in which motoric and verbal processes are naturally intertwined. Our paradigm opens new avenues for fine-grained explorations of embodied language processes.

Time to Face Language: Embodied Mechanisms Underpin the Inception of Face-Related Meanings in the Human Brain

In construing meaning, the brain recruits multimodal (conceptual) systems and embodied (modality-specific) mechanisms. Yet, no consensus exists on how crucial the latter are for the inception of semantic distinctions. To address this issue, we combined electroencephalographic (EEG) and intracranial EEG (iEEG) to examine when nouns denoting facial body parts (FBPs) and nonFBPs are discriminated in face-processing and multimodal networks. First, FBP words increased N170 amplitude (a hallmark of early facial processing). Second, they triggered fast (~100 ms) activity boosts within the face-processing network, alongside later (~275 ms) effects in multimodal circuits. Third, iEEG recordings from face-processing hubs allowed decoding ~80% of items before 200 ms, while classification based on multimodal-network activity only surpassed ~70% after 250 ms. Finally, EEG and iEEG connectivity between both networks proved greater in early (0-200 ms) than later (200-400 ms) windows. Collectively, our findings indicate that, at least for some lexico-semantic categories, meaning is construed through fast reenactments of modality-specific experience.

Action fluency identifies different sex, age, global cognition, executive function and brain activation profile in non-demented patients with Parkinson's disease

Patients with Parkinson's disease (PD) have difficulties processing action words, which could be related to early cognitive decline. The action fluency test can be used to quickly and easily assess the processing of action words in PD. The goal of this study was to characterize how the action fluency test relates to personal characteristics, disease factors, cognition, and neural activity in PD. Forty-eight participants with PD (34 male, 14 female) and 35 control participants (16 male, 19 female) completed functional neuroimaging using a set-shifting task and a neuropsychological assessment including the action fluency test. PD participants with a score one standard deviation below the norm or lower on the action fluency test were identified. All PD participants with poor performance (PD-P, n = 15) were male. They were compared to male PD participants with scores within the normal range (PD-N, n = 19) and male healthy controls (HC, n = 16). PD-P were older, had lower global cognition scores, lower executive functions scores, and decreased activity in fronto-temporal regions compared with PD-N. There was no difference between the two PD groups in terms of the duration of the disease, dose of dopaminergic medication, and severity of motor symptoms. PD-N were younger than HC, but there was no other significant difference between these groups. The action fluency test identified a subgroup of PD patients with distinct sex, age, global cognition, executive functions, and brain activity characteristics. Implications for the evaluation of cognition are discussed.

From discourse to pathology: Automatic identification of Parkinson's disease patients via morphological measures across three languages

Embodied cognition research on Parkinson's disease (PD) points to disruptions of frontostriatal language functions as sensitive targets for clinical assessment. However, no existing approach has been tested for crosslinguistic validity, let alone by combining naturalistic tasks with machine-learning tools. To address these issues, we conducted the first classifier-based examination of morphological processing (a core frontostriatal function) in spontaneous monologues from PD patients across three typologically different languages. The study comprised 330 participants, encompassing speakers of Spanish (61 patients, 57 matched controls), German (88 patients, 88 matched controls), and Czech (20 patients, 16 matched controls). All subjects described the activities they perform during a regular day, and their monologues were automatically coded via morphological tagging, a computerized method that labels each word with a part-of-speech tag (e.g., noun, verb) and specific morphological tags (e.g., person, gender, number, tense). The ensuing data were subjected to machine-learning analyses to assess whether differential morphological patterns could classify between patients and controls and reflect the former's degree of motor impairment. Results showed robust classification rates, with over 80% of patients being discriminated from controls in each language separately. Moreover, the most discriminative morphological features were associated with the patients' motor compromise (as indicated by Pearson r correlations between predicted and collected motor impairment scores that ranged from moderate to moderate-to-strong across languages). Taken together, our results suggest that morphological patterning, an embodied frontostriatal domain, may be distinctively affected in PD across languages and even under ecological testing conditions.

Action sound-shape congruencies explain sound symbolism

Sound symbolism, the surprising semantic relationship between meaningless pseudowords (e.g., 'maluma', 'takete') and abstract (round vs. sharp) shapes, is a hitherto unexplained human-specific knowledge domain. Here we explore whether abstract sound symbolic links can be explained by those between the sounds and shapes of bodily actions. To this end, we asked human subjects to match pseudowords with abstract shapes and, in a different experimental block, the sounds of actions with the shapes of the trajectories of the actions causing these same sounds. Crucially, both conditions were also crossed. Our findings reveal concordant matching in the sound symbolic and action domains, and, importantly, significant correlations between them. We conclude that the sound symbolic knowledge interlinking speech sounds and abstract shapes is explained by audiovisual information immanent to action experience along with acoustic similarities between speech and action sounds. These results demonstrate a fundamental role of action knowledge for abstract sound symbolism, which may have been key to human symbol-manipulation ability.

How words ripple through bilingual hands: Motor-language coupling during L1 and L2 writing

The speed of our hand movements can be affected by concurrent processing of manual action verbs (MaVs). Whereas this phenomenon is well established for native languages (L1s), it remains underexplored in late foreign languages (L2s), especially during highly automatized tasks. Here we timed keystroke activity while Spanish-English bilinguals copied MaVs, non-manual action verbs, and non-action verbs in their L1 and L2. Motor planning and execution dynamics were indexed by first-letter lag (the time-lapse between word presentation and first keystroke) and whole-word lag (the time-lapse between first and last keystroke), respectively. Despite yielding no effects on motor planning, MaVs facilitated typing execution in L1 but delayed it in L2, irrespective of the subjects' typing skills, age of L2 learning, and L2 competence. Therefore, motor-language coupling effects seem to be present in both languages though they can arise differently in each. These results extend language grounding models, illuminating the role of embodied mechanisms throughout life.

Semantic and attentional networks in bilingual processing: fMRI connectivity signatures of translation directionality

Comparisons between backward and forward translation (BT, FT) have long illuminated the organization of bilingual memory, with neuroscientific evidence indicating that FT would involve greater linguistic and attentional demands. However, no study has directly assessed the functional interaction between relevant mechanisms. Against this background, we conducted the first fMRI investigation of functional connectivity (FC) differences between BT and FT. In addition to yielding lower behavioral outcomes, FT was characterized by increased FC between a core semantic hub (the left anterior temporal lobe, ATL) and key nodes of attentional and vigilance networks (left inferior frontal, left orbitofrontal, and bilateral parietal clusters). Instead, distinct FC patterns for BT emerged only between the left ATL and the right thalamus, a region implicated in automatic relaying of sensory information to cortical regions. Therefore, FT seems to involve enhanced coupling between semantic and attentional mechanisms, suggesting that asymmetries in cross-language processing reflect dynamic interactions between linguistic and domain-general systems.

Early Motor Trajectories Predict Motor but not Cognitive Function in Preterm- and Term-Born Adults without Pre-existing Neurological Conditions

Very preterm (VP; <32 weeks gestation) and/or very low birth weight (VLBW; <1500 g) birth has been associated with an increased risk of adverse motor and cognitive outcomes that may persist into adulthood. The aim of this study was to determine whether motor development in the first five years of life is associated with motor and cognitive outcomes in adulthood. A prospective observational study in Germany followed 260 VP/VLBW and 229 term-born individuals from birth into adulthood. Early motor trajectories (i.e., high and low degree of motor difficulties) were determined from neurological examinations from birth to 56 months. Adult motor and cognitive outcomes were determined from information from multiple instruments and IQ tests, respectively. Associations of VP/VLBW birth and early motor difficulties on adult outcomes were assessed using regression analyses. VP/VLBW individuals had an increased risk for early motor difficulties (Relative Risk: 11.77, 95% confidence interval (CI): 4.28, 32.35). Early motor difficulties were associated with poorer motor competence in adulthood (β = 0.22, p < 0.001), independent of VP/VLBW birth. Adult IQ was predicted by VP/VLBW (β = −0.12, p < 0.05) and child IQ (β = 0.51, p < 0.001), while early motor difficulties ceased to be associated with adult IQ once participants with a neurological impairment were excluded (β = 0.02, p > 0.05). Motor problems in childhood were homotypically associated with poorer motor competence in adulthood. Similarly, early cognitive problems were homotypically associated with adult cognitive outcomes. Thus, both motor and cognitive function should be assessed in routine follow-up during childhood.

Motor-system dynamics during naturalistic reading of action narratives in first and second language

Do embodied semantic systems play different roles depending on when and how well a given language was learned? Emergent evidence suggests that this is the case for isolated, decontextualized stimuli, but no study has addressed the issue considering naturalistic narratives. Seeking to bridge this gap, we assessed motor-system dynamics in 26 Spanish-English bilinguals as they engaged in free, unconstrained reading of naturalistic action texts (ATs, highlighting the characters’ movements) and neutral texts (NTs, featuring low motility) in their first and second language (L1, L2). To explore functional connectivity spread over each reading session, we recorded ongoing high-density electroencephalographic signals and subjected them to functional connectivity analysis via a spatial clustering approach. Results showed that, in L1, AT (relative to NT) reading involved increased connectivity between left and right central electrodes consistently implicated in action-related processes, as well as distinct source-level modulations in motor regions. In L2, despite null group-level effects, enhanced motor-related connectivity during AT reading correlated positively with L2 proficiency and negatively with age of L2 learning. Taken together, these findings suggest that action simulations during unconstrained narrative reading involve neural couplings between motor-sensitive mechanisms, in proportion to how consolidated a language is. More generally, such evidence addresses recent calls to test the ecological validity of motor-resonance effects while offering new insights on their relation with experiential variables.

Muscle activation in semantic processing: An electromyography approach

In this study, we focus on the spontaneous activity related to manual verbs to determine the extent to which semantic processing of manual verbs affects spontaneous arm muscle activity. For this purpose, we recorded the arm's electromyographic activity while participants read manual and non-manual verbs, focusing their attention on the semantic content or a specific letter. In addition, we manipulated the arm position (in front of the body or behind the back) to observe postural priming effects for spontaneous muscle activity. Our results show that when arms were placed forward and the attention was directed to the semantic content, there was an enhanced spontaneous activation. Our results suggest that spontaneous motor responses are related to the involvement of the motor system in action language comprehension as suggested by language embodiment theories.

The Nonverbal Processing of Actions Is an Area of Relative Strength in the Semantic Variant of Primary Progressive Aphasia

Purpose Better performance for actions compared to objects has been reported in the semantic variant of primary progressive aphasia (svPPA). This study investigated the influence of the assessment task (naming, semantic picture matching) over the dissociation between objects and actions. Method Ten individuals with svPPA and 17 matched controls completed object and action naming tests, and object and action semantic picture matching tests. Performance was compared between the svPPA and control groups, within the svPPA group, and for each participant with svPPA versus the control group individually. Results Compared to controls, participants with svPPA were impaired on object and action naming, and object and action semantic picture matching. As a group, participants with svPPA had an advantage for actions over objects and for semantic picture matching tests over naming tests. Eight participants had a better performance for actions compared to objects in naming, with three showing a significant difference. Nine participants had a better performance for actions compared to objects in semantic picture matching, with six showing a significant difference. For objects, semantic picture matching was better than naming in nine participants, with five showing a significant difference. For actions, semantic picture matching was better than naming in all 10 participants, with nine showing a significant difference. Conclusion The nonverbal processing of actions, as assessed with a semantic picture matching test, is an area of relative strength in svPPA. Clinical implications for assessment planning and interpretation and theoretical implications for current models of semantic cognition are discussed.

Distal engagement: Intentions in perception

Non-representational approaches to cognition have struggled to provide accounts of long-term planning that forgo the use of representations. An explanation comes easier for cognitivist accounts, which hold that we concoct and use contentful mental representations as guides to coordinate a series of actions towards an end state. One non-representational approach, ecological-enactivism, has recently seen several proposals that account for "high-level" or "representation-hungry" capacities, including long-term planning and action coordination. In this paper, we demonstrate the explanatory gap in these accounts that stems from avoiding the incorporation of long-term intentions, as they play an important role both in action coordination and perception on the ecological account. Using recent enactive accounts of language, we argue for a non-representational conception of intentions, their formation, and their role in coordinating pre-reflective action. We provide an account for the coordination of our present actions towards a distant goal, a skill we call distal engagement. Rather than positing intentions as an actual cognitive entity in need of explanation, we argue that we take them up in this way as a practice due to linguistically scaffolded attitudes towards language use.

Functional and resting-state characterizations of a periventricular heterotopic nodule associated with epileptogenic activity

The object of this study was to extensively characterize a region of periventricular nodular heterotopia (PVNH) in an epilepsy patient to reveal its possible neurocognitive functional role(s). The authors used 3-T MRI approaches to exhaustively characterize a single, right hemisphere heterotopion in a high-functioning adult male with medically responsive epilepsy, which had manifested during late adolescence. The heterotopion proved to be spectroscopically consistent with a cortical-like composition and was interconnected with nearby ipsilateral cortical fundi, as revealed by fiber tractography (diffusion-weighted imaging) and resting-state functional connectivity MRI (rsfMRI). Moreover, the region of PVNH demonstrated two novel characterizations for a heterotopion. First, functional MRI (fMRI), as distinct from rsfMRI, showed that the heterotopion was significantly modulated while the patient watched animated video scenes of biological motion (i.e., cartoons). Second, rsfMRI, which demonstrated correlated brain activity during a task-negative state, uniquely showed directionality within an interconnected network, receiving positive path effects from patent cortical and cerebellar foci while outputting only negative path effects to specific brain foci.These findings are addressed in the context of the impact on noninvasive presurgical brain mapping strategies for adult and pediatric patient workups, as well as the impact of this study on an understanding of the functional cortical architecture underlying cognition from a neurodiversity and evolutionary perspective.

Simulation across representation: The interplay of schemas and simulation-based inference on different levels of abstraction

Language comprehension of action verbs recruits embodied representations in the brain that are assumed to invoke a mental simulation (e.g., “grasping a peanut”). This extends to abstract concepts, as well (“grasping an idea”). We, therefore, argue that mental simulation works across levels of abstractness and involves higher-level schematic structures that subsume a generic structure of actions and events.

Too late to be grounded? Motor resonance for action words acquired after middle childhood

Though well established for languages acquired in infancy, the role of embodied mechanisms remains poorly understood for languages learned in middle childhood and adulthood. To bridge this gap, we examined 34 experiments that assessed sensorimotor resonance during processing of action-related words in real and artificial languages acquired since age 7 and into adulthood. Evidence from late bilinguals indicates that foreign-language action words modulate neural activity in motor circuits and predictably facilitate or delay physical movements (even in an effector-specific fashion), with outcomes that prove partly sensitive to language proficiency. Also, data from newly learned vocabularies suggest that embodied effects emerge after brief periods of adult language exposure, remain stable through time, and hinge on the performance of bodily movements (and, seemingly, on action observation, too). In sum, our work shows that infant language exposure is not indispensable for the recruitment of embodied mechanisms during language processing, a finding that carries non-trivial theoretical, pedagogical, and clinical implications for neurolinguistics, in general, and bilingualism research, in particular.

Prediction and Mismatch Negativity Responses Reflect Impairments in Action Semantic Processing in Adults With Autism Spectrum Disorders

The neurophysiological mechanisms underlying motor and language difficulties in autism spectrum disorders (ASD) are still largely unclear. The present work investigates biological indicators of sound processing, (action-) semantic understanding and predictive coding and their correlation with clinical symptoms of ASD. Twenty-two adults with high-functioning ASD and 25 typically developed (TD) participants engaged in an auditory, passive listening, Mismatch Negativity (MMN) task while high-density electroencephalography (EEG) was recorded. Action and non-action words were presented in the context of sounds, which were either semantically congruent with regard to the body part they relate to or semantically incongruent or unrelated. The anticipatory activity before sound onset, the Prediction Potential (PP), was significantly reduced in the ASD group specifically for action, but not for non-action sounds. The early-MMN-like responses to words (latency: 120 ms) were differentially modulated across groups: controls showed larger amplitudes for words in action-sound compared to non-action contexts, whereas ASD participants demonstrated enlarged early-MMN-like responses only in a pure tone context, with no other modulation dependent on action sound context. Late-MMN-like responses around 560 ms post-stimulus onset revealed body-part-congruent action-semantic priming for words in control participants, but not in the ASD group. Importantly, neurophysiological indices of semantic priming in ASD participants correlated with the extent of autistic traits as revealed by the Autism Spectrum Quotient (AQ). The data suggest that high-functioning adults with ASD show a specific deficit in semantic processing and predictive coding of sounds and words related to action, which is absent for neutral, non-action, sounds.

Neurophysiological evidence for rapid processing of verbal and gestural information in understanding communicative actions

During everyday social interaction, gestures are a fundamental part of human communication. The communicative pragmatic role of hand gestures and their interaction with spoken language has been documented at the earliest stage of language development, in which two types of indexical gestures are most prominent: the pointing gesture for directing attention to objects and the give-me gesture for making requests. Here we study, in adult human participants, the neurophysiological signatures of gestural-linguistic acts of communicating the pragmatic intentions of naming and requesting by simultaneously presenting written words and gestures. Already at ~150 ms, brain responses diverged between naming and request actions expressed by word-gesture combination, whereas the same gestures presented in isolation elicited their earliest neurophysiological dissociations significantly later (at ~210 ms). There was an early enhancement of request-evoked brain activity as compared with naming, which was due to sources in the frontocentral cortex, consistent with access to action knowledge in request understanding. In addition, an enhanced N400-like response indicated late semantic integration of gesture-language interaction. The present study demonstrates that word-gesture combinations used to express communicative pragmatic intentions speed up the brain correlates of comprehension processes – compared with gesture-only understanding – thereby calling into question current serial linguistic models viewing pragmatic function decoding at the end of a language comprehension cascade. Instead, information about the social-interactive role of communicative acts is processed instantaneously.