Syntactic gender processing in the human brain: a review and a model

Evaluating Speaker-Listener Cognitive Effort in Speech Communication Through Brain-to-Brain Synchrony: A Pilot Functional Near-Infrared Spectroscopy Investigation

PURPOSE

We explore a new approach to the study of cognitive effort involved in listening to speech by measuring the brain activity in a listener in relation to the brain activity in a speaker. We hypothesize that the strength of this brain-to-brain synchrony (coupling) reflects the magnitude of cognitive effort involved in verbal communication and includes both listening effort and speaking effort. We investigate whether interbrain synchrony is greater in native-to-native versus native-to-nonnative communication using functional near-infrared spectroscopy (fNIRS).

METHOD

Two speakers participated, a native speaker of American English and a native speaker of Korean who spoke English as a second language. Each speaker was fitted with the fNIRS cap and told short stories. The native English speaker provided the English narratives, and the Korean speaker provided both the nonnative (accented) English and Korean narratives. In separate sessions, fNIRS data were obtained from seven English monolingual participants ages 20-24 years who listened to each speaker's stories. After listening to each story in native and nonnative English, they retold the content, and their transcripts and audio recordings were analyzed for comprehension and discourse fluency, measured in the number of hesitations and articulation rate. No story retellings were obtained for narratives in Korean (an incomprehensible language for English listeners). Utilizing fNIRS technique termed sequential scanning, we quantified the brain-to-brain synchronization in each speaker-listener dyad.

RESULTS

For native-to-native dyads, multiple brain regions associated with various linguistic and executive functions were activated. There was a weaker coupling for native-to-nonnative dyads, and only the brain regions associated with higher order cognitive processes and functions were synchronized. All listeners understood the content of all stories, but they hesitated significantly more when retelling stories told in accented English. The nonnative speaker hesitated significantly more often than the native speaker and had a significantly slower articulation rate. There was no brain-to-brain coupling during listening to Korean, indicating a break in communication when listeners failed to comprehend the speaker.

CONCLUSIONS

We found that effortful speech processing decreased interbrain synchrony and delayed comprehension processes. The obtained brain-based and behavioral patterns are consistent with our proposal that cognitive effort in verbal communication pertains to both the listener and the speaker and that brain-to-brain synchrony can be an indicator of differences in their cumulative communicative effort.

SUPPLEMENTAL MATERIAL

https://doi.org/10.23641/asha.25452142.

Neural classification maps for distinct word combinations in Broca's area

Humans are equipped with the remarkable ability to comprehend an infinite number of utterances. Relations between grammatical categories restrict the way words combine into phrases and sentences. How the brain recognizes different word combinations remains largely unknown, although this is a necessary condition for combinatorial unboundedness in language. Here, we used functional magnetic resonance imaging and multivariate pattern analysis to explore whether distinct neural populations of a known language network hub-Broca's area-are specialized for recognizing distinct simple word combinations. The phrases consisted of a noun (flag) occurring either with a content word, an adjective (green flag), or with a function word, a determiner (that flag). The key result is that the distribution of neural populations classifying word combination in Broca's area seems sensitive to neuroanatomical subdivisions within this area, irrespective of task. The information patterns for adjective + noun were localized in its anterior part (BA45) whereas those for determiner + noun were localized in its posterior part (BA44). Our findings provide preliminary answers to the fundamental question of how lexical and grammatical category information interact during simple word combination, with the observation that Broca's area is sensitive to the recognition of categorical relationships during combinatory processing, based on different demands placed on syntactic and semantic information. This supports the hypothesis that the combinatorial power of language consists of some neural computation capturing phrasal differences when processing linguistic input.

The neural substrate of noun morphological inflection: A rapid event-related fMRI study in Italian

The present research investigated the neural correlates of nominal inflection and aimed at disclosing their possible link with the frequency distribution of noun inflectional features: grammatical gender, inflectional suffixes and inflectional classes. The properties of the Italian nominal system were exploited since it allows to explore exhaustively fine-grained phenomena in the inflectional processing. An event-related functional magnetic resonance imaging (fMRI) experiment was carried out where Italian masculine and feminine nouns were visually presented to 50 healthy participants in an overt inflectional task: the generation of the plural from the singular and vice versa. The grammatical gender and the citation form suffix of nouns were manipulated in a factorial design. Functional data showed that inflectional operations for nouns activate an extensive cortical network involving the left inferior and right superior frontal gyri, the left and right middle temporal gyri, the posterior cingulate cortex and the cerebellum. Activations were variably modulated by the distributional features of gender-dependent properties of nouns. Particularly, cortical activity increased during inflectional operations for small and/or scarcely consistent inflectional classes. These findings demonstrate the relevance of specific morphological (inflectional suffixes) and distributional features (size and consistency) shared by groups of words (inflectional classes) in a language, particularly when implementing cognitive operations required for language processing.

Coupled neural systems underlie the production and comprehension of naturalistic narrative speech

Neuroimaging studies of language have typically focused on either production or comprehension of single speech utterances such as syllables, words, or sentences. In this study we used a new approach to functional MRI acquisition and analysis to characterize the neural responses during production and comprehension of complex real-life speech. First, using a time-warp based intrasubject correlation method, we identified all areas that are reliably activated in the brains of speakers telling a 15-min-long narrative. Next, we identified areas that are reliably activated in the brains of listeners as they comprehended that same narrative. This allowed us to identify networks of brain regions specific to production and comprehension, as well as those that are shared between the two processes. The results indicate that production of a real-life narrative is not localized to the left hemisphere but recruits an extensive bilateral network, which overlaps extensively with the comprehension system. Moreover, by directly comparing the neural activity time courses during production and comprehension of the same narrative we were able to identify not only the spatial overlap of activity but also areas in which the neural activity is coupled across the speaker's and listener's brains during production and comprehension of the same narrative. We demonstrate widespread bilateral coupling between production- and comprehension-related processing within both linguistic and nonlinguistic areas, exposing the surprising extent of shared processes across the two systems.

Activation of syntax in lexical production in healthy speakers and in aphasia

Theories of spoken word production agree that semantic and phonological representations are activated in spoken word production. There is less agreement concerning the role of syntax. In this study we investigated noun syntax activation in English bare noun naming, using mass and count nouns. Fourteen healthy controls and 13 speakers with aphasia took part. Participants named mass and count nouns, and completed a related noun syntax judgement task. We analysed speakers' noun syntax knowledge when naming accurately, and when making errors in production. Healthy speakers' noun syntax judgement was accurate for words they named correctly, but this did not correlate with naming accuracy. Speakers with aphasia varied in their noun syntax judgement, and this also did not correlate with naming accuracy. Healthy speakers' syntax for semantic errors was less accurate, as was that for speakers with aphasia. For phonological errors half the participants with aphasia could access syntax, half could not, indicating two types of phonological error. Individual differences were found in no responses. Finally, we found no effect of frequency for any of the above. The lack of a relationship between syntax and naming accuracy suggests that syntax is available, but access is not obligatory. This finding supports theories incorporating non-obligatory syntactic processing, which is independent of phonological access. The semantic error data are best explained within such a theory where there is damage to phonological access and hence to independent syntax. For the aphasia group we identify two types of phonological error, one implicating syntax and phonology, and one implicating phonology only, again supporting independent access to these systems. Overall the data support a model within which syntax is independent of phonology, and activation of syntax operates flexibly dependent on task demands and integrity of other processing routines.

The neural correlates of agrammatism: Evidence from aphasic and healthy speakers performing an overt picture description task

Functional brain imaging studies have improved our knowledge of the neural localization of language functions and the functional reorganization after a lesion. However, the neural correlates of agrammatic symptoms in aphasia remain largely unknown. The present fMRI study examined the neural correlates of morpho-syntactic encoding and agrammatic errors in continuous language production by combining three approaches. First, the neural mechanisms underlying natural morpho-syntactic processing in a picture description task were analyzed in 15 healthy speakers. Second, agrammatic-like speech behavior was induced in the same group of healthy speakers to study the underlying functional processes by limiting the utterance length. In a third approach, five agrammatic participants performed the picture description task to gain insights in the neural correlates of agrammatism and the functional reorganization of language processing after stroke. In all approaches, utterances were analyzed for syntactic completeness, complexity, and morphology. Event-related data analysis was conducted by defining every clause-like unit (CLU) as an event with its onset-time and duration. Agrammatic and correct CLUs were contrasted. Due to the small sample size as well as heterogeneous lesion sizes and sites with lesion foci in the insula lobe, inferior frontal, superior temporal and inferior parietal areas the activation patterns in the agrammatic speakers were analyzed on a single subject level. In the group of healthy speakers, posterior temporal and inferior parietal areas were associated with greater morpho-syntactic demands in complete and complex CLUs. The intentional manipulation of morpho-syntactic structures and the omission of function words were associated with additional inferior frontal activation. Overall, the results revealed that the investigation of the neural correlates of agrammatic language production can be reasonably conducted with an overt language production paradigm.

Toward a neural basis of interactive alignment in conversation

The interactive-alignment account of dialogue proposes that interlocutors achieve conversational success by aligning their understanding of the situation under discussion. Such alignment occurs because they prime each other at different levels of representation (e.g., phonology, syntax, semantics), and this is possible because these representations are shared across production and comprehension. In this paper, we briefly review the behavioral evidence, and then consider how findings from cognitive neuroscience might lend support to this account, on the assumption that alignment of neural activity corresponds to alignment of mental states. We first review work supporting representational parity between production and comprehension, and suggest that neural activity associated with phonological, lexical, and syntactic aspects of production and comprehension are closely related. We next consider evidence for the neural bases of the activation and use of situation models during production and comprehension, and how these demonstrate the activation of non-linguistic conceptual representations associated with language use. We then review evidence for alignment of neural mechanisms that are specific to the act of communication. Finally, we suggest some avenues of further research that need to be explored to test crucial predictions of the interactive alignment account.

fMRI evidence for strategic decision-making during resolution of pronoun reference

Pronouns are extraordinarily common in daily language yet little is known about the neural mechanisms that support decisions about pronoun reference. We propose a large-scale neural network for resolving pronoun reference that consists of two components. First, a core language network in peri-Sylvian cortex supports syntactic and semantic resources for interpreting pronoun meaning in sentences. Second, a frontal-parietal network that supports strategic decision-making is recruited to support probabilistic and risk-related components of resolving a pronoun's referent. In an fMRI study of healthy young adults, we observed activation of left inferior frontal and superior temporal cortex, consistent with a language network. We also observed activation of brain regions not associated with traditional language areas. By manipulating the context of the pronoun, we were able to demonstrate recruitment of dorsolateral prefrontal cortex during probabilistic evaluation of a pronoun's reference, and orbital frontal activation when a pronoun must adopt a risky referent. Together, these findings are consistent with a two-component model for resolving a pronoun's reference that includes neuroanatomic regions supporting core linguistic and decision-making mechanisms.

Shared language: overlap and segregation of the neuronal infrastructure for speaking and listening revealed by functional MRI

Whether the brain's speech-production system is also involved in speech comprehension is a topic of much debate. Research has focused on whether motor areas are involved in listening, but overlap between speaking and listening might occur not only at primary sensory and motor levels, but also at linguistic levels (where semantic, lexical, and syntactic processes occur). Using functional MRI adaptation during speech comprehension and production, we found that the brain areas involved in semantic, lexical, and syntactic processing are mostly the same for speaking and for listening. Effects of primary processing load (indicative of sensory and motor processes) overlapped in auditory cortex and left inferior frontal cortex, but not in motor cortex, where processing load affected activity only in speaking. These results indicate that the linguistic parts of the language system are used for both speaking and listening, but that the motor system does not seem to provide a crucial contribution to listening.

Left cytoarchitectonic BA 44 processes syntactic gender violations in determiner phrases

Recent neuroimaging studies make contradictory predictions about the involvement of left Brodmann's area (BA) 44 in processing local syntactic violations in determiner phrases (DPs). Some studies suggest a role for BA 44 in detecting local syntactic violations, whereas others attribute this function to the left premotor cortex. Therefore, the present event-related functional magnetic resonance imaging (fMRI) study investigated whether left-cytoarchitectonic BA 44 was activated when German DPs involving syntactic gender violations were compared with correct DPs (correct: 'der Baum'-the[masculine] tree[masculine]; violated: 'das Baum'--the[neuter] tree[masculine]). Grammaticality judgements were made for both visual and auditory DPs to be able to generalize the results across modalities. Grammaticality judgements involved, among others, left BA 44 and left BA 6 in the premotor cortex for visual and auditory stimuli. Most importantly, activation in left BA 44 was consistently higher for violated than for correct DPs. This finding was behaviourally corroborated by longer reaction times for violated versus correct DPs. Additional brain regions, showing the same effect, included left premotor cortex, supplementary motor area, right middle and superior frontal cortex, and left cerebellum. Based on earlier findings from the literature, the results indicate the involvement of left BA 44 in processing local syntactic violations when these include morphological features, whereas left premotor cortex seems crucial for the detection of local word category violations.

Double dissociation between syntactic gender and picture naming processing: a brain stimulation mapping study

Neural foundations of syntactic gender processing remain poorly understood. We used electrostimulation mapping in nine right-handed awake patients during surgery for a glioma within the left hemisphere, to study whether the cortico-subcortical structures involved in naming versus syntactic gender processing are common or distinct. In French, the article determines the grammatical gender. Thus, the patient was asked to perform a picture naming task and to give the appropriate article for each picture, with and without stimulation. Cortical stimulation elicited reproducible syntactic gender disturbances in six patients, in the inferior frontal gyrus (three cases), and in the posterior middle temporal gyrus (three cases). Interestingly, no naming disorders were generated during stimulation of the syntactic sites, while cortical areas inducing naming disturbances never elicited grammatical gender errors when stimulated. Moreover, at the subcortical level, stimulation of the white matter lateral to the caudate nucleus induced gender errors in three patients, with no naming disorders. Using cortico-subcortical electrical mapping in awake patients, we demonstrate for the first time (1) a double dissociation between syntactic gender and naming processing, supporting independent network model rather than serial theory, (2) the involvement of the left inferior frontal gyrus, especially the pars triangularis, and the posterior left middle temporal gyrus in grammatical gender processing, (3) the existence of white matter pathways, likely a sub-part of the left superior longitudinal fasciculus, underlying a large-scale distributed cortico-subcortical circuit which might selectively sub-serve syntactic gender processing, even if interconnected with parallel sub-networks involved in naming (semantic and phonological) processing.