Coupled neural systems underlie the production and comprehension of naturalistic narrative speech

The Language Network Reliably "Tracks" Naturalistic Meaningful Nonverbal Stimuli

The language network, comprised of brain regions in the left frontal and temporal cortex, responds robustly and reliably during language comprehension but shows little or no response during many nonlinguistic cognitive tasks (e.g., Fedorenko & Blank, 2020). However, one domain whose relationship with language remains debated is semantics-our conceptual knowledge of the world. Given that the language network responds strongly to meaningful linguistic stimuli, could some of this response be driven by the presence of rich conceptual representations encoded in linguistic inputs? In this study, we used a naturalistic cognition paradigm to test whether the cognitive and neural resources that are responsible for language processing are also recruited for processing semantically rich nonverbal stimuli. To do so, we measured BOLD responses to a set of ∼5-minute-long video and audio clips that consisted of meaningful event sequences but did not contain any linguistic content. We then used the intersubject correlation (ISC) approach (Hasson et al., 2004) to examine the extent to which the language network "tracks" these stimuli, that is, exhibits stimulus-related variation. Across all the regions of the language network, meaningful nonverbal stimuli elicited reliable ISCs. These ISCs were higher than the ISCs elicited by semantically impoverished nonverbal stimuli (e.g., a music clip), but substantially lower than the ISCs elicited by linguistic stimuli. Our results complement earlier findings from controlled experiments (e.g., Ivanova et al., 2021) in providing further evidence that the language network shows some sensitivity to semantic content in nonverbal stimuli.

Predictive language processing: integrating comprehension and production, and what atypical populations can tell us

Predictive processing, a crucial aspect of human cognition, is also relevant for language comprehension. In everyday situations, we exploit various sources of information to anticipate and therefore facilitate processing of upcoming linguistic input. In the literature, there are a variety of models that aim at accounting for such ability. One group of models propose a strict relationship between prediction and language production mechanisms. In this review, we first introduce very briefly the concept of predictive processing during language comprehension. Secondly, we focus on models that attribute a prominent role to language production and sensorimotor processing in language prediction ("prediction-by-production" models). Contextually, we provide a summary of studies that investigated the role of speech production and auditory perception on language comprehension/prediction tasks in healthy, typical participants. Then, we provide an overview of the limited existing literature on specific atypical/clinical populations that may represent suitable testing ground for such models-i.e., populations with impaired speech production and auditory perception mechanisms. Ultimately, we suggest a more widely and in-depth testing of prediction-by-production accounts, and the involvement of atypical populations both for model testing and as targets for possible novel speech/language treatment approaches.

Evaluating Speaker-Listener Cognitive Effort in Speech Communication Through Brain-to-Brain Synchrony: A Pilot Functional Near-Infrared Spectroscopy Investigation

PURPOSE

We explore a new approach to the study of cognitive effort involved in listening to speech by measuring the brain activity in a listener in relation to the brain activity in a speaker. We hypothesize that the strength of this brain-to-brain synchrony (coupling) reflects the magnitude of cognitive effort involved in verbal communication and includes both listening effort and speaking effort. We investigate whether interbrain synchrony is greater in native-to-native versus native-to-nonnative communication using functional near-infrared spectroscopy (fNIRS).

METHOD

Two speakers participated, a native speaker of American English and a native speaker of Korean who spoke English as a second language. Each speaker was fitted with the fNIRS cap and told short stories. The native English speaker provided the English narratives, and the Korean speaker provided both the nonnative (accented) English and Korean narratives. In separate sessions, fNIRS data were obtained from seven English monolingual participants ages 20-24 years who listened to each speaker's stories. After listening to each story in native and nonnative English, they retold the content, and their transcripts and audio recordings were analyzed for comprehension and discourse fluency, measured in the number of hesitations and articulation rate. No story retellings were obtained for narratives in Korean (an incomprehensible language for English listeners). Utilizing fNIRS technique termed sequential scanning, we quantified the brain-to-brain synchronization in each speaker-listener dyad.

RESULTS

For native-to-native dyads, multiple brain regions associated with various linguistic and executive functions were activated. There was a weaker coupling for native-to-nonnative dyads, and only the brain regions associated with higher order cognitive processes and functions were synchronized. All listeners understood the content of all stories, but they hesitated significantly more when retelling stories told in accented English. The nonnative speaker hesitated significantly more often than the native speaker and had a significantly slower articulation rate. There was no brain-to-brain coupling during listening to Korean, indicating a break in communication when listeners failed to comprehend the speaker.

CONCLUSIONS

We found that effortful speech processing decreased interbrain synchrony and delayed comprehension processes. The obtained brain-based and behavioral patterns are consistent with our proposal that cognitive effort in verbal communication pertains to both the listener and the speaker and that brain-to-brain synchrony can be an indicator of differences in their cumulative communicative effort.

SUPPLEMENTAL MATERIAL

https://doi.org/10.23641/asha.25452142.

The language network as a natural kind within the broader landscape of the human brain

Language behaviour is complex, but neuroscientific evidence disentangles it into distinct components supported by dedicated brain areas or networks. In this Review, we describe the 'core' language network, which includes left-hemisphere frontal and temporal areas, and show that it is strongly interconnected, independent of input and output modalities, causally important for language and language-selective. We discuss evidence that this language network plausibly stores language knowledge and supports core linguistic computations related to accessing words and constructions from memory and combining them to interpret (decode) or generate (encode) linguistic messages. We emphasize that the language network works closely with, but is distinct from, both lower-level - perceptual and motor - mechanisms and higher-level systems of knowledge and reasoning. The perceptual and motor mechanisms process linguistic signals, but, in contrast to the language network, are sensitive only to these signals' surface properties, not their meanings; the systems of knowledge and reasoning (such as the system that supports social reasoning) are sometimes engaged during language use but are not language-selective. This Review lays a foundation both for in-depth investigations of these different components of the language processing pipeline and for probing inter-component interactions.

Natural language processing models reveal neural dynamics of human conversation

Through conversation, humans relay complex information through the alternation of speech production and comprehension. The neural mechanisms that underlie these complementary processes or through which information is precisely conveyed by language, however, remain poorly understood. Here, we used pretrained deep learning natural language processing models in combination with intracranial neuronal recordings to discover neural signals that reliably reflect speech production, comprehension, and their transitions during natural conversation between individuals. Our findings indicate that neural activities that encoded linguistic information were broadly distributed throughout frontotemporal areas across multiple frequency bands. We also find that these activities were specific to the words and sentences being conveyed and that they were dependent on the word's specific context and order. Finally, we demonstrate that these neural patterns partially overlapped during language production and comprehension and that listener-speaker transitions were associated with specific, time-aligned changes in neural activity. Collectively, our findings reveal a dynamical organization of neural activities that subserve language production and comprehension during natural conversation and harness the use of deep learning models in understanding the neural mechanisms underlying human language.

How does the human brain process noisy speech in real life? Insights from the second-person neuroscience perspective

Comprehending speech with the existence of background noise is of great importance for human life. In the past decades, a large number of psychological, cognitive and neuroscientific research has explored the neurocognitive mechanisms of speech-in-noise comprehension. However, as limited by the low ecological validity of the speech stimuli and the experimental paradigm, as well as the inadequate attention on the high-order linguistic and extralinguistic processes, there remains much unknown about how the brain processes noisy speech in real-life scenarios. A recently emerging approach, i.e., the second-person neuroscience approach, provides a novel conceptual framework. It measures both of the speaker's and the listener's neural activities, and estimates the speaker-listener neural coupling with regarding of the speaker's production-related neural activity as a standardized reference. The second-person approach not only promotes the use of naturalistic speech but also allows for free communication between speaker and listener as in a close-to-life context. In this review, we first briefly review the previous discoveries about how the brain processes speech in noise; then, we introduce the principles and advantages of the second-person neuroscience approach and discuss its implications to unravel the linguistic and extralinguistic processes during speech-in-noise comprehension; finally, we conclude by proposing some critical issues and calls for more research interests in the second-person approach, which would further extend the present knowledge about how people comprehend speech in noise.

Diverging neural dynamics for syntactic structure building in naturalistic speaking and listening

The neural correlates of sentence production are typically studied using task paradigms that differ considerably from the experience of speaking outside of an experimental setting. In this fMRI study, we aimed to gain a better understanding of syntactic processing in spontaneous production versus naturalistic comprehension in three regions of interest (BA44, BA45, and left posterior middle temporal gyrus). A group of participants (n = 16) was asked to speak about the events of an episode of a TV series in the scanner. Another group of participants (n = 36) listened to the spoken recall of a participant from the first group. To model syntactic processing, we extracted word-by-word metrics of phrase-structure building with a top-down and a bottom-up parser that make different hypotheses about the timing of structure building. While the top-down parser anticipates syntactic structure, sometimes before it is obvious to the listener, the bottom-up parser builds syntactic structure in an integratory way after all of the evidence has been presented. In comprehension, neural activity was found to be better modeled by the bottom-up parser, while in production, it was better modeled by the top-down parser. We additionally modeled structure building in production with two strategies that were developed here to make different predictions about the incrementality of structure building during speaking. We found evidence for highly incremental and anticipatory structure building in production, which was confirmed by a converging analysis of the pausing patterns in speech. Overall, this study shows the feasibility of studying the neural dynamics of spontaneous language production.

Comparing high definition transcranial direct current stimulation to left temporoparietal junction and left inferior frontal gyrus for logopenic primary progressive aphasia: A single-case study

Logopenic variant primary progressive aphasia (lvPPA) is characterized by word-finding deficits and phonologic errors in fluent speech. Transcranial direct current stimulation (tDCS) targeting either left temporoparietal junction (TPJ) or left inferior frontal gyrus (IFG) show evidence of improving language function in lvPPA. The present case study evaluated the effects of two separate rounds of high definition tDCS (HD-tDCS) (4 mA; 30 sessions) on language and functional neuroimaging in a 57-year-old woman with lvPPA. Stimulation was centred on two different regions across rounds: (1) left TPJ, and (2) left (IFG). Results showed an improved proportion of content to floorholder words during a naturalistic speech task through both rounds as well as change in confrontation naming after TPJ (improvement) and IFG (worsened) stimulation. fMRI connectivity during task showed left lateralized positive correlations following round 1 and anti-correlations with components of the default mode network following round 2. Resting state segregation of a language-associated functional network increased following both rounds, and task-based segregation of the same network increased following IFG stimulation. These results suggest that stimulation to both regions using HD-tDCS may improve language function in lvPPA, while simultaneously eliciting widespread changes beyond the targeted area in neuronal activity and functional connectivity.

Common and distinct patterns underlying different linguistic tasks: multivariate disconnectome symptom mapping in poststroke patients

Numerous studies have been devoted to neural mechanisms of a variety of linguistic tasks (e.g. speech comprehension and production). To date, however, whether and how the neural patterns underlying different linguistic tasks are similar or differ remains elusive. In this study, we compared the neural patterns underlying 3 linguistic tasks mainly concerning speech comprehension and production. To address this, multivariate regression approaches with lesion/disconnection symptom mapping were applied to data from 216 stroke patients with damage to the left hemisphere. The results showed that lesion/disconnection patterns could predict both poststroke scores of speech comprehension and production tasks; these patterns exhibited shared regions on the temporal pole of the left hemisphere as well as unique regions contributing to the prediction for each domain. Lower scores in speech comprehension tasks were associated with lesions/abnormalities in the superior temporal gyrus and middle temporal gyrus, while lower scores in speech production tasks were associated with lesions/abnormalities in the left inferior parietal lobe and frontal lobe. These results suggested an important role of the ventral and dorsal stream pathways in speech comprehension and production (i.e. supporting the dual stream model) and highlighted the applicability of the novel multivariate disconnectome-based symptom mapping in cognitive neuroscience research.

How a speaker herds the audience: Multi-brain neural convergence over time during naturalistic storytelling

Storytelling-an ancient way for humans to share individual experiences with others-has been found to induce neural synchronization among listeners. In our exploration of the dynamic fluctuations in listener-listener (LL) coupling throughout stories, we uncover a significant correlation between LL and lag-speaker-listener (lag-SL) couplings over time. Using the analogy of neural pattern (dis)similarity as distances between participants, we term this phenomenon the "herding effect": like a shepherd guiding a group of sheep, the more closely listeners follow the speaker's prior brain activity patterns (higher lag-SL similarity), the more tightly they cluster together (higher LL similarity). This herding effect is particularly pronounced in brain regions where neural synchronization among listeners tracks with behavioral ratings of narrative engagement, highlighting the mediating role of narrative content in the observed multi-brain neural coupling dynamics. By integrating LL and SL neural couplings, this study illustrates how unfolding stories shape a dynamic multi-brain functional network and how the configuration of this network may be associated with moment-by-moment efficacy of communication.

Interpersonal alignment of neural evidence accumulation to social exchange of confidence

Private, subjective beliefs about uncertainty have been found to have idiosyncratic computational and neural substrates yet, humans share such beliefs seamlessly and cooperate successfully. Bringing together decision making under uncertainty and interpersonal alignment in communication, in a discovery plus pre-registered replication design, we examined the neuro-computational basis of the relationship between privately held and socially shared uncertainty. Examining confidence-speed-accuracy trade-off in uncertainty-ridden perceptual decisions under social vs isolated context, we found that shared (i.e. reported confidence) and subjective (inferred from pupillometry) uncertainty dynamically followed social information. An attractor neural network model incorporating social information as top-down additive input captured the observed behavior and demonstrated the emergence of social alignment in virtual dyadic simulations. Electroencephalography showed that social exchange of confidence modulated the neural signature of perceptual evidence accumulation in the central parietal cortex. Our findings offer a neural population model for interpersonal alignment of shared beliefs.

Cognitive Reappraisal and Expressive Suppression Evoke Distinct Neural Connections during Interpersonal Emotion Regulation

Interpersonal emotion regulation is the dynamic process where the regulator aims to change the target's emotional state, which is presumed to engage three neural systems: cognitive control (i.e., dorsal and ventral lateral PFC, etc.), empathy/social cognition (i.e., dorsal premotor regions, temporal-parietal junction, etc.), and affective response (i.e., insula, amygdala, etc.). This study aimed to identify the underlying neural correlate (especially the interpersonal one), of interpersonal emotion regulation based on two typical strategies (cognitive appraisal, expressive suppression). Thirty-four female dyads (friends) were randomly assigned into two strategy groups, with one assigned as the target and the other as the regulator to downregulate the target's negative emotions using two strategies. A functional near-infrared spectroscopy system was used to simultaneously measure participants' neural activity. Results showed that these two strategies could successfully downregulate the targets' negative emotions. Both strategies evoked intrapersonal and interpersonal neural couplings between the cognitive control, social cognition, and mirror neuron systems (e.g., PFC, temporal-parietal junction, premotor cortex, etc.), whereas cognitive reappraisal (vs expressive suppression) evoked a broader pattern. Further, cognitive reappraisal involved increased interpersonal brain synchronization between the prefrontal and temporal areas at the sharing stage, whereas expressive suppression evoked increased interpersonal brain synchronization associated with the PFC at the regulation stage. These findings indicate that intrapersonal and interpersonal neural couplings associated with regions within the abovementioned systems, possibly involving mental processes, such as cognitive control, mentalizing, and observing, underlie interpersonal emotion regulation based on cognitive reappraisal or expressive suppression.SIGNIFICANCE STATEMENT As significant as intrapersonal emotion regulation, interpersonal emotion regulation subserves parent-child, couple, and leader-follower relationships. Despite enormous growth in research on intrapersonal emotion regulation, the field lacks insight into the neural correlates underpinning interpersonal emotion regulation. This study aimed to probe the underlying neural correlates of interpersonal emotion regulation using a multibrain neuroimaging (i.e., hyperscanning) based on functional near-infrared spectroscopy. Results showed that both cognitive reappraisal and expressive suppression strategies successfully downregulated the target's negative emotions. More importantly, they evoked intrapersonal and interpersonal neural couplings associated with regions within the cognitive control, social cognition, and mirror neuron systems, possibly involving mental processes, such as cognitive control, mentalizing, and observing. These findings deepen our understanding of the neural correlates underpinning interpersonal emotion regulation.

When the Brain Cares: Personal interests amplify engagement of language, self-reference, and reward regions in the brains of children with and without autism

Human language is shaped by individual experiences and interests. However, to study language in the brain, researchers use generic stimuli, avoiding the variable personal interests that typically animate language. Thus, it is unknown how personal interests affect language function in the brain. We conducted personalized functional magnetic resonance imaging (fMRI) in 20 typically-developing children as they listened to personalized narratives about their specific interest and non-personalized, generic narratives. Personally-interesting narratives amplified engagement of language regions, producing more consistent activation patterns across individuals - even though each narrative was unique - than the generic narratives. The personalized narratives also engaged self-reference and reward areas of the brain associated with motivation. Amplification of brain responses to personally-interesting narratives was also observed in 15 autistic children, a condition characterized by both intense specific interests and difficulties with communication. Here we show that personal interests significantly affect language processing in the human brain.

Prediction during language comprehension: what is next?

Prediction is often regarded as an integral aspect of incremental language comprehension, but little is known about the cognitive architectures and mechanisms that support it. We review studies showing that listeners and readers use all manner of contextual information to generate multifaceted predictions about upcoming input. The nature of these predictions may vary between individuals owing to differences in language experience, among other factors. We then turn to unresolved questions which may guide the search for the underlying mechanisms. (i) Is prediction essential to language processing or an optional strategy? (ii) Are predictions generated from within the language system or by domain-general processes? (iii) What is the relationship between prediction and memory? (iv) Does prediction in comprehension require simulation via the production system? We discuss promising directions for making progress in answering these questions and for developing a mechanistic understanding of prediction in language.

Grounded Cognition Entails Linguistic Relativity: Response to Commentators

In this paper, I respond to eight commentaries on my target article called "Grounded cognition entails linguistic relativity: A neglected implication of a major semantic theory." The commentaries span a broad range of disciplines and perspectives. I have organized my response around the following topics: (1) an introductory synopsis of my main argument; (2) grounded versus amodal theories of concepts; (3) language-specific versus language-independent concepts; (4) language, culture, and cognition; (5) language itself as a source of conceptual grounding; (6) abstract concepts, linguistic relativity, and contextual and individual variability; (7) word meanings as language-specific predictions; and (8) some final remarks about the importance of cross-linguistic diversity.

Neural synchrony underlies the positive effect of shared reading on children's language ability

Although it is well recognized that parent-child shared reading produces positive effects on children's language ability, the underlying neurocognitive mechanisms are not well understood. Here, we addressed this issue by measuring brain activities from mother-child dyads simultaneously during a shared book reading task using functional near infrared spectroscopy hyperscanning. The behavioral results showed that the long-term experience of shared reading significantly predicted children's language ability. Interestingly, the prediction was moderated by children's age: for older children over 30 months, the more the shared reading experience, the better the language performance; for younger children below 30 months, however, no significant relationship was observed. The brain results showed significant interpersonal neural synchronization between mothers and children at the superior temporal cortex, which was closely associated with older children's language ability through the mediation of long-term experience of shared reading. Finally, the results showed that the instantaneous quality of shared reading contributed to children's language ability through enhancing interpersonal neural synchronization and increasing long-term experience. Based on these findings, we tentatively proposed a theoretical model for the relationship among interpersonal neural synchronization, shared reading and children's language ability. These findings will facilitate our understanding on the role of shared reading in children's language development.

The language network is not engaged in object categorization

The relationship between language and thought is the subject of long-standing debate. One claim states that language facilitates categorization of objects based on a certain feature (e.g. color) through the use of category labels that reduce interference from other, irrelevant features. Therefore, language impairment is expected to affect categorization of items grouped by a single feature (low-dimensional categories, e.g. "Yellow Things") more than categorization of items that share many features (high-dimensional categories, e.g. "Animals"). To test this account, we conducted two behavioral studies with individuals with aphasia and an fMRI experiment with healthy adults. The aphasia studies showed that selective low-dimensional categorization impairment was present in some, but not all, individuals with severe anomia and was not characteristic of aphasia in general. fMRI results revealed little activity in language-responsive brain regions during both low- and high-dimensional categorization; instead, categorization recruited the domain-general multiple-demand network (involved in wide-ranging cognitive tasks). Combined, results demonstrate that the language system is not implicated in object categorization. Instead, selective low-dimensional categorization impairment might be caused by damage to brain regions responsible for cognitive control. Our work adds to the growing evidence of the dissociation between the language system and many cognitive tasks in adults.

Disruption to left inferior frontal cortex modulates semantic prediction effects in reading and subsequent memory: Evidence from simultaneous TMS-EEG

Readers use prior context to predict features of upcoming words. When predictions are accurate, this increases the efficiency of comprehension. However, little is known about the fate of predictable and unpredictable words in memory or the neural systems governing these processes. Several theories suggest that the speech production system, including the left inferior frontal cortex (LIFC), is recruited for prediction but evidence that LIFC plays a causal role is lacking. We first examined the effects of predictability on memory and then tested the role of posterior LIFC using transcranial magnetic stimulation (TMS). In Experiment 1, participants read category cues, followed by a predictable, unpredictable, or incongruent target word for later recall. We observed a predictability benefit to memory, with predictable words remembered better than unpredictable words. In Experiment 2, participants performed the same task with electroencephalography (EEG) while undergoing event-related TMS over posterior LIFC using a protocol known to disrupt speech production, or over the right hemisphere homologue as an active control site. Under control stimulation, predictable words were better recalled than unpredictable words, replicating Experiment 1. This predictability benefit to memory was eliminated under LIFC stimulation. Moreover, while an a priori ROI-based analysis did not yield evidence for a reduction in the N400 predictability effect, mass-univariate analyses did suggest that the N400 predictability effect was reduced in spatial and temporal extent under LIFC stimulation. Collectively, these results provide causal evidence that the LIFC is recruited for prediction during silent reading, consistent with prediction-through-production accounts.

20 years of the default mode network: A review and synthesis

The discovery of the default mode network (DMN) has revolutionized our understanding of the workings of the human brain. Here, I review developments that led to the discovery of the DMN, offer a personal reflection, and consider how our ideas of DMN function have evolved over the past two decades. I summarize literature examining the role of the DMN in self-reference, social cognition, episodic and autobiographical memory, language and semantic memory, and mind wandering. I identify unifying themes and propose new perspectives on the DMN's role in human cognition. I argue that the DMN integrates and broadcasts memory, language, and semantic representations to create a coherent "internal narrative" reflecting our individual experiences. This narrative is central to the construction of a sense of self, shapes how we perceive ourselves and interact with others, may have ontogenetic origins in self-directed speech during childhood, and forms a vital component of human consciousness.

Shared intentionality modulates interpersonal neural synchronization at the establishment of communication system

Whether and how shared intentionality (SI) influences the establishment of a novel interpersonal communication system is poorly understood. To investigate this issue, we designed a coordinating symbolic communication game (CSCG) and applied behavioral, functional near-infrared spectroscopy (fNIRS)-based hyperscanning, and hyper-transcranial alternating current stimulation (hyper-tACS) methods. Here we show that SI is a strong contributor to communicative accuracy. Moreover, SI, communicative accuracy, and interpersonal neural synchronization (INS) in the right superior temporal gyrus (rSTG) are higher when dyads successfully establish a novel communication system. Furthermore, the SI influences communicative accuracy by increasing INS. Additionally, using time series and long short-term memory neural network analyses, we find that the INS can predict communicative accuracy at the early formation stage of the communication system. Importantly, the INS partially mediates the relationship between the SI and the communicative accuracy only at the formation stage of the communication system. In contrast, when the communication system is established, SI and INS no longer contribute to communicative accuracy. Finally, the hyper-tACS experiment confirms that INS has a causal effect on communicative accuracy. These findings suggest a behavioral and neural mechanism, subserved by the SI and INS, that underlies the establishment of a novel interpersonal communication system.

Functional connectivity explains how neuronavigated TMS of posterior temporal subregions differentially affect language processing

BACKGROUND

"Wernicke's area" is most often used to describe the posterior superior temporal gyrus (STG) and refers to a region traditionally thought to support language comprehension. However, the posterior STG additionally plays a critical role in language production. The purpose of the current study was to determine to what extent regions within the posterior STG are selectively recruited during language production.

METHODS

23 healthy right-handed participants completed an auditory fMRI localizer task, resting-state fMRI and underwent neuronavigated TMS language mapping. We applied repetitive TMS bursts during a picture naming paradigm to probe speech disruptions of different categories (anomia, speech arrest, semantic paraphasia and phonological paraphasia). We combined an in-house built high precision stimulation software suite with E-field modeling to map the naming errors to cortical regions and revealed a dissociation of language functions within the temporal gyrus. Resting state fMRI was used to explain how E-field peaks of different categories differentially affected language production.

RESULTS

Peaks for phonological and semantic errors were found in the STG while those for anomia and speech arrest were located in the MTG. Seed-based connectivity analysis revealed a local connectivity pattern for phonological and semantic errors, while anomia and speech arrest seeds resulted in a larger network between IFG and posterior MTG.

CONCLUSIONS

Our study provides important insights into the functional neuroanatomy of language production and might help to increase the current understanding of specific language production difficulties on a causal level.

A shared linguistic space for transmitting our thoughts from brain to brain in natural conversations

Effective communication hinges on a mutual understanding of word meaning in different contexts. The embedding space learned by large language models can serve as an explicit model of the shared, context-rich meaning space humans use to communicate their thoughts. We recorded brain activity using electrocorticography during spontaneous, face-to-face conversations in five pairs of epilepsy patients. We demonstrate that the linguistic embedding space can capture the linguistic content of word-by-word neural alignment between speaker and listener. Linguistic content emerged in the speaker's brain before word articulation, and the same linguistic content rapidly reemerged in the listener's brain after word articulation. These findings establish a computational framework to study how human brains transmit their thoughts to one another in real-world contexts.

Divergent interpersonal neural synchronization patterns in the first, second language and interlingual communication

An accumulating number of studies have highlighted the importance of interpersonal neural synchronization (INS) between interlocutors in successful verbal communications. The opportunities for communication across different language contexts are rapidly expanding, thanks to the frequent interactions among people all over the world. However, whether the INS changes in different language contexts and how language choice affects the INS remain scarcely explored. The study recruited twenty pairs of participants to communicate in the first language (L1), second language (L2) and interlingual contexts. Using functional near-infrared spectroscopy (fNIRS), we examined the neural activities of interlocutors and analyzed their wavelet transform coherence to assess the INS of dyads. Results showed that as compared to the resting state, stronger INS was observed at the left inferior temporal gyrus, middle temporal gyrus, pre-motor and supplementary motor cortex, dorsolateral prefrontal cortex, and inferior frontal gyrus in L1; at the left middle temporal gyrus, superior temporal gyrus, and inferior frontal gyrus in L2; at the left inferior temporal gyrus and inferior frontal gyrus in interlingual context. Additionally, INS at the left inferior frontal gyrus was significantly stronger in L2 than in L1. These findings reveal the differences of the INS in different language contexts and confirm the importance of language choice for the INS changes.

Semantic reconstruction of continuous language from non-invasive brain recordings

A brain-computer interface that decodes continuous language from non-invasive recordings would have many scientific and practical applications. Currently, however, non-invasive language decoders can only identify stimuli from among a small set of words or phrases. Here we introduce a non-invasive decoder that reconstructs continuous language from cortical semantic representations recorded using functional magnetic resonance imaging (fMRI). Given novel brain recordings, this decoder generates intelligible word sequences that recover the meaning of perceived speech, imagined speech and even silent videos, demonstrating that a single decoder can be applied to a range of tasks. We tested the decoder across cortex and found that continuous language can be separately decoded from multiple regions. As brain-computer interfaces should respect mental privacy, we tested whether successful decoding requires subject cooperation and found that subject cooperation is required both to train and to apply the decoder. Our findings demonstrate the viability of non-invasive language brain-computer interfaces.

Neural signatures of second language proficiency in narrative processing

Making sense of speech in a second language relies on multiple abilities. Differences in brain activity related to proficiency in language tasks have often been attributed to processing demands. However, during naturalistic narrative comprehension, listeners at different proficiency levels may form different representations of the same speech. We hypothesized that the intersubject synchronization of these representations could be used to measure second-language proficiency. Using a searchlight-shared response model, we found highly proficient participants showed synchronization in regions similar to those of native speakers, including in the default mode network and the lateral prefrontal cortex. In contrast, participants with low proficiency showed more synchronization in auditory cortex and word-level semantic processing areas in the temporal lobe. Moderate proficiency showed the greatest neural diversity, suggesting lower consistency in the source of this partial proficiency. Based on these synchronization differences, we were able to classify the proficiency level or predict behavioral performance on an independent English test in held-out participants, suggesting the identified neural systems represented proficiency-sensitive information that was generalizable to other individuals. These findings suggest higher second-language proficiency leads to more native-like neural processing of naturalistic language, including in systems beyond the cognitive control network or the core language network.

Beyond speaking: neurocognitive perspectives on language production in social interaction

The human faculty to speak has evolved, so has been argued, for communicating with others and for engaging in social interactions. Hence the human cognitive system should be equipped to address the demands that social interaction places on the language production system. These demands include the need to coordinate speaking with listening, the need to integrate own (verbal) actions with the interlocutor's actions, and the need to adapt language flexibly to the interlocutor and the social context. In order to meet these demands, core processes of language production are supported by cognitive processes that enable interpersonal coordination and social cognition. To fully understand the cognitive architecture and its neural implementation enabling humans to speak in social interaction, our understanding of how humans produce language needs to be connected to our understanding of how humans gain insights into other people's mental states and coordinate in social interaction. This article reviews theories and neurocognitive experiments that make this connection and can contribute to advancing our understanding of speaking in social interaction. This article is part of a discussion meeting issue 'Face2face: advancing the science of social interaction'.

Precision fMRI reveals that the language-selective network supports both phrase-structure building and lexical access during language production

A fronto-temporal brain network has long been implicated in language comprehension. However, this network's role in language production remains debated. In particular, it remains unclear whether all or only some language regions contribute to production, and which aspects of production these regions support. Across 3 functional magnetic resonance imaging experiments that rely on robust individual-subject analyses, we characterize the language network's response to high-level production demands. We report 3 novel results. First, sentence production, spoken or typed, elicits a strong response throughout the language network. Second, the language network responds to both phrase-structure building and lexical access demands, although the response to phrase-structure building is stronger and more spatially extensive, present in every language region. Finally, contra some proposals, we find no evidence of brain regions-within or outside the language network-that selectively support phrase-structure building in production relative to comprehension. Instead, all language regions respond more strongly during production than comprehension, suggesting that production incurs a greater cost for the language network. Together, these results align with the idea that language comprehension and production draw on the same knowledge representations, which are stored in a distributed manner within the language-selective network and are used to both interpret and generate linguistic utterances.

Revealing the neurobiology underlying interpersonal neural synchronization with multimodal data fusion

Humans synchronize with one another to foster successful interactions. Here, we use a multimodal data fusion approach with the aim of elucidating the neurobiological mechanisms by which interpersonal neural synchronization (INS) occurs. Our meta-analysis of 22 functional magnetic resonance imaging and 69 near-infrared spectroscopy hyperscanning experiments (740 and 3721 subjects) revealed robust brain regional correlates of INS in the right temporoparietal junction and left ventral prefrontal cortex. Integrating this meta-analytic information with public databases, biobehavioral and brain-functional association analyses suggested that INS involves sensory-integrative hubs with functional connections to mentalizing and attention networks. On the molecular and genetic levels, we found INS to be associated with GABAergic neurotransmission and layer IV/V neuronal circuits, protracted developmental gene expression patterns, and disorders of neurodevelopment. Although limited by the indirect nature of phenotypic-molecular association analyses, our findings generate new testable hypotheses on the neurobiological basis of INS.

The hyper-brain neural couplings distinguishing high-creative group dynamics: an fNIRS hyperscanning study

This hyperscanning study aimed to identify a neural coupling profile that distinguishes high-creative group dynamics through functional near infrared spectroscopy. A total of 123 dyads completed one creativity task (alternative uses task, AUT) and contrast task (objective characteristics task). A K-means clustering analysis on AUT performance grouped 31/29 dyads into high/low-creative group, respectively. In comparison with the low-creative group, the high-creative group showed: (i) higher collective flexibility and delayed perspective-taking behaviors, but lower immediate perspective-taking behaviors; (ii) enhanced interpersonal brain synchronization (IBS) between the left inferior frontal gyrus (lIFG) and right motor cortex, and nodal Eloc at the right superior temporal gyrus (rSTG); (iii) declined intrapersonal functional connectivity between the right angular gyrus (rAG) and rSTG, and IBS between the lIFG and rAG. The enhanced neural couplings positively correlated with group creative performance, whereas a reverse correlation pattern existed in the declined ones. A leave-one-out cross-validation analysis showed these neural couplings reliably predicted group creative performance within the sample. These indicate that high-creative group dynamics are characterized by utilizing partners' shared information when necessary (e.g. encountering idea exhaustion). A neural coupling profile consisting of sophisticated interplays between regions within frontal, temporal, and parietal lobes may underlie high-creative creative dynamics.

Evaluating the impact of short educational videos on the cortical networks for mathematics

Many teaching websites, such as the Khan Academy, propose vivid videos illustrating a mathematical concept. Using functional magnetic resonance imaging, we asked whether watching such a video suffices to rapidly change the brain networks for mathematical knowledge. We capitalized on the finding that, when judging the truth of short spoken statements, distinct semantic regions activate depending on whether the statements bear on mathematical knowledge or on other domains of semantic knowledge. Here, participants answered such questions before and after watching a lively 5-min video, which taught them the rudiments of a new domain. During the video, a distinct math-responsive network, comprising anterior intraparietal and inferior temporal nodes, showed intersubject synchrony when viewing mathematics course rather than control courses in biology or law. However, this experience led to minimal subsequent changes in the activity of those domain-specific areas when answering questions on the same topics a few minutes later. All taught facts, whether mathematical or not, led to domain-general repetition enhancement, particularly prominent in the cuneus, posterior cingulate, and posterior parietal cortices. We conclude that short videos do not suffice to induce a meaningful lasting change in the brain's math-responsive network, but merely engage domain-general regions possibly involved in episodic short-term memory.

Letting leaders spontaneously emerge yields better creative outcomes and higher leader-follower interbrain synchrony during creative group communication

This study aimed to investigate how the ways leaders arise (appointed vs. emergent) affect the leader-follower interaction during creative group communication. Hyperscanning technique was adopted to reveal the underlying interpersonal neural correlates using functional near-infrared spectroscopy. Participants were assigned into 3-person groups to complete a creative problem-solving task. These groups were randomly split into conditions of appointed (condition A) and emergent (condition E) leaders. Creative group outcomes were better in condition E, accompanied by more frequent perspective-taking behaviors between leaders and followers. The interpersonal brain synchronization (IBS) increment for leader-follower pairs was significantly higher at the right angular gyrus (rAG), between the rAG and the right supramarginal gyrus (rSMG), and between the right middle temporal gyrus and the right motor cortex in condition E and positively correlated with perspective-taking behaviors between leaders and followers. The graph-based analysis showed higher nodal betweenness of the rAG and the rSMG in condition E. These results indicated the neural coupling of brain regions involved in mentalizing, semantic processing and motor imagery may underlie the dynamic information transmission between leaders and followers during creative group communication.

Inefficient speech-motor control affects predictive speech comprehension: atypical electrophysiological correlates in stuttering

Listeners predict upcoming information during language comprehension. However, how this ability is implemented is still largely unknown. Here, we tested the hypothesis proposing that language production mechanisms have a role in prediction. We studied 2 electroencephalographic correlates of predictability during speech comprehension-pre-target alpha-beta (8-30 Hz) power decrease and the post-target N400 event-related potential effect-in a population with impaired speech-motor control, i.e. adults who stutter (AWS), compared to typically fluent adults (TFA). Participants listened to sentences that could either constrain towards a target word or not, modulating its predictability. As a complementary task, participants also performed context-driven word production. Compared to TFA, AWS not only displayed atypical neural responses in production, but, critically, they showed a different pattern also in comprehension. Specifically, while TFA showed the expected pre-target power decrease, AWS showed a power increase in frontal regions, associated with speech-motor control. In addition, the post-target N400 effect was reduced for AWS with respect to TFA. Finally, we found that production and comprehension power changes were positively correlated in TFA, but not in AWS. Overall, the results support the idea that processes and neural structures prominently devoted to speech planning also support prediction during speech comprehension.

Computational meaningfulness as the source of beneficial cognitive biases

The human brain has evolved to solve the problems it encounters in multiple environments. In solving these challenges, it forms mental simulations about multidimensional information about the world. These processes produce context-dependent behaviors. The brain as overparameterized modeling organ is an evolutionary solution for producing behavior in a complex world. One of the most essential characteristics of living creatures is that they compute the values of information they receive from external and internal contexts. As a result of this computation, the creature can behave in optimal ways in each environment. Whereas most other living creatures compute almost exclusively biological values (e.g., how to get food), the human as a cultural creature computes meaningfulness from the perspective of one's activity. The computational meaningfulness means the process of the human brain, with the help of which an individual tries to make the respective situation comprehensible to herself to know how to behave optimally. This paper challenges the bias-centric approach of behavioral economics by exploring different possibilities opened up by computational meaningfulness with insight into wider perspectives. We concentrate on confirmation bias and framing effect as behavioral economics examples of cognitive biases. We conclude that from the computational meaningfulness perspective of the brain, the use of these biases are indispensable property of an optimally designed computational system of what the human brain is like. From this perspective, cognitive biases can be rational under some conditions. Whereas the bias-centric approach relies on small-scale interpretable models which include only a few explanatory variables, the computational meaningfulness perspective emphasizes the behavioral models, which allow multiple variables in these models. People are used to working in multidimensional and varying environments. The human brain is at its best in such an environment and scientific study should increasingly take place in such situations simulating the real environment. By using naturalistic stimuli (e.g., videos and VR) we can create more realistic, life-like contexts for research purposes and analyze resulting data using machine learning algorithms. In this manner, we can better explain, understand and predict human behavior and choice in different contexts.

Neural correlates of linguistic collocations during continuous speech perception

Language is fundamentally predictable, both on a higher schematic level as well as low-level lexical items. Regarding predictability on a lexical level, collocations are frequent co-occurrences of words that are often characterized by high strength of association. So far, psycho- and neurolinguistic studies have mostly employed highly artificial experimental paradigms in the investigation of collocations by focusing on the processing of single words or isolated sentences. In contrast, here we analyze EEG brain responses recorded during stimulation with continuous speech, i.e., audio books. We find that the N400 response to collocations is significantly different from that of non-collocations, whereas the effect varies with respect to cortical region (anterior/posterior) and laterality (left/right). Our results are in line with studies using continuous speech, and they mostly contradict those using artificial paradigms and stimuli. To the best of our knowledge, this is the first neurolinguistic study on collocations using continuous speech stimulation.

Information flow across the cortical timescale hierarchy during narrative construction

When listening to spoken narratives, we must integrate information over multiple, concurrent timescales, building up from words to sentences to paragraphs to a coherent narrative. Recent evidence suggests that the brain relies on a chain of hierarchically organized areas with increasing temporal receptive windows to process naturalistic narratives. We hypothesized that the structure of this cortical processing hierarchy should result in an observable sequence of response lags between networks comprising the hierarchy during narrative comprehension. This study uses functional MRI to estimate the response lags between functional networks during narrative comprehension. We use intersubject cross-correlation analysis to capture network connectivity driven by the shared stimulus. We found a fixed temporal sequence of response lags-on the scale of several seconds-starting in early auditory areas, followed by language areas, the attention network, and lastly the default mode network. This gradient is consistent across eight distinct stories but absent in data acquired during rest or using a scrambled story stimulus, supporting our hypothesis that narrative construction gives rise to internetwork lags. Finally, we build a simple computational model for the neural dynamics underlying the construction of nested narrative features. Our simulations illustrate how the gradual accumulation of information within the boundaries of nested linguistic events, accompanied by increased activity at each level of the processing hierarchy, can give rise to the observed lag gradient.

Speech decoding from a small set of spatially segregated minimally invasive intracranial EEG electrodes with a compact and interpretable neural network

Objective. Speech decoding, one of the most intriguing brain-computer interface applications, opens up plentiful opportunities from rehabilitation of patients to direct and seamless communication between human species. Typical solutions rely on invasive recordings with a large number of distributed electrodes implanted through craniotomy. Here we explored the possibility of creating speech prosthesis in a minimally invasive setting with a small number of spatially segregated intracranial electrodes.Approach. We collected one hour of data (from two sessions) in two patients implanted with invasive electrodes. We then used only the contacts that pertained to a single stereotactic electroencephalographic (sEEG) shaft or an electrocorticographic (ECoG) stripe to decode neural activity into 26 words and one silence class. We employed a compact convolutional network-based architecture whose spatial and temporal filter weights allow for a physiologically plausible interpretation.Mainresults. We achieved on average 55% accuracy using only six channels of data recorded with a single minimally invasive sEEG electrode in the first patient and 70% accuracy using only eight channels of data recorded for a single ECoG strip in the second patient in classifying 26+1 overtly pronounced words. Our compact architecture did not require the use of pre-engineered features, learned fast and resulted in a stable, interpretable and physiologically meaningful decision rule successfully operating over a contiguous dataset collected during a different time interval than that used for training. Spatial characteristics of the pivotal neuronal populations corroborate with active and passive speech mapping results and exhibit the inverse space-frequency relationship characteristic of neural activity. Compared to other architectures our compact solution performed on par or better than those recently featured in neural speech decoding literature.Significance. We showcase the possibility of building a speech prosthesis with a small number of electrodes and based on a compact feature engineering free decoder derived from a small amount of training data.

Unraveling the functional attributes of the language connectome: crucial subnetworks, flexibility and variability

Language processing is a highly integrative function, intertwining linguistic operations (processing the language code intentionally used for communication) and extra-linguistic processes (e.g., attention monitoring, predictive inference, long-term memory). This synergetic cognitive architecture requires a distributed and specialized neural substrate. Brain systems have mainly been examined at rest. However, task-related functional connectivity provides additional and valuable information about how information is processed when various cognitive states are involved. We gathered thirteen language fMRI tasks in a unique database of one hundred and fifty neurotypical adults (InLang [Interactive networks of Language] database), providing the opportunity to assess language features across a wide range of linguistic processes. Using this database, we applied network theory as a computational tool to model the task-related functional connectome of language (LANG atlas). The organization of this data-driven neurocognitive atlas of language was examined at multiple levels, uncovering its major components (or crucial subnetworks), and its anatomical and functional correlates. In addition, we estimated its reconfiguration as a function of linguistic demand (flexibility) or several factors such as age or gender (variability). We observed that several discrete networks could be specifically shaped to promote key functional features of language: coding-decoding (Net1), control-executive (Net2), abstract-knowledge (Net3), and sensorimotor (Net4) functions. The architecture of these systems and the functional connectivity of the pivotal brain regions varied according to the nature of the linguistic process, gender, or age. By accounting for the multifaceted nature of language and modulating factors, this study can contribute to enriching and refining existing neurocognitive models of language. The LANG atlas can also be considered a reference for comparative or clinical studies involving various patients and conditions.

The early adolescent brain on music: Analysis of functional dynamics reveals engagement of orbitofrontal cortex reward system

Music listening plays a pivotal role for children and adolescents, yet it remains unclear how music modulates brain activity at the level of functional networks in this young population. Analysing the dynamics of brain networks occurring and dissolving over time in response to music can provide a better understanding of the neural underpinning of music listening. We collected functional magnetic resonance imaging (fMRI) data from 17 preadolescents aged 10-11 years while listening to two similar music pieces separated by periods without music. We subsequently tracked the occurrence of functional brain networks over the recording time using a recent method that detects recurrent patterns of phase-locking in the fMRI signals: the leading eigenvector dynamics analysis (LEiDA). The probabilities of occurrence and switching profiles of different functional networks were compared between periods of music and no music. Our results showed significantly increased occurrence of a specific functional network during the two music pieces compared to no music, involving the medial orbitofrontal and ventromedial prefrontal cortices-a brain subsystem associated to reward processing. Moreover, the higher the musical reward sensitivity of the preadolescents, the more this network was preceded by a pattern involving the insula. Our findings highlight the involvement of a brain subsystem associated with hedonic and emotional processing during music listening in the early adolescent brain. These results offer novel insight into the neural underpinnings of musical reward in early adolescence, improving our understanding of the important role and the potential benefits of music at this delicate age.

Probabilistic atlas for the language network based on precision fMRI data from >800 individuals

Two analytic traditions characterize fMRI language research. One relies on averaging activations across individuals. This approach has limitations: because of inter-individual variability in the locations of language areas, any given voxel/vertex in a common brain space is part of the language network in some individuals but in others, may belong to a distinct network. An alternative approach relies on identifying language areas in each individual using a functional ‘localizer’. Because of its greater sensitivity, functional resolution, and interpretability, functional localization is gaining popularity, but it is not always feasible, and cannot be applied retroactively to past studies. To bridge these disjoint approaches, we created a probabilistic functional atlas using fMRI data for an extensively validated language localizer in 806 individuals. This atlas enables estimating the probability that any given location in a common space belongs to the language network, and thus can help interpret group-level activation peaks and lesion locations, or select voxels/electrodes for analysis. More meaningful comparisons of findings across studies should increase robustness and replicability in language research.

Measurement(s)	Brain activity measurement
Technology Type(s)	fMRI
Sample Characteristic - Organism	Homo sapiens

Speaker-listener neural coupling reveals a right-lateralized mechanism for non-native speech-in-noise comprehension

While the increasingly globalized world has brought more and more demands for non-native language communication, the prevalence of background noise in everyday life poses a great challenge to non-native speech comprehension. The present study employed an interbrain approach based on functional near-infrared spectroscopy (fNIRS) to explore how people adapt to comprehend non-native speech information in noise. A group of Korean participants who acquired Chinese as their non-native language was invited to listen to Chinese narratives at 4 noise levels (no noise, 2 dB, -6 dB, and - 9 dB). These narratives were real-life stories spoken by native Chinese speakers. Processing of the non-native speech was associated with significant fNIRS-based listener-speaker neural couplings mainly over the right hemisphere at both the listener's and the speaker's sides. More importantly, the neural couplings from the listener's right superior temporal gyrus, the right middle temporal gyrus, as well as the right postcentral gyrus were found to be positively correlated with their individual comprehension performance at the strongest noise level (-9 dB). These results provide interbrain evidence in support of the right-lateralized mechanism for non-native speech processing and suggest that both an auditory-based and a sensorimotor-based mechanism contributed to the non-native speech-in-noise comprehension.

Event-related responses reflect chunk boundaries in natural speech

Chunking language has been proposed to be vital for comprehension enabling the extraction of meaning from a continuous stream of speech. However, neurocognitive mechanisms of chunking are poorly understood. The present study investigated neural correlates of chunk boundaries intuitively identified by listeners in natural speech drawn from linguistic corpora using magneto- and electroencephalography (MEEG). In a behavioral experiment, subjects marked chunk boundaries in the excerpts intuitively, which revealed highly consistent chunk boundary markings across the subjects. We next recorded brain activity to investigate whether chunk boundaries with high and medium agreement rates elicit distinct evoked responses compared to non-boundaries. Pauses placed at chunk boundaries elicited a closure positive shift with the sources over bilateral auditory cortices. In contrast, pauses placed within a chunk were perceived as interruptions and elicited a biphasic emitted potential with sources located in the bilateral primary and non-primary auditory areas with right-hemispheric dominance, and in the right inferior frontal cortex. Furthermore, pauses placed at stronger boundaries elicited earlier and more prominent activation over the left hemisphere suggesting that brain responses to chunk boundaries of natural speech can be modulated by the relative strength of different linguistic cues, such as syntactic structure and prosody.

Differential Tracking of Linguistic vs. Mental State Content in Naturalistic Stimuli by Language and Theory of Mind (ToM) Brain Networks

Language and social cognition, especially the ability to reason about mental states, known as theory of mind (ToM), are deeply related in development and everyday use. However, whether these cognitive faculties rely on distinct, overlapping, or the same mechanisms remains debated. Some evidence suggests that, by adulthood, language and ToM draw on largely distinct-though plausibly interacting-cortical networks. However, the broad topography of these networks is similar, and some have emphasized the importance of social content / communicative intent in the linguistic signal for eliciting responses in the language areas. Here, we combine the power of individual-subject functional localization with the naturalistic-cognition inter-subject correlation approach to illuminate the language-ToM relationship. Using functional magnetic resonance imaging (fMRI), we recorded neural activity as participants (n = 43) listened to stories and dialogues with mental state content (+linguistic, +ToM), viewed silent animations and live action films with mental state content but no language (-linguistic, +ToM), or listened to an expository text (+linguistic, -ToM). The ToM network robustly tracked stimuli rich in mental state information regardless of whether mental states were conveyed linguistically or non-linguistically, while tracking a +linguistic / -ToM stimulus only weakly. In contrast, the language network tracked linguistic stimuli more strongly than (a) non-linguistic stimuli, and than (b) the ToM network, and showed reliable tracking even for the linguistic condition devoid of mental state content. These findings suggest that in spite of their indisputably close links, language and ToM dissociate robustly in their neural substrates-and thus plausibly cognitive mechanisms-including during the processing of rich naturalistic materials.

Dynamic Inter-Brain Networks Correspond With Specific Communication Behaviors: Using Functional Near-Infrared Spectroscopy Hyperscanning During Creative and Non-creative Communication

Social interaction is a dynamic and variable process. However, most hyperscanning studies implicitly assume that inter-brain synchrony (IBS) is constant and rarely investigate the temporal variability of the multi-brain networks. In this study, we used sliding windows and k-mean clustering to obtain a set of representative inter-brain network states during different group communication tasks. By calculating the network parameters and temporal occurrence of the inter-brain states, we found that dense efficient interbrain states and sparse inefficient interbrain states appeared alternately and periodically, and the occurrence of efficient interbrain states was positively correlated with collaborative behaviors and group performance. Moreover, compared to common communication, the occurrence of efficient interbrain states and state transitions were significantly higher during creative communication, indicating a more active and intertwined neural network. These findings may indicate that there is a close correspondence between inter-brain network states and social behaviors, contributing to the flourishing literature on group communication.

Resting-State Neural-Activity Alterations in Subacute Aphasia after Stroke

Linguistic deficits are frequent symptoms among stroke survivors. The neural mechanism of post-stroke aphasia (PSA) was incompletely understood. Recently, resting-state functional magnetic resonance imaging (rs-fMRI) was widely used among several neuropsychological disorders. However, previous rs-fMRI studies of PSA were limited to very small sample size and the absence of reproducibility with different neuroimaging indexes. The present study performed comparisons with static and dynamic amplitude of low-frequency fluctuations (ALFF) and functional connectivity (FC) based on modest sample size (40 PSA and 37 healthy controls). Compared with controls, PSA showed significantly increased static ALFF predominantly in the bilateral supplementary motor area (SMA) and right hippocampus-parahippocampus (R HIP-ParaHip) and decreased static ALFF in right cerebellum. The increased dynamic ALFF in SMA and decreased dynamic ALFF in right cerebellum were also found in PSA. The static and dynamic ALFF in right cerebellum was positively correlated with spontaneous speech. The FC between the SMA and R HIP-ParaHip was significantly stronger in patients than controls and positively correlated with ALFF in bilateral SMA. In addition, the FC between the R HIP-ParaHip and the right temporal was also enhanced in patients and negatively correlated with repetition, naming, and comprehension score. These findings revealed consistently abnormal intrinsic neural activity in SMA and cerebellum, which may underlie linguistic deficits in PSA.

Processing of visual and non-visual naturalistic spatial information in the "parahippocampal place area"

The "parahippocampal place area" (PPA) in the human ventral visual stream exhibits increased hemodynamic activity correlated with the perception of landscape photos compared to faces or objects. Here, we investigate the perception of scene-related, spatial information embedded in two naturalistic stimuli. The same 14 participants were watching a Hollywood movie and listening to its audio-description as part of the open-data resource studyforrest.org. We model hemodynamic activity based on annotations of selected stimulus features, and compare results to a block-design visual localizer. On a group level, increased activation correlating with visual spatial information occurring in the movie is overlapping with a traditionally localized PPA. Activation correlating with semantic spatial information occurring in the audio-description is more restricted to the anterior PPA. On an individual level, we find significant bilateral activity in the PPA of nine individuals and unilateral activity in one individual. Results suggest that activation in the PPA generalizes to spatial information embedded in a movie and an auditory narrative, and may call for considering a functional subdivision of the PPA.

Cerebellar tDCS does not modulate language processing performance in healthy individuals

Clinical and neuroscientific studies in healthy volunteers have established that the cerebellum contributes to language comprehension and production. Yet most evidence is correlational and the exact role of the cerebellum remains unclear. The aim of this study was to investigate the role of the right cerebellum in unimpaired language comprehension and production using non-invasive brain stimulation. In this double-blind, sham-controlled experiment, thirty-six healthy participants received anodal or sham transcranial direct current (tDCS) stimulation to the right cerebellum while performing a lexical decision, sentence comprehension, verbal fluency and a non-language control task. Active tDCS did not modulate performance in any of the tasks. Additional exploratory analyses suggest difficulty-specific performance modulation in the sentence comprehension and lexical decision task, with tDCS improving performance in easy trials of the sentence comprehension task and difficult trials in the lexical decision task. Overall, our findings provide no evidence for the involvement of the right posterior cerebellum in language processing. Further research is needed to dissociate the influence of task difficulty of the underlying cognitive processes.

Interpersonal Neural Synchronization Predicting Learning Outcomes From Teaching-Learning Interaction: A Meta-Analysis

In school education, teaching-learning interaction is deemed as a core process in the classroom. The fundamental neural basis underlying teaching-learning interaction is proposed to be essential for tuning learning outcomes. However, the neural basis of this process as well as the relationship between the neural dynamics and the learning outcomes are largely unclear. With non-invasive technologies such as fNIRS (functional near-infrared spectroscopy), hyperscanning techniques have been developed since the last decade and been applied to the field of educational neuroscience for simultaneous multi-brain scanning. Hyperscanning studies suggest that the interpersonal neural synchronization (INS) during teaching-learning interaction might be an ideal neural biomarker for predicting learning outcomes. To systematically evaluate such a relationship, this meta-analysis ran on a random-effects model on 16 studies with 23 independent samples (effect sizes). Further moderator analyses were also performed to examine the potential influences of the style, mode, content, and the assessment method of learning outcomes. The random-effects modeling results confirmed a robust positive correlation between INS and learning outcomes. Subsequent analyses revealed that such relationship was mainly affected by both interaction style and mode. Therefore, the present meta-analysis provided a confirmatory neurocognitive foundation for teaching-learning interaction, as well as its relation to the learning outcomes, consolidated future learning and teaching studies in various disciplines including second language education with a firm methodological reference.

Directed coupling in multi-brain networks underlies generalized synchrony during social exchange

Advances in social neuroscience have made neural signatures of social exchange measurable simultaneously across people. This has identified brain regions differentially active during social interaction between human dyads, but the underlying systems-level mechanisms are incompletely understood. This paper introduces dynamic causal modeling and Bayesian model comparison to assess the causal and directed connectivity between two brains in the context of hyperscanning (h-DCM). In this setting, correlated neuronal responses become the data features that have to be explained by models with and without between-brain (effective) connections. Connections between brains can be understood in the context of generalized synchrony, which explains how dynamical systems become synchronized when they are coupled to each another. Under generalized synchrony, each brain state can be predicted by the other brain or a mixture of both. Our results show that effective connectivity between brains is not a feature within dyads per se but emerges selectively during social exchange. We demonstrate a causal impact of the sender's brain activity on the receiver of information, which explains previous reports of two-brain synchrony. We discuss the implications of this work; in particular, how characterizing generalized synchrony enables the discovery of between-brain connections in any social contact, and the advantage of h-DCM in studying brain function on the subject level, dyadic level, and group level within a directed model of (between) brain function.

Patient-clinician brain concordance underlies causal dynamics in nonverbal communication and negative affective expressivity

Patient-clinician concordance in behavior and brain activity has been proposed as a potential key mediator of mutual empathy and clinical rapport in the therapeutic encounter. However, the specific elements of patient-clinician communication that may support brain-to-brain concordance and therapeutic alliance are unknown. Here, we investigated how pain-related, directional facial communication between patients and clinicians is associated with brain-to-brain concordance. Patient-clinician dyads interacted in a pain-treatment context, during synchronous assessment of brain activity (fMRI hyperscanning) and online video transfer, enabling face-to-face social interaction. In-scanner videos were used for automated individual facial action unit (AU) time-series extraction. First, an interpretable machine-learning classifier of patients' facial expressions, from an independent fMRI experiment, significantly distinguished moderately painful leg pressure from innocuous pressure stimuli. Next, we estimated neural-network causality of patient-to-clinician directional information flow of facial expressions during clinician-initiated treatment of patients' evoked pain. We identified a leader-follower relationship in which patients predominantly led the facial communication while clinicians responded to patients' expressions. Finally, analyses of dynamic brain-to-brain concordance showed that patients' mid/posterior insular concordance with the clinicians' anterior insula cortex, a region identified in previously published data from this study¹, was associated with therapeutic alliance, and self-reported and objective (patient-to-clinician-directed causal influence) markers of negative-affect expressivity. These results suggest a role of patient-clinician concordance of the insula, a social-mirroring and salience-processing brain node, in mediating directional dynamics of pain-directed facial communication during therapeutic encounters.