Coupled neural systems underlie the production and comprehension of naturalistic narrative speech

The Effect of Speech Repetition Rate on Neural Activation in Healthy Adults: Implications for Treatment of Aphasia and Other Fluency Disorders

Functional imaging studies have provided insight into the effect of rate on production of syllables, pseudowords, and naturalistic speech, but the influence of rate on repetition of commonly-used words/phrases suitable for therapeutic use merits closer examination. Aim: To identify speech-motor regions responsive to rate and test the hypothesis that those regions would provide greater support as rates increase, we used an overt speech repetition task and functional magnetic resonance imaging (fMRI) to capture rate-modulated activation within speech-motor regions and determine whether modulations occur linearly and/or show hemispheric preference. Methods: Twelve healthy, right-handed adults participated in an fMRI task requiring overt repetition of commonly-used words/phrases at rates of 1, 2, and 3 syllables/second (syll./sec.). Results: Across all rates, bilateral activation was found both in ventral portions of primary sensorimotor cortex and middle and superior temporal regions. A repeated measures analysis of variance with pairwise comparisons revealed an overall difference between rates in temporal lobe regions of interest (ROIs) bilaterally (p < 0.001); all six comparisons reached significance (p < 0.05). Five of the six were highly significant (p < 0.008), while the left-hemisphere 2- vs. 3-syll./sec. comparison, though still significant, was less robust (p = 0.037). Temporal ROI mean beta-values increased linearly across the three rates bilaterally. Significant rate effects observed in the temporal lobes were slightly more pronounced in the right-hemisphere. No significant overall rate differences were seen in sensorimotor ROIs, nor was there a clear hemispheric effect. Conclusion: Linear effects in superior temporal ROIs suggest that sensory feedback corresponds directly to task demands. The lesser degree of significance in left-hemisphere activation at the faster, closer-to-normal rate may represent an increase in neural efficiency (and therefore, decreased demand) when the task so closely approximates a highly-practiced function. The presence of significant bilateral activation during overt repetition of words/phrases at all three rates suggests that repetition-based speech production may draw support from either or both hemispheres. This bihemispheric redundancy in regions associated with speech-motor control and their sensitivity to changes in rate may play an important role in interventions for nonfluent aphasia and other fluency disorders, particularly when right-hemisphere structures are the sole remaining pathway for production of meaningful speech.

Prediction of psychosis across protocols and risk cohorts using automated language analysis

Language and speech are the primary source of data for psychiatrists to diagnose and treat mental disorders. In psychosis, the very structure of language can be disturbed, including semantic coherence (e.g., derailment and tangentiality) and syntactic complexity (e.g., concreteness). Subtle disturbances in language are evident in schizophrenia even prior to first psychosis onset, during prodromal stages. Using computer-based natural language processing analyses, we previously showed that, among English-speaking clinical (e.g., ultra) high-risk youths, baseline reduction in semantic coherence (the flow of meaning in speech) and in syntactic complexity could predict subsequent psychosis onset with high accuracy. Herein, we aimed to cross-validate these automated linguistic analytic methods in a second larger risk cohort, also English-speaking, and to discriminate speech in psychosis from normal speech. We identified an automated machine-learning speech classifier - comprising decreased semantic coherence, greater variance in that coherence, and reduced usage of possessive pronouns - that had an 83% accuracy in predicting psychosis onset (intra-protocol), a cross-validated accuracy of 79% of psychosis onset prediction in the original risk cohort (cross-protocol), and a 72% accuracy in discriminating the speech of recent-onset psychosis patients from that of healthy individuals. The classifier was highly correlated with previously identified manual linguistic predictors. Our findings support the utility and validity of automated natural language processing methods to characterize disturbances in semantics and syntax across stages of psychotic disorder. The next steps will be to apply these methods in larger risk cohorts to further test reproducibility, also in languages other than English, and identify sources of variability. This technology has the potential to improve prediction of psychosis outcome among at-risk youths and identify linguistic targets for remediation and preventive intervention. More broadly, automated linguistic analysis can be a powerful tool for diagnosis and treatment across neuropsychiatry.

Social affective context reveals altered network dynamics in schizophrenia patients

Impairments in social cognition and interactions are core psychopathologies in schizophrenia, often manifesting as an inability to appropriately relate to the intentions and feelings of others. Neuroimaging has helped to demarcate the dynamics of two distinct functional connectivity circuits underlying the social-affective processes related to mentalization (known as Theory of Mind, ToM) and somatic-affiliation (known as Embodied Simulation, ES). While evidence points to abnormal activation patterns within these networks among those suffering from schizophrenia, it is yet unclear however, if these patients exhibit this abnormal functional connectivity in the context of social-affective experiences. The current fMRI study, investigated functional connectivity dynamics within ToM and ES networks as subjects experienced evolving cinematic portrayals of fear. During scanning, schizophrenia patients and healthy controls passively watched a cinematic scene in which a mother and her son face various threatening events. Participants then provided a continuous and retrospective report of their fear intensity during a second viewing outside the scanner. Using network cohesion index (NCI) analysis, we examined modulations of ES-related and ToM-related functional connectivity dynamics and their relation to symptom severity and the continuous emotional ratings of the induced cinematic fear. Compared to patients, healthy controls showed higher ES-NCI and marginally lower ToM-NCI during emotional peaks. Cross-correlation analysis revealed an intriguing dynamic between NCI and the inter-group difference of reported fear. Schizophrenia patients rated their fear as lower relative to healthy controls, shortly after exhibiting lower ES connectivity. This increased difference in rating was also followed by higher ToM connectivity among schizophrenia patients. The clinical relevance of these findings is further highlighted by the following two results: (a) ToM-NCI was found to have a strong correlation with the severity of general symptoms during one of the two main emotional peaks (Spearman R = 0.77); and (b) k-mean clustering demonstrated that the networks' NCI dynamic during the social-affective context reliably differentiated between patients and controls. Together, these findings point to a possible neural marker for abnormal social-affective processing in schizophrenia, manifested as the disturbed balance between two functional networks involved in social-affective affiliation. This in turn suggests that exaggerated mentalization over somatic-affiliative processing, in response to another's' distress may underlie social-affective deficits in schizophrenia.

Prediction is Production: The missing link between language production and comprehension

Language comprehension often involves the generation of predictions. It has been hypothesized that such prediction-for-comprehension entails actual language production. Recent studies provided evidence that the production system is recruited during language comprehension, but the link between production and prediction during comprehension remains hypothetical. Here, we tested this hypothesis by comparing prediction during sentence comprehension (primary task) in participants having the production system either available or not (non-verbal versus verbal secondary task). In the primary task, sentences containing an expected or unexpected target noun-phrase were presented during electroencephalography recording. Prediction, measured as the magnitude of the N400 effect elicited by the article (expected versus unexpected), was hindered only when the production system was taxed during sentence context reading. The present study provides the first direct evidence that the availability of the speech production system is necessary for generating lexical prediction during sentence comprehension. Furthermore, these important results provide an explanation for the recruitment of language production during comprehension.

Scale-Free Amplitude Modulation of Neuronal Oscillations Tracks Comprehension of Accelerated Speech

Speech comprehension is preserved up to a threefold acceleration, but deteriorates rapidly at higher speeds. Current models posit that perceptual resilience to accelerated speech is limited by the brain's ability to parse speech into syllabic units using δ/θ oscillations. Here, we investigated whether the involvement of neuronal oscillations in processing accelerated speech also relates to their scale-free amplitude modulation as indexed by the strength of long-range temporal correlations (LRTC). We recorded MEG while 24 human subjects (12 females) listened to radio news uttered at different comprehensible rates, at a mostly unintelligible rate and at this same speed interleaved with silence gaps. δ, θ, and low-γ oscillations followed the nonlinear variation of comprehension, with LRTC rising only at the highest speed. In contrast, increasing the rate was associated with a monotonic increase in LRTC in high-γ activity. When intelligibility was restored with the insertion of silence gaps, LRTC in the δ, θ, and low-γ oscillations resumed the low levels observed for intelligible speech. Remarkably, the lower the individual subject scaling exponents of δ/θ oscillations, the greater the comprehension of the fastest speech rate. Moreover, the strength of LRTC of the speech envelope decreased at the maximal rate, suggesting an inverse relationship with the LRTC of brain dynamics when comprehension halts. Our findings show that scale-free amplitude modulation of cortical oscillations and speech signals are tightly coupled to speech uptake capacity.SIGNIFICANCE STATEMENT One may read this statement in 20-30 s, but reading it in less than five leaves us clueless. Our minds limit how much information we grasp in an instant. Understanding the neural constraints on our capacity for sensory uptake is a fundamental question in neuroscience. Here, MEG was used to investigate neuronal activity while subjects listened to radio news played faster and faster until becoming unintelligible. We found that speech comprehension is related to the scale-free dynamics of δ and θ bands, whereas this property in high-γ fluctuations mirrors speech rate. We propose that successful speech processing imposes constraints on the self-organization of synchronous cell assemblies and their scale-free dynamics adjusts to the temporal properties of spoken language.

Dopamine drives left-hemispheric lateralization of neural networks during human speech

Although the concept of left-hemispheric lateralization of neural processes during speech production has been known since the times of Broca, its physiological underpinnings still remain elusive. We sought to assess the modulatory influences of a major neurotransmitter, dopamine, on hemispheric lateralization during real-life speaking using a multimodal analysis of functional MRI, intracranial EEG recordings, and large-scale neural population simulations based on diffusion-weighted MRI. We demonstrate that speech-induced phasic dopamine release into the dorsal striatum and speech motor cortex exerts direct modulation of neuronal activity in these regions and drives left-hemispheric lateralization of speech production network. Dopamine-induced lateralization of functional activity and networks during speaking is not dependent on lateralization of structural nigro-striatal and nigro-motocortical pathways. Our findings provide the first mechanistic explanation for left-hemispheric lateralization of human speech that is due to left-lateralized dopaminergic modulation of brain activity and functional networks.

Exploring the neural substrates of misinformation processing

It is well known that information that is initially thought to be correct but then revealed to be false, often continues to influence human judgement and decision making despite people being aware of the retraction. Yet little research has examined the underlying neural substrates of this phenomenon, which is known as the 'continued influence effect of misinformation' (CIEM). It remains unclear how the human brain processes critical information that retracts prior claims. To address this question in further detail, 26 healthy adults underwent functional magnetic resonance imaging (fMRI) while listening to brief narratives which either involved a retraction of prior information or not. Following each narrative, subjects' comprehension of the narrative, including their inclination to rely on retracted information, was probed. As expected, it was found that retracted information continued to affect participants' narrative-related reasoning. In addition, the fMRI data indicated that the continued influence of retracted information may be due to a breakdown of narrative-level integration and coherence-building mechanisms implemented by the precuneus and posterior cingulate gyrus.

Persuasion, Influence, and Value: Perspectives from Communication and Social Neuroscience

Opportunities to persuade and be persuaded are ubiquitous. What determines whether influence spreads and takes hold? This review provides an overview of evidence for the central role of subjective valuation in persuasion and social influence for both propagators and receivers of influence. We first review evidence that decisions to communicate information are determined by the subjective value a communicator expects to gain from sharing. We next review evidence that the effects of social influence and persuasion on receivers, in turn, arise from changes in the receiver's subjective valuation of objects, ideas, and behaviors. We then review evidence that self-related and social considerations are two key inputs to the value calculation in both communicators and receivers. Finally, we highlight biological coupling between communicators and receivers as a mechanism through which perceptions of value can be transmitted.

Discovering Event Structure in Continuous Narrative Perception and Memory

During realistic, continuous perception, humans automatically segment experiences into discrete events. Using a novel model of cortical event dynamics, we investigate how cortical structures generate event representations during narrative perception and how these events are stored to and retrieved from memory. Our data-driven approach allows us to detect event boundaries as shifts between stable patterns of brain activity without relying on stimulus annotations and reveals a nested hierarchy from short events in sensory regions to long events in high-order areas (including angular gyrus and posterior medial cortex), which represent abstract, multimodal situation models. High-order event boundaries are coupled to increases in hippocampal activity, which predict pattern reinstatement during later free recall. These areas also show evidence of anticipatory reinstatement as subjects listen to a familiar narrative. Based on these results, we propose that brain activity is naturally structured into nested events, which form the basis of long-term memory representations.

Domain-General Brain Regions Do Not Track Linguistic Input as Closely as Language-Selective Regions

Language comprehension engages a cortical network of left frontal and temporal regions. Activity in this network is language-selective, showing virtually no modulation by nonlinguistic tasks. In addition, language comprehension engages a second network consisting of bilateral frontal, parietal, cingulate, and insular regions. Activity in this "multiple demand" (MD) network scales with comprehension difficulty, but also with cognitive effort across a wide range of nonlinguistic tasks in a domain-general fashion. Given the functional dissociation between the language and MD networks, their respective contributions to comprehension are likely distinct, yet such differences remain elusive. Prior neuroimaging studies have suggested that activity in each network covaries with some linguistic features that, behaviorally, influence on-line processing and comprehension. This sensitivity of the language and MD networks to local input characteristics has often been interpreted, implicitly or explicitly, as evidence that both networks track linguistic input closely, and in a manner consistent across individuals. Here, we used fMRI to directly test this assumption by comparing the BOLD signal time courses in each network across different people (n = 45, men and women) listening to the same story. Language network activity showed fewer individual differences, indicative of closer input tracking, whereas MD network activity was more idiosyncratic and, moreover, showed lower reliability within an individual across repetitions of a story. These findings constrain cognitive models of language comprehension by suggesting a novel distinction between the processes implemented in the language and MD networks.SIGNIFICANCE STATEMENT Language comprehension recruits both language-specific mechanisms and domain-general mechanisms that are engaged in many cognitive processes. In the human cortex, language-selective mechanisms are implemented in the left-lateralized "core language network", whereas domain-general mechanisms are implemented in the bilateral "multiple demand" (MD) network. Here, we report the first direct comparison of the respective contributions of these networks to naturalistic story comprehension. Using a novel combination of neuroimaging approaches we find that MD regions track stories less closely than language regions. This finding constrains the possible contributions of the MD network to comprehension, contrasts with accounts positing that this network has continuous access to linguistic input, and suggests a new typology of comprehension processes based on their extent of input tracking.

Brain and Social Networks: Fundamental Building Blocks of Human Experience

How do brains shape social networks, and how do social ties shape the brain? Social networks are complex webs by which ideas spread among people. Brains comprise webs by which information is processed and transmitted among neural units. While brain activity and structure offer biological mechanisms for human behaviors, social networks offer external inducers or modulators of those behaviors. Together, these two axes represent fundamental contributors to human experience. Integrating foundational knowledge from social and developmental psychology and sociology on how individuals function within dyads, groups, and societies with recent advances in network neuroscience can offer new insights into both domains. Here, we use the example of how ideas and behaviors spread to illustrate the potential of multilayer network models.

Connectome-based lesion-symptom mapping (CLSM): A novel approach to map neurological function

Lesion-symptom mapping is a key tool in understanding the relationship between structure and function in neuroscience as it can provide objective evidence about which regions are crucial for a given process. Initial limitations with this approach were largely overcome by voxel-based lesion-symptom mapping (VLSM), a method introduced in the early 2000s, which allows for a whole-brain approach to study the association between damaged areas and behavioral impairment by applying an independent statistical test at every voxel. By doing so, this technique eliminated the need to predefine regions of interest or classify patients into groups based on arbitrary cutoff scores. VLSM has nonetheless its own limitations; chiefly, a bias towards recognizing cortical necrosis/gliosis but with poor sensitivity for detecting injury along long white matter tracts, thus ignoring cortical disconnection, which can per se lead to behavioral impairment. Here, we propose a complementary method that, instead, establishes a statistical relationship between the strength of connections between all brain regions of the brain (as defined by a standard brain atlas) and the array of behavioral performance seen in patients with brain injury: connectome-based lesion-symptom mapping (CLSM). Whole-brain CLSM therefore has the potential to identify key connections for behavior independently of a priori assumptions with applicability across a broad spectrum of neurological and psychiatric diseases. We propose that this approach can further our understanding of brain-structure relationships and is worth exploring in clinical and theoretical contexts.

•

Lesion-symptom mapping has been crucial to understand brain-function relations

•

VLSM eliminated the need to predefine regions of interest or biased patient groups.

•

Main limitations of VLSM relate cortical necrosis/gliosis and white matter tracts

•

CLSM can identify key connections for behavior independently of a priori assumptions

•

CLSM has applicability across several neurological and psychiatric diseases

Transcranial electric stimulation for the investigation of speech perception and comprehension

Transcranial electric stimulation (tES), comprising transcranial direct current stimulation (tDCS) and transcranial alternating current stimulation (tACS), involves applying weak electrical current to the scalp, which can be used to modulate membrane potentials and thereby modify neural activity. Critically, behavioural or perceptual consequences of this modulation provide evidence for a causal role of neural activity in the stimulated brain region for the observed outcome. We present tES as a tool for the investigation of which neural responses are necessary for successful speech perception and comprehension. We summarise existing studies, along with challenges that need to be overcome, potential solutions, and future directions. We conclude that, although standardised stimulation parameters still need to be established, tES is a promising tool for revealing the neural basis of speech processing. Future research can use this method to explore the causal role of brain regions and neural processes for the perception and comprehension of speech.

Key considerations in designing a speech brain-computer interface

Restoring communication in case of aphasia is a key challenge for neurotechnologies. To this end, brain-computer strategies can be envisioned to allow artificial speech synthesis from the continuous decoding of neural signals underlying speech imagination. Such speech brain-computer interfaces do not exist yet and their design should consider three key choices that need to be made: the choice of appropriate brain regions to record neural activity from, the choice of an appropriate recording technique, and the choice of a neural decoding scheme in association with an appropriate speech synthesis method. These key considerations are discussed here in light of (1) the current understanding of the functional neuroanatomy of cortical areas underlying overt and covert speech production, (2) the available literature making use of a variety of brain recording techniques to better characterize and address the challenge of decoding cortical speech signals, and (3) the different speech synthesis approaches that can be considered depending on the level of speech representation (phonetic, acoustic or articulatory) envisioned to be decoded at the core of a speech BCI paradigm.

A multimodal encoding model applied to imaging decision-related neural cascades in the human brain

Perception and cognition in the brain are naturally characterized as spatiotemporal processes. Decision-making, for example, depends on coordinated patterns of neural activity cascading across the brain, running in time from stimulus to response and in space from primary sensory regions to the frontal lobe. Measuring this cascade is key to developing an understanding of brain function. Here we report on a novel methodology that employs multi-modal imaging for inferring this cascade in humans at unprecedented spatiotemporal resolution. Specifically, we develop an encoding model to link simultaneously measured electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) signals to infer high-resolution spatiotemporal brain dynamics during a perceptual decision. After demonstrating replication of results from the literature, we report previously unobserved sequential reactivation of a substantial fraction of the pre-response network whose magnitude correlates with a proxy for decision confidence. Our encoding model, which temporally tags BOLD activations using time localized EEG variability, identifies a coordinated and spatially distributed neural cascade that is associated with a perceptual decision. In general the methodology illuminates complex brain dynamics that would otherwise be unobservable using fMRI or EEG acquired separately.

Brain-to-brain entrainment: EEG interbrain synchronization while speaking and listening

Electroencephalographic hyperscanning was used to investigate interbrain synchronization patterns in dyads of participants interacting through speech. Results show that brain oscillations are synchronized between listener and speaker during oral narratives. This interpersonal synchronization is mediated in part by a lower-level sensory mechanism of speech-to-brain synchronization, but also by the interactive process that takes place in the situation per se. These results demonstrate the existence of brain-to-brain entrainment which is not merely an epiphenomenon of auditory processing, during listening to one speaker. The study highlights the validity of the two-person neuroscience framework for understanding induced brain activity, and suggests that verbal information exchange cannot be fully understood by examining the listener’s or speaker’s brain activity in isolation.

Social Context-Dependent Activity in Marmoset Frontal Cortex Populations during Natural Conversations

Communication is an inherently interactive process that weaves together the fabric of both human and nonhuman primate societies. To investigate the properties of the primate brain during active social signaling, we recorded the responses of frontal cortex neurons as freely moving marmosets engaged in conversational exchanges with a visually occluded virtual marmoset. We found that small changes in firing rate (∼1 Hz) occurred across a broadly distributed population of frontal cortex neurons when marmosets heard a conspecific vocalization, and that these changes corresponded to subjects' likelihood of producing or withholding a vocal reply. Although the contributions of individual neurons were relatively small, large populations of neurons were able to clearly distinguish between these social contexts. Most significantly, this social context-dependent change in firing rate was evident even before subjects heard the vocalization, indicating that the probability of a conversational exchange was determined by the state of the frontal cortex at the time a vocalization was heard, and not by a decision driven by acoustic characteristics of the vocalization. We found that changes in neural activity scaled with the length of the conversation, with greater changes in firing rate evident for longer conversations. These data reveal specific and important facets of this neural activity that constrain its possible roles in active social signaling, and we hypothesize that the close coupling between frontal cortex activity and this natural, active primate social-signaling behavior facilitates social-monitoring mechanisms critical to conversational exchanges.SIGNIFICANCE STATEMENT We provide evidence for a novel pattern of neural activity in the frontal cortex of freely moving, naturally behaving, marmoset monkeys that may facilitate natural primate conversations. We discovered small (∼1 Hz), but reliable, changes in neural activity that occurred before marmosets even heard a conspecific vocalization that, as a population, almost perfectly predicted whether subjects would produce a vocalization in response. The change in the state of the frontal cortex persisted throughout the conversation and its magnitude scaled linearly with the length of the interaction. We hypothesize that this social context-dependent change in frontal cortex activity is supported by several mechanisms, such as social arousal and attention, and facilitates social monitoring critical for vocal coordination characteristic of human and nonhuman primate conversations.

Neural Mechanism Underling Comprehension of Narrative Speech and Its Heritability: Study in a Large Population

Comprehension of narratives constitutes a fundamental part of our everyday life experience. Although the neural mechanism of auditory narrative comprehension has been investigated in some studies, the neural correlates underlying this mechanism and its heritability remain poorly understood. We investigated comprehension of naturalistic speech in a large, healthy adult population (n = 429; 176/253 M/F; 22-36 years of age) consisting of 192 twin pairs (49 monozygotic and 47 dizygotic pairs) and 237 of their siblings. We used high quality functional MRI datasets from the Human Connectome Project (HCP) in which a story-based paradigm was utilized for the auditory narrative comprehension. Our results revealed that narrative comprehension was associated with activations of the classical language regions including superior temporal gyrus (STG), middle temporal gyrus (MTG), and inferior frontal gyrus (IFG) in both hemispheres, though STG and MTG were activated symmetrically and activation in IFG were left-lateralized. Our results further showed that the narrative comprehension was associated with activations in areas beyond the classical language regions, e.g. medial superior frontal gyrus (SFGmed), middle frontal gyrus (MFG), and supplementary motor area (SMA). Of subcortical structures, only the hippocampus was involved. The results of heritability analysis revealed that the oral reading recognition and picture vocabulary comprehension were significantly heritable (h ²  > 0.56, p < 10^- 13). In addition, the extent of activation of five areas in the left hemisphere, i.e. STG, IFG pars opercularis, SFGmed, SMA, and precuneus, and one area in the right hemisphere, i.e. MFG, were significantly heritable (h ²  > 0.33, p < 0.0004). The current study, to the best of our knowledge, is the first to investigate auditory narrative comprehension and its heritability in a large healthy population. Referring to the excellent quality of the HCP data, our results can clarify the functional contributions of linguistic and extra-linguistic cortices during narrative comprehension.

Resting State fMRI Functional Connectivity Analysis Using Dynamic Time Warping

Traditional resting-state network concept is based on calculating linear dependence of spontaneous low frequency fluctuations of the BOLD signals of different brain areas, which assumes temporally stable zero-lag synchrony across regions. However, growing amount of experimental findings suggest that functional connectivity exhibits dynamic changes and a complex time-lag structure, which cannot be captured by the static zero-lag correlation analysis. Here we propose a new approach applying Dynamic Time Warping (DTW) distance to evaluate functional connectivity strength that accounts for non-stationarity and phase-lags between the observed signals. Using simulated fMRI data we found that DTW captures dynamic interactions and it is less sensitive to linearly combined global noise in the data as compared to traditional correlation analysis. We tested our method using resting-state fMRI data from repeated measurements of an individual subject and showed that DTW analysis results in more stable connectivity patterns by reducing the within-subject variability and increasing robustness for preprocessing strategies. Classification results on a public dataset revealed a superior sensitivity of the DTW analysis to group differences by showing that DTW based classifiers outperform the zero-lag correlation and maximal lag cross-correlation based classifiers significantly. Our findings suggest that analysing resting-state functional connectivity using DTW provides an efficient new way for characterizing functional networks.

Interpersonal synchrony enhanced through 20 Hz phase-coupled dual brain stimulation

Synchronous movement is a key component of social behaviour in several species including humans. Recent theories have suggested a link between interpersonal synchrony of brain oscillations and interpersonal movement synchrony. The present study investigated this link. Using transcranial alternating current stimulation (tACS) applied over the left motor cortex, we induced beta band (20 Hz) oscillations in pairs of individuals who both performed a finger-tapping task with the right hand. In-phase or anti-phase oscillations were delivered during a preparatory period prior to movement and while the tapping task was performed. In-phase 20 Hz stimulation enhanced interpersonal movement synchrony, compared to anti-phase or sham stimulation, particularly for the initial taps following the preparatory period. This was confirmed in an analysis comparing real vs. pseudo pair surrogate data. No enhancement was observed for stimulation frequencies of 2 Hz (matching the target movement frequency) or 10 Hz (alpha band). Thus, phase-coupling of beta band neural oscillations across two individuals' (resting) motor cortices supports the interpersonal alignment of sensorimotor processes that regulate rhythmic action initiation, thereby facilitating the establishment of synchronous movement. Phase-locked dual brain stimulation provides a promising method to study causal effects of interpersonal brain synchrony on social, sensorimotor and cognitive processes.

Shared memories reveal shared structure in neural activity across individuals

Our lives revolve around sharing experiences and memories with others. When different people recount the same events, how similar are their underlying neural representations? Participants viewed a 50-min movie, then verbally described the events during functional MRI, producing unguided detailed descriptions lasting up to 40 min. As each person spoke, event-specific spatial patterns were reinstated in default-network, medial-temporal, and high-level visual areas. Individual event patterns were both highly discriminable from one another and similar among people, suggesting consistent spatial organization. In many high-order areas, patterns were more similar between people recalling the same event than between recall and perception, indicating systematic reshaping of percept into memory. These results reveal the existence of a common spatial organization for memories in high-level cortical areas, where encoded information is largely abstracted beyond sensory constraints, and that neural patterns during perception are altered systematically across people into shared memory representations for real-life events.

Mirroring and beyond: coupled dynamics as a generalized framework for modelling social interactions

When people observe one another, behavioural alignment can be detected at many levels, from the physical to the mental. Likewise, when people process the same highly complex stimulus sequences, such as films and stories, alignment is detected in the elicited brain activity. In early sensory areas, shared neural patterns are coupled to the low-level properties of the stimulus (shape, motion, volume, etc.), while in high-order brain areas, shared neural patterns are coupled to high-levels aspects of the stimulus, such as meaning. Successful social interactions require such alignments (both behavioural and neural), as communication cannot occur without shared understanding. However, we need to go beyond simple, symmetric (mirror) alignment once we start interacting. Interactions are dynamic processes, which involve continuous mutual adaptation, development of complementary behaviour and division of labour such as leader-follower roles. Here, we argue that interacting individuals are dynamically coupled rather than simply aligned. This broader framework for understanding interactions can encompass both processes by which behaviour and brain activity mirror each other (neural alignment), and situations in which behaviour and brain activity in one participant are coupled (but not mirrored) to the dynamics in the other participant. To apply these more sophisticated accounts of social interactions to the study of the underlying neural processes we need to develop new experimental paradigms and novel methods of data analysis.

Differences in Behavior and Brain Activity during Hypothetical and Real Choices

Real behaviors are binding consequential commitments to a course of action, such as harming another person, buying an Apple watch, or fleeing from danger. Cognitive scientists are generally interested in the psychological and neural processes that cause such real behavior. However, for practical reasons, many scientific studies measure behavior using only hypothetical or imagined stimuli. Generalizing from such studies to real behavior implicitly assumes that the processes underlying the two types of behavior are similar. We review evidence of similarity and differences in hypothetical and real mental processes. In many cases, hypothetical choice tasks give an incomplete picture of brain circuitry that is active during real choice.

Neural pattern change during encoding of a narrative predicts retrospective duration estimates

What mechanisms support our ability to estimate durations on the order of minutes? Behavioral studies in humans have shown that changes in contextual features lead to overestimation of past durations. Based on evidence that the medial temporal lobes and prefrontal cortex represent contextual features, we related the degree of fMRI pattern change in these regions with people's subsequent duration estimates. After listening to a radio story in the scanner, participants were asked how much time had elapsed between pairs of clips from the story. Our ROI analyses found that duration estimates were correlated with the neural pattern distance between two clips at encoding in the right entorhinal cortex. Moreover, whole-brain searchlight analyses revealed a cluster spanning the right anterior temporal lobe. Our findings provide convergent support for the hypothesis that retrospective time judgments are driven by 'drift' in contextual representations supported by these regions.

Brain Activity in Patients With Adductor Spasmodic Dysphonia Detected by Functional Magnetic Resonance Imaging

OBJECTIVES

Spasmodic dysphonia (SD) is considered a focal dystonia. However, the detailed pathophysiology of SD remains unclear, despite the detection of abnormal activity in several brain regions. The aim of this study was to clarify the pathophysiological background of SD.

STUDY DESIGN

This is a case-control study.

METHODS

Both task-related brain activity measured by functional magnetic resonance imaging by reading the five-digit numbers and resting-state functional connectivity (FC) measured by 150 T2-weighted echo planar images acquired without any task were investigated in 12 patients with adductor SD and in 16 healthy controls.

RESULTS

The patients with SD showed significantly higher task-related brain activation in the left middle temporal gyrus, left thalamus, bilateral primary motor area, bilateral premotor area, bilateral cerebellum, bilateral somatosensory area, right insula, and right putamen compared with the controls. Region of interest voxel FC analysis revealed many FC changes within the cerebellum-basal ganglia-thalamus-cortex loop in the patients with SD. Of the significant connectivity changes between the patients with SD and the controls, the FC between the left thalamus and the left caudate nucleus was significantly correlated with clinical parameters in SD.

CONCLUSION

The higher task-related brain activity in the insula and cerebellum was consistent with previous neuroimaging studies, suggesting that these areas are one of the unique characteristics of phonation-induced brain activity in SD. Based on FC analysis and their significant correlations with clinical parameters, the basal ganglia network plays an important role in the pathogenesis of SD.

Structural Organization of the Laryngeal Motor Cortical Network and Its Implication for Evolution of Speech Production

The laryngeal motor cortex (LMC) is essential for the production of learned vocal behaviors because bilateral damage to this area renders humans unable to speak but has no apparent effect on innate vocalizations such as human laughing and crying or monkey calls. Several hypotheses have been put forward attempting to explain the evolutionary changes from monkeys to humans that potentially led to enhanced LMC functionality for finer motor control of speech production. These views, however, remain limited to the position of the larynx area within the motor cortex, as well as its connections with the phonatory brainstem regions responsible for the direct control of laryngeal muscles. Using probabilistic diffusion tractography in healthy humans and rhesus monkeys, we show that, whereas the LMC structural network is largely comparable in both species, the LMC establishes nearly 7-fold stronger connectivity with the somatosensory and inferior parietal cortices in humans than in macaques. These findings suggest that important "hard-wired" components of the human LMC network controlling the laryngeal component of speech motor output evolved from an already existing, similar network in nonhuman primates. However, the evolution of enhanced LMC-parietal connections likely allowed for more complex synchrony of higher-order sensorimotor coordination, proprioceptive and tactile feedback, and modulation of learned voice for speech production.

SIGNIFICANCE STATEMENT

The role of the primary motor cortex in the formation of a comprehensive network controlling speech and language has been long underestimated and poorly studied. Here, we provide comparative and quantitative evidence for the significance of this region in the control of a highly learned and uniquely human behavior: speech production. From the viewpoint of structural network organization, we discuss potential evolutionary advances of enhanced temporoparietal cortical connections with the laryngeal motor cortex in humans compared with nonhuman primates that may have contributed to the development of finer vocal motor control necessary for speech production.

An examination of the language construct in NIMH's research domain criteria: Time for reconceptualization!

The National Institute of Mental Health's Research Domain Criteria (RDoC) Initiative "calls for the development of new ways of classifying psychopathology based on dimensions of observable behavior." As a result of this ambitious initiative, language has been identified as an independent construct in the RDoC matrix. In this article, we frame language within an evolutionary and neuropsychological context and discuss some of the limitations to the current measurements of language. Findings from genomics and the neuroimaging of performance during language tasks are discussed in relation to serious mental illness and within the context of caveats regarding measuring language. Indeed, the data collection and analysis methods employed to assay language have been both aided and constrained by the available technologies, methodologies, and conceptual definitions. Consequently, different fields of language research show inconsistent definitions of language that have become increasingly broad over time. Individually, they have also shown significant improvements in conceptual resolution, as well as in experimental and analytic techniques. More recently, language research has embraced collaborations across disciplines, notably neuroscience, cognitive science, and computational linguistics and has ultimately re-defined classical ideas of language. As we move forward, the new models of language with their remarkably multifaceted constructs force a re-examination of the NIMH RDoC conceptualization of language and thus the neuroscience and genetics underlying this concept. © 2016 The Authors. American Journal of Medical Genetics Part B: Neuropsychiatric Genetics Published by Wiley Periodicals, Inc.

Phonetic variation in bilingual speech: A lens for studying the production-comprehension link

We exploit the unique phonetic properties of bilingual speech to ask how processes occurring during planning affect speech articulation, and whether listeners can use the phonetic modulations that occur in anticipation of a codeswitch to help restrict their lexical search to the appropriate language. An analysis of spontaneous bilingual codeswitching in the Bangor Miami Corpus (Deuchar et al., 2014) reveals that in anticipation of switching languages, Spanish-English bilinguals produce slowed speech rate and cross-language phonological influence on consonant voice onset time. A study of speech comprehension using the visual world paradigm demonstrates that bilingual listeners can indeed exploit these low-level phonetic cues to anticipate that a codeswitch is coming and to suppress activation of the non-target language. We discuss the implications of these results for current theories of bilingual language regulation, and situate them in terms of recent proposals relating the coupling of the production and comprehension systems more generally.

What can we learn from a two-brain approach to verbal interaction?

Verbal interaction is one of the most frequent social interactions humans encounter on a daily basis. In the current paper, we zoom in on what the multi-brain approach has contributed, and can contribute in the future, to our understanding of the neural mechanisms supporting verbal interaction. Indeed, since verbal interaction can only exist between individuals, it seems intuitive to focus analyses on inter-individual neural markers, i.e. between-brain neural coupling. To date, however, there is a severe lack of theoretically-driven, testable hypotheses about what between-brain neural coupling actually reflects. In this paper, we develop a testable hypothesis in which between-pair variation in between-brain neural coupling is of key importance. Based on theoretical frameworks and empirical data, we argue that the level of between-brain neural coupling reflects speaker-listener alignment at different levels of linguistic and extra-linguistic representation. We discuss the possibility that between-brain neural coupling could inform us about the highest level of inter-speaker alignment: mutual understanding.

Automated segmentation of chronic stroke lesions using LINDA: Lesion identification with neighborhood data analysis

The gold standard for identifying stroke lesions is manual tracing, a method that is known to be observer dependent and time consuming, thus impractical for big data studies. We propose LINDA (Lesion Identification with Neighborhood Data Analysis), an automated segmentation algorithm capable of learning the relationship between existing manual segmentations and a single T1-weighted MRI. A dataset of 60 left hemispheric chronic stroke patients is used to build the method and test it with k-fold and leave-one-out procedures. With respect to manual tracings, predicted lesion maps showed a mean dice overlap of 0.696 ± 0.16, Hausdorff distance of 17.9 ± 9.8 mm, and average displacement of 2.54 ± 1.38 mm. The manual and predicted lesion volumes correlated at r = 0.961. An additional dataset of 45 patients was utilized to test LINDA with independent data, achieving high accuracy rates and confirming its cross-institutional applicability. To investigate the cost of moving from manual tracings to automated segmentation, we performed comparative lesion-to-symptom mapping (LSM) on five behavioral scores. Predicted and manual lesions produced similar neuro-cognitive maps, albeit with some discussed discrepancies. Of note, region-wise LSM was more robust to the prediction error than voxel-wise LSM. Our results show that, while several limitations exist, our current results compete with or exceed the state-of-the-art, producing consistent predictions, very low failure rates, and transferable knowledge between labs. This work also establishes a new viewpoint on evaluating automated methods not only with segmentation accuracy but also with brain-behavior relationships. LINDA is made available online with trained models from over 100 patients.

Acquiring and Producing Sentences: Whether Learners Use Verb-Specific or Verb-General Information Depends on Cue Validity

Learning to produce sentences involves learning patterns that enable the generation of new utterances. Language contains both verb-specific and verb-general regularities that are relevant to this capacity. Previous research has focused on whether one source is more important than the other. We tested whether the production system can flexibly learn to use either source, depending on the predictive validity of different cues in the input. Participants learned new sentence structures in a miniature language paradigm. In three experiments, we manipulated whether individual verbs or verb-general mappings better predicted the structures heard during learning. Evaluation of participants' subsequent production revealed that they could use either the structural preferences of individual verbs or abstract meaning-to-form mappings to construct new sentences. Further, this choice varied according to cue validity. These results demonstrate flexibility within the production architecture and the importance of considering how language was learned when discussing how language is used.

Ventral and dorsal streams for choosing word order during sentence production

Proficient language use requires speakers to vary word order and choose between different ways of expressing the same meaning. Prior statistical associations between individual verbs and different word orders are known to influence speakers' choices, but the underlying neural mechanisms are unknown. Here we show that distinct neural pathways are used for verbs with different statistical associations. We manipulated statistical experience by training participants in a language containing novel verbs and two alternative word orders (agent-before-patient, AP; patient-before-agent, PA). Some verbs appeared exclusively in AP, others exclusively in PA, and yet others in both orders. Subsequently, we used sparse sampling neuroimaging to examine the neural substrates as participants generated new sentences in the scanner. Behaviorally, participants showed an overall preference for AP order, but also increased PA order for verbs experienced in that order, reflecting statistical learning. Functional activation and connectivity analyses revealed distinct networks underlying the increased PA production. Verbs experienced in both orders during training preferentially recruited a ventral stream, indicating the use of conceptual processing for mapping meaning to word order. In contrast, verbs experienced solely in PA order recruited dorsal pathways, indicating the use of selective attention and sensorimotor integration for choosing words in the right order. These results show that the brain tracks the structural associations of individual verbs and that the same structural output may be achieved via ventral or dorsal streams, depending on the type of regularities in the input.

Neural systems involved in processing novel linguistic constructions and their visual referents

In language, abstract phrasal patterns provide an important source of meaning, but little is known about whether or how such constructions are used to predict upcoming visual scenes. Findings from two fMRI studies indicate that initial exposure to a novel construction allows its semantics to be used for such predictions. Specifically, greater activity in the ventral striatum, a region sensitive to prediction errors, was linked to worse overall comprehension of a novel construction. Moreover, activity in occipital cortex was attenuated when a visual event could be inferred from a learned construction, which may reflect predictive coding of the event. These effects disappeared when predictions were unlikely: that is, when phrases provided no additional information about visual events. These findings support the idea that learners create and evaluate predictions about new instances during comprehension of novel linguistic constructions.

Neuronal bases of structural coherence in contemporary dance observation

The neuronal processes underlying dance observation have been the focus of an increasing number of brain imaging studies over the past decade. However, the existing literature mainly dealt with effects of motor and visual expertise, whereas the neural and cognitive mechanisms that underlie the interpretation of dance choreographies remained unexplored. Hence, much attention has been given to the action observation network (AON) whereas the role of other potentially relevant neuro-cognitive mechanisms such as mentalizing (theory of mind) or language (narrative comprehension) in dance understanding is yet to be elucidated. We report the results of an fMRI study where the structural coherence of short contemporary dance choreographies was manipulated parametrically using the same taped movement material. Our participants were all trained dancers. The whole-brain analysis argues that the interpretation of structurally coherent dance phrases involves a subpart (superior parietal) of the AON as well as mentalizing regions in the dorsomedial prefrontal cortex. An ROI analysis based on a similar study using linguistic materials (Pallier et al., 2011) suggests that structural processing in language and dance might share certain neural mechanisms.

The Functional Connectome of Speech Control

In the past few years, several studies have been directed to understanding the complexity of functional interactions between different brain regions during various human behaviors. Among these, neuroimaging research installed the notion that speech and language require an orchestration of brain regions for comprehension, planning, and integration of a heard sound with a spoken word. However, these studies have been largely limited to mapping the neural correlates of separate speech elements and examining distinct cortical or subcortical circuits involved in different aspects of speech control. As a result, the complexity of the brain network machinery controlling speech and language remained largely unknown. Using graph theoretical analysis of functional MRI (fMRI) data in healthy subjects, we quantified the large-scale speech network topology by constructing functional brain networks of increasing hierarchy from the resting state to motor output of meaningless syllables to complex production of real-life speech as well as compared to non-speech-related sequential finger tapping and pure tone discrimination networks. We identified a segregated network of highly connected local neural communities (hubs) in the primary sensorimotor and parietal regions, which formed a commonly shared core hub network across the examined conditions, with the left area 4p playing an important role in speech network organization. These sensorimotor core hubs exhibited features of flexible hubs based on their participation in several functional domains across different networks and ability to adaptively switch long-range functional connectivity depending on task content, resulting in a distinct community structure of each examined network. Specifically, compared to other tasks, speech production was characterized by the formation of six distinct neural communities with specialized recruitment of the prefrontal cortex, insula, putamen, and thalamus, which collectively forged the formation of the functional speech connectome. In addition, the observed capacity of the primary sensorimotor cortex to exhibit operational heterogeneity challenged the established concept of unimodality of this region.

This study uses graph theory to analyze functional MRI data recorded from speakers as they produce single syllables or whole sentences, revealing the complexity of the brain network machinery that controls speech and language.

Speech production is a complex process that requires the orchestration of multiple brain regions. However, our current understanding of the large-scale neural architecture during speaking remains scant, as research has mostly focused on examining distinct brain circuits involved in distinct aspects of speech control. Here, we performed graph theoretical analyses of functional MRI data acquired from healthy subjects in order to reveal how brain regions relate to one another while speaking. We constructed functional brain networks of increasing hierarchy from rest to simple vocal motor output to the production of real-life speech, and compared these to nonspeech control tasks such as finger tapping and pure tone discrimination. We discovered a specialized network of densely connected sensorimotor regions, which formed a common processing core across all conditions. Specifically, the primary sensorimotor cortex participated in multiple functional domains across different networks and modulated long-range connections depending on task content, which challenges the established concept of low-order unimodal function of this region. Compared to other tasks, speech production was characterized by the formation of six distinct neural communities with specialized recruitment of the prefrontal cortex, insula, putamen, and thalamus, which collectively formed the functional speech connectome.

Second language lexical development and cognitive control: A longitudinal fMRI study

In this paper we report a longitudinal functional magnetic resonance imaging (fMRI) study that tested contrasting predictions about the time course of cognitive control in second language (L2) acquisition. We examined the neural correlates of lexical processing in L2 learners twice over the course of one academic year. Specifically, while in the scanner, participants were asked to judge the language membership of unambiguous first and second language words, as well as interlingual homographs. Our ROI and connectivity analyses reveal that with increased exposure to the L2, overall activation in control areas such as the anterior cingulate cortex decrease while connectivity with semantic processing regions such as the middle temporal gyrus increase. These results suggest that cognitive control is more important initially in L2 acquisition, and have significant implications for understanding developmental and neurocognitive models of second language lexical processing.

The Now-or-Never bottleneck: A fundamental constraint on language

Memory is fleeting. New material rapidly obliterates previous material. How, then, can the brain deal successfully with the continual deluge of linguistic input? We argue that, to deal with this "Now-or-Never" bottleneck, the brain must compress and recode linguistic input as rapidly as possible. This observation has strong implications for the nature of language processing: (1) the language system must "eagerly" recode and compress linguistic input; (2) as the bottleneck recurs at each new representational level, the language system must build a multilevel linguistic representation; and (3) the language system must deploy all available information predictively to ensure that local linguistic ambiguities are dealt with "Right-First-Time"; once the original input is lost, there is no way for the language system to recover. This is "Chunk-and-Pass" processing. Similarly, language learning must also occur in the here and now, which implies that language acquisition is learning to process, rather than inducing, a grammar. Moreover, this perspective provides a cognitive foundation for grammaticalization and other aspects of language change. Chunk-and-Pass processing also helps explain a variety of core properties of language, including its multilevel representational structure and duality of patterning. This approach promises to create a direct relationship between psycholinguistics and linguistic theory. More generally, we outline a framework within which to integrate often disconnected inquiries into language processing, language acquisition, and language change and evolution.

Linking Neuroimaging with Functional Linguistic Analysis to Understand Processes of Successful Communication

Functional linguistic models posit a systematic link between language FORM and the FUNCTIONS for which language is used. This is a systematic (and therefore quantifiable) relationship. Yet, many open questions remain about the mechanisms that link form, function and communication relevant outcomes. Neuroimaging methods can provide insight into such processes that are not apparent from other methods. We argue that the combination of neural and linguistic measures will allow insight into both individual and population-level communication processes that would not be possible using either method in isolation. We present examples illustrating this methodological integration and notes regarding the most amenable linguistic tools. We summarize a framework in which language presented to and produced by participants undergoing neuroimaging is correlated with the resulting neural data and other proximal communication outcomes allowing the triangulation of individual experimental with population level outcomes, thereby linking between micro and macro levels of analysis.

Speech networks at rest and in action: interactions between functional brain networks controlling speech production

Speech production is one of the most complex human behaviors. Although brain activation during speaking has been well investigated, our understanding of interactions between the brain regions and neural networks remains scarce. We combined seed-based interregional correlation analysis with graph theoretical analysis of functional MRI data during the resting state and sentence production in healthy subjects to investigate the interface and topology of functional networks originating from the key brain regions controlling speech, i.e., the laryngeal/orofacial motor cortex, inferior frontal and superior temporal gyri, supplementary motor area, cingulate cortex, putamen, and thalamus. During both resting and speaking, the interactions between these networks were bilaterally distributed and centered on the sensorimotor brain regions. However, speech production preferentially recruited the inferior parietal lobule (IPL) and cerebellum into the large-scale network, suggesting the importance of these regions in facilitation of the transition from the resting state to speaking. Furthermore, the cerebellum (lobule VI) was the most prominent region showing functional influences on speech-network integration and segregation. Although networks were bilaterally distributed, interregional connectivity during speaking was stronger in the left vs. right hemisphere, which may have underlined a more homogeneous overlap between the examined networks in the left hemisphere. Among these, the laryngeal motor cortex (LMC) established a core network that fully overlapped with all other speech-related networks, determining the extent of network interactions. Our data demonstrate complex interactions of large-scale brain networks controlling speech production and point to the critical role of the LMC, IPL, and cerebellum in the formation of speech production network.

Cerebral coherence between communicators marks the emergence of meaning

How can we understand each other during communicative interactions? An influential suggestion holds that communicators are primed by each other's behaviors, with associative mechanisms automatically coordinating the production of communicative signals and the comprehension of their meanings. An alternative suggestion posits that mutual understanding requires shared conceptualizations of a signal's use, i.e., "conceptual pacts" that are abstracted away from specific experiences. Both accounts predict coherent neural dynamics across communicators, aligned either to the occurrence of a signal or to the dynamics of conceptual pacts. Using coherence spectral-density analysis of cerebral activity simultaneously measured in pairs of communicators, this study shows that establishing mutual understanding of novel signals synchronizes cerebral dynamics across communicators' right temporal lobes. This interpersonal cerebral coherence occurred only within pairs with a shared communicative history, and at temporal scales independent from signals' occurrences. These findings favor the notion that meaning emerges from shared conceptualizations of a signal's use.