Spatiotemporal dynamics of word retrieval in speech production revealed by cortical high-frequency band activity

The characteristics and reproducibility of motor speech functional neuroimaging in healthy controls

Introduction

Functional magnetic resonance imaging (fMRI) can improve our understanding of neural processes subserving motor speech function. Yet its reproducibility remains unclear. This study aimed to evaluate the reproducibility of fMRI using a word repetition task across two time points.

Methods

Imaging data from 14 healthy controls were analysed using a multi-level general linear model.

Results

Significant activation was observed during the task in the right hemispheric cerebellar lobules IV-V, right putamen, and bilateral sensorimotor cortices. Activation between timepoints was found to be moderately reproducible across time in the cerebellum but not in other brain regions.

Discussion

Preliminary findings highlight the involvement of the cerebellum and connected cerebral regions during a motor speech task. More work is needed to determine the degree of reproducibility of speech fMRI before this could be used as a reliable marker of changes in brain activity.

Common and distinct patterns underlying different linguistic tasks: multivariate disconnectome symptom mapping in poststroke patients

Numerous studies have been devoted to neural mechanisms of a variety of linguistic tasks (e.g. speech comprehension and production). To date, however, whether and how the neural patterns underlying different linguistic tasks are similar or differ remains elusive. In this study, we compared the neural patterns underlying 3 linguistic tasks mainly concerning speech comprehension and production. To address this, multivariate regression approaches with lesion/disconnection symptom mapping were applied to data from 216 stroke patients with damage to the left hemisphere. The results showed that lesion/disconnection patterns could predict both poststroke scores of speech comprehension and production tasks; these patterns exhibited shared regions on the temporal pole of the left hemisphere as well as unique regions contributing to the prediction for each domain. Lower scores in speech comprehension tasks were associated with lesions/abnormalities in the superior temporal gyrus and middle temporal gyrus, while lower scores in speech production tasks were associated with lesions/abnormalities in the left inferior parietal lobe and frontal lobe. These results suggested an important role of the ventral and dorsal stream pathways in speech comprehension and production (i.e. supporting the dual stream model) and highlighted the applicability of the novel multivariate disconnectome-based symptom mapping in cognitive neuroscience research.

Recurrent connectivity supports higher-level visual and semantic object representations in the brain

Visual object recognition has been traditionally conceptualised as a predominantly feedforward process through the ventral visual pathway. While feedforward artificial neural networks (ANNs) can achieve human-level classification on some image-labelling tasks, it's unclear whether computational models of vision alone can accurately capture the evolving spatiotemporal neural dynamics. Here, we probe these dynamics using a combination of representational similarity and connectivity analyses of fMRI and MEG data recorded during the recognition of familiar, unambiguous objects. Modelling the visual and semantic properties of our stimuli using an artificial neural network as well as a semantic feature model, we find that unique aspects of the neural architecture and connectivity dynamics relate to visual and semantic object properties. Critically, we show that recurrent processing between the anterior and posterior ventral temporal cortex relates to higher-level visual properties prior to semantic object properties, in addition to semantic-related feedback from the frontal lobe to the ventral temporal lobe between 250 and 500 ms after stimulus onset. These results demonstrate the distinct contributions made by semantic object properties in explaining neural activity and connectivity, highlighting it as a core part of object recognition not fully accounted for by current biologically inspired neural networks.

Spatio-temporal evolution of human neural activity during visually cued hand movements

Making hand movements in response to visual cues is common in daily life. It has been well known that this process activates multiple areas in the brain, but how these neural activations progress across space and time remains largely unknown. Taking advantage of intracranial electroencephalographic (iEEG) recordings using depth and subdural electrodes from 36 human subjects using the same task, we applied single-trial and cross-trial analyses to high-frequency iEEG activity. The results show that the neural activation was widely distributed across the human brain both within and on the surface of the brain, and focused specifically on certain areas in the parietal, frontal, and occipital lobes, where parietal lobes present significant left lateralization on the activation. We also demonstrate temporal differences across these brain regions. Finally, we evaluated the degree to which the timing of activity within these regions was related to sensory or motor function. The findings of this study promote the understanding of task-related neural processing of the human brain, and may provide important insights for translational applications.

Intra- and inter-hemispheric network dynamics supporting object recognition and speech production

We built normative brain atlases that animate millisecond-scale intra- and inter-hemispheric white matter-level connectivity dynamics supporting object recognition and speech production. We quantified electrocorticographic modulations during three naming tasks using event-related high-gamma activity from 1,114 nonepileptogenic intracranial electrodes (i.e., non-lesional areas unaffected by epileptiform discharges). Using this electrocorticography data, we visualized functional connectivity modulations defined as significant naming-related high-gamma modulations occurring simultaneously at two sites connected by direct white matter streamlines on diffusion-weighted imaging tractography. Immediately after stimulus onset, intra- and inter-hemispheric functional connectivity enhancements were confined mainly across modality-specific perceptual regions. During response preparation, left intra-hemispheric connectivity enhancements propagated in a posterior-to-anterior direction, involving the left precentral and prefrontal areas. After overt response onset, inter- and intra-hemispheric connectivity enhancements mainly encompassed precentral, postcentral, and superior-temporal (STG) gyri. We found task-specific connectivity enhancements during response preparation as follows. Picture naming enhanced activity along the left arcuate fasciculus between the inferior-temporal and precentral/posterior inferior-frontal (pIFG) gyri. Nonspeech environmental sound naming augmented functional connectivity via the left inferior longitudinal and fronto-occipital fasciculi between the medial-occipital and STG/pIFG. Auditory descriptive naming task enhanced usage of the left frontal U-fibers, involving the middle-frontal gyrus. Taken together, the commonly observed network enhancements include inter-hemispheric connectivity optimizing perceptual processing exerted in each hemisphere, left intra-hemispheric connectivity supporting semantic and lexical processing, and inter-hemispheric connectivity for symmetric oral movements during overt speech. Our atlases improve the currently available models of object recognition and speech production by adding neural dynamics via direct intra- and inter-hemispheric white matter tracts.

A narrative review of the anatomy and function of the white matter tracts in language production and comprehension

Much is known about the role of cortical areas in language processing. The shift towards network approaches in recent years has highlighted the importance of uncovering the role of white matter in connecting these areas. However, despite a large body of research, many of these tracts’ functions are not well-understood. We present a comprehensive review of the empirical evidence on the role of eight major tracts that are hypothesized to be involved in language processing (inferior longitudinal fasciculus, inferior fronto-occipital fasciculus, uncinate fasciculus, extreme capsule, middle longitudinal fasciculus, superior longitudinal fasciculus, arcuate fasciculus, and frontal aslant tract). For each tract, we hypothesize its role based on the function of the cortical regions it connects. We then evaluate these hypotheses with data from three sources: studies in neurotypical individuals, neuropsychological data, and intraoperative stimulation studies. Finally, we summarize the conclusions supported by the data and highlight the areas needing further investigation.

Taxonomic and thematic semantic relationships in picture naming as revealed by Laplacian-transformed event-related potentials

Semantically related concepts co-activate when we speak. Prior research reported both behavioral interference and facilitation due to co-activation during picture naming. Different word relationships may account for some of this discrepancy. Taxonomically related words (e.g., WOLF-DOG) have been associated with semantic interference; thematically related words (e.g., BONE-DOG) have been associated with facilitation. Although these different semantic relationships have been associated with opposite behavioral outcomes, electrophysiological studies have found inconsistent effects on event-related potentials. We conducted a picture-word interference electroencephalography experiment to examine word retrieval dynamics in these different semantic relationships. Importantly, we used traditional monopolar analysis as well as Laplacian transformation allowing us to examine spatially deblurred event-related components. Both analyses revealed greater negativity (150-250 ms) for unrelated than related taxonomic pairs, though more restricted in space for thematic pairs. Critically, Laplacian analyses revealed a larger negative-going component in the 300 to 500 ms time window in taxonomically related versus unrelated pairs which were restricted to a left frontal recording site. In parallel, an opposite effect was found in the same time window but localized to a left parietal site. Finding these opposite effects in the same time window was feasible thanks to the use of the Laplacian transformation and suggests that frontal control processes are concurrently engaged with cascading effects of the spread of activation through semantically related representations.

Inter-individual variability in dorsal stream dynamics during word production

The current standard model of language production involves a sensorimotor dorsal stream connecting areas in the temporo-parietal junction with those in the inferior frontal gyrus and lateral premotor cortex. These regions have been linked to various aspects of word production such as phonological processing or articulatory programming, primarily through neuropsychological and functional imaging group studies. Most if not all the theoretical descriptions of this model imply that the same network should be identifiable across individual speakers. We tested this hypothesis by quantifying the variability of activation observed across individuals within each dorsal stream anatomical region. This estimate was based on electrical activity recorded directly from the cerebral cortex with millisecond accuracy in awake epileptic patients clinically implanted with intracerebral depth electrodes for pre-surgical diagnosis. Each region's activity was quantified using two different metrics-intra-cerebral evoked related potentials and high gamma activity-at the level of the group, the individual and the recording contact. The two metrics show simultaneous activation of parietal and frontal regions during a picture naming task, in line with models that posit interactive processing during word retrieval. They also reveal different levels of between-patient variability across brain regions, except in core auditory and motor regions. The independence and non-uniformity of cortical activity estimated through the two metrics push the current model towards sub-second and sub-region explorations focused on individualized language speech production. Several hypotheses are considered for this within-region heterogeneity.

Clinical neuroscience and neurotechnology: An amazing symbiosis

In the last decades, clinical neuroscience found a novel ally in neurotechnologies, devices able to record and stimulate electrical activity in the nervous system. These technologies improved the ability to diagnose and treat neural disorders. Neurotechnologies are concurrently enabling a deeper understanding of healthy and pathological dynamics of the nervous system through stimulation and recordings during brain implants. On the other hand, clinical neurosciences are not only driving neuroengineering toward the most relevant clinical issues, but are also shaping the neurotechnologies thanks to clinical advancements. For instance, understanding the etiology of a disease informs the location of a therapeutic stimulation, but also the way stimulation patterns should be designed to be more effective/naturalistic. Here, we describe cases of fruitful integration such as Deep Brain Stimulation and cortical interfaces to highlight how this symbiosis between clinical neuroscience and neurotechnology is closer to a novel integrated framework than to a simple interdisciplinary interaction.

Opportunities, pitfalls and trade-offs in designing protocols for measuring the neural correlates of speech

Decoding speech and speech-related processes directly from the human brain has intensified in studies over recent years as such a decoder has the potential to positively impact people with limited communication capacity due to disease or injury. Additionally, it can present entirely new forms of human-computer interaction and human-machine communication in general and facilitate better neuroscientific understanding of speech processes. Here, we synthesize the literature on neural speech decoding pertaining to how speech decoding experiments have been conducted, coalescing around a necessity for thoughtful experimental design aimed at specific research goals, and robust procedures for evaluating speech decoding paradigms. We examine the use of different modalities for presenting stimuli to participants, methods for construction of paradigms including timings and speech rhythms, and possible linguistic considerations. In addition, novel methods for eliciting naturalistic speech and validating imagined speech task performance in experimental settings are presented based on recent research. We also describe the multitude of terms used to instruct participants on how to produce imagined speech during experiments and propose methods for investigating the effect of these terms on imagined speech decoding. We demonstrate that the range of experimental procedures used in neural speech decoding studies can have unintended consequences which can impact upon the efficacy of the knowledge obtained. The review delineates the strengths and weaknesses of present approaches and poses methodological advances which we anticipate will enhance experimental design, and progress toward the optimal design of movement independent direct speech brain-computer interfaces.

Early beta oscillations in multisensory association areas underlie crossmodal performance enhancement

The combination of signals from different sensory modalities can enhance perception and facilitate behavioral responses. While previous research described crossmodal influences in a wide range of tasks, it remains unclear how such influences drive performance enhancements. In particular, the neural mechanisms underlying performance-relevant crossmodal influences, as well as the latency and spatial profile of such influences are not well understood. Here, we examined data from high-density electroencephalography (N = 30) recordings to characterize the oscillatory signatures of crossmodal facilitation of response speed, as manifested in the speeding of visual responses by concurrent task-irrelevant auditory information. Using a data-driven analysis approach, we found that individual gains in response speed correlated with larger beta power difference (13-25 Hz) between the audiovisual and the visual condition, starting within 80 ms after stimulus onset in the secondary visual cortex and in multisensory association areas in the parietal cortex. In addition, we examined data from electrocorticography (ECoG) recordings in four epileptic patients in a comparable paradigm. These ECoG data revealed reduced beta power in audiovisual compared with visual trials in the superior temporal gyrus (STG). Collectively, our data suggest that the crossmodal facilitation of response speed is associated with reduced early beta power in multisensory association and secondary visual areas. The reduced early beta power may reflect an auditory-driven feedback signal to improve visual processing through attentional gating. These findings improve our understanding of the neural mechanisms underlying crossmodal response speed facilitation and highlight the critical role of beta oscillations in mediating behaviorally relevant multisensory processing.

An open-source toolbox for Multi-patient Intracranial EEG Analysis (MIA)

Intracranial EEG (iEEG) performed during the pre-surgical evaluation of refractory epilepsy provides a great opportunity to investigate the neurophysiology of human cognitive functions with exceptional spatial and temporal precisions. A difficulty of the iEEG approach for cognitive neuroscience, however, is the potential variability across patients in the anatomical location of implantations and in the functional responses therein recorded. In this context, we designed, implemented, and tested a user-friendly and efficient open-source toolbox for Multi-Patient Intracranial data Analysis (MIA), which can be used as standalone program or as a Brainstorm plugin. MIA helps analyzing event related iEEG signals while following good scientific practice recommendations, such as building reproducible analysis pipelines and applying robust statistics. The signals can be analyzed in the temporal and time-frequency domains, and the similarity of time courses across patients or contacts can be assessed within anatomical regions. MIA allows visualizing all these results in a variety of formats at every step of the analysis. Here, we present the toolbox architecture and illustrate the different steps and features of the analysis pipeline using a group dataset collected during a language task.

Language Tasks and the Network Control Role of the Left Inferior Frontal Gyrus

Recent work has combined cognitive neuroscience and control theory to make predictions about cognitive control functions. Here, we test a link between whole-brain theories of semantics and the role of the left inferior frontal gyrus (LIFG) in controlled language performance using network control theory (NCT), a branch of systems engineering. Specifically, we examined whether two properties of node controllability, boundary and modal controllability, were linked to semantic selection and retrieval on sentence completion and verb generation tasks. We tested whether the controllability of the left IFG moderated language selection and retrieval costs and the effects of continuous θ burst stimulation (cTBS), an inhibitory form of transcranial magnetic stimulation (TMS) on behavior in 41 human subjects (25 active, 16 sham). We predicted that boundary controllability, a measure of the theoretical ability of a node to integrate and segregate brain networks, would be linked to word selection in the contextually-rich sentence completion task. In contrast, we expected that modal controllability, a measure of the theoretical ability of a node to drive the brain into specifically hard-to-reach states, would be linked to retrieval on the low-context verb generation task. Boundary controllability was linked to selection and to the ability of TMS to reduce response latencies on the sentence completion task. In contrast, modal controllability was not linked to performance on the tasks or TMS effects. Overall, our results suggest a link between the network integrating role of the LIFG and selection and the overall semantic demands of sentence completion.

The Same Ultra-Rapid Parallel Brain Dynamics Underpin the Production and Perception of Speech

The temporal dynamics by which linguistic information becomes available is one of the key properties to understand how language is organized in the brain. An unresolved debate between different brain language models is whether words, the building blocks of language, are activated in a sequential or parallel manner. In this study, we approached this issue from a novel perspective by directly comparing the time course of word component activation in speech production versus perception. In an overt object naming task and a passive listening task, we analyzed with mixed linear models at the single-trial level the event-related brain potentials elicited by the same lexico-semantic and phonological word knowledge in the two language modalities. Results revealed that both word components manifested simultaneously as early as 75 ms after stimulus onset in production and perception; differences between the language modalities only became apparent after 300 ms of processing. The data provide evidence for ultra-rapid parallel dynamics of language processing and are interpreted within a neural assembly framework where words recruit the same integrated cell assemblies across production and perception. These word assemblies ignite early on in parallel and only later on reverberate in a behavior-specific manner.

Characterizing multi-word speech production using event-related potentials

Event-related potentials (ERPs) derived from electroencephalography (EEG) have proven useful for understanding linguistic processes during language perception and production. Words are commonly produced in sequences, yet most ERP studies have used single-word experimental designs. Single-word designs reduce potential ERP overlap in word sequence production. However, word sequence production engages brain mechanisms in different ways than single word production. In particular, speech monitoring and planning mechanisms are more engaged than for single words since several words must be produced in a short period of time. This study evaluates the feasibility of recording ERP components in the context of word sequence production, and whether separate components could be isolated for each word. Scalp EEG data were acquired, while participants recited word sequences from memory at a regular pace, using a tongue-twister paradigm. The results revealed fronto-central error-related negativity, previously associated with speech monitoring, which could be distinguished for each word. Its peak amplitude was sensitive to Cycle and Phonological Similarity. However, an effect of sequential production was also observable on baseline measures, indicating baseline shifts throughout the word sequence due to concurrent sustained medial-frontal EEG activity. We also report a late left anterior negativity (LLAN), associated with verbal response planning and execution, onsetting around 100 ms before the first word in each cycle and sustained throughout the rest of the cycle. This work underlines the importance of considering the contribution of transient and sustained EEG activity on ERPs, and provides evidence that ERPs can be used to study sequential word production.

Intracranial recording in patients with aphasia using nanomaterial-based flexible electronics: promises and challenges

In recent years, researchers have studied how nanotechnology could enhance neuroimaging techniques. The application of nanomaterial-based flexible electronics has the potential to advance conventional intracranial electroencephalography (iEEG) by utilising brain-compatible soft nanomaterials. The resultant technique has significantly high spatial and temporal resolution, both of which enhance the localisation of brain functions and the mapping of dynamic language processing. This review presents findings on aphasia, an impairment in language and communication, and discusses how different brain imaging techniques, including positron emission tomography, magnetic resonance imaging, and iEEG, have advanced our understanding of the neural networks underlying language and reading processing. We then outline the strengths and weaknesses of iEEG in studying human cognition and the development of intracranial recordings that use brain-compatible flexible electrodes. We close by discussing the potential advantages and challenges of future investigations adopting nanomaterial-based flexible electronics for intracranial recording in patients with aphasia.

NetDI: Methodology Elucidating the Role of Power and Dynamical Brain Network Features That Underpin Word Production

Canonical language models describe eloquent function as the product of a series of cognitive processes, typically characterized by the independent activation profiles of focal brain regions. In contrast, more recent work has suggested that the interactions between these regions, the cortical networks of language, are critical for understanding speech production. We investigated the cortical basis of picture naming (PN) with human intracranial electrocorticography (ECoG) recordings and direct cortical stimulation (DCS), adjudicating between two competing hypotheses: are task-specific cognitive functions discretely computed within well-localized brain regions or rather by distributed networks? The time resolution of ECoG allows direct comparison of intraregional activation measures [high gamma (h_γ) power] with graph theoretic measures of interregional dynamics. We developed an analysis framework, network dynamics using directed information (NetDI), using information and graph theoretic tools to reveal spatiotemporal dynamics at multiple scales: coarse, intermediate, and fine. Our analysis found novel relationships between the power profiles and network measures during the task. Furthermore, validation using DCS indicates that such network parameters combined with h_γ power are more predictive than h_γ power alone, for identifying critical language regions in the brain. NetDI reveals a high-dimensional space of network dynamics supporting cortical language function, and to account for disruptions to language function observed after neurosurgical resection, traumatic injury, and degenerative disease.

Word frequency effect in written production: Evidence from ERPs and neural oscillations

It has been widely documented that word frequency (WF) modulates language processing in various input and output modalities. WF effect has also been reported in the domain of written production; however, how WF affects written production is a controversial issue. The present study attempts to investigate the time course of and neural oscillation underlying the WF effect in handwritten production. Participants were asked to handwrite pictures names of high versus low WF, while concurrently recording EEG. EEG trials were extracted time-locked to picture onsets and then submitted to event-related potential analysis and time-frequency analysis. WF affected ERPs in the time windows of around 98-160 and 282-360 ms after picture onsets. More importantly, WF modulated the evoked and induced theta-band (4-8 Hz) neural oscillations in the time window of around 36-240 and 244-472 ms, respectively. Considering the time course of language production and the role of theta-band oscillation in long-term memory retrieval, we suggest that the two stages of the WF effect, respectively, reflect conceptual preparation and retrieval of orthographic word-forms in written production.

Lack of selectivity for syntax relative to word meanings throughout the language network

To understand what you are reading now, your mind retrieves the meanings of words and constructions from a linguistic knowledge store (lexico-semantic processing) and identifies the relationships among them to construct a complex meaning (syntactic or combinatorial processing). Do these two sets of processes rely on distinct, specialized mechanisms or, rather, share a common pool of resources? Linguistic theorizing, empirical evidence from language acquisition and processing, and computational modeling have jointly painted a picture whereby lexico-semantic and syntactic processing are deeply inter-connected and perhaps not separable. In contrast, many current proposals of the neural architecture of language continue to endorse a view whereby certain brain regions selectively support syntactic/combinatorial processing, although the locus of such "syntactic hub", and its nature, vary across proposals. Here, we searched for selectivity for syntactic over lexico-semantic processing using a powerful individual-subjects fMRI approach across three sentence comprehension paradigms that have been used in prior work to argue for such selectivity: responses to lexico-semantic vs. morpho-syntactic violations (Experiment 1); recovery from neural suppression across pairs of sentences differing in only lexical items vs. only syntactic structure (Experiment 2); and same/different meaning judgments on such sentence pairs (Experiment 3). Across experiments, both lexico-semantic and syntactic conditions elicited robust responses throughout the left fronto-temporal language network. Critically, however, no regions were more strongly engaged by syntactic than lexico-semantic processing, although some regions showed the opposite pattern. Thus, contra many current proposals of the neural architecture of language, syntactic/combinatorial processing is not separable from lexico-semantic processing at the level of brain regions-or even voxel subsets-within the language network, in line with strong integration between these two processes that has been consistently observed in behavioral and computational language research. The results further suggest that the language network may be generally more strongly concerned with meaning than syntactic form, in line with the primary function of language-to share meanings across minds.

Lack of selectivity for syntax relative to word meanings throughout the language network

To understand what you are reading now, your mind retrieves the meanings of words and constructions from a linguistic knowledge store (lexico-semantic processing) and identifies the relationships among them to construct a complex meaning (syntactic or combinatorial processing). Do these two sets of processes rely on distinct, specialized mechanisms or, rather, share a common pool of resources? Linguistic theorizing, empirical evidence from language acquisition and processing, and computational modeling have jointly painted a picture whereby lexico-semantic and syntactic processing are deeply inter-connected and perhaps not separable. In contrast, many current proposals of the neural architecture of language continue to endorse a view whereby certain brain regions selectively support syntactic/combinatorial processing, although the locus of such "syntactic hub", and its nature, vary across proposals. Here, we searched for selectivity for syntactic over lexico-semantic processing using a powerful individual-subjects fMRI approach across three sentence comprehension paradigms that have been used in prior work to argue for such selectivity: responses to lexico-semantic vs. morpho-syntactic violations (Experiment 1); recovery from neural suppression across pairs of sentences differing in only lexical items vs. only syntactic structure (Experiment 2); and same/different meaning judgments on such sentence pairs (Experiment 3). Across experiments, both lexico-semantic and syntactic conditions elicited robust responses throughout the left fronto-temporal language network. Critically, however, no regions were more strongly engaged by syntactic than lexico-semantic processing, although some regions showed the opposite pattern. Thus, contra many current proposals of the neural architecture of language, syntactic/combinatorial processing is not separable from lexico-semantic processing at the level of brain regions-or even voxel subsets-within the language network, in line with strong integration between these two processes that has been consistently observed in behavioral and computational language research. The results further suggest that the language network may be generally more strongly concerned with meaning than syntactic form, in line with the primary function of language-to share meanings across minds.

A Vector Space Model for Neural Network Functions: Inspirations From Similarities Between the Theory of Connectivity and the Logarithmic Time Course of Word Production

The present report examines the coinciding results of two study groups each presenting a power-of-two function to describe network structures underlying perceptual processes in one case and word production during verbal fluency tasks in the other. The former is theorized as neural cliques organized according to the function N = 2 ⁱ - 1, whereas the latter assumes word conglomerations thinkable as tuples following the function N = 2 ⁱ . Both theories assume the innate optimization of energy efficiency to cause the specific connectivity structure. The vast resemblance between both formulae motivated the development of a common formulation. This was obtained by using a vector space model, in which the configuration of neural cliques or connected words is represented by a N-dimensional state vector. A further analysis of the model showed that the entire time course of word production could be derived using basically one single minimal transformation-matrix. This again seems in line with the principle of maximum energy efficiency.

Electroencephalographic characteristics of word finding during phonological and semantic verbal fluency tasks

Aims

Verbal Fluency is sensitive to brain damage and is employed to assess language abilities like the size of vocabulary and the semantic‐lexical networks’ integrity and executive functioning abilities particularly inhibition, working memory, and self‐monitoring. Various studies revealed oscillatory changes related to word retrieval during different tasks. However, there are not enough studies on electroencephalographic characteristics of word retrieval routes (phonological or semantic pathway) during free recall. The purpose of our study was to investigate electroencephalography power relationship with semantic and phonological word finding routes during verbal fluency.

Methods

In this within‐subject study, the electroencephalography of 20 healthy participants was recorded during written category and letter fluency tasks and compared with the rest state. Absolute power of the signals in delta (1‐3.5 Hz), theta (4‐7.5 Hz), alpha (8‐12 Hz), and beta (12.5‐30 Hz) was calculated in three lobes (frontal, parietal, and temporal).

Results

A repeated measures ANOVA showed significant interaction of condition × lobe × frequency × side (P < .001). Post hoc test for each lobe showed significant changes in the absolute power of delta, theta and beta for frontal, delta and theta for parietal, and theta and beta for temporal lobes (P‐values < .05).

Conclusion

Searching the words by phonological entries is associated with decreased beta and increased theta in left frontal lobe. These changes are not necessary for semantic word retrieval strategy.

Word retrieval either by phonological entries or semantic categories is accompanied by increased delta in frontal and parietal lobes. Decreased beta and increased theta bands in the left frontal lobe are associated with phonological word retrieval strategy while during semantic word finding, increased beta was observed in the left temporal lobe.

Electrocorticography reveals spatiotemporal neuronal activation patterns of verbal fluency in patients with epilepsy

Verbal fluency is commonly used to evaluate cognitive dysfunction in a variety of neuropsychiatric diseases, yet the neurobiology underlying performance of this task is incompletely understood. Electrocorticography (ECoG) provides a unique opportunity to investigate temporal activation patterns during cognitive tasks with high spatial and temporal precision. We used ECoG to study high gamma activity (HGA) patterns in patients undergoing presurgical evaluation for intractable epilepsy as they completed an overt, free-recall verbal fluency task. We examined regions demonstrating changes in HGA during specific timeframes relative to speech onset. Early pre-speech high gamma activity was present in left frontal regions during letter fluency and in bifrontal regions during category fluency. During timeframes typically associated with word planning, a distributed network was engaged including left inferior frontal, orbitofrontal and posterior temporal regions. Peri-Rolandic activation was observed during speech onset, and there was post-speech activation in the bilateral posterior superior temporal regions. Based on these observations in the context of prior studies, we propose a model of neocortical activity patterns underlying verbal fluency.

Network dynamics of Broca's area during word selection

Current models of word-production in Broca's area (i.e. left ventro-lateral prefrontal cortex, VLPFC) posit that sequential and staggered semantic, lexical, phonological and articulatory processes precede articulation. Using millisecond-resolution intra-cranial recordings, we evaluated spatiotemporal dynamics and high frequency functional interconnectivity between left VLPFC regions during single-word production. Through the systematic variation of retrieval, selection, and phonological loads, we identified specific activation profiles and functional coupling patterns between these regions that fit within current psycholinguistic theories of word production. However, network interactions underpinning these processes activate in parallel (not sequentially), while the processes themselves are indexed by specific changes in network state. We found evidence that suggests that pars orbitalis is coupled with pars triangularis during lexical retrieval, while lexical selection is terminated via coupled activity with M1 at articulation onset. Taken together, this work reveals that speech production relies on very specific inter-regional couplings in rapid sequence in the language dominant hemisphere.

Workshops of the Seventh International Brain-Computer Interface Meeting: Not Getting Lost in Translation

The Seventh International Brain-Computer Interface (BCI) Meeting was held May 21-25th, 2018 at the Asilomar Conference Grounds, Pacific Grove, California, United States. The interactive nature of this conference was embodied by 25 workshops covering topics in BCI (also called brain-machine interface) research. Workshops covered foundational topics such as hardware development and signal analysis algorithms, new and imaginative topics such as BCI for virtual reality and multi-brain BCIs, and translational topics such as clinical applications and ethical assumptions of BCI development. BCI research is expanding in the diversity of applications and populations for whom those applications are being developed. BCI applications are moving toward clinical readiness as researchers struggle with the practical considerations to make sure that BCI translational efforts will be successful. This paper summarizes each workshop, providing an overview of the topic of discussion, references for additional information, and identifying future issues for research and development that resulted from the interactions and discussion at the workshop.

Cortical Dynamics of Semantic Priming and Interference during Word Production: An Intracerebral Study

Language production requires that semantic representations are mapped to lexical representations on the basis of the ongoing context to select the appropriate words. This mapping is thought to generate two opposing phenomena, “semantic priming,” where multiple word candidates are activated, and “interference,” where these word activities are differentiated to make a goal-relevant selection. In previous neuroimaging and neurophysiological research, priming and interference have been associated to activity in regions of a left frontotemporal network. Most of such studies relied on recordings that either have high temporal or high spatial resolution, but not both. Here, we employed intracerebral EEG techniques to explore with both high resolutions, the neural activity associated with these phenomena. The data came from nine epileptic patients who were stereotactically implanted for presurgical diagnostics. They performed a cyclic picture-naming task contrasting semantically homogeneous and heterogeneous contexts. Of the 84 brain regions sampled, 39 showed task-evoked activity that was significant and consistent across two patients or more. In nine of these regions, activity was significantly modulated by the semantic manipulation. It was reduced for semantically homogeneous contexts (i.e., priming) in eight of these regions, located in the temporal ventral pathway as well as frontal areas. Conversely, it was increased only in the pre-SMA, notably at an early poststimulus temporal window (200–300 msec) and a preresponse temporal window (700–800 msec). These temporal effects respectively suggest the pre-SMA's role in initial conflict detection (e.g., increased response caution) and in preresponse control. Such roles of the pre-SMA are traditional from a history of neural evidence in simple perceptual tasks, yet are also consistent with recent cognitive lexicosemantic theories that highlight top–down processes in language production. Finally, although no significant semantic modulation was found in the ACC, future intracerebral EEG work should continue to inspect ACC with the pre-SMA.

Speech synthesis from ECoG using densely connected 3D convolutional neural networks

OBJECTIVE

Direct synthesis of speech from neural signals could provide a fast and natural way of communication to people with neurological diseases. Invasively-measured brain activity (electrocorticography; ECoG) supplies the necessary temporal and spatial resolution to decode fast and complex processes such as speech production. A number of impressive advances in speech decoding using neural signals have been achieved in recent years, but the complex dynamics are still not fully understood. However, it is unlikely that simple linear models can capture the relation between neural activity and continuous spoken speech.

APPROACH

Here we show that deep neural networks can be used to map ECoG from speech production areas onto an intermediate representation of speech (logMel spectrogram). The proposed method uses a densely connected convolutional neural network topology which is well-suited to work with the small amount of data available from each participant.

MAIN RESULTS

In a study with six participants, we achieved correlations up to r  =  0.69 between the reconstructed and original logMel spectrograms. We transfered our prediction back into an audible waveform by applying a Wavenet vocoder. The vocoder was conditioned on logMel features that harnessed a much larger, pre-existing data corpus to provide the most natural acoustic output.

SIGNIFICANCE

To the best of our knowledge, this is the first time that high-quality speech has been reconstructed from neural recordings during speech production using deep neural networks.

Morphometric correlates of anomia in patients with small left temporopolar lesions

Visual object naming is a complex cognitive process that engages an interconnected network of cortical regions moving from occipitotemporal to anterior-inferior temporal cortices, and extending into the inferior frontal cortex. Naming can fail for diverse reasons, and different stages of the naming multi-step process appear to be reliant upon the integrity of different neuroanatomical locations. While the neural correlates of semantic errors have been extensively studied, the neural basis of omission errors remains relatively unspecified. Although a strong line of evidence supports an association between anterior temporal lobe damage and semantic errors, there are some studies suggesting that the anterior temporal lobe could be also associated with omissions. However, support for this hypothesis comes from studies with patients in whom damage affected extensive brain regions, sometimes bilaterally. Here, we availed of a group of 12 patients with epilepsy associated with a small lesion at the tip of the left temporal pole. Using an unbiased surface-based morphometry methodology, we correlated two morphological features with errors observed during visual naming. Analyses revealed a correlation between omission errors and reduced local gyrification index in three cortical clusters: one in the left anteromedial temporal lobe region (AMTL) and two in the left anterior cingulate cortex (ACC). Our findings support the view that regions in ACC and AMTL are critical structures within a network engaged in word selection from semantics.

Human Verbal Memory Encoding Is Hierarchically Distributed in a Continuous Processing Stream

Processing of memory is supported by coordinated activity in a network of sensory, association, and motor brain regions. It remains a major challenge to determine where memory is encoded for later retrieval. Here, we used direct intracranial brain recordings from epilepsy patients performing free recall tasks to determine the temporal pattern and anatomical distribution of verbal memory encoding across the entire human cortex. High γ frequency activity (65–115 Hz) showed consistent power responses during encoding of subsequently recalled and forgotten words on a subset of electrodes localized in 16 distinct cortical areas activated in the tasks. More of the high γ power during word encoding, and less power before and after the word presentation, was characteristic of successful recall and observed across multiple brain regions. Latencies of the induced power changes and this subsequent memory effect (SME) between the recalled and forgotten words followed an anatomical sequence from visual to prefrontal cortical areas. Finally, the magnitude of the memory effect was unexpectedly found to be the largest in selected brain regions both at the top and at the bottom of the processing stream. These included the language processing areas of the prefrontal cortex and the early visual areas at the junction of the occipital and temporal lobes. Our results provide evidence for distributed encoding of verbal memory organized along a hierarchical posterior-to-anterior processing stream.

The dual origin of semantic errors in access deficit: activation vs. inhibition deficit

According to most models of language production, to name a picture one must first map semantic features onto lexical items. Even if both sets of representations are intact, problems in mapping semantic to lexical representations can impair production. Individuals with this problem, sometimes referred to as "access deficit", often demonstrate evidence of preserved semantic knowledge (e.g., good comprehension), increased rate of lexical (usually semantic) errors in production, and inconsistent accuracy on naming the same picture on different occasions. In this paper, I argue that access deficit can have two distinct etiologies. I will present a case of double dissociation between two individuals with chronic post-stroke aphasia, one of whom shows a profile compatible with impaired activation of the target lexical item from semantic features (activation deficit), while the other shows a profile compatible with impaired inhibition of competing lexical items (inhibition deficit). These results have three key implications: (a) they provide support for the theoretical separation between activation and selection processes in computational models of word production, (b) they point to the critical role of inhibitory control in lexical selection, and (c) they invite a closer inspection of the origin of semantic errors in individuals with access deficit in order to choose the best treatment option.

Neurolinguistics Research Advancing Development of a Direct-Speech Brain-Computer Interface

A direct-speech brain-computer interface (DS-BCI) acquires neural signals corresponding to imagined speech, then processes and decodes these signals to produce a linguistic output in the form of phonemes, words, or sentences. Recent research has shown the potential of neurolinguistics to enhance decoding approaches to imagined speech with the inclusion of semantics and phonology in experimental procedures. As neurolinguistics research findings are beginning to be incorporated within the scope of DS-BCI research, it is our view that a thorough understanding of imagined speech, and its relationship with overt speech, must be considered an integral feature of research in this field. With a focus on imagined speech, we provide a review of the most important neurolinguistics research informing the field of DS-BCI and suggest how this research may be utilized to improve current experimental protocols and decoding techniques. Our review of the literature supports a cross-disciplinary approach to DS-BCI research, in which neurolinguistics concepts and methods are utilized to aid development of a naturalistic mode of communication.

To select or to wait? The importance of criterion setting in debates of competitive lexical selection

Competitive accounts of lexical selection propose that the activation of competitors slows down the selection of the target. Non-competitive accounts, on the other hand, posit that target response latencies are independent of the activation of competing items. In this paper, we propose a signal detection framework for lexical selection and show how a flexible selection criterion affects claims of competitive selection. Specifically, we review evidence from neurotypical and brain-damaged speakers and demonstrate that task goals and the state of the production system determine whether a competitive or a non-competitive selection profile arises. We end by arguing that there is conclusive evidence for a flexible criterion in lexical selection, and that integrating criterion shifts into models of language production is critical for evaluating theoretical claims regarding (non-)competitive selection.

The cortical dynamics of speaking: Lexical and phonological knowledge simultaneously recruit the frontal and temporal cortex within 200 ms

Language production models typically assume that retrieving a word for articulation is a sequential process with substantial functional delays between conceptual, lexical, phonological and motor processing, respectively. Nevertheless, explicit evidence contrasting the spatiotemporal dynamics between different word production components is scarce. Here, using anatomically constrained magnetoencephalography during overt meaningful speech production, we explore the speed with which lexico-semantic versus acoustic-articulatory information of a to-be-uttered word become first neurophysiologically manifest in the cerebral cortex. We demonstrate early modulations of brain activity by the lexical frequency of a word in the temporal cortex and the left inferior frontal gyrus, simultaneously with activity in the motor and the posterior superior temporal cortex reflecting articulatory-acoustic phonological features (+LABIAL vs. +CORONAL) of the word-initial speech sounds (e.g., Monkey vs. Donkey). The specific nature of the spatiotemporal pattern correlating with a word's frequency and initial phoneme demonstrates that, in the course of speech planning, lexico-semantic and phonological-articulatory processes emerge together rapidly, drawing in parallel on temporal and frontal cortex. This novel finding calls for revisions of current brain language theories of word production.