Somatotopic mapping of natural upper- and lower-extremity movements and speech production with high gamma electrocorticography

Motor somatotopy impacts imagery strategy success in human intracortical brain-computer interfaces

Objective:The notion of a somatotopically organized motor cortex, with movements of different body parts being controlled by spatially distinct areas of cortex, is well known. However, recent studies have challenged this notion and suggested a more distributed representation of movement control. This shift in perspective has significant implications, particularly when considering the implantation location of electrode arrays for intracortical brain-computer interfaces (iBCIs). We sought to evaluate whether the location of neural recordings from the precentral gyrus, and thus the underlying somatotopy, has any impact on the imagery strategies that can enable successful iBCI control.Approach:Three individuals with a spinal cord injury were enrolled in an ongoing clinical trial of an iBCI. Participants had two intracortical microelectrode arrays implanted in the arm and/or hand areas of the precentral gyrus based on presurgical functional imaging. Neural data were recorded while participants attempted to perform movements of the hand, wrist, elbow, and shoulder.Main results:We found that electrode arrays that were located more medially recorded significantly more activity during attempted proximal arm movements (elbow, shoulder) than did lateral arrays, which captured more activity related to attempted distal arm movements (hand, wrist). We also evaluated the relative contribution from the two arrays implanted in each participant to decoding accuracy during calibration of an iBCI decoder for translation and grasping tasks. For both task types, imagery strategy (e.g. reaching vs wrist movements) had a significant impact on the relative contributions of each array to decoding. Overall, we found some evidence of broad tuning to arm and hand movements; however, there was a clear bias in the amount of information accessible about each movement type in spatially distinct areas of cortex.Significance:These results demonstrate that classical concepts of somatotopy can have real consequences for iBCI use, and highlight the importance of considering somatotopy when planning iBCI implantation.

A mosaic of whole-body representations in human motor cortex

Understanding how the body is represented in motor cortex is key to understanding how the brain controls movement. The precentral gyrus (PCG) has long been thought to contain largely distinct regions for the arm, leg and face (represented by the "motor homunculus"). However, mounting evidence has begun to reveal a more intermixed, interrelated and broadly tuned motor map. Here, we revisit the motor homunculus using microelectrode array recordings from 20 arrays that broadly sample PCG across 8 individuals, creating a comprehensive map of human motor cortex at single neuron resolution. We found whole-body representations throughout all sampled points of PCG, contradicting traditional leg/arm/face boundaries. We also found two speech-preferential areas with a broadly tuned, orofacial-dominant area in between them, previously unaccounted for by the homunculus. Throughout PCG, movement representations of the four limbs were interlinked, with homologous movements of different limbs (e.g., toe curl and hand close) having correlated representations. Our findings indicate that, while the classic homunculus aligns with each area's preferred body region at a coarse level, at a finer scale, PCG may be better described as a mosaic of functional zones, each with its own whole-body representation.

Motor somatotopy impacts imagery strategy success in human intracortical brain-computer interfaces

The notion of a somatotopically organized motor cortex, with movements of different body parts being controlled by spatially distinct areas of cortex, is well known. However, recent studies have challenged this notion and suggested a more distributed representation of movement control. This shift in perspective has significant implications, particularly when considering the implantation location of electrode arrays for intracortical brain-computer interfaces (iBCIs). We sought to evaluate whether the location of neural recordings from the precentral gyrus, and thus the underlying somatotopy, has any impact on the imagery strategies that can enable successful iBCI control. Three individuals with a spinal cord injury were enrolled in an ongoing clinical trial of an iBCI. Participants had two intracortical microelectrode arrays implanted in the arm and/or hand areas of the precentral gyrus based on presurgical functional imaging. Neural data were recorded while participants attempted to perform movements of the hand, wrist, elbow, and shoulder. We found that electrode arrays that were located more medially recorded significantly more activity during attempted proximal arm movements (elbow, shoulder) than did lateral arrays, which captured more activity related to attempted distal arm movements (hand, wrist). We also evaluated the relative contribution from the two arrays implanted in each participant to decoding accuracy during calibration of an iBCI decoder for translation and grasping tasks. For both task types, imagery strategy (e.g., reaching vs. wrist movements) had a significant impact on the relative contributions of each array to decoding. Overall, we found some evidence of broad tuning to arm and hand movements; however, there was a clear bias in the amount of information accessible about each movement type in spatially distinct areas of cortex. These results demonstrate that classical concepts of somatotopy can have real consequences for iBCI use, and highlight the importance of considering somatotopy when planning iBCI implantation.

Characterization of High-Gamma Activity in Electrocorticographic Signals

Introduction

Electrocorticographic (ECoG) high-gamma activity (HGA) is a widely recognized and robust neural correlate of cognition and behavior. However, fundamental signal properties of HGA, such as the high-gamma frequency band or temporal dynamics of HGA, have never been systematically characterized. As a result, HGA estimators are often poorly adjusted, such that they miss valuable physiological information.

Methods

To address these issues, we conducted a thorough qualitative and quantitative characterization of HGA in ECoG signals. Our study is based on ECoG signals recorded from 18 epilepsy patients while performing motor control, listening, and visual perception tasks. In this study, we first categorize HGA into HGA types based on the cognitive/behavioral task. For each HGA type, we then systematically quantify three fundamental signal properties of HGA: the high-gamma frequency band, the HGA bandwidth, and the temporal dynamics of HGA.

Results

The high-gamma frequency band strongly varies across subjects and across cognitive/behavioral tasks. In addition, HGA time courses have lowpass character, with transients limited to 10 Hz. The task-related rise time and duration of these HGA time courses depend on the individual subject and cognitive/behavioral task. Task-related HGA amplitudes are comparable across the investigated tasks.

Discussion

This study is of high practical relevance because it provides a systematic basis for optimizing experiment design, ECoG acquisition and processing, and HGA estimation. Our results reveal previously unknown characteristics of HGA, the physiological principles of which need to be investigated in further studies.

A Study of Word Complexity Under Conditions of Non-experimental, Natural Overt Speech Production Using ECoG

The linguistic complexity of words has largely been studied on the behavioral level and in experimental settings. Only little is known about the neural processes underlying it in uninstructed, spontaneous conversations. We built up a multimodal neurolinguistic corpus composed of synchronized audio, video, and electrocorticographic (ECoG) recordings from the fronto-temporo-parietal cortex to address this phenomenon based on uninstructed, spontaneous speech production. We performed extensive linguistic annotations of the language material and calculated word complexity using several numeric parameters. We orthogonalized the parameters with the help of a linear regression model. Then, we correlated the spectral components of neural activity with the individual linguistic parameters and with the residuals of the linear regression model, and compared the results. The proportional relation between the number of consonants and vowels, which was the most informative parameter with regard to the neural representation of word complexity, showed effects in two areas: the frontal one was at the junction of the premotor cortex, the prefrontal cortex, and Brodmann area 44. The postcentral one lay directly above the lateral sulcus and comprised the ventral central sulcus, the parietal operculum and the adjacent inferior parietal cortex. Beyond the physiological findings summarized here, our methods may be useful for those interested in ways of studying neural effects related to natural language production and in surmounting the intrinsic problem of collinearity between multiple features of spontaneously spoken material.

High-gamma modulation language mapping with stereo-EEG: A novel analytic approach and diagnostic validation

OBJECTIVE

A novel analytic approach for task-related high-gamma modulation (HGM) in stereo-electroencephalography (SEEG) was developed and evaluated for language mapping.

METHODS

SEEG signals, acquired from drug-resistant epilepsy patients during a visual naming task, were analyzed to find clusters of 50-150 Hz power modulations in time-frequency domain. Classifier models to identify electrode contacts within the reference neuroanatomy and electrical stimulation mapping (ESM) speech/language sites were developed and validated.

RESULTS

In 21 patients (9 females), aged 4.8-21.2 years, SEEG HGM model predicted electrode locations within Neurosynth language parcels with high diagnostic odds ratio (DOR 10.9, p < 0.0001), high specificity (0.85), and fair sensitivity (0.66). Another SEEG HGM model classified ESM speech/language sites with significant DOR (5.0, p < 0.0001), high specificity (0.74), but insufficient sensitivity. Time to largest power change reliably localized electrodes within Neurosynth language parcels, while, time to center-of-mass power change identified ESM sites.

CONCLUSIONS

SEEG HGM mapping can accurately localize neuroanatomic and ESM language sites.

SIGNIFICANCE

Predictive modelling incorporating time, frequency, and magnitude of power change is a useful methodology for task-related HGM, which offers insights into discrepancies between HGM language maps and neuroanatomy or ESM.

An Energy-Efficient CMOS Dual-Mode Array Architecture for High-Density ECoG-Based Brain-Machine Interfaces

This article presents an energy-efficient electrocorticography (ECoG) array architecture for fully-implantable brain machine interface systems. A novel dual-mode analog signal processing method is introduced that extracts neural features from high- γ band (80-160 Hz) at the early stages of signal acquisition. Initially, brain activity across the full-spectrum is momentarily observed to compute the feature weights in the digital back-end during full-band mode operation. Subsequently, these weights are fed back to the front-end and the system reverts to base-band mode to perform feature extraction. This approach utilizes a distinct optimized signal pathway based on power envelope extraction, resulting in 1.72× power reduction in the analog blocks and up to 50× potential power savings for digitization and processing (implemented off-chip in this article). A prototype incorporating a 32-channel ultra-low power signal acquisition front-end is fabricated in 180 nm CMOS process with 0.8 V supply. This chip consumes 1.05  μW (0.205  μW for feature extraction only) power and occupies 0.245 [Formula: see text] die area per channel. The chip measurement shows better than 76.5-dB common-mode rejection ratio (CMRR), 4.09 noise efficiency factor (NEF), and 10.04 power efficiency factor (PEF). In-vivo human tests have been carried out with electroencephalography and ECoG signals to validate the performance and dual-mode operation in comparison to commercial acquisition systems.

Hand Knob Area of Premotor Cortex Represents the Whole Body in a Compositional Way

Decades after the motor homunculus was first proposed, it is still unknown how different body parts are intermixed and interrelated in human motor cortical areas at single-neuron resolution. Using multi-unit recordings, we studied how face, head, arm, and leg movements are represented in the hand knob area of premotor cortex (precentral gyrus) in people with tetraplegia. Contrary to traditional expectations, we found strong representation of all movements and a partially "compositional" neural code that linked together all four limbs. The code consisted of (1) a limb-coding component representing the limb to be moved and (2) a movement-coding component where analogous movements from each limb (e.g., hand grasp and toe curl) were represented similarly. Compositional coding might facilitate skill transfer across limbs, and it provides a useful framework for thinking about how the motor system constructs movement. Finally, we leveraged these results to create a whole-body intracortical brain-computer interface that spreads targets across all limbs.

Electrocorticographic Encoding of Human Gait in the Leg Primary Motor Cortex

While prior noninvasive (e.g., electroencephalographic) studies suggest that the human primary motor cortex (M1) is active during gait processes, the limitations of noninvasive recordings make it impossible to determine whether M1 is involved in high-level motor control (e.g., obstacle avoidance, walking speed), low-level motor control (e.g., coordinated muscle activation), or only nonmotor processes (e.g., integrating/relaying sensory information). This study represents the first invasive electroneurophysiological characterization of the human leg M1 during walking. Two subjects with an electrocorticographic grid over the interhemispheric M1 area were recruited. Both exhibited generalized γ-band (40-200 Hz) synchronization across M1 during treadmill walking, as well as periodic γ-band changes within each stride (across multiple walking speeds). Additionally, these changes appeared to be of motor, rather than sensory, origin. However, M1 activity during walking shared few features with M1 activity during individual leg muscle movements, and was not highly correlated with lower limb trajectories on a single channel basis. These findings suggest that M1 primarily encodes high-level gait motor control (i.e., walking duration and speed) instead of the low-level patterns of leg muscle activation or movement trajectories. Therefore, M1 likely interacts with subcortical/spinal networks, which are responsible for low-level motor control, to produce normal human walking.

Spatiotemporal dynamics of auditory and picture naming-related high-gamma modulations: A study of Japanese-speaking patients

OBJECTIVE

To characterize the spatiotemporal dynamics of auditory and picture naming-related cortical activation in Japanese-speaking patients.

METHODS

Ten patients were assigned auditory naming and picture naming tasks during extraoperative intracranial EEG recording in a tertiary epilepsy center. Time-frequency analysis determined at what electrode sites and at what time windows during each task the amplitude of high-gamma activity (65-95 Hz) was modulated.

RESULTS

The superior-temporal gyrus on each hemisphere showed high-gamma augmentation during sentence listening, whereas the left middle-temporal and inferior-frontal gyri showed high-gamma augmentation peaking around stimulus offset. Auditory naming-specific high-gamma augmentation was noted in the bilateral superior-temporal gyri as well as left frontal-parietal-temporal perisylvian network regions, whereas picture naming-specific augmentation was noted in the occipital-fusiform regions, bilaterally. The inferior pre- and postcentral gyri on each hemisphere showed modality-common high-gamma augmentation time-locked to overt responses.

CONCLUSIONS

The spatiotemporal dynamics of auditory and picture naming-related high-gamma augmentation in Japanese-speaking patients were qualitatively similar to those previously reported in studies of English-speaking patients.

SIGNIFICANCE

The cortical dynamics for auditory sentence recognition are at least partly shared by cohorts speaking two distinct languages. Multicenter studies regarding the clinical utility of high-gamma language mapping across Eastern and Western hemispheres may be feasible.

Human motor cortex relies on sparse and action-specific activation during laughing, smiling and speech production

Smiling, laughing, and overt speech production are fundamental to human everyday communication. However, little is known about how the human brain achieves the highly accurate and differentiated control of such orofacial movement during natural conditions. Here, we utilized the high spatiotemporal resolution of subdural recordings to elucidate how human motor cortex is functionally engaged during control of real-life orofacial motor behaviour. For each investigated movement class-lip licking, speech production, laughing and smiling-our findings reveal a characteristic brain activity pattern within the mouth motor cortex with both spatial segregation and overlap between classes. Our findings thus show that motor cortex relies on sparse and action-specific activation during real-life orofacial behaviour, apparently organized in distinct but overlapping subareas that control different types of natural orofacial movements.

Development of information sharing in language neocortex in childhood-onset drug-resistant epilepsy

OBJECTIVE

We studied age-related dynamics of information sharing among cortical language regions with electrocorticographic high-gamma modulation during picture-naming and story-listening tasks.

METHODS

Seventeen epilepsy patients aged 4-19 years, undergoing extraoperative monitoring with left-hemispheric subdural electrodes, were included. Mutual information (MI), a nondirectional measure of shared information, between 16 pairs of cortical regions of interest, was computed from trial-averaged 70-150 Hz power modulations during language tasks. Impact of age on pairwise MI between language regions and their determinants were ascertained with regression analysis.

RESULTS

During picture naming, significant increase in MI with age was seen between pairwise combinations of Broca's area, inferior precentral gyrus (iPreC), and frontal association cortex (FAC); Wernicke's area and posterior association cortex (PAC); and Broca's and Wernicke's areas. During story listening, significant age-related increase in MI was seen between Wernicke's area and either Broca's area, FAC, or PAC; and between Broca's area and FAC. Significant impact of baseline intelligence quotient was seen on the relationship between age and MI for all pairs, except between Broca's area and iPreC. The mean MI was higher during naming compared to listening for pairs including iPreC with Broca's area, FAC, or PAC and was lower for pairs of Wernicke's area or PAC with anterior language regions.

SIGNIFICANCE

Information sharing matures with age "within" frontal and temporoparietal language cortices, and "between" Broca's and Wernicke's areas. This study provides evidence for distinct patterns of developmental plasticity within perisylvian language cortex and has implications for planning epilepsy surgery.

Neurolinguistics Research Advancing Development of a Direct-Speech Brain-Computer Interface

A direct-speech brain-computer interface (DS-BCI) acquires neural signals corresponding to imagined speech, then processes and decodes these signals to produce a linguistic output in the form of phonemes, words, or sentences. Recent research has shown the potential of neurolinguistics to enhance decoding approaches to imagined speech with the inclusion of semantics and phonology in experimental procedures. As neurolinguistics research findings are beginning to be incorporated within the scope of DS-BCI research, it is our view that a thorough understanding of imagined speech, and its relationship with overt speech, must be considered an integral feature of research in this field. With a focus on imagined speech, we provide a review of the most important neurolinguistics research informing the field of DS-BCI and suggest how this research may be utilized to improve current experimental protocols and decoding techniques. Our review of the literature supports a cross-disciplinary approach to DS-BCI research, in which neurolinguistics concepts and methods are utilized to aid development of a naturalistic mode of communication.

The contribution of different frequency bands in class separability of covert speech tasks for BCIs

Several recent studies demonstrate the possibility of using user initiated covert speech mental tasks in brain computer interfaces with varying degrees of success, but details of the best frequency features had not been investigated. In this work, ten volunteers in the age range of 22-70 years participated in the experiment. Eight of them were neurologically healthy, one user was dyslexic, and another was autistic. The four words "back", "forward", "left", and "right" were shortened into "BA", "FO", "LE", and "RY", which are phonetically dissimilar and cognitively relevant directional commands. Participants were asked to covertly speak each as soon as the letters appeared on a screen. Volunteers completed five recording runs. During each run the four words were presented in random succession to avoid sequence bias. The recorded EEG data from the ten users were analysed to discover the best features within a Gabor Transform of the signals, i.e., those yielding the highest word-pair classification accuracy for this specific type of linguistic mental activity. Using this BCI, suitable class separability of covert speech tasks is confirmed for all, including disabled users, with consistently high classification accuracy from 72% to 88% in all cases. Like motor imagery tasks, Alpha and Beta band activity were found to contain 12% and 31% of the most important features respectively. Gamma band activity, which indicates high mental functions, contains 57% of the most important features in this study.

Real-life speech production and perception have a shared premotor-cortical substrate

Motor-cognitive accounts assume that the articulatory cortex is involved in language comprehension, but previous studies may have observed such an involvement as an artefact of experimental procedures. Here, we employed electrocorticography (ECoG) during natural, non-experimental behavior combined with electrocortical stimulation mapping to study the neural basis of real-life human verbal communication. We took advantage of ECoG’s ability to capture high-gamma activity (70–350 Hz) as a spatially and temporally precise index of cortical activation during unconstrained, naturalistic speech production and perception conditions. Our findings show that an electrostimulation-defined mouth motor region located in the superior ventral premotor cortex is consistently activated during both conditions. This region became active early relative to the onset of speech production and was recruited during speech perception regardless of acoustic background noise. Our study thus pinpoints a shared ventral premotor substrate for real-life speech production and perception with its basic properties.

ECoG high-gamma modulation versus electrical stimulation for presurgical language mapping

OBJECTIVE

This meta-analysis compared diagnostic validity of electrocorticographic (ECoG) high-γ modulation (HGM) with electrical stimulation mapping (ESM) for presurgical language localization.

METHODS

From a structured literature search, studies with electrode level data comparing ECoG HGM and ESM for language localization were included in the meta-analysis. Outcomes included global measures of diagnostic validity: area under the summary receiver operating characteristic (SROC) curve (AUC), and diagnostic odds ratio (DOR); as well as pooled estimates of sensitivity and specificity. Clinical and technical determinants of sensitivity/specificity were explored.

RESULTS

Fifteen studies were included in qualitative synthesis, and 10 studies included in the meta-analysis (number of patients 1-17, mean age 10.3-53.6years). Overt picture naming was the most commonly used task for language mapping with either method. Electrocorticographic high-γ modulation was analyzed at 50-400Hz with different bandwidths in individual studies. For ESM, pulse duration, train duration, and maximum current varied greatly among studies. Sensitivity (0.23-0.99), specificity (0.48-0.96), and DOR (1.45-376.28) varied widely across studies. The pooled estimates are: sensitivity 0.61 (95% CI 0.44, 0.76), specificity 0.79 (95% CI 0.68, 0.88), and DOR 6.44 (95% CI 3.47, 11.94). Area under the SROC curve was 0.77. Results of bivariate meta-regression were limited by small samples for individual variables.

CONCLUSION

Electrocorticographic high-γ modulation is a specific but not sensitive method for language localization compared with gold-standard ESM. Given the pooled DOR of 6.44 and AUC of 0.77, ECoG HGM can fairly reliably ascertain electrodes overlying ESM cortical language sites.

Presurgical language mapping using event-related high-gamma activity: The Detroit procedure

A number of investigators have reported that event-related augmentation of high-gamma activity at 70-110 Hz on electrocorticography (ECoG) can localize functionally-important brain regions in children and adults who undergo epilepsy surgery. The advantages of ECoG-based language mapping over the gold-standard stimulation include: (i) lack of stimulation-induced seizures, (ii) better sensitivity of localization of language areas in young children, and (iii) shorter patient participant time. Despite its potential utility, ECoG-based language mapping is far less commonly practiced than stimulation mapping. Here, we have provided video presentations to explain, point-by-point, our own hardware setting and time-frequency analysis procedures. We also have provided standardized auditory stimuli, in multiple languages, ready to be used for ECoG-based language mapping. Finally, we discussed the technical aspects of ECoG-based mapping, including its pitfalls, to facilitate appropriate interpretation of the data.

Three- and four-dimensional mapping of speech and language in patients with epilepsy

We have provided 3-D and 4D mapping of speech and language function based upon the results of direct cortical stimulation and event-related modulation of electrocorticography signals. Patients estimated to have right-hemispheric language dominance were excluded. Thus, 100 patients who underwent two-stage epilepsy surgery with chronic electrocorticography recording were studied. An older group consisted of 84 patients at least 10 years of age (7367 artefact-free non-epileptic electrodes), whereas a younger group included 16 children younger than age 10 (1438 electrodes). The probability of symptoms transiently induced by electrical stimulation was delineated on a 3D average surface image. The electrocorticography amplitude changes of high-gamma (70-110 Hz) and beta (15-30 Hz) activities during an auditory-naming task were animated on the average surface image in a 4D manner. Thereby, high-gamma augmentation and beta attenuation were treated as summary measures of cortical activation. Stimulation data indicated the causal relationship between (i) superior-temporal gyrus of either hemisphere and auditory hallucination; (ii) left superior-/middle-temporal gyri and receptive aphasia; (iii) widespread temporal/frontal lobe regions of the left hemisphere and expressive aphasia; and (iv) bilateral precentral/left posterior superior-frontal regions and speech arrest. On electrocorticography analysis, high-gamma augmentation involved the bilateral superior-temporal and precentral gyri immediately following question onset; at the same time, high-gamma activity was attenuated in the left orbitofrontal gyrus. High-gamma activity was augmented in the left temporal/frontal lobe regions, as well as left inferior-parietal and cingulate regions, maximally around question offset, with high-gamma augmentation in the left pars orbitalis inferior-frontal, middle-frontal, and inferior-parietal regions preceded by high-gamma attenuation in the contralateral homotopic regions. Immediately before verbal response, high-gamma augmentation involved the posterior superior-frontal and pre/postcentral regions, bilaterally. Beta-attenuation was spatially and temporally correlated with high-gamma augmentation in general but with exceptions. The younger and older groups shared similar spatial-temporal profiles of high-gamma and beta modulation; except, the younger group failed to show left-dominant activation in the rostral middle-frontal and pars orbitalis inferior-frontal regions around stimulus offset. The human brain may rapidly and alternately activate and deactivate cortical areas advantageous or obtrusive to function directed toward speech and language at a given moment. Increased left-dominant activation in the anterior frontal structures in the older age group may reflect developmental consolidation of the language system. The results of our functional mapping may be useful in predicting, across not only space but also time and patient age, sites specific to language function for presurgical evaluation of focal epilepsy.

Spatio-temporal dynamics of working memory maintenance and scanning of verbal information

OBJECTIVE

During verbal communication, humans briefly maintain mental representations of speech sounds conveying verbal information, and constantly scan these representations for comparison to incoming information. We determined the spatio-temporal dynamics of such short-term maintenance and subsequent scanning of verbal information, by intracranially measuring high-gamma activity at 70-110Hz during a working memory task.

METHODS

Patients listened to a stimulus set of two or four spoken letters and were instructed to remember those letters over a two-second interval, following which they were asked to determine if a subsequent target letter had been presented earlier in that trial's stimulus set.

RESULTS

Auditory presentation of letter stimuli sequentially elicited high-gamma augmentation bilaterally in the superior-temporal and pre-central gyri. During the two-second maintenance period, high-gamma activity was augmented in the left pre-central gyrus, and this effect was larger during the maintenance of stimulus sets consisting of four compared to two letters. During the scanning period following target presentation, high-gamma augmentation involved the left inferior-frontal and supra-marginal gyri.

CONCLUSIONS

Short-term maintenance of verbal information is, at least in part, supported by the left pre-central gyrus, whereas scanning by the left inferior-frontal and supra-marginal gyri.

SIGNIFICANCE

The cortical structures involved in short-term maintenance and scanning of speech stimuli were segregated with an excellent temporal resolution.

Spatio-Temporal Progression of Cortical Activity Related to Continuous Overt and Covert Speech Production in a Reading Task

How the human brain plans, executes, and monitors continuous and fluent speech has remained largely elusive. For example, previous research has defined the cortical locations most important for different aspects of speech function, but has not yet yielded a definition of the temporal progression of involvement of those locations as speech progresses either overtly or covertly. In this paper, we uncovered the spatio-temporal evolution of neuronal population-level activity related to continuous overt speech, and identified those locations that shared activity characteristics across overt and covert speech. Specifically, we asked subjects to repeat continuous sentences aloud or silently while we recorded electrical signals directly from the surface of the brain (electrocorticography (ECoG)). We then determined the relationship between cortical activity and speech output across different areas of cortex and at sub-second timescales. The results highlight a spatio-temporal progression of cortical involvement in the continuous speech process that initiates utterances in frontal-motor areas and ends with the monitoring of auditory feedback in superior temporal gyrus. Direct comparison of cortical activity related to overt versus covert conditions revealed a common network of brain regions involved in speech that may implement orthographic and phonological processing. Our results provide one of the first characterizations of the spatiotemporal electrophysiological representations of the continuous speech process, and also highlight the common neural substrate of overt and covert speech. These results thereby contribute to a refined understanding of speech functions in the human brain.

Unsupervised Decoding of Long-Term, Naturalistic Human Neural Recordings with Automated Video and Audio Annotations

Fully automated decoding of human activities and intentions from direct neural recordings is a tantalizing challenge in brain-computer interfacing. Implementing Brain Computer Interfaces (BCIs) outside carefully controlled experiments in laboratory settings requires adaptive and scalable strategies with minimal supervision. Here we describe an unsupervised approach to decoding neural states from naturalistic human brain recordings. We analyzed continuous, long-term electrocorticography (ECoG) data recorded over many days from the brain of subjects in a hospital room, with simultaneous audio and video recordings. We discovered coherent clusters in high-dimensional ECoG recordings using hierarchical clustering and automatically annotated them using speech and movement labels extracted from audio and video. To our knowledge, this represents the first time techniques from computer vision and speech processing have been used for natural ECoG decoding. Interpretable behaviors were decoded from ECoG data, including moving, speaking and resting; the results were assessed by comparison with manual annotation. Discovered clusters were projected back onto the brain revealing features consistent with known functional areas, opening the door to automated functional brain mapping in natural settings.

Cortical subnetwork dynamics during human language tasks

Language tasks require the coordinated activation of multiple subnetworks-groups of related cortical interactions involved in specific components of task processing. Although electrocorticography (ECoG) has sufficient temporal and spatial resolution to capture the dynamics of event-related interactions between cortical sites, it is difficult to decompose these complex spatiotemporal patterns into functionally discrete subnetworks without explicit knowledge of each subnetwork's timing. We hypothesized that subnetworks corresponding to distinct components of task-related processing could be identified as groups of interactions with co-varying strengths. In this study, five subjects implanted with ECoG grids over language areas performed word repetition and picture naming. We estimated the interaction strength between each pair of electrodes during each task using a time-varying dynamic Bayesian network (tvDBN) model constructed from the power of high gamma (70-110Hz) activity, a surrogate for population firing rates. We then reduced the dimensionality of this model using principal component analysis (PCA) to identify groups of interactions with co-varying strengths, which we term functional network components (FNCs). This data-driven technique estimates both the weight of each interaction's contribution to a particular subnetwork, and the temporal profile of each subnetwork's activation during the task. We found FNCs with temporal and anatomical features consistent with articulatory preparation in both tasks, and with auditory and visual processing in the word repetition and picture naming tasks, respectively. These FNCs were highly consistent between subjects with similar electrode placement, and were robust enough to be characterized in single trials. Furthermore, the interaction patterns uncovered by FNC analysis correlated well with recent literature suggesting important functional-anatomical distinctions between processing external and self-produced speech. Our results demonstrate that subnetwork decomposition of event-related cortical interactions is a powerful paradigm for interpreting the rich dynamics of large-scale, distributed cortical networks during human cognitive tasks.

CortiQ-based Real-Time Functional Mapping for Epilepsy Surgery

PURPOSE

To evaluate the use of the cortiQ-based mapping system (g.tec medication engineering GmbH, Austria) for real-time functional mapping (RTFM) and to compare it to results from electrical cortical stimulation mapping (ESM) and functional magnetic resonance imaging (fMRI).

METHODS

Electrocorticographic activity was recorded in 3 male patients with intractable epilepsy by using cortiQ mapping system and analyzed in real time. Activation related to motor, sensory, and receptive language tasks was determined by evaluating the power of the high gamma frequency band (60-170 Hz). The sensitivity and specificity of RTFM were tested against ESM and fMRI results.

RESULTS

"Next-neighbor" approach demonstrated [sensitivity/specificity %] (1) RTFM against ESM: 100.00/79.70 for hand motor; 100.00/73.87 for hand sensory; -/87 for language (it was not identified by the ESM); (2) RTFM against fMRI: 100.00/84.4 for hand motor; 66.70/85.35 for hand sensory; and 87.85/77.70 for language.

CONCLUSIONS

The results of the quantitative "next-neighbor" RTFM evaluation were concordant to those from ESM and fMRI. The RTFM correlates well with localization of hand motor function provided by ESM and fMRI, which may offer added localization in the operating room and guidance for extraoperative ESM mapping. Real-time functional mapping correlates with fMRI language activation when ESM findings are negative. It has fewer limitations than ESM and greater flexibility in activation paradigms and measuring responses.

Spatial-temporal functional mapping of language at the bedside with electrocorticography

OBJECTIVE

To investigate the feasibility and clinical utility of using passive electrocorticography (ECoG) for online spatial-temporal functional mapping (STFM) of language cortex in patients being monitored for epilepsy surgery.

METHODS

We developed and tested an online system that exploits ECoG's temporal resolution to display the evolution of statistically significant high gamma (70-110 Hz) responses across all recording sites activated by a discrete cognitive task. We illustrate how this spatial-temporal evolution can be used to study the function of individual recording sites engaged during different language tasks, and how this approach can be particularly useful for mapping eloquent cortex.

RESULTS

Using electrocortical stimulation mapping (ESM) as the clinical gold standard for localizing language cortex, the average sensitivity and specificity of online STFM across 7 patients were 69.9% and 83.5%, respectively. Moreover, relative to regions of interest where discrete cortical lesions have most reliably caused language impairments in the literature, the sensitivity of STFM was significantly greater than that of ESM, while its specificity was also greater than that of ESM, though not significantly so.

CONCLUSIONS

This study supports the feasibility and clinical utility of online STFM for mapping human language function, particularly under clinical circumstances in which time is limited and comprehensive ESM is impractical.

Language mapping using high gamma electrocorticography, fMRI, and TMS versus electrocortical stimulation

OBJECTIVE

The aim of the present study was to compare localization of the language cortex using cortical stimulation mapping (CSM), high gamma electrocorticography (hgECoG), functional magnetic resonance imaging (fMRI), and transcranial magnetic stimulation (TMS).

METHODS

Language mapping using CSM, hgECoG, fMRI, and TMS were compared in nine patients with epilepsy. Considering CSM as reference, we compared language mapping approaches based on hgECoG, fMRI, and TMS using their sensitivity, specificity, and the results of receiver operating characteristic (ROC) analyses.

RESULTS

Our results show that areas involved in language processing can be identified by hgECoG, fMRI, and TMS. The average sensitivity/specificity of hgECoG, fMRI, and TMS across all patients was 100%/85%, 50%/80%, and 67%/66%, respectively. The average area under the ROC curve of hgECoG, fMRI, and TMS across CSM-positive patients was 0.98, 0.76, and 0.68, respectively.

CONCLUSIONS

There is considerable concordance between CSM, hgECoG, fMRI, and TMS language mapping. Our results reveal that hgECoG, fMRI, and TMS are valuable tools for presurgical language mapping.

SIGNIFICANCE

Language mapping on the basis of hgECoG, fMRI, and TMS can provide important additional information, therefore, these methods can be used in conjunction with CSM or as an alternative, when the latter is deemed impractical.

Neuro-oscillatory phase alignment drives speeded multisensory response times: an electro-corticographic investigation

Even simple tasks rely on information exchange between functionally distinct and often relatively distant neuronal ensembles. Considerable work indicates oscillatory synchronization through phase alignment is a major agent of inter-regional communication. In the brain, different oscillatory phases correspond to low- and high-excitability states. Optimally aligned phases (or high-excitability states) promote inter-regional communication. Studies have also shown that sensory stimulation can modulate or reset the phase of ongoing cortical oscillations. For example, auditory stimuli can reset the phase of oscillations in visual cortex, influencing processing of a simultaneous visual stimulus. Such cross-regional phase reset represents a candidate mechanism for aligning oscillatory phase for inter-regional communication. Here, we explored the role of local and inter-regional phase alignment in driving a well established behavioral correlate of multisensory integration: the redundant target effect (RTE), which refers to the fact that responses to multisensory inputs are substantially faster than to unisensory stimuli. In a speeded detection task, human epileptic patients (N = 3) responded to unisensory (auditory or visual) and multisensory (audiovisual) stimuli with a button press, while electrocorticography was recorded over auditory and motor regions. Visual stimulation significantly modulated auditory activity via phase reset in the delta and theta bands. During the period between stimulation and subsequent motor response, transient synchronization between auditory and motor regions was observed. Phase synchrony to multisensory inputs was faster than to unisensory stimulation. This sensorimotor phase alignment correlated with behavior such that stronger synchrony was associated with faster responses, linking the commonly observed RTE with phase alignment across a sensorimotor network.

New whole-body sensory-motor gradients revealed using phase-locked analysis and verified using multivoxel pattern analysis and functional connectivity

Topographic organization is one of the main principles of organization in the human brain. Specifically, whole-brain topographic mapping using spectral analysis is responsible for one of the greatest advances in vision research. Thus, it is intriguing that although topography is a key feature also in the motor system, whole-body somatosensory-motor mapping using spectral analysis has not been conducted in humans outside M1/SMA. Here, using this method, we were able to map a homunculus in the globus pallidus, a key target area for deep brain stimulation, which has not been mapped noninvasively or in healthy subjects. The analysis clarifies contradictory and partial results regarding somatotopy in the caudal-cingulate zone and rostral-cingulate zone in the medial wall and in the putamen. Most of the results were confirmed at the single-subject level and were found to be compatible with results from animal studies. Using multivoxel pattern analysis, we could predict movements of individual body parts in these homunculi, thus confirming that they contain somatotopic information. Using functional connectivity, we demonstrate interhemispheric functional somatotopic connectivity of these homunculi, such that the somatotopy in one hemisphere could have been found given the connectivity pattern of the corresponding regions of interest in the other hemisphere. When inspecting the somatotopic and nonsomatotopic connectivity patterns, a similarity index indicated that the pattern of connected and nonconnected regions of interest across different homunculi is similar for different body parts and hemispheres. The results show that topographical gradients are even more widespread than previously assumed in the somatosensory-motor system. Spectral analysis can thus potentially serve as a gold standard for defining somatosensory-motor system areas for basic research and clinical applications.

Electrocorticographic language mapping in children by high-gamma synchronization during spontaneous conversation: comparison with conventional electrical cortical stimulation

INTRODUCTION

This study describes development of a novel language mapping approach using high-γ modulation in electrocorticograph (ECoG) during spontaneous conversation, and its comparison with electrical cortical stimulation (ECS) in childhood-onset drug-resistant epilepsy.

METHODS

Patients undergoing invasive pre-surgical monitoring and able to converse with the investigator were eligible. ECoG signals and synchronized audio were acquired during quiet baseline and during natural conversation between investigator and the patient. Using Signal Modeling for Real-time Identification and Event Detection (SIGFRIED) procedure, a statistical model for baseline high-γ (70-116 Hz) power, and a single score for each channel representing the probability that the power features in the experimental signal window belonged to the baseline model, were calculated. Electrodes with significant high-γ responses (HGS) were plotted on the 3D cortical model. Sensitivity, specificity, positive and negative predictive values (PPV, NPV), and classification accuracy were calculated compared to ECS.

RESULTS

Seven patients were included (4 males, mean age 10.28 ± 4.07 years). Significant high-γ responses were observed in classic language areas in the left hemisphere plus in some homologous right hemispheric areas. Compared with clinical standard ECS mapping, the sensitivity and specificity of HGS mapping was 88.89% and 63.64%, respectively, and PPV and NPV were 35.29% and 96.25%, with an overall accuracy of 68.24%. HGS mapping was able to correctly determine all ECS+ sites in 6 of 7 patients and all false-sites (ECS+, HGS- for visual naming, n = 3) were attributable to only 1 patient.

CONCLUSIONS

This study supports the feasibility of language mapping with ECoG HGS during spontaneous conversation, and its accuracy compared to traditional ECS. Given long-standing concerns about ecological validity of ECS mapping of cued language tasks, and difficulties encountered with its use in children, ECoG mapping of spontaneous language may provide a valid alternative for clinical use.

From speech to thought: the neuronal basis of cognitive units in non-experimental, real-life communication investigated using ECoG

Exchange of thoughts by means of expressive speech is fundamental to human communication. However, the neuronal basis of real-life communication in general, and of verbal exchange of ideas in particular, has rarely been studied until now. Here, our aim was to establish an approach for exploring the neuronal processes related to cognitive “idea” units (IUs) in conditions of non-experimental speech production. We investigated whether such units corresponding to single, coherent chunks of speech with syntactically-defined borders, are useful to unravel the neuronal mechanisms underlying real-world human cognition. To this aim, we employed simultaneous electrocorticography (ECoG) and video recordings obtained in pre-neurosurgical diagnostics of epilepsy patients. We transcribed non-experimental, daily hospital conversations, identified IUs in transcriptions of the patients' speech, classified the obtained IUs according to a previously-proposed taxonomy focusing on memory content, and investigated the underlying neuronal activity. In each of our three subjects, we were able to collect a large number of IUs which could be assigned to different functional IU subclasses with a high inter-rater agreement. Robust IU-onset-related changes in spectral magnitude could be observed in high gamma frequencies (70–150 Hz) on the inferior lateral convexity and in the superior temporal cortex regardless of the IU content. A comparison of the topography of these responses with mouth motor and speech areas identified by electrocortical stimulation showed that IUs might be of use for extraoperative mapping of eloquent cortex (average sensitivity: 44.4%, average specificity: 91.1%). High gamma responses specific to memory-related IU subclasses were observed in the inferior parietal and prefrontal regions. IU-based analysis of ECoG recordings during non-experimental communication thus elicits topographically- and functionally-specific effects. We conclude that segmentation of spontaneous real-world speech in linguistically-motivated units is a promising strategy for elucidating the neuronal basis of mental processing during non-experimental communication.

Comparison of high gamma electrocorticography and fMRI with electrocortical stimulation for localization of somatosensory and language cortex

OBJECTIVE

We investigated the contribution of electrocortical stimulation (ECS), induced high gamma electrocorticography (hgECoG) and functional magnetic resonance imaging (fMRI) for the localization of somatosensory and language cortex.

METHODS

23 Epileptic patients with subdural electrodes underwent a protocol of somatosensory stimulation and/or an auditory semantic decision task. 14 Patients did the same protocol with fMRI prior to implantation.

RESULTS

ECS resulted in the identification of thumb somatosensory cortex in 12/16 patients. Taking ECS as a gold standard, hgECoG and fMRI identified 53.6/33% of true positive and 4/12% of false positive contacts, respectively. The hgECoG false positive sites were all found in the hand area of the post-central gyrus. ECS localized language-related sites in 7/12 patients with hgECoG and fMRI showing 50/64% of true positive and 8/23% of false positive contacts, respectively. All but one of the hgECoG/fMRI false positive contacts were located in plausible language areas. Four patients showed post-surgical impairments: the resection included the sites positively indicated by ECS, hgECoG and fMRI in 3 patients and a positive hgECoG site in one patient.

CONCLUSIONS

HgECoG and fMRI provide additional localization information in patients who cannot sufficiently collaborate during ECS.

SIGNIFICANCE

HgECoG and fMRI make the cortical mapping procedure more flexible not only by identifying priority cortical sites for ECS or when ECS is not feasible, but also when ECS does not provide any result.

Towards effective non-invasive brain-computer interfaces dedicated to gait rehabilitation systems

In the last few years, significant progress has been made in the field of walk rehabilitation. Motor cortex signals in bipedal monkeys have been interpreted to predict walk kinematics. Epidural electrical stimulation in rats and in one young paraplegic has been realized to partially restore motor control after spinal cord injury. However, these experimental trials are far from being applicable to all patients suffering from motor impairments. Therefore, it is thought that more simple rehabilitation systems are desirable in the meanwhile. The goal of this review is to describe and summarize the progress made in the development of non-invasive brain-computer interfaces dedicated to motor rehabilitation systems. In the first part, the main principles of human locomotion control are presented. The paper then focuses on the mechanisms of supra-spinal centers active during gait, including results from electroencephalography, functional brain imaging technologies [near-infrared spectroscopy (NIRS), functional magnetic resonance imaging (fMRI), positron-emission tomography (PET), single-photon emission-computed tomography (SPECT)] and invasive studies. The first brain-computer interface (BCI) applications to gait rehabilitation are then presented, with a discussion about the different strategies developed in the field. The challenges to raise for future systems are identified and discussed. Finally, we present some proposals to address these challenges, in order to contribute to the improvement of BCI for gait rehabilitation.

Electrocorticographic correlates of overt articulation of 44 English phonemes: intracranial recording in children with focal epilepsy

OBJECTIVE

We determined the temporal-spatial patterns of electrocorticography (ECoG) signal modulation during overt articulation of 44 American English phonemes.

METHODS

We studied two children with focal epilepsy who underwent extraoperative ECoG recording. Using animation movies, we delineated 'when' and 'where' gamma- (70-110 Hz) and low-frequency-band activities (10-30 Hz) were modulated during self-paced articulation.

RESULTS

Regardless of the classes of phoneme articulated, gamma-augmentation initially involved a common site within the left inferior Rolandic area. Subsequently, gamma-augmentation and/or attenuation involved distinct sites within the left oral-sensorimotor area with a timing variable across phonemes. Finally, gamma-augmentation in a larynx-sensorimotor area took place uniformly at the onset of sound generation, and effectively distinguished voiced and voiceless phonemes. Gamma-attenuation involved the left inferior-frontal and superior-temporal regions simultaneously during articulation. Low-frequency band attenuation involved widespread regions including the frontal, temporal, and parietal regions.

CONCLUSIONS

Our preliminary results support the notion that articulation of distinct phonemes recruits specific sensorimotor activation and deactivation. Gamma attenuation in the left inferior-frontal and superior-temporal regions may reflect transient functional suppression in these cortical regions during automatic, self-paced vocalization of phonemes containing no semantic or syntactic information.

SIGNIFICANCE

Further studies are warranted to determine if measurement of event-related modulations of gamma-band activity, compared to that of the low-frequency-band, is more useful for decoding the underlying articulatory functions.