Don't put words in my mouth: Speech perception can generate False Positive activation of a speech BCI

Recent studies have demonstrated that speech can be decoded from brain activity and used for brain-computer interface (BCI)-based communication. It is however also known that the area often used as a signal source for speech decoding BCIs, the sensorimotor cortex (SMC), is also engaged when people perceive speech, thus making speech perception a potential source of false positive activation of the BCI. The current study investigated if and how speech perception may interfere with reliable speech BCI control. We recorded high-density electrocorticography (HD-ECoG) data from five subjects while they performed a speech perception and speech production task and trained a support-vector machine (SVM) on the produced speech data. Our results show that decoders that are highly reliable at detecting self-produced speech from brain signals also generate false positives during the perception of speech. We conclude that speech perception interferes with reliable BCI control, and that efforts to limit the occurrence of false positives during daily-life BCI use should be implemented in BCI design to increase the likelihood of successful adaptation by end users.

Decoding attempted phantom hand movements from ipsilateral sensorimotor areas after amputation

Objective.The sensorimotor cortex is often selected as target in the development of a Brain-Computer Interface, as activation patterns from this region can be robustly decoded to discriminate between different movements the user executes. Up until recently, such BCIs were primarily based on activity in the contralateral hemisphere, where decoding movements still works even years after denervation. However, there is increasing evidence for a role of the sensorimotor cortex in controlling the ipsilateral body. The aim of this study is to investigate the effects of denervation on the movement representation on the ipsilateral sensorimotor cortex.Approach.Eight subjects with acquired above-elbow arm amputation and nine controls performed a task in which they made (or attempted to make with their phantom hand) six different gestures from the American Manual Alphabet. Brain activity was measured using 7T functional MRI, and a classifier was trained to discriminate between activation patterns on four different regions of interest (ROIs) on the ipsilateral sensorimotor cortex.Main results.Classification scores showed that decoding was possible and significantly better than chance level for both the phantom and intact hands from all ROIs. Decoding both the left (intact) and right (phantom) hand from the same hemisphere was also possible with above-chance level classification score.Significance.The possibility to decode both hands from the same hemisphere, even years after denervation, indicates that implantation of motor-electrodes for BCI control possibly need only cover a single hemisphere, making surgery less invasive, and increasing options for people with lateralized damage to motor cortex like after stroke.

Classification of Articulator Movements and Movement Direction from Sensorimotor Cortex Activity

For people suffering from severe paralysis, communication can be difficult or nearly impossible. Technology systems called brain-computer interfaces (BCIs) are being developed to assist these people with communication by using their brain activity to control a computer without any muscle activity. To benefit the development of BCIs that employ neural activity related to speech, we investigated if neural activity patterns related to different articulator movements can be distinguished from each other. We recorded with electrocorticography (ECoG), the neural activity related to different articulator movements in 4 epilepsy patients and classified which articulator participants moved based on the sensorimotor cortex activity patterns. The same was done for different movement directions of a single articulator, the tongue. In both experiments highly accurate classification was obtained, on average 92% for different articulators and 85% for different tongue directions. Furthermore, the data show that only a small part of the sensorimotor cortex is needed for classification (ca. 1 cm2). We show that recordings from small parts of the sensorimotor cortex contain information about different articulator movements which might be used for BCI control. Our results are of interest for BCI systems that aim to decode neural activity related to (actual or attempted) movements from a contained cortical area.

Spatial-Temporal Dynamics of the Sensorimotor Cortex: Sustained and Transient Activity

How the sensorimotor cortex is organized with respect to controlling different features of movement is unclear. One unresolved question concerns the relation between the duration of an action and the duration of the associated neuronal activity change in the sensorimotor cortex. Using subdural electrocorticography electrodes, we investigated in five subjects, whether high frequency band (HFB; 75-135 Hz) power changes have a transient or sustained relation to speech duration, during pronunciation of the Dutch /i/ vowel with different durations. We showed that the neuronal activity patterns recorded from the sensorimotor cortex can be directly related to action duration in some locations, whereas in other locations, during the same action, neuronal activity is transient, with a peak in HFB activity at movement onset and/or offset. This data sheds light on the neural underpinnings of motor actions and we discuss the possible mechanisms underlying these different response types.

The influence of prior pronunciations on sensorimotor cortex activity patterns during vowel production

OBJECTIVE

In recent years, brain-computer interface (BCI) systems have been investigated for their potential as a communication device to assist people with severe paralysis. Decoding speech sensorimotor cortex activity is a promising avenue for the generation of BCI control signals, but is complicated by variability in neural patterns, leading to suboptimal decoding. We investigated whether neural pattern variability associated with sound pronunciation can be explained by prior pronunciations and determined to what extent prior speech affects BCI decoding accuracy.

APPROACH

Neural patterns in speech motor areas were evaluated with electrocorticography in five epilepsy patients, who performed a simple speech task that involved pronunciation of the /i/ sound, preceded by either silence, the /a/ sound or the /u/ sound.

MAIN RESULTS

The neural pattern related to the /i/ sound depends on previous sounds and is therefore associated with multiple distinct sensorimotor patterns, which is likely to reflect differences in the movements towards this sound. We also show that these patterns still contain a commonality that is distinct from the other vowel sounds (/a/ and /u/). Classification accuracies for the decoding of different sounds do increase, however, when the multiple patterns for the /i/ sound are taken into account. Simply including multiple forms of the /i/ vowel in the training set for the creation of a single /i/ model performs as well as training individual models for each /i/ variation.

SIGNIFICANCE

Our results are of interest for the development of BCIs that aim to decode speech sounds from the sensorimotor cortex, since they argue that a multitude of cortical activity patterns associated with speech movements can be reduced to a basis set of models which reflect meaningful language units (vowels), yet it is important to account for the variety of neural patterns associated with a single sound in the training process.

Repeated Vowel Production Affects Features of Neural Activity in Sensorimotor Cortex

The sensorimotor cortex is responsible for the generation of movements and interest in the ability to use this area for decoding speech by brain–computer interfaces has increased recently. Speech decoding is challenging however, since the relationship between neural activity and motor actions is not completely understood. Non-linearity between neural activity and movement has been found for instance for simple finger movements. Despite equal motor output, neural activity amplitudes are affected by preceding movements and the time between movements. It is unknown if neural activity is also affected by preceding motor actions during speech. We addressed this issue, using electrocorticographic high frequency band (HFB; 75–135 Hz) power changes in the sensorimotor cortex during discrete vowel generation. Three subjects with temporarily implanted electrode grids produced the /i/ vowel at repetition rates of 1, 1.33 and 1.66 Hz. For every repetition, the HFB power amplitude was determined. During the first utterance, most electrodes showed a large HFB power peak, which decreased for subsequent utterances. This result could not be explained by differences in performance. With increasing duration between utterances, more electrodes showed an equal response to all repetitions, suggesting that the duration between vowel productions influences the effect of previous productions on sensorimotor cortex activity. Our findings correspond with previous studies for finger movements and bear relevance for the development of brain-computer interfaces that employ speech decoding based on brain signals, in that past utterances will need to be taken into account for these systems to work accurately.

Give me a sign: decoding four complex hand gestures based on high-density ECoG

The increasing understanding of human brain functions makes it possible to directly interact with the brain for therapeutic purposes. Implantable brain computer interfaces promise to replace or restore motor functions in patients with partial or complete paralysis. We postulate that neuronal states associated with gestures, as they are used in the finger spelling alphabet of sign languages, provide an excellent signal for implantable brain computer interfaces to restore communication. To test this, we evaluated decodability of four gestures using high-density electrocorticography in two participants. The electrode grids were located subdurally on the hand knob area of the sensorimotor cortex covering a surface of 2.5–5.2 cm². Using a pattern-matching classification approach four types of hand gestures were classified based on their pattern of neuronal activity. In the two participants the gestures were classified with 97 and 74 % accuracy. The high frequencies (>65 Hz) allowed for the best classification results. This proof-of-principle study indicates that the four gestures are associated with a reliable and discriminable spatial representation on a confined area of the sensorimotor cortex. This robust representation on a small area makes hand gestures an interesting control feature for an implantable BCI to restore communication for severely paralyzed people.

Brain implants for substituting lost motor function: state of the art and potential impact on the lives of motor-impaired seniors

Recent scientific achievements bring the concept of neural prosthetics for reinstating lost motor function closer to medical application. Current research involves severely paralyzed people under the age of 65, but implications for seniors with stroke or trauma-induced impairments are clearly on the horizon. Demographic changes will lead to a shortage of personnel to care for an increasing population of senior citizens, threatening maintenance of an acceptable level of care and urging ways for people to live longer at their home independent from personal assistance. This is particularly challenging when people suffer from disabilities such as partial paralysis after stroke or trauma, where daily personal assistance is required. For some of these people, neural prosthetics can reinstate some lost motor function and/or lost communication, thereby increasing independence and possibly quality of life. In this viewpoint article, we present the state of the art in decoding brain activity in the service of brain-computer interfacing. Although some noninvasive applications produce good results, we focus on brain implants that benefit from better quality brain signals. Fully implantable neural prostheses for home use are not available yet, but clinical trials are being prepared. More sophisticated systems are expected to follow in the years to come, with capabilities of interest for less severe paralysis. Eventually the combination of smart robotics and brain implants is expected to enable people to interact well enough with their environment to live an independent life in spite of motor disabilities.

Task-free electrocorticography frequency mapping of the motor cortex

OBJECTIVE

Electrocortical stimulation mapping (ESM) is the current gold standard for functional mapping of the eloquent cortex prior to epilepsy surgery. The procedure is, however, time-consuming and quite demanding for patients. Electrocorticography frequency mapping (ECoG mapping) has been suggested as an adjunct method. Here, we investigated whether it is possible to perform mapping of motor regions using ECoG data of spontaneous movements.

METHODS

Using the video registration of seven epilepsy patients who underwent electrocorticography and ESM, we selected periods of spontaneous hand and arm movements and periods of rest. Frequency analysis was performed, and electrodes showing a significant change in power (4-7, 8-14, 15-25, 26-45 or 65-95 Hz) were compared with those being identified as relevant for hand and/or arm movement by ESM.

RESULTS

All frequency bands showed a high specificity (>0.80), and the 65-95 Hz frequency band additionally had a high sensitivity (0.82) for identifying ESM positive electrodes.

CONCLUSIONS

Our data show a good match between ECoG mapping of spontaneous movements and ESM data.

SIGNIFICANCE

The accurate match suggests that ECoG mapping of the motor cortex using spontaneous movements may be a valuable complement to ESM, especially when other options requiring patient cooperation fail.

Frequency specific spatial interactions in human electrocorticography: V1 alpha oscillations reflect surround suppression

Electrical brain signals are often decomposed into frequency ranges that are implicated in different functions. Using subdural electrocorticography (ECoG, intracranial EEG) and functional magnetic resonance imaging (fMRI), we measured frequency spectra and BOLD responses in primary visual cortex (V1) and intraparietal sulcus (IPS). In V1 and IPS, 30-120 Hz (gamma, broadband) oscillations allowed population receptive field (pRF) reconstruction comparable to fMRI estimates. Lower frequencies, however, responded very differently in V1 and IPS. In V1, broadband activity extends down to 3 Hz. In the 4-7 Hz (theta) and 18-30 Hz (beta) ranges broadband activity increases power during stimulation within the pRF. However, V1 9-12 Hz (alpha) frequency oscillations showed a different time course. The broadband power here is exceeded by a frequency-specific power increase during stimulation of the area outside the pRF. As such, V1 alpha oscillations reflected surround suppression of the pRF, much like negative fMRI responses. They were consequently highly localized, depending on stimulus and pRF position, and independent between nearby electrodes. In IPS, all 3-25 Hz oscillations were strongest during baseline recording and correlated between nearby electrodes, consistent with large-scale disengagement. These findings demonstrate V1 alpha oscillations result from locally active functional processes and relate these alpha oscillations to negative fMRI signals. They highlight that similar oscillations in different areas reflect processes with different functional roles. However, both of these roles of alpha seem to reflect suppression of spiking activity.

The effects of blood vessels on electrocorticography

Electrocorticography, primarily used in a clinical context, is becoming increasingly important for fundamental neuroscientific research, as well as for brain-computer interfaces. Recordings from these implanted electrodes have a number of advantages over non-invasive recordings in terms of band width, spatial resolution, smaller vulnerability to artifacts and overall signal quality. However, an unresolved issue is that signals vary greatly across electrodes. Here, we examine the effect of blood vessels lying between an electrode and the cortex on signals recorded from subdural grid electrodes. Blood vessels of different sizes cover extensive parts of the cortex causing variations in the electrode-cortex connection across grids. The power spectral density of electrodes located on the cortex and electrodes located on blood vessels obtained from eight epilepsy patients is compared. We find that blood vessels affect the power spectral density of the recorded signal in a frequency-band-specific way, in that frequencies between 30 and 70 Hz are attenuated the most. Here, the signal is attenuated on average by 30-40% compared to electrodes directly on the cortex. For lower frequencies this attenuation effect is less pronounced. We conclude that blood vessels influence the signal properties in a non-uniform manner.