Text and image generation from intracranial electroencephalography using an embedding space for text and images

OBJECTIVE

Invasive brain-computer interfaces (BCIs) are promising communication devices for severely paralyzed patients. Recent advances in intracranial electroencephalography (iEEG) coupled with natural language processing have enhanced communication speed and accuracy. It should be noted that such a speech BCI uses signals from the motor cortex. However, BCIs based on motor cortical activities may experience signal deterioration in users with motor cortical degenerative diseases such as amyotrophic lateral sclerosis (ALS). An alternative approach to using iEEG of the motor cortex is necessary to support patients with such conditions. Approach: In this study, a multimodal embedding of text and images was used to decode visual semantic information from iEEG signals of the visual cortex to generate text and images. We used contrastive language-image pretraining (CLIP) embedding to represent images presented to 17 patients implanted with electrodes in the occipital and temporal cortexes. A CLIP image vector was inferred from the high-γ power of the iEEG signals recorded while viewing the images. Main results: Text was generated by CLIPCAP from the inferred CLIP vector with better-than-chance accuracy. Then, an image was created from the generated text using StableDiffusion with significant accuracy. Significance: The text and images generated from iEEG through the CLIP embedding vector can be used for improved communication. .

A deep learning model for the detection of various dementia and MCI pathologies based on resting-state electroencephalography data: A retrospective multicentre study

Dementia and mild cognitive impairment (MCI) represent significant health challenges in an aging population. As the search for noninvasive, precise and accessible diagnostic methods continues, the efficacy of electroencephalography (EEG) combined with deep convolutional neural networks (DCNNs) in varied clinical settings remains unverified, particularly for pathologies underlying MCI such as Alzheimer's disease (AD), dementia with Lewy bodies (DLB) and idiopathic normal-pressure hydrocephalus (iNPH). Addressing this gap, our study evaluates the generalizability of a DCNN trained on EEG data from a single hospital (Hospital #1). For data from Hospital #1, the DCNN achieved a balanced accuracy (bACC) of 0.927 in classifying individuals as healthy (n = 69) or as having AD, DLB, or iNPH (n = 188). The model demonstrated robustness across institutions, maintaining bACCs of 0.805 for data from Hospital #2 (n = 73) and 0.920 at Hospital #3 (n = 139). Additionally, the model could differentiate AD, DLB, and iNPH cases with bACCs of 0.572 for data from Hospital #1 (n = 188), 0.619 for Hospital #2 (n = 70), and 0.508 for Hospital #3 (n = 139). Notably, it also identified MCI pathologies with a bACC of 0.715 for Hospital #1 (n = 83), despite being trained on overt dementia cases instead of MCI cases. These outcomes confirm the DCNN's adaptability and scalability, representing a significant stride toward its clinical application. Additionally, our findings suggest a potential for identifying shared EEG signatures between MCI and dementia, contributing to the field's understanding of their common pathophysiological mechanisms.

Magnetoencephalographic neurofeedback training decreases β-low-γ phase-amplitude coupling of the motor cortex of healthy adults: A double-blinded randomized crossover feasibility study

Objective
The coupling between the beta (13-30 Hz) phase and low gamma (50-100 Hz) amplitude in the motor cortex is thought to regulate motor performance. Abnormal phase-amplitude coupling (PAC) of beta-low gamma (β-low-γ PAC) is associated with motor symptoms of Parkinson's disease. However, the causal relationship between β-low-γ PAC and motor performance in healthy subjects is unknown. We hypothesized that healthy subjects could change the strength of the β-low-γ PAC in the resting state by neurofeedback training (NFT) to control the β-low-γ PAC, such that the motor performance changes in accordance with the changes in β-low-γ PAC in the resting state. 
Methods
We developed an NFT to control the strength of the β-low-γ PAC in the motor cortex, which was evaluated by magnetoencephalography (MEG) using a current source estimation technique. Twenty subjects were enrolled in a double-blind randomized crossover trial to test the feasibility of the MEG NFT. In the NFT for 2 days, the subjects were instructed to reduce the size of a black circle whose radius was proportional (down-training) or inversely proportional (up-training) to the strength of the β-low-γ PAC. The reaction times (RTs) to press a button according to some cues were evaluated before and after training. This study was registered at ClinicalTrials.gov (***) and UMIN-CTR (***).
Results
The β-low-γ PAC during the resting state was significantly decreased after down-training, although not significantly after up-training. RTs tended to decrease after both trainings, however the differences were not statistically significant. There was no significant correlation between the changes in β-low-γ PAC during rest and RTs.
Conclusion
The proposed MEG NFT was demonstrated to change the β-low-γ PAC of the motor cortex in healthy subjects. However, a relationship between PAC and RT has not yet been demonstrated.

Precise Discrimination for Multiple Etiologies of Dementia Cases Based on Deep Learning with Electroencephalography

INTRODUCTION

It is critical to develop accurate and universally available biomarkers for dementia diseases to appropriately deal with the dementia problems under world-wide rapid increasing of patients with dementia. In this sense, electroencephalography (EEG) has been utilized as a promising examination to screen and assist in diagnosing dementia, with advantages of sensitiveness to neural functions, inexpensiveness, and high availability. Moreover, the algorithm-based deep learning can expand EEG applicability, yielding accurate and automatic classification easily applied even in general hospitals without any research specialist.

METHODS

We utilized a novel deep neural network, with which high accuracy of discrimination was archived in neurological disorders in the previous study. Based on this network, we analyzed EEG data of healthy volunteers (HVs, N = 55), patients with Alzheimer's disease (AD, N = 101), dementia with Lewy bodies (DLB, N = 75), and idiopathic normal pressure hydrocephalus (iNPH, N = 60) to evaluate the discriminative accuracy of these diseases.

RESULTS

High discriminative accuracies were archived between HV and patients with dementia, yielding 81.7% (vs. AD), 93.9% (vs. DLB), 93.1% (vs. iNPH), and 87.7% (vs. AD, DLB, and iNPH).

CONCLUSION

This study revealed that the EEG data of patients with dementia were successfully discriminated from HVs based on a novel deep learning algorithm, which could be useful for automatic screening and assisting diagnosis of dementia diseases.

Neurofeedback Training without Explicit Phantom Hand Movements and Hand-Like Visual Feedback to Modulate Pain: A Randomized Crossover Feasibility Trial

Phantom limb pain is attributed to abnormal sensorimotor cortical representations, although the causal relationship between phantom limb pain and sensorimotor cortical representations suffers from the potentially confounding effects of phantom hand movements. We developed neurofeedback training to change sensorimotor cortical representations without explicit phantom hand movements or hand-like visual feedback. We tested the feasibility of neurofeedback training in fourteen patients with phantom limb pain. Neurofeedback training was performed in a single-blind, randomized, crossover trial using two decoders constructed using motor cortical currents measured during phantom hand movements; the motor cortical currents contralateral or ipsilateral to the phantom hand (contralateral and ipsilateral training) were estimated from magnetoencephalograms. Patients were instructed to control the size of a disk, which was proportional to the decoding results, but to not move their phantom hands or other body parts. The pain assessed by the visual analogue scale was significantly greater after contralateral training than after ipsilateral training. Classification accuracy of phantom hand movements significantly increased only after contralateral training. These results suggested that the proposed neurofeedback training changed phantom hand representation and modulated pain without explicit phantom hand movements or hand-like visual feedback, thus showing the relation between the phantom hand representations and pain. PERSPECTIVE: Our work demonstrates the feasibility of using neurofeedback training to change phantom hand representation and modulate pain perception without explicit phantom hand movements and hand-like visual feedback. The results enhance the mechanistic understanding of certain treatments, such as mirror therapy, that change the sensorimotor cortical representation.

Abnormal phase-amplitude coupling characterizes the interictal state in epilepsy

OBJECTIVE

 Diagnosing epilepsy still requires visual interpretation of electroencephalography and magnetoencephalography (MEG) by specialists, which prevents quantification and standardization of diagnosis. Previous studies proposed automated diagnosis by combining various features from electroencephalography and MEG, such as relative power (Power) and functional connectivity. However, the usefulness of interictal phase-amplitude coupling (PAC) in diagnosing epilepsy is still unknown. We hypothesized that resting-state PAC would be different for patients with epilepsy in the interictal state and for healthy participants such that it would improve discrimination between the groups.

METHODS

 We obtained resting-state MEG and magnetic resonance imaging in 90 patients with epilepsy during their preoperative evaluation and in 90 healthy participants. We used the cortical currents estimated from MEG and magnetic resonance imaging to calculate Power in the δ (1-3 Hz), θ (4-7 Hz), α (8-13 Hz), β (13-30 Hz), low γ (35-55 Hz), and high γ (65-90 Hz) bands and functional connectivity in the θ band. PAC was evaluated using the synchronization index (SI) for eight frequency band pairs: the phases of δ, θ, α, and β and the amplitudes of low and high γ. First, we compared the mean SI values for the patients with epilepsy and the healthy participants. Then, using features such as PAC, Power, functional connectivity, and features extracted by deep learning individually or combined, we tested whether PAC improves discrimination accuracy for the two groups.

RESULTS

 The mean SI values were significantly different for the patients with epilepsy and the healthy participants. The SI value difference was highest for θ/low γ in the temporal lobe. Discrimination accuracy was the highest, at 90%, using the combination of PAC and deep learning.

SIGNIFICANCE

 Abnormal PAC characterized the patients with epilepsy in the interictal state compared with the healthy participants, potentially improving the discrimination of epilepsy.

Data-driven electrophysiological feature based on deep learning to detect epileptic seizures

Objective. To identify a new electrophysiological feature characterising the epileptic seizures, which is commonly observed in different types of epilepsy.Methods. We recorded the intracranial electroencephalogram (iEEG) of 21 patients (12 women and 9 men) with multiple types of refractory epilepsy. The raw iEEG signals of the early phase of epileptic seizures and interictal states were classified by a convolutional neural network (Epi-Net). For comparison, the same signals were classified by a support vector machine (SVM) using the spectral power and phase-amplitude coupling. The features learned by Epi-Net were derived by a modified integrated gradients method. We considered the product of powers multiplied by the relative contribution of each frequency amplitude as a data-driven epileptogenicity index (d-EI). We compared the d-EI and other conventional features in terms of accuracy to detect the epileptic seizures. Finally, we compared the d-EI among the electrodes to evaluate its relationship with the resected area and the Engel classification.Results. Epi-Net successfully identified the epileptic seizures, with an area under the receiver operating characteristic curve of 0.944 ± 0.067, which was significantly larger than that of the SVM (0.808 ± 0.253,n =21;p =0.025). The learned iEEG signals were characterised by increased powers of 17-92 Hz and >180 Hz in addition to decreased powers of other frequencies. The proposed d-EI detected them with better accuracy than the other iEEG features. Moreover, the surgical resection of areas with a larger increase in d-EI was observed for all nine patients with Engel class ⩽1, but not for the 4 of 12 patients with Engel class >1, demonstrating the significant association with seizure outcomes.Significance.We derived an iEEG feature from the trained Epi-Net, which identified the epileptic seizures with improved accuracy and might contribute to identification of the epileptogenic zone.

BCI training to move a virtual hand reduces phantom limb pain: A randomized crossover trial

OBJECTIVE

To determine whether training with a brain-computer interface (BCI) to control an image of a phantom hand, which moves based on cortical currents estimated from magnetoencephalographic signals, reduces phantom limb pain.

METHODS

Twelve patients with chronic phantom limb pain of the upper limb due to amputation or brachial plexus root avulsion participated in a randomized single-blinded crossover trial. Patients were trained to move the virtual hand image controlled by the BCI with a real decoder, which was constructed to classify intact hand movements from motor cortical currents, by moving their phantom hands for 3 days ("real training"). Pain was evaluated using a visual analogue scale (VAS) before and after training, and at follow-up for an additional 16 days. As a control, patients engaged in the training with the same hand image controlled by randomly changing values ("random training"). The 2 trainings were randomly assigned to the patients. This trial is registered at UMIN-CTR (UMIN000013608).

RESULTS

VAS at day 4 was significantly reduced from the baseline after real training (mean [SD], 45.3 [24.2]-30.9 [20.6], 1/100 mm; p = 0.009 < 0.025), but not after random training (p = 0.047 > 0.025). Compared to VAS at day 1, VAS at days 4 and 8 was significantly reduced by 32% and 36%, respectively, after real training and was significantly lower than VAS after random training (p < 0.01).

CONCLUSION

Three-day training to move the hand images controlled by BCI significantly reduced pain for 1 week.

CLASSIFICATION OF EVIDENCE

This study provides Class III evidence that BCI reduces phantom limb pain.

Prediction of IDH and TERT promoter mutations in low-grade glioma from magnetic resonance images using a convolutional neural network

Identification of genotypes is crucial for treatment of glioma. Here, we developed a method to predict tumor genotypes using a pretrained convolutional neural network (CNN) from magnetic resonance (MR) images and compared the accuracy to that of a diagnosis based on conventional radiomic features and patient age. Multisite preoperative MR images of 164 patients with grade II/III glioma were grouped by IDH and TERT promoter (pTERT) mutations as follows: (1) IDH wild type, (2) IDH and pTERT co-mutations, (3) IDH mutant and pTERT wild type. We applied a CNN (AlexNet) to four types of MR sequence and obtained the CNN texture features to classify the groups with a linear support vector machine. The classification was also performed using conventional radiomic features and/or patient age. Using all features, we succeeded in classifying patients with an accuracy of 63.1%, which was significantly higher than the accuracy obtained from using either the radiomic features or patient age alone. In particular, prediction of the pTERT mutation was significantly improved by the CNN texture features. In conclusion, the pretrained CNN texture features capture the information of IDH and TERT genotypes in grade II/III gliomas better than the conventional radiomic features.

Automatic diagnosis of neurological diseases using MEG signals with a deep neural network

The application of deep learning to neuroimaging big data will help develop computer-aided diagnosis of neurological diseases. Pattern recognition using deep learning can extract features of neuroimaging signals unique to various neurological diseases, leading to better diagnoses. In this study, we developed MNet, a novel deep neural network to classify multiple neurological diseases using resting-state magnetoencephalography (MEG) signals. We used the MEG signals of 67 healthy subjects, 26 patients with spinal cord injury, and 140 patients with epilepsy to train and test the network using 10-fold cross-validation. The trained MNet succeeded in classifying the healthy subjects and those with the two neurological diseases with an accuracy of 70.7 ± 10.6%, which significantly exceeded the accuracy of 63.4 ± 12.7% calculated from relative powers of six frequency bands (δ: 1-4 Hz; θ: 4-8 Hz; low-α: 8-10 Hz; high-α: 10-13 Hz; β: 13-30 Hz; low-γ: 30-50 Hz) for each channel using a support vector machine as a classifier (p = 4.2 × 10-2). The specificity of classification for each disease ranged from 86-94%. Our results suggest that this technique would be useful for developing a classifier that will improve neurological diagnoses and allow high specificity in identifying diseases.

MEG-BMI to Control Phantom Limb Pain

A brachial plexus root avulsion (BPRA) causes intractable pain in the insensible affected hands. Such pain is partly due to phantom limb pain, which is neuropathic pain occurring after the amputation of a limb and partial or complete deafferentation. Previous studies suggested that the pain was attributable to maladaptive plasticity of the sensorimotor cortex. However, there is little evidence to demonstrate the causal links between the pain and the cortical representation, and how much cortical factors affect the pain. Here, we applied lesioning of the dorsal root entry zone (DREZotomy) and training with a brain–machine interface (BMI) based on real-time magnetoencephalography signals to reconstruct affected hand movements with a robotic hand. The DREZotomy successfully reduced the shooting pain after BPRA, but a part of the pain remained. The BMI training successfully induced some plastic changes in the sensorimotor representation of the phantom hand movements and helped control the remaining pain. When the patient tried to control the robotic hand by moving their phantom hand through association with the representation of the intact hand, this especially decreased the pain while decreasing the classification accuracy of the phantom hand movements. These results strongly suggested that pain after the BPRA was partly attributable to cortical representation of phantom hand movements and that the BMI training controlled the pain by inducing appropriate cortical reorganization. For the treatment of chronic pain, we need to know how to modulate the cortical representation by novel methods.

Training in Use of Brain-Machine Interface-Controlled Robotic Hand Improves Accuracy Decoding Two Types of Hand Movements

Objective: Brain-machine interfaces (BMIs) are useful for inducing plastic changes in cortical representation. A BMI first decodes hand movements using cortical signals and then converts the decoded information into movements of a robotic hand. By using the BMI robotic hand, the cortical representation decoded by the BMI is modulated to improve decoding accuracy. We developed a BMI based on real-time magnetoencephalography (MEG) signals to control a robotic hand using decoded hand movements. Subjects were trained to use the BMI robotic hand freely for 10 min to evaluate plastic changes in the cortical representation due to the training.

Method: We trained nine young healthy subjects with normal motor function. In open-loop conditions, they were instructed to grasp or open their right hands during MEG recording. Time-averaged MEG signals were then used to train a real decoder to control the robotic arm in real time. Then, subjects were instructed to control the BMI-controlled robotic hand by moving their right hands for 10 min while watching the robot's movement. During this closed-loop session, subjects tried to improve their ability to control the robot. Finally, subjects performed the same offline task to compare cortical activities related to the hand movements. As a control, we used a random decoder trained by the MEG signals with shuffled movement labels. We performed the same experiments with the random decoder as a crossover trial. To evaluate the cortical representation, cortical currents were estimated using a source localization technique. Hand movements were also decoded by a support vector machine using the MEG signals during the offline task. The classification accuracy of the movements was compared among offline tasks.

Results: During the BMI training with the real decoder, the subjects succeeded in improving their accuracy in controlling the BMI robotic hand with correct rates of 0.28 ± 0.13 to 0.50 ± 0.11 (p = 0.017, n = 8, paired Student's t-test). Moreover, the classification accuracy of hand movements during the offline task was significantly increased after BMI training with the real decoder from 62.7 ± 6.5 to 70.0 ± 11.1% (p = 0.022, n = 8, t₍₇₎ = 2.93, paired Student's t-test), whereas accuracy did not significantly change after BMI training with the random decoder from 63.0 ± 8.8 to 66.4 ± 9.0% (p = 0.225, n = 8, t₍₇₎ = 1.33).

Conclusion: BMI training is a useful tool to train the cortical activity necessary for BMI control and to induce some plastic changes in the activity.

Mapping ECoG channel contributions to trajectory and muscle activity prediction in human sensorimotor cortex

Studies on brain-machine interface techniques have shown that electrocorticography (ECoG) is an effective modality for predicting limb trajectories and muscle activity in humans. Motor control studies have also identified distributions of “extrinsic-like” and “intrinsic-like” neurons in the premotor (PM) and primary motor (M1) cortices. Here, we investigated whether trajectories and muscle activity predicted from ECoG were obtained based on signals derived from extrinsic-like or intrinsic-like neurons. Three participants carried objects of three different masses along the same counterclockwise path on a table. Trajectories of the object and upper arm muscle activity were predicted using a sparse linear regression. Weight matrices for the predictors were then compared to determine if the ECoG channels contributed more information about trajectory or muscle activity. We found that channels over both PM and M1 contributed highly to trajectory prediction, while a channel over M1 was the highest contributor for muscle activity prediction.

Induced sensorimotor brain plasticity controls pain in phantom limb patients

The cause of pain in a phantom limb after partial or complete deafferentation is an important problem. A popular but increasingly controversial theory is that it results from maladaptive reorganization of the sensorimotor cortex, suggesting that experimental induction of further reorganization should affect the pain, especially if it results in functional restoration. Here we use a brain–machine interface (BMI) based on real-time magnetoencephalography signals to reconstruct affected hand movements with a robotic hand. BMI training induces significant plasticity in the sensorimotor cortex, manifested as improved discriminability of movement information and enhanced prosthetic control. Contrary to our expectation that functional restoration would reduce pain, the BMI training with the phantom hand intensifies the pain. In contrast, BMI training designed to dissociate the prosthetic and phantom hands actually reduces pain. These results reveal a functional relevance between sensorimotor cortical plasticity and pain, and may provide a novel treatment with BMI neurofeedback.

Pain in a phantom limb after limb deafferentation may be due to maladaptive sensorimotor representation. Here the authors find that sensorimotor plasticity induced by BMI training with the phantom hand, contrary to expectation, increased pain while dissociating prosthetic movements from the phantom arm relieved the pain.

Real-Time Control of a Neuroprosthetic Hand by Magnetoencephalographic Signals from Paralysed Patients

Neuroprosthetic arms might potentially restore motor functions for severely paralysed patients. Invasive measurements of cortical currents using electrocorticography have been widely used for neuroprosthetic control. Moreover, magnetoencephalography (MEG) exhibits characteristic brain signals similar to those of invasively measured signals. However, it remains unclear whether non-invasively measured signals convey enough motor information to control a neuroprosthetic hand, especially for severely paralysed patients whose sensorimotor cortex might be reorganized. We tested an MEG-based neuroprosthetic system to evaluate the accuracy of using cortical currents in the sensorimotor cortex of severely paralysed patients to control a prosthetic hand. The patients attempted to grasp with or open their paralysed hand while the slow components of MEG signals (slow movement fields; SMFs) were recorded. Even without actual movements, the SMFs of all patients indicated characteristic spatiotemporal patterns similar to actual movements, and the SMFs were successfully used to control a neuroprosthetic hand in a closed-loop condition. These results demonstrate that the slow components of MEG signals carry sufficient information to classify movement types. Successful control by paralysed patients suggests the feasibility of using an MEG-based neuroprosthetic hand to predict a patient's ability to control an invasive neuroprosthesis via the same signal sources as the non-invasive method.

Categorical discrimination of human body parts by magnetoencephalography

Humans recognize body parts in categories. Previous studies have shown that responses in the fusiform body area (FBA) and extrastriate body area (EBA) are evoked by the perception of the human body, when presented either as whole or as isolated parts. These responses occur approximately 190 ms after body images are visualized. The extent to which body-sensitive responses show specificity for different body part categories remains to be largely clarified. We used a decoding method to quantify neural responses associated with the perception of different categories of body parts. Nine subjects underwent measurements of their brain activities by magnetoencephalography (MEG) while viewing 14 images of feet, hands, mouths, and objects. We decoded categories of the presented images from the MEG signals using a support vector machine (SVM) and calculated their accuracy by 10-fold cross-validation. For each subject, a response that appeared to be a body-sensitive response was observed and the MEG signals corresponding to the three types of body categories were classified based on the signals in the occipitotemporal cortex. The accuracy in decoding body-part categories (with a peak at approximately 48%) was above chance (33.3%) and significantly higher than that for random categories. According to the time course and location, the responses are suggested to be body-sensitive and to include information regarding the body-part category. Finally, this non-invasive method can decode category information of a visual object with high temporal and spatial resolution and this result may have a significant impact in the field of brain-machine interface research.

Decoding fingertip trajectory from electrocorticographic signals in humans

Seeking to apply brain-machine interface technology in neuroprosthetics, a number of methods for predicting trajectory of the elbow and wrist have been proposed and have shown remarkable results. Recently, the prediction of hand trajectory and classification of hand gestures or grasping types have attracted considerable attention. However, trajectory prediction for precise finger motion has remained a challenge. We proposed a method for the prediction of fingertip motions from electrocorticographic signals in human cortex. A patient performed extension/flexion tasks with three fingers. Average Pearson's correlation coefficients and normalized root-mean-square errors between decoded and actual trajectories were 0.83-0.90 and 0.24-0.48, respectively. To confirm generalizability to other users, we applied our method to the BCI Competition IV open data sets. Our method showed that the prediction accuracy of fingertip trajectory could be equivalent to that of other results in the competition.

Prediction of three-dimensional arm trajectories based on ECoG signals recorded from human sensorimotor cortex

Brain-machine interface techniques have been applied in a number of studies to control neuromotor prostheses and for neurorehabilitation in the hopes of providing a means to restore lost motor function. Electrocorticography (ECoG) has seen recent use in this regard because it offers a higher spatiotemporal resolution than non-invasive EEG and is less invasive than intracortical microelectrodes. Although several studies have already succeeded in the inference of computer cursor trajectories and finger flexions using human ECoG signals, precise three-dimensional (3D) trajectory reconstruction for a human limb from ECoG has not yet been achieved. In this study, we predicted 3D arm trajectories in time series from ECoG signals in humans using a novel preprocessing method and a sparse linear regression. Average Pearson’s correlation coefficients and normalized root-mean-square errors between predicted and actual trajectories were 0.44∼0.73 and 0.18∼0.42, respectively, confirming the feasibility of predicting 3D arm trajectories from ECoG. We foresee this method contributing to future advancements in neuroprosthesis and neurorehabilitation technology.

Electrocorticographic control of a prosthetic arm in paralyzed patients

OBJECTIVE

Paralyzed patients may benefit from restoration of movement afforded by prosthetics controlled by electrocorticography (ECoG). Although ECoG shows promising results in human volunteers, it is unclear whether ECoG signals recorded from chronically paralyzed patients provide sufficient motor information, and if they do, whether they can be applied to control a prosthetic.

METHODS

We recorded ECoG signals from sensorimotor cortices of 12 patients while they executed or attempted to execute 3 to 5 simple hand and elbow movements. Sensorimotor function was severely impaired in 3 patients due to peripheral nervous system lesion or amputation, moderately impaired due to central nervous system lesions sparing the cortex in 4 patients, and normal in 5 patients. Time frequency and decoding analyses were performed with the patients' ECoG signals.

RESULTS

In all patients, the high gamma power (80-150 Hz) of the ECoG signals during movements was clearly responsive to movement types and provided the best information for classifying different movement types. The classification performance was significantly better than chance in all patients, although differences between ECoG power modulations during different movement types were significantly less in patients with severely impaired motor function. In the impaired patients, cortical representations tended to overlap each other. Finally, using the classification method in real time, a moderately impaired patient and 3 nonparalyzed patients successfully controlled a prosthetic arm.

INTERPRETATION

ECoG signals appear useful for prosthetic arm control and may provide clinically feasible motor restoration for patients with paralysis but no injury of the sensorimotor cortex.

Real-time control of a prosthetic hand using human electrocorticography signals

OBJECT

A brain-machine interface (BMI) offers patients with severe motor disabilities greater independence by controlling external devices such as prosthetic arms. Among the available signal sources for the BMI, electrocorticography (ECoG) provides a clinically feasible signal with long-term stability and low clinical risk. Although ECoG signals have been used to infer arm movements, no study has examined its use to control a prosthetic arm in real time. The authors present an integrated BMI system for the control of a prosthetic hand using ECoG signals in a patient who had suffered a stroke. This system used the power modulations of the ECoG signal that are characteristic during movements of the patient's hand and enabled control of the prosthetic hand with movements that mimicked the patient's hand movements.

METHODS

A poststroke patient with subdural electrodes placed over his sensorimotor cortex performed 3 types of simple hand movements following a sound cue (calibration period). Time-frequency analysis was performed with the ECoG signals to select 3 frequency bands (1-8, 25-40, and 80-150 Hz) that revealed characteristic power modulation during the movements. Using these selected features, 2 classifiers (decoders) were trained to predict the movement state--that is, whether the patient was moving his hand or not--and the movement type based on a linear support vector machine. The decoding accuracy was compared among the 3 frequency bands to identify the most informative features. With the trained decoders, novel ECoG signals were decoded online while the patient performed the same task without cues (free-run period). According to the results of the real-time decoding, the prosthetic hand mimicked the patient's hand movements.

RESULTS

Offline cross-validation analysis of the ECoG data measured during the calibration period revealed that the state and movement type of the patient's hand were predicted with an accuracy of 79.6% (chance 50%) and 68.3% (chance 33.3%), respectively. Using the trained decoders, the onset of the hand movement was detected within 0.37 ± 0.29 seconds of the actual movement. At the detected onset timing, the type of movement was inferred with an accuracy of 69.2%. In the free-run period, the patient's hand movements were faithfully mimicked by the prosthetic hand in real time.

CONCLUSIONS

The present integrated BMI system successfully decoded the hand movements of a poststroke patient and controlled a prosthetic hand in real time. This success paves the way for the restoration of the patient's motor function using a prosthetic arm controlled by a BMI using ECoG signals.