Electrospinography for non-invasively recording spinal sensorimotor networks in humans

Objective. Currently, few non-invasive measures exist for directly measuring spinal sensorimotor networks. Electrospinography (ESG) is one non-invasive method but is primarily used to measure evoked responses or for monitoring the spinal cord during surgery. Our objectives were to evaluate the feasibility of ESG to measure spinal sensorimotor networks by determining spatiotemporal and functional connectivity changes during single-joint movements at the spinal and cortical levels.Approach. We synchronously recorded electroencephalography (EEG), electromyography, and ESG in ten neurologically intact adults while performing one of three lower-limb tasks (no movement, plantar-flexion and knee flexion) in the prone position. A multi-pronged approach was applied for removing artifacts usingH∞filtering, artifact subspace reconstruction and independent component (IC) analysis. Next, data were segmented by task and ICs of EEG were clustered across participants. Within-participant analysis of ICs and ESG data was conducted, and ESG was characterized in the time and frequency domains. Generalized partial directed coherence analysis was performed within ICs and between ICs and ESG data by participant and task.Results.K-means clustering resulted in five clusters of ICs at Brodmann areas (BAs) 9, BA 8, BA 39, BA 4, and BA 22. Areas associated with motor planning, working memory, visual processing, movement, and attention, respectively. Time-frequency analysis of ESG data found localized changes during movement execution when compared to no movement. Lastly, we found bi-directional changes in functional connectivity (p < 0.05, adjusted for multiple comparisons) within IC's and between IC's and ESG sensors during movement when compared to the no movement condition.Significance. To our knowledge this is the first report using high density ESG for characterizing single joint lower limb movements. Our findings provide support that ESG contains information about efferent and afferent signaling in neurologically intact adults and suggests that we can utilize ESG to directly study the spinal cord.

Task-dependent Alteration in Delta Band Corticomuscular Coherence during Standing in Chronic Stroke Survivors

Balance control is an important indicator of mobility and independence in activities of daily living. How the changes in functional integrity of corticospinal tract due to stroke affects the maintenance of upright stance remains to be known. We investigated the changes in functional coupling between the cortex and lower limb muscles during a challenging balance task over multiple frequency bands in chronic stroke survivors. Eleven stroke patients and nine healthy controls performed a challenging balance task. They stood on a computerized platform with/without somatosensory input distortion created by sway-referencing the support surface, thereby varying the difficulty levels of the task. We computed corticomuscular coherence between Cz (electroencephalography) and leg muscles and assessed balance performance using Berg Balance scale (BBS), Timed-up and go (TUG) and center of pressure (COP) measures. We found lower delta frequency band coherence in stroke patients when compared with healthy controls under medium difficulty condition for distal but not proximal leg muscles. For both groups, we found similar coherence at other frequency bands. On BBS and TUG, stroke patients showed poor balance. However, similar group differences were not consistently observed across COP measures. The presence of distal versus proximal effect suggests differences in the (re)organization of the corticospinal connections across the two muscles groups for balance control. We argue that the observed group difference in the delta coherence might be due to altered mechanisms for the detection of somatosensory modulation resulting from sway-referencing of the support platform for balance control.

Transfer Learning with CNN Models for Brain-Machine Interfaces to command lower-limb exoskeletons: A Solution for Limited Data. Annu Int Conf IEEE Eng Med Biol Soc 2023;2023:1-4. [PMID: 38083615 DOI: 10.1109/embc40787.2023.10340008] [Citation(s) in RCA: 0] [Impact Index Per Article: 0]

This study evaluates the performance of two convolutional neural networks (CNNs) in a brain-machine interface (BMI) based on motor imagery (MI) by using a small dataset collected from five participants wearing a lower-limb exoskeleton. To address the issue of limited data availability, transfer learning was employed by training models on EEG signals from other subjects and subsequently fine-tuning them to specific users. A combination of common spatial patterns (CSP) and linear discriminant analysis (LDA) was used as a benchmark for comparison. The study's primary aim is to examine the potential of CNNs and transfer learning in the development of an automatic neural classification system for a BMI based on MI to command a lower-limb exoskeleton that can be used by individuals without specialized training.Clinical Relevance- BMI can be used in rehabilitation for patients with motor impairment by using mental simulation of movement to activate robotic exoskeletons. This can promote neural plasticity and aid in recovery.

Design and Validation of a Low-Cost Mobile EEG-Based Brain-Computer Interface

Objective: We designed and validated a wireless, low-cost, easy-to-use, mobile, dry-electrode headset for scalp electroencephalography (EEG) recordings for closed-loop brain-computer (BCI) interface and internet-of-things (IoT) applications. Approach: The EEG-based BCI headset was designed from commercial off-the-shelf (COTS) components using a multi-pronged approach that balanced interoperability, cost, portability, usability, form factor, reliability, and closed-loop operation. Main Results: The adjustable headset was designed to accommodate 90% of the population. A patent-pending self-positioning dry electrode bracket allowed for vertical self-positioning while parting the user's hair to ensure contact of the electrode with the scalp. In the current prototype, five EEG electrodes were incorporated in the electrode bracket spanning the sensorimotor cortices bilaterally, and three skin sensors were included to measure eye movement and blinks. An inertial measurement unit (IMU) provides monitoring of head movements. The EEG amplifier operates with 24-bit resolution up to 500 Hz sampling frequency and can communicate with other devices using 802.11 b/g/n WiFi. It has high signal-to-noise ratio (SNR) and common-mode rejection ratio (CMRR) (121 dB and 110 dB, respectively) and low input noise. In closed-loop BCI mode, the system can operate at 40 Hz, including real-time adaptive noise cancellation and 512 MB of processor memory. It supports LabVIEW as a backend coding language and JavaScript (JS), Cascading Style Sheets (CSS), and HyperText Markup Language (HTML) as front-end coding languages and includes training and optimization of support vector machine (SVM) neural classifiers. Extensive bench testing supports the technical specifications and human-subject pilot testing of a closed-loop BCI application to support upper-limb rehabilitation and provides proof-of-concept validation for the device's use at both the clinic and at home. Significance: The usability, interoperability, portability, reliability, and programmability of the proposed wireless closed-loop BCI system provides a low-cost solution for BCI and neurorehabilitation research and IoT applications.

Lateralized Neural Entropy modulates with Grip Force during Precision Grasping

When holding a coffee mug filled to the brim, we strive to avoid spilling the coffee. This ability relies on the neural processes underlying the control of finger forces on a moment-to-moment basis. The brain activity lateralized to the contralateral hemisphere averaged over a trial and across the trials is known to be associated with the magnitude of grip force applied on an object. However, the mechanistic involvement of the variability in neural signals during grip force control remains unclear. In this study, we examined the dependence of neural variability over the frontal, central, and parietal regions assessed using noninvasive electroencephalography (EEG) on grip force magnitude during an isometric force control task. We hypothesized laterally specific modulation in EEG variability with higher magnitude of the grip force exerted during grip force control. We utilized an existing EEG dataset (64 channel) comprised of healthy young adults, who performed an isometric force control task while receiving visual feedback of the force applied. The force magnitude to be exerted on the instrumented object was cued to participants during the task, and varied pseudorandomly among 5, 10, and 15% of their maximum voluntary contraction (MVC) across the trials. We quantified neural variability via sample entropy (sequence-dependent measure) and standard deviation (sequence-independent measure) of the temporal EEG signal over the frontal, central, and parietal electrodes. The EEG sample entropy over the central electrodes showed lateralized, nonlinear, localized, modulation with force magnitude. Similar modulation was not observed over frontal or parietal EEG activity, nor for standard deviation in the EEG activity. Our findings highlight specificity in neural control of grip forces by demonstrating the modulation in sequence-dependent but not sequence-independent component of EEG variability. This modulation appeared to be lateralized, spatially constrained, and functionally dependent on the grip force magnitude. We discuss the relevance of these findings in scenarios where a finer precision is essential to enable grasp application, such as prosthesis and associated neural signal integration, and propose directions for future studies investigating the mechanistic role of neural entropy in grip force control.

Effects of transcutaneous spinal stimulation on spatiotemporal cortical activation patterns: A proof-of-concept EEG study

OBJECTIVE

Transcutaneous spinal cord stimulation (TSS) has been shown to be a promising non-invasive alternative to epidural spinal cord stimulation (ESS) for improving outcomes of people with spinal cord injury (SCI). However, studies on the effects of TSS on cortical activation are limited. Our objectives were to evaluate the spatiotemporal effects of TSS on brain activity, and determine changes in functional connectivity under several different stimulation conditions. As a control, we also assessed the effects of functional electrical stimulation (FES) on cortical activity.

APPROACH

Non-invasive scalp electroencephalography (EEG) was recorded during TSS or FES while five neurologically intact participants performed one of three lower-limb tasks while in the supine position: (1) A no contraction control task, (2) a rhythmic contraction task, or (3) a tonic contraction task. After EEG denoising and segmentation, independent components were clustered across subjects to characterize sensorimotor networks in the time and frequency domains. Independent components of the event related potentials (ERPs) were calculated for each cluster and condition. Next, a Generalized Partial Directed Coherence (gPDC) analysis was performed on each cluster to compare the functional connectivity between conditions and tasks.

RESULTS

Independent Component analysis of EEG during TSS resulted in three clusters identified at Brodmann areas (BA) 9, BA 6, and BA 4, which are areas associated with working memory, planning, and movement control. Lastly, we found significant (p < 0.05, adjusted for multiple comparisons) increases and decreases in functional connectivity of clusters during TSS, but not during FES when compared to the no stimulation conditions.

SIGNIFICANCE

The findings from this study provide evidence of how TSS recruits cortical networks during tonic and rhythmic lower limb movements. These results have implications for the development of spinal cord-based computer interfaces, and the design of neural stimulation devices for the treatment of pain and sensorimotor deficit.

Assessment of Biomechanical Predictors of Occurrence of Low-Amplitude N1 Potentials Evoked by Naturally Occurring Postural Instabilities

Naturally occurring postural instabilities that occur while standing and walking elicit specific cortical responses in the fronto-central regions (N1 potentials) followed by corrective balance responses to prevent falling. However, no framework could simultaneously track different biomechanical parameters preceding N1s, predict N1s, and assess their predictive power. Here, we propose a framework and show its utility by examining cortical activity (through electroencephalography [EEG]), ground reaction forces, and head acceleration in the anterior-posterior (AP) direction. Ten healthy young adults carried out a balance task of standing on a support surface with or without sway referencing in the AP direction, amplifying, or dampening natural body sway. Using independent components from the fronto-central cortical region obtained from subject-specific head models, we first robustly validated a prior approach on identifying low-amplitude N1 potentials before early signs of balance corrections. Then, a machine learning algorithm was used to evaluate different biomechanical parameters obtained before N1 potentials, to predict the occurrence of N1s. When different biomechanical parameters were directly compared, the time to boundary (TTB) was found to be the best predictor of the occurrence of upcoming low-amplitude N1 potentials during a balance task. Based on these findings, we confirm that the spatio-temporal characteristics of the center of pressure (COP) might serve as an essential parameter that can facilitate the early detection of postural instability in a balance task. Extending our framework to identify such biomarkers in dynamic situations like walking might improve the implementation of corrective balance responses through brain-machine-interfaces to reduce falls in the elderly.

A Mixed Filtering Approach for Real-Time Seizure State Tracking Using Multi-Channel Electroencephalography Data

Real-time continuous tracking of seizure state is necessary to develop feedback neuromodulation therapy that can prevent or terminate a seizure early. Due to its high temporal resolution, high scalp coverage, and non-invasive applicability, electroencephalography (EEG) is a good candidate for seizure tracking. In this research, we make multiple seizure state estimations using a mixed-filter and multiple channels found over the entire sensor space; then by applying a Kalman filter, we produce a single seizure state estimation made up of these individual estimations. Using a modified wrapper feature selection, we determine two optimal features of mixed data type, one continuous and one binary analyzing all available channels. These features are used in a state-space framework to model the continuous hidden seizure state. Expectation maximization is performed offline on the training and validation data sets to estimate unknown parameters. The seizure state estimation process is performed for multiple channels, and the seizure state estimation is derived using a square-root Kalman filter. A second expectation maximization step is utilized to estimate the unknown square-root Kalman filter parameters. This method is tested in a real-time applicable way for seizure state estimation. Applying this approach, we obtain a single seizure state estimation with quantitative information about the likelihood of a seizure occurring, which we call seizure probability. Our results on the experimental data (CHB-MIT EEG database) validate the proposed estimation method and we achieve an average accuracy, sensitivity, and specificity of 92.7%, 92.8%, and 93.4%, respectively. The potential applications of this seizure estimation model are for closed-loop neuromodulation and long-term quantitative analysis of seizure treatment efficacy.

Concerns in the Blurred Divisions between Medical and Consumer Neurotechnology

Neurotechnology has traditionally been central to the diagnosis and treatment of neurological disorders. While these devices have initially been utilized in clinical and research settings, recent advancements in neurotechnology have yielded devices that are more portable, user-friendly, and less expensive. These improvements allow laypeople to monitor their brain waves and interface their brains with external devices. Such improvements have led to the rise of wearable neurotechnology that is marketed to the consumer. While many of the consumer devices are marketed for innocuous applications, such as use in video games, there is potential for them to be repurposed for medical use. How do we manage neurotechnologies that skirt the line between medical and consumer applications and what can be done to ensure consumer safety? Here, we characterize neurotechnology based on medical and consumer applications and summarize currently marketed uses of consumer-grade wearable headsets. We lay out concerns that may arise due to the similar claims associated with both medical and consumer devices, the possibility of consumer devices being repurposed for medical uses, and the potential for medical uses of neurotechnology to influence commercial markets related to employment and self-enhancement.

Characterization of the Stages of Creative Writing With Mobile EEG Using Generalized Partial Directed Coherence

Two stages of the creative writing process were characterized through mobile scalp electroencephalography (EEG) in a 16-week creative writing workshop. Portable dry EEG systems (four channels: TP09, AF07, AF08, TP10) with synchronized head acceleration, video recordings, and journal entries, recorded mobile brain-body activity of Spanish heritage students. Each student's brain-body activity was recorded as they experienced spaces in Houston, Texas (“Preparation” stage), and while they worked on their creative texts (“Generation” stage). We used Generalized Partial Directed Coherence (gPDC) to compare the functional connectivity among both stages. There was a trend of higher gPDC in the Preparation stage from right temporo-parietal (TP10) to left anterior-frontal (AF07) brain scalp areas within 1–50 Hz, not reaching statistical significance. The opposite directionality was found for the Generation stage, with statistical significant differences (p < 0.05) restricted to the delta band (1–4 Hz). There was statistically higher gPDC observed for the inter-hemispheric connections AF07–AF08 in the delta and theta bands (1–8 Hz), and AF08 to TP09 in the alpha and beta (8–30 Hz) bands. The left anterior-frontal (AF07) recordings showed higher power localized to the gamma band (32–50 Hz) for the Generation stage. An ancillary analysis of Sample Entropy did not show significant difference. The information transfer from anterior-frontal to temporal-parietal areas of the scalp may reflect multisensory interpretation during the Preparation stage, while brain signals originating at temporal-parietal toward frontal locations during the Generation stage may reflect the final decision making process to translate the multisensory experience into a creative text.

A Roadmap Towards Standards for Neurally Controlled End Effectors

The control and manipulation of various types of end effectors such as powered exoskeletons, prostheses, and ‘neural’ cursors by brain-machine interface (BMI) systems has been the target of many research projects. A seamless “plug and play” interface between any BMI and end effector is desired, wherein similar user's intent cause similar end effectors to behave identically. This report is based on the outcomes of an IEEE Standards Association Industry Connections working group on End Effectors for Brain-Machine Interfacing that convened to identify and address gaps in the existing standards for BMI-based solutions with a focus on the end-effector component. A roadmap towards standardization of end effectors for BMI systems is discussed by identifying current device standards that are applicable for end effectors. While current standards address basic electrical and mechanical safety, and to some extent, performance requirements, several gaps exist pertaining to unified terminologies, data communication protocols, patient safety and risk mitigation.

Neural activity modulations and motor recovery following brain-exoskeleton interface mediated stroke rehabilitation

•

Motor intention based arm training targets activity-dependent neuroplasticity.

•

80% of stroke participants recovered clinically relevant functional movements.

•

Ipsi-lesional, delta-band EEG activity was highly correlated with motor recovery.

•

Results suggest higher activation of ipsi-lesional hemisphere post-intervention.

Brain-machine interfaces (BMI) based on scalp EEG have the potential to promote cortical plasticity following stroke, which has been shown to improve motor recovery outcomes. However, the efficacy of BMI enabled robotic training for upper-limb recovery is seldom quantified using clinical, EEG-based, and kinematics-based metrics. Further, a movement related neural correlate that can predict the extent of motor recovery still remains elusive, which impedes the clinical translation of BMI-based stroke rehabilitation.

To address above knowledge gaps, 10 chronic stroke individuals with stable baseline clinical scores were recruited to participate in 12 therapy sessions involving a BMI enabled powered exoskeleton for elbow training. On average, 132 ± 22 repetitions were performed per participant, per session. BMI accuracy across all sessions and subjects was 79 ± 18% with a false positives rate of 23 ± 20%.

Post-training clinical assessments found that FMA for upper extremity and ARAT scores significantly improved over baseline by 3.92 ± 3.73 and 5.35 ± 4.62 points, respectively. Also, 80% participants (7 with moderate-mild impairment, 1 with severe impairment) achieved minimal clinically important difference (MCID: FMA-UE >5.2 or ARAT >5.7) during the course of the study. Kinematic measures indicate that, on average, participants’ movements became faster and smoother. Moreover, modulations in movement related cortical potentials, an EEG-based neural correlate measured contralateral to the impaired arm, were significantly correlated with ARAT scores (ρ = 0.72, p < 0.05) and marginally correlated with FMA-UE (ρ = 0.63, p = 0.051). This suggests higher activation of ipsi-lesional hemisphere post-intervention or inhibition of competing contra-lesional hemisphere, which may be evidence of neuroplasticity and cortical reorganization following BMI mediated rehabilitation therapy.

Towards a Portable Magnetoencephalography Based Brain Computer Interface with Optically-Pumped Magnetometers

Brain Computer Interfaces (BCIs) allow individuals to control devices, machines and prostheses with their thoughts. Most feasibility studies with BCIs have utilized scalp electroencephalography (EEG), due to it being accessible, noninvasive, and portable. While BCIs have been studied with magnetoencephalography (MEG), the modality has limited applications due to the large immobile hardware. Here we propose that room-temperature, optically-pumped magnetometers (OPMs) can potentially serve a portable modality that can be used for BCIs. OPMs have the added advantage that low-frequency neuromagnetic fields are not affected by volume conduction, which is known to distort EEG signals. In this feasibility study, we tested an OPM system with a real-time BCI where able bodied participants controlled a cursor to reach two targets. This BCI system used alpha and beta-band power modulations associated with hand movements. Our preliminary results show significant alpha and beta-band desynchronization due to movement, as found in previous literature.

Classification and Transfer Learning of EEG during a Kinesthetic Motor Imagery Task using Deep Convolutional Neural Networks

The reliable classification of Electroencephalography (EEG) signals is a crucial step towards making EEG-controlled non-invasive neuro-exoskeleton rehabilitation a practical reality. EEG signals collected during motor imagery tasks have been proposed to act as a control signal for exoskeleton applications. Here, a Deep Convolutional Neural Network (DCNN) was optimized to classify a two-class kinesthetic motor imagery EEG dataset, leading to an optimized architecture consisting of four convolutional layers and three fully connected layers. Transfer learning, or the leveraging of data from past subjects to classify the intentions of a new subject, is important for rehabilitation as it helps to minimize the number of training sessions required from subjects who lack full motor functionality. The transfer learning training paradigm investigated through this study utilized region criticality trends to reduce the number of new subject training sessions and increase the classification performance when compared against a single-subject non-transfer-learning classifier.

An empirical comparison of neural networks and machine learning algorithms for EEG gait decoding

Previous studies of Brain Computer Interfaces (BCI) based on scalp electroencephalography (EEG) have demonstrated the feasibility of decoding kinematics for lower limb movements during walking. In this computational study, we investigated offline decoding analysis with different models and conditions to assess how they influence the performance and stability of the decoder. Specifically, we conducted three computational decoding experiments that investigated decoding accuracy: (1) based on delta band time-domain features, (2) when downsampling data, (3) of different frequency band features. In each experiment, eight different decoder algorithms were compared including the current state-of-the-art. Different tap sizes (sample window sizes) were also evaluated for a real-time applicability assessment. A feature of importance analysis was conducted to ascertain which features were most relevant for decoding; moreover, the stability to perturbations was assessed to quantify the robustness of the methods. Results indicated that generally the Gated Recurrent Unit (GRU) and Quasi Recurrent Neural Network (QRNN) outperformed other methods in terms of decoding accuracy and stability. Previous state-of-the-art Unscented Kalman Filter (UKF) still outperformed other decoders when using smaller tap sizes, with fast convergence in performance, but occurred at a cost to noise vulnerability. Downsampling and the inclusion of other frequency band features yielded overall improvement in performance. The results suggest that neural network-based decoders with downsampling or a wide range of frequency band features could not only improve decoder performance but also robustness with applications for stable use of BCIs.

Towards a Unified Framework for De-noising Neural Signals

Neural signals provide key information for decision-making processes in multiple disciplines including medicine, engineering, and neuroscience. The correct interpretation of these signals, however, requires substantial processing, especially when the signals exhibit low Signal to Noise Ratio (SNR). Electroencephalographic (EEG) signals are considered among this group and require effective handling of multiple types of artifactual components. Unfortunately, most available de-noising tools are suitable only for offline signal processing. For some artifacts (e.g., EEG motion artifacts), no established method of effective denoising exists for offline or real-time applications. Thus, there is a critical need for methods that can handle artifacts in neural signals with high performance, reliability and real-time capability. Here, we propose novel methods for handling some of the most challenging artifacts that exhibit highly complex dynamics, including motion artifacts. Having the same core sample-adaptive processing tool used for handling different types of artifacts, we present our efforts towards a unified framework for neural data artifact denoising with real-time compatibility.

Brain-machine interfaces for controlling lower-limb powered robotic systems

OBJECTIVE

Lower-limb, powered robotics systems such as exoskeletons and orthoses have emerged as novel robotic interventions to assist or rehabilitate people with walking disabilities. These devices are generally controlled by certain physical maneuvers, for example pressing buttons or shifting body weight. Although effective, these control schemes are not what humans naturally use. The usability and clinical relevance of these robotics systems could be further enhanced by brain-machine interfaces (BMIs). A number of preliminary studies have been published on this topic, but a systematic understanding of the experimental design, tasks, and performance of BMI-exoskeleton systems for restoration of gait is lacking.

APPROACH

To address this gap, we applied standard systematic review methodology for a literature search in PubMed and EMBASE databases and identified 11 studies involving BMI-robotics systems. The devices, user population, input and output of the BMIs and robot systems respectively, neural features, decoders, denoising techniques, and system performance were reviewed and compared.

MAIN RESULTS

Results showed BMIs classifying walk versus stand tasks are the most common. The results also indicate that electroencephalography (EEG) is the only recording method for humans. Performance was not clearly presented in most of the studies. Several challenges were summarized, including EEG denoising, safety, responsiveness and others.

SIGNIFICANCE

We conclude that lower-body powered exoskeletons with automated gait intention detection based on BMIs open new possibilities in the assistance and rehabilitation fields, although the current performance, clinical benefits and several key challenging issues indicate that additional research and development is required to deploy these systems in the clinic and at home. Moreover, rigorous EEG denoising techniques, suitable performance metrics, consistent trial reporting, and more clinical trials are needed to advance the field.

Deep learning for electroencephalogram (EEG) classification tasks: a review

OBJECTIVE

Electroencephalography (EEG) analysis has been an important tool in neuroscience with applications in neuroscience, neural engineering (e.g. Brain-computer interfaces, BCI's), and even commercial applications. Many of the analytical tools used in EEG studies have used machine learning to uncover relevant information for neural classification and neuroimaging. Recently, the availability of large EEG data sets and advances in machine learning have both led to the deployment of deep learning architectures, especially in the analysis of EEG signals and in understanding the information it may contain for brain functionality. The robust automatic classification of these signals is an important step towards making the use of EEG more practical in many applications and less reliant on trained professionals. Towards this goal, a systematic review of the literature on deep learning applications to EEG classification was performed to address the following critical questions: (1) Which EEG classification tasks have been explored with deep learning? (2) What input formulations have been used for training the deep networks? (3) Are there specific deep learning network structures suitable for specific types of tasks?

APPROACH

A systematic literature review of EEG classification using deep learning was performed on Web of Science and PubMed databases, resulting in 90 identified studies. Those studies were analyzed based on type of task, EEG preprocessing methods, input type, and deep learning architecture.

MAIN RESULTS

For EEG classification tasks, convolutional neural networks, recurrent neural networks, deep belief networks outperform stacked auto-encoders and multi-layer perceptron neural networks in classification accuracy. The tasks that used deep learning fell into five general groups: emotion recognition, motor imagery, mental workload, seizure detection, event related potential detection, and sleep scoring. For each type of task, we describe the specific input formulation, major characteristics, and end classifier recommendations found through this review.

SIGNIFICANCE

This review summarizes the current practices and performance outcomes in the use of deep learning for EEG classification. Practical suggestions on the selection of many hyperparameters are provided in the hope that they will promote or guide the deployment of deep learning to EEG datasets in future research.

Deep learning for electroencephalogram (EEG) classification tasks: a review

OBJECTIVE

Electroencephalography (EEG) analysis has been an important tool in neuroscience with applications in neuroscience, neural engineering (e.g. Brain-computer interfaces, BCI's), and even commercial applications. Many of the analytical tools used in EEG studies have used machine learning to uncover relevant information for neural classification and neuroimaging. Recently, the availability of large EEG data sets and advances in machine learning have both led to the deployment of deep learning architectures, especially in the analysis of EEG signals and in understanding the information it may contain for brain functionality. The robust automatic classification of these signals is an important step towards making the use of EEG more practical in many applications and less reliant on trained professionals. Towards this goal, a systematic review of the literature on deep learning applications to EEG classification was performed to address the following critical questions: (1) Which EEG classification tasks have been explored with deep learning? (2) What input formulations have been used for training the deep networks? (3) Are there specific deep learning network structures suitable for specific types of tasks?

APPROACH

A systematic literature review of EEG classification using deep learning was performed on Web of Science and PubMed databases, resulting in 90 identified studies. Those studies were analyzed based on type of task, EEG preprocessing methods, input type, and deep learning architecture.

MAIN RESULTS

For EEG classification tasks, convolutional neural networks, recurrent neural networks, deep belief networks outperform stacked auto-encoders and multi-layer perceptron neural networks in classification accuracy. The tasks that used deep learning fell into five general groups: emotion recognition, motor imagery, mental workload, seizure detection, event related potential detection, and sleep scoring. For each type of task, we describe the specific input formulation, major characteristics, and end classifier recommendations found through this review.

SIGNIFICANCE

This review summarizes the current practices and performance outcomes in the use of deep learning for EEG classification. Practical suggestions on the selection of many hyperparameters are provided in the hope that they will promote or guide the deployment of deep learning to EEG datasets in future research.

Improving robotic stroke rehabilitation by incorporating neural intent detection: Preliminary results from a clinical trial

This paper presents the preliminary findings of a multi-year clinical study evaluating the effectiveness of adding a brain-machine interface (BMI) to the MAHI-Exo II, a robotic upper limb exoskeleton, for elbow flexion/extension rehabilitation in chronic stroke survivors. The BMI was used to trigger robot motion when movement intention was detected from subjects' neural signals, thus requiring that subjects be mentally engaged during robotic therapy. The first six subjects to complete the program have shown improvements in both Fugl-Meyer Upper-Extremity scores as well as in kinematic movement quality measures that relate to movement planning, coordination, and control. These results are encouraging and suggest that increasing subject engagement during therapy through the addition of an intent-detecting BMI enhances the effectiveness of standard robotic rehabilitation.

Full body mobile brain-body imaging data during unconstrained locomotion on stairs, ramps, and level ground

Human locomotion is a complex process that requires the integration of central and peripheral nervous signalling. Understanding the brain's involvement in locomotion is challenging and is traditionally investigated during locomotor imagination or observation. However, stationary imaging methods lack the ability to infer information about the peripheral and central signalling during actual task execution. In this report, we present a dataset containing simultaneously recorded electroencephalography (EEG), lower-limb electromyography (EMG), and full body motion capture recorded from ten able-bodied individuals. The subjects completed an average of twenty trials on an experimental gait course containing level-ground, ramps, and stairs. We recorded 60-channel EEG from the scalp and 4-channel EOG from the face and temples. Surface EMG was recorded from six muscle sites bilaterally on the thigh and shank. The motion capture system consisted of seventeen wireless IMUs, allowing for unconstrained ambulation in the experimental space. In this report, we present the rationale for collecting these data, a detailed explanation of the experimental setup, and a brief validation of the data quality.

Modulation of Neural Activity during Guided Viewing of Visual Art

Mobile Brain-Body Imaging (MoBI) technology was deployed to record multi-modal data from 209 participants to examine the brain's response to artistic stimuli at the Museo de Arte Contemporáneo (MARCO) in Monterrey, México. EEG signals were recorded as the subjects walked through the exhibit in guided groups of 6-8 people. Moreover, guided groups were either provided with an explanation of each art piece (Guided-E), or given no explanation (Guided-NE). The study was performed using portable Muse (InteraXon, Inc, Toronto, ON, Canada) headbands with four dry electrodes located at AF7, AF8, TP9, and TP10. Each participant performed a baseline (BL) control condition devoid of artistic stimuli and selected his/her favorite piece of art (FP) during the guided tour. In this study, we report data related to participants' demographic information and aesthetic preference as well as effects of art viewing on neural activity (EEG) in a select subgroup of 18-30 year-old subjects (Nc = 25) that generated high-quality EEG signals, on both BL and FP conditions. Dependencies on gender, sensor placement, and presence or absence of art explanation were also analyzed. After denoising, clustering of spectral EEG models was used to identify neural patterns associated with BL and FP conditions. Results indicate statistically significant suppression of beta band frequencies (15-25 Hz) in the prefrontal electrodes (AF7 and AF8) during appreciation of subjects' favorite painting, compared to the BL condition, which was significantly different from EEG responses to non-favorite paintings (NFP). No significant differences in brain activity in relation to the presence or absence of explanation during exhibit tours were found. Moreover, a frontal to posterior asymmetry in neural activity was observed, for both BL and FP conditions. These findings provide new information about frequency-related effects of preferred art viewing in brain activity, and support the view that art appreciation is independent of the artists' intent or original interpretation and related to the individual message that viewers themselves provide to each piece.

Electrocortical correlates of human level-ground, slope, and stair walking

This study investigated electrocortical dynamics of human walking across different unconstrained walking conditions (i.e., level ground (LW), ramp ascent (RA), and stair ascent (SA)). Non-invasive active-electrode scalp electroencephalography (EEG) signals were recorded and a systematic EEG processing method was implemented to reduce artifacts. Source localization combined with independent component analysis and k-means clustering revealed the involvement of four clusters in the brain during the walking tasks: Left and Right Occipital Lobe (LOL, ROL), Posterior Parietal Cortex (PPC), and Central Sensorimotor Cortex (SMC). Results showed that the changes of spectral power in the PPC and SMC clusters were associated with the level of motor task demands. Specifically, we observed α and β suppression at the beginning of the gait cycle in both SA and RA walking (relative to LW) in the SMC. Additionally, we observed significant β rebound (synchronization) at the initial swing phase of the gait cycle, which may be indicative of active cortical signaling involved in maintaining the current locomotor state. An increase of low γ band power in this cluster was also found in SA walking. In the PPC, the low γ band power increased with the level of task demands (from LW to RA and SA). Additionally, our results provide evidence that electrocortical amplitude modulations (relative to average gait cycle) are correlated with the level of difficulty in locomotion tasks. Specifically, the modulations in the PPC shifted to higher frequency bands when the subjects walked in RA and SA conditions. Moreover, low γ modulations in the central sensorimotor area were observed in the LW walking and shifted to lower frequency bands in RA and SA walking. These findings extend our understanding of cortical dynamics of human walking at different level of locomotion task demands and reinforces the growing body of literature supporting a shared-control paradigm between spinal and cortical networks during locomotion.

Deployment of Mobile EEG Technology in an Art Museum Setting: Evaluation of Signal Quality and Usability

Electroencephalography (EEG) has emerged as a powerful tool for quantitatively studying the brain that enables natural and mobile experiments. Recent advances in EEG have allowed for the use of dry electrodes that do not require a conductive medium between the recording electrode and the scalp. The overall goal of this research was to gain an understanding of the overall usability and signal quality of dry EEG headsets compared to traditional gel-based systems in an unconstrained environment. EEG was used to collect Mobile Brain-body Imaging (MoBI) data from 432 people as they experienced an art exhibit in a public museum. The subjects were instrumented with either one of four dry electrode EEG systems or a conventional gel electrode EEG system. Each of the systems was evaluated based on the signal quality and usability in a real-world setting. First, we describe the various artifacts that were characteristic of each of the systems. Second, we report on each system's usability and their limitations in a mobile setting. Third, to evaluate signal quality for task discrimination and characterization, we employed a data driven clustering approach on the data from 134 of the 432 subjects (those with reliable location tracking information and usable EEG data) to evaluate the power spectral density (PSD) content of the EEG recordings. The experiment consisted of a baseline condition in which the subjects sat quietly facing a white wall for 1 min. Subsequently, the participants were encouraged to explore the exhibit for as long as they wished (piece-viewing). No constraints were placed upon the individual in relation to action, time, or navigation of the exhibit. In this freely-behaving approach, the EEG systems varied in their capacity to record characteristic modulations in the EEG data, with the gel-based system more clearly capturing stereotypical alpha and beta-band modulations.

Unscented Kalman filter for neural decoding of human treadmill walking from non-invasive electroencephalography

The feasibility of decoding lower limb kinematics in human treadmill walking from noninvasive electroencephalography (EEG) has been demonstrated with linear Wiener filter. However, nonlinear relationship between neural activities and limb movements may challenge the linear decoders in real-time brain computer interface (BCI) applications. In this study, we propose a nonlinear neural decoder using an Unscented Kalman Filter (UKF) to infer lower limb joint angles from noninvasive scalp EEG signals during human treadmill walking. Our results demonstrate that lower limb joint angles during treadmill walking can be decoded from the fluctuations in the amplitude of slow cortical potentials in the delta band (0.1-3Hz). Overall, the average decoding accuracy were 0.43 ± 0.18 for Pearson's r value and 1.82 ± 3.07 for signal to noise ratio (SNR), and robust to ocular, muscle, or movement artifacts. Moreover, the signal preprocessing scheme and the design of UKF allow the implementation of the proposed EEG-based BCI for real-time applications. This has implications for the development of closed-loop EEG-based BCI systems for gait rehabilitation after stroke.

Electrocortical amplitude modulations of human level-ground, slope, and stair walking

This study investigates if the electrocortical amplitude modulations relative to the mean gait cycle are different across walking conditions (i.e., level-ground (LW), ramp ascent (RA), and stair ascent (SA)). Non-invasive electroencephalography (EEG) signals were recorded and a systematic EEG processing method was implemented to reduce artifacts. Source localization using independent component analysis and k-means clustering revealed the involvement of four clusters in the brain (Left and Right Occipital Lobe, Posterior Parietal Cortex (PPC), and Sensorimotor Area) during the walking tasks. We found that electrocortical amplitude modulations varied across different walking conditions. Specifically, our results showed that the modulations in the PPC shifted to higher frequency bands when the subjects walked in RA and SA conditions. Moreover, we found low γ modulations in the sensorimotor area in LW walking and the modulations in this cluster shifted to lower frequency bands in RA and SA walking. These results are a promising step toward the development of a non-invasive Neural-machine Interface (NMI) for locomotion mode recognition.

Multi-Trial Gait Adaptation of Healthy Individuals during Visual Kinematic Perturbations

Optimizing rehabilitation strategies requires understanding the effects of contextual cues on adaptation learning. Prior studies have examined these effects on the specificity of split-belt walking adaptation, showing that contextual visual cues can be manipulated to modulate the magnitude, transfer, and washout of split-belt-induced learning in humans. Specifically, manipulating the availability of vision during training or testing phases of learning resulted in differences in adaptive mechanisms for temporal and spatial features of walking. However, multi-trial locomotor training has been rarely explored when using visual kinematic gait perturbations. In this study, we investigated multi-trial locomotor adaptation in ten healthy individuals while applying visual kinematic perturbations. Subjects were instructed to control a moving cursor, which represented the position of their heel, to follow a prescribed heel path profile displayed on a monitor. The perturbations were introduced by scaling all of the lower limb joint angles by a factor of 0.7 (i.e., a gain change), resulting in visual feedback errors between subjects' heel trajectories and the prescribed path profiles. Our findings suggest that, with practice, the subjects learned, albeit with different strategies, to reduce the tracking errors and showed faster response time in later trials. Moreover, the gait symmetry indices, in both the spatial and temporal domains, changed significantly during gait adaptation (P < 0.001). After-effects were present in the temporal gait symmetry index whens the visual perturbations were removed in the post-exposure period (P < 0.001), suggesting adaptation learning. These findings may have implications for developing novel gait rehabilitation interventions.

Noninvasive EEG correlates of overground and stair walking

Automated walking intention detection remains a challenge in lower-limb neuroprosthetic systems. Here, we assess the feasibility of extracting motor intent from scalp electroencephalography (EEG). First, we evaluated the corticomuscular coherence between central EEG electrodes (C1, Cz, C2) and muscles of the shank and thigh during walking on level ground and stairs. Second, we trained decoders to predict the linear envelope of the surface electromyogram (EMG). We observed significant EEG-led corticomuscular coupling between electrodes and sEMG (tibialis anterior) in the high delta (3-4 Hz) and low theta (4-5 Hz) frequency bands during level walking, indicating efferent signaling from the cortex to peripheral motor neurons. The coherence was increased between EEG and vastus lateralis and tibialis anterior in the delta band (<; 2 Hz) during stair ascent, indicating a task specific modulation in corticomuscular coupling. However, EMG was the leading signal for biceps femoris and gastrocnemius coherence during stair ascent, possibly representing afferent feedback loops from periphery to the motor cortex. Decoder validation showed that EEG signals contained information about the sEMG patterns during over ground walking, however, the accuracy of the predicted sEMG patterns decreased during the stair condition. Overall, these initial findings support the feasibility of integrating sEMG and EEG into a hybrid decoder for volitional control of lower limb neuroprostheses.

Risk management and regulations for lower limb medical exoskeletons: a review

Gait disability is a major health care problem worldwide. Powered exoskeletons have recently emerged as devices that can enable users with gait disabilities to ambulate in an upright posture, and potentially bring other clinical benefits. In 2014, the US Food and Drug Administration approved marketing of the ReWalk™ Personal Exoskeleton as a class II medical device with special controls. Since then, Indego™ and Ekso™ have also received regulatory approval. With similar trends worldwide, this industry is likely to grow rapidly. On the other hand, the regulatory science of powered exoskeletons is still developing. The type and extent of probable risks of these devices are yet to be understood, and industry standards are yet to be developed. To address this gap, Manufacturer and User Facility Device Experience, Clinicaltrials.gov, and PubMed databases were searched for reports of adverse events and inclusion and exclusion criteria involving the use of lower limb powered exoskeletons. Current inclusion and exclusion criteria, which can determine probable risks, were found to be diverse. Reported adverse events and identified risks of current devices are also wide-ranging. In light of these findings, current regulations, standards, and regulatory procedures for medical device applications in the USA, Europe, and Japan were also compared. There is a need to raise awareness of probable risks associated with the use of powered exoskeletons and to develop adequate countermeasures, standards, and regulations for these human-machine systems. With appropriate risk mitigation strategies, adequate standards, comprehensive reporting of adverse events, and regulatory oversight, powered exoskeletons may one day allow individuals with gait disabilities to safely and independently ambulate.

Multiple Kernel Based Region Importance Learning for Neural Classification of Gait States from EEG Signals

With the development of Brain Machine Interface (BMI) systems, people with motor disabilities are able to control external devices to help them restore movement abilities. Longitudinal validation of these systems is critical not only to assess long-term performance reliability but also to investigate adaptations in electrocortical patterns due to learning to use the BMI system. In this paper, we decode the patterns of user's intended gait states (e.g., stop, walk, turn left, and turn right) from scalp electroencephalography (EEG) signals and simultaneously learn the relative importance of different brain areas by using the multiple kernel learning (MKL) algorithm. The region of importance (ROI) is identified during training the MKL for classification. The efficacy of the proposed method is validated by classifying different movement intentions from two subjects—an able-bodied and a spinal cord injury (SCI) subject. The preliminary results demonstrate that frontal and fronto-central regions are the most important regions for the tested subjects performing gait movements, which is consistent with the brain regions hypothesized to be involved in the control of lower-limb movements. However, we observed some regional changes comparing the able-bodied and the SCI subject. Moreover, in the longitudinal experiments, our findings exhibit the cortical plasticity triggered by the BMI use, as the classification accuracy and the weights for important regions—in sensor space—generally increased, as the user learned to control the exoskeleton for movement over multiple sessions.

Graphonomics and its contribution to the field of motor behavior: A position statement

The term graphonomics was conceived in the early 1980s; it defined a multidisciplinary emerging field focused on handwriting and drawing movements. Researchers in the field of graphonomics have made important contribution to the field of motor behavior by developing models aimed to conceptualize the production of fine motor movements using graphical tools. Although skeptics have argued that recent technological advancements would reduce the impact of graphonomic research, a shift of focus within in the field of graphonomics into fine motor tasks in general proves the resilience of the field. Moreover, it has been suggested that the use of fine motor movements due to technological advances has increased in importance in everyday life. It is concluded that the International Graphonomics Society can have a leading role in fostering collaborative multidisciplinary efforts and can help with the dissemination of findings contributing to the field of human movement sciences to a larger public.

Gait adaptation to visual kinematic perturbations using a real-time closed-loop brain-computer interface to a virtual reality avatar

OBJECTIVE

The control of human bipedal locomotion is of great interest to the field of lower-body brain-computer interfaces (BCIs) for gait rehabilitation. While the feasibility of closed-loop BCI systems for the control of a lower body exoskeleton has been recently shown, multi-day closed-loop neural decoding of human gait in a BCI virtual reality (BCI-VR) environment has yet to be demonstrated. BCI-VR systems provide valuable alternatives for movement rehabilitation when wearable robots are not desirable due to medical conditions, cost, accessibility, usability, or patient preferences.

APPROACH

In this study, we propose a real-time closed-loop BCI that decodes lower limb joint angles from scalp electroencephalography (EEG) during treadmill walking to control a walking avatar in a virtual environment. Fluctuations in the amplitude of slow cortical potentials of EEG in the delta band (0.1-3 Hz) were used for prediction; thus, the EEG features correspond to time-domain amplitude modulated potentials in the delta band. Virtual kinematic perturbations resulting in asymmetric walking gait patterns of the avatar were also introduced to investigate gait adaptation using the closed-loop BCI-VR system over a period of eight days.

MAIN RESULTS

Our results demonstrate the feasibility of using a closed-loop BCI to learn to control a walking avatar under normal and altered visuomotor perturbations, which involved cortical adaptations. The average decoding accuracies (Pearson's r values) in real-time BCI across all subjects increased from (Hip: 0.18 ± 0.31; Knee: 0.23 ± 0.33; Ankle: 0.14 ± 0.22) on Day 1 to (Hip: 0.40 ± 0.24; Knee: 0.55 ± 0.20; Ankle: 0.29 ± 0.22) on Day 8.

SIGNIFICANCE

These findings have implications for the development of a real-time closed-loop EEG-based BCI-VR system for gait rehabilitation after stroke and for understanding cortical plasticity induced by a closed-loop BCI-VR system.

Powered exoskeletons for bipedal locomotion after spinal cord injury

OBJECTIVE

Powered exoskeletons promise to increase the quality of life of people with lower-body paralysis or weakened legs by assisting or restoring legged mobility while providing health benefits across multiple physiological systems. Here, a systematic review of the literature on powered exoskeletons addressed critical questions: What is the current evidence of clinical efficacy for lower-limb powered exoskeletons? What are the benefits and risks for individuals with spinal cord injury (SCI)? What are the levels of injury considered in such studies? What are their outcome measures? What are the opportunities for the next generation exoskeletons?

APPROACH

A systematic search of online databases was performed to identify clinical trials and safety or efficacy studies with lower-limb powered exoskeletons for individuals with SCI. Twenty-two studies with eight powered exoskeletons thus selected, were analyzed based on the protocol design, subject demographics, study duration, and primary/secondary outcome measures for assessing exoskeleton's performance in SCI subjects.

MAIN RESULTS

Findings show that the level of injury varies across studies, with T10 injuries being represented in 45.4% of the studies. A categorical breakdown of outcome measures revealed 63% of these measures were gait and ambulation related, followed by energy expenditure (16%), physiological improvements (13%), and usability and comfort (8%). Moreover, outcome measures varied across studies, and none had measures spanning every category, making comparisons difficult.

SIGNIFICANCE

This review of the literature shows that a majority of current studies focus on thoracic level injury as well as there is an emphasis on ambulatory-related primary outcome measures. Future research should: 1) develop criteria for optimal selection and training of patients most likely to benefit from this technology, 2) design multimodal gait intention detection systems that engage and empower the user, 3) develop real-time monitoring and diagnostic capabilities, and 4) adopt comprehensive metrics for assessing safety, benefits, and usability.

Design and Optimization of an EEG-Based Brain Machine Interface (BMI) to an Upper-Limb Exoskeleton for Stroke Survivors

This study demonstrates the feasibility of detecting motor intent from brain activity of chronic stroke patients using an asynchronous electroencephalography (EEG)-based brain machine interface (BMI). Intent was inferred from movement related cortical potentials (MRCPs) measured over an optimized set of EEG electrodes. Successful intent detection triggered the motion of an upper-limb exoskeleton (MAHI Exo-II), to guide movement and to encourage active user participation by providing instantaneous sensory feedback. Several BMI design features were optimized to increase system performance in the presence of single-trial variability of MRCPs in the injured brain: (1) an adaptive time window was used for extracting features during BMI calibration; (2) training data from two consecutive days were pooled for BMI calibration to increase robustness to handle the day-to-day variations typical of EEG, and (3) BMI predictions were gated by residual electromyography (EMG) activity from the impaired arm, to reduce the number of false positives. This patient-specific BMI calibration approach can accommodate a broad spectrum of stroke patients with diverse motor capabilities. Following BMI optimization on day 3, testing of the closed-loop BMI-MAHI exoskeleton, on 4th and 5th days of the study, showed consistent BMI performance with overall mean true positive rate (TPR) = 62.7 ± 21.4% on day 4 and 67.1 ± 14.6% on day 5. The overall false positive rate (FPR) across subjects was 27.74 ± 37.46% on day 4 and 27.5 ± 35.64% on day 5; however for two subjects who had residual motor function and could benefit from the EMG-gated BMI, the mean FPR was quite low (< 10%). On average, motor intent was detected -367 ± 328 ms before movement onset during closed-loop operation. These findings provide evidence that closed-loop EEG-based BMI for stroke patients can be designed and optimized to perform well across multiple days without system recalibration.

Negligible Motion Artifacts in Scalp Electroencephalography (EEG) During Treadmill Walking

Recent mobile brain/body imaging (MoBI) techniques based on active electrode scalp electroencephalogram (EEG) allow the acquisition and real-time analysis of brain dynamics during active unrestrained motor behavior involving whole body movements such as treadmill walking, over-ground walking and other locomotive and non-locomotive tasks. Unfortunately, MoBI protocols are prone to physiological and non-physiological artifacts, including motion artifacts that may contaminate the EEG recordings. A few attempts have been made to quantify these artifacts during locomotion tasks but with inconclusive results due in part to methodological pitfalls. In this paper, we investigate the potential contributions of motion artifacts in scalp EEG during treadmill walking at three different speeds (1.5, 3.0, and 4.5 km/h) using a wireless 64 channel active EEG system and a wireless inertial sensor attached to the subject’s head. The experimental setup was designed according to good measurement practices using state-of-the-art commercially available instruments, and the measurements were analyzed using Fourier analysis and wavelet coherence approaches. Contrary to prior claims, the subjects’ motion did not significantly affect their EEG during treadmill walking although precaution should be taken when gait speeds approach 4.5 km/h. Overall, these findings suggest how MoBI methods may be safely deployed in neural, cognitive, and rehabilitation engineering applications.

Classification of finger vibrotactile input using scalp EEG

While there are many output brain-computer interface (output BCIs) studies, few have examined the input pathway, namely decoding the sensory input. To examine the possibility of building a BCI with sensory input using scalp electroencephalography (EEG), this study builds a classifier based on Local Fisher Discriminant Analysis (LFDA) and Gaussian Mixture Model (GMM) to classify neural activity generated by vibrotactile sensory stimuli delivered to the fingers. Small vibrators were placed on the fingertips of the participant. They vibrated one by one in a random sequence while the participant sat still with eyes closed. EEG data were recorded and later used to classify which finger was vibrated. There were two tasks: one focusing on differentiating between ipsilateral fingers, the other one focusing on differentiating contralateral fingers. Decoding accuracies were high in both tasks: 97.6% and 99.3% respectively. Event-related EEG features in both amplitude and power domain are discussed.

Human-Centered Design of Wearable Neuroprostheses and Exoskeletons

Human-centered design of wearable robots involves the development of innovative science and technologies that minimize the mismatch between humans’ and machines’ capabilities, leading to their intuitive integration and confluent interaction. Here, we summarize our human-centered approach to the design of closed-loop brain-machine interfaces (BMI) to powered prostheses and exoskeletons that allow people to act beyond their impaired or diminished physical or sensory-motor capabilities. The goal is to develop multifunctional human-machine interfaces with integrated diagnostic, assistive and therapeutic functions. Moreover, these complex human-machine systems should be effective, reliable, safe and engaging and support the patient in performing intended actions with minimal effort and errors with adequate interaction time. To illustrate our approach, we review an example of a user-in-the-loop, patient-centered, non-invasive BMI system to a powered exoskeleton for persons with paraplegia. We conclude with a summary of challenges to the translation of these complex human-machine systems to the end-user.

A Closed-loop Brain Computer Interface to a Virtual Reality Avatar: Gait Adaptation to Visual Kinematic Perturbations

The control of human bipedal locomotion is of great interest to the field of lower-body brain computer interfaces (BCIs) for rehabilitation of gait. While the feasibility of a closed-loop BCI system for the control of a lower body exoskeleton has been recently shown, multi-day closed-loop neural decoding of human gait in a virtual reality (BCI-VR) environment has yet to be demonstrated. In this study, we propose a real-time closed-loop BCI that decodes lower limb joint angles from scalp electroencephalography (EEG) during treadmill walking to control the walking movements of a virtual avatar. Moreover, virtual kinematic perturbations resulting in asymmetric walking gait patterns of the avatar were also introduced to investigate gait adaptation using the closed-loop BCI-VR system over a period of eight days. Our results demonstrate the feasibility of using a closed-loop BCI to learn to control a walking avatar under normal and altered visuomotor perturbations, which involved cortical adaptations. These findings have implications for the development of BCI-VR systems for gait rehabilitation after stroke and for understanding cortical plasticity induced by a closed-loop BCI system.

Observation-based training for neuroprosthetic control of grasping by amputees

Current brain-machine interfaces (BMIs) allow upper limb amputees to position robotic arms with a high degree of accuracy, but lack the ability to control hand pre-shaping for grasping different objects. We have previously shown that low frequency (0.1-1 Hz) time domain cortical activity recorded at the scalp via electroencephalography (EEG) encodes information about grasp pre-shaping. To transfer this technology to clinical populations such as amputees, the challenge lies in constructing BMI models in the absence of overt training hand movements. Here we show that it is possible to train BMI models using observed grasping movements performed by a robotic hand attached to amputees' residual limb. Three transradial amputees controlled the grasping motion of an attached robotic hand via their EEG, following the action-observation training phase. Over multiple sessions, subjects successfully grasped the presented object (a bottle or a credit card) in 53±16 % of trials, demonstrating the validity of the BMI models. Importantly, the validation of the BMI model was through closed-loop performance, which demonstrates generalization of the model to unseen data. These results suggest `mirror neuron system' properties captured by delta band EEG that allows neural representation for action observation to be used for action control in an EEG-based BMI system.

An integrated neuro-robotic interface for stroke rehabilitation using the NASA X1 powered lower limb exoskeleton

Stroke remains a leading cause of disability, limiting independent ambulation in survivors, and consequently affecting quality of life (QOL). Recent technological advances in neural interfacing with robotic rehabilitation devices are promising in the context of gait rehabilitation. Here, the X1, NASA's powered robotic lower limb exoskeleton, is introduced as a potential diagnostic, assistive, and therapeutic tool for stroke rehabilitation. Additionally, the feasibility of decoding lower limb joint kinematics and kinetics during walking with the X1 from scalp electroencephalographic (EEG) signals--the first step towards the development of a brain-machine interface (BMI) system to the X1 exoskeleton--is demonstrated.

An exploration of grip force regulation with a low-impedance myoelectric prosthesis featuring referred haptic feedback

Background

Haptic display technologies are well suited to relay proprioceptive, force, and contact cues from a prosthetic terminal device back to the residual limb and thereby reduce reliance on visual feedback. The ease with which an amputee interprets these haptic cues, however, likely depends on whether their dynamic signal behavior corresponds to expected behaviors—behaviors consonant with a natural limb coupled to the environment. A highly geared motor in a terminal device along with the associated high back-drive impedance influences dynamic interactions with the environment, creating effects not encountered with a natural limb. Here we explore grasp and lift performance with a backdrivable (low backdrive impedance) terminal device placed under proportional myoelectric position control that features referred haptic feedback.

Methods

We fabricated a back-drivable terminal device that could be used by amputees and non-amputees alike and drove aperture (or grip force, when a stiff object was in its grasp) in proportion to a myoelectric signal drawn from a single muscle site in the forearm. In randomly ordered trials, we assessed the performance of N=10 participants (7 non-amputee, 3 amputee) attempting to grasp and lift an object using the terminal device under three feedback conditions (no feedback, vibrotactile feedback, and joint torque feedback), and two object weights that were indiscernible by vision.

Results

Both non-amputee and amputee participants scaled their grip force according to the object weight. Our results showed only minor differences in grip force, grip/load force coordination, and slip as a function of sensory feedback condition, though the grip force at the point of lift-off for the heavier object was significantly greater for amputee participants in the presence of joint torque feedback. An examination of grip/load force phase plots revealed that our amputee participants used larger safety margins and demonstrated less coordination than our non-amputee participants.

Conclusions

Our results suggest that a backdrivable terminal device may hold advantages over non-backdrivable devices by allowing grip/load force coordination consistent with behaviors observed in the natural limb. Likewise, the inconclusive effect of referred haptic feedback on grasp and lift performance suggests the need for additional testing that includes adequate training for participants.

Your Brain on Art: Emergent Cortical Dynamics During Aesthetic Experiences

The brain response to conceptual art was studied with mobile electroencephalography (EEG) to examine the neural basis of aesthetic experiences. In contrast to most studies of perceptual phenomena, participants were moving and thinking freely as they viewed the exhibit The Boundary of Life is Quietly Crossed by Dario Robleto at the Menil Collection-Houston. The brain activity of over 400 subjects was recorded using dry-electrode and one reference gel-based EEG systems over a period of 3 months. Here, we report initial findings based on the reference system. EEG segments corresponding to each art piece were grouped into one of three classes (complex, moderate, and baseline) based on analysis of a digital image of each piece. Time, frequency, and wavelet features extracted from EEG were used to classify patterns associated with viewing art, and ranked based on their relevance for classification. The maximum classification accuracy was 55% (chance = 33%) with delta and gamma features the most relevant for classification. Functional analysis revealed a significant increase in connection strength in localized brain networks while subjects viewed the most aesthetically pleasing art compared to viewing a blank wall. The direction of signal flow showed early recruitment of broad posterior areas followed by focal anterior activation. Significant differences in the strength of connections were also observed across age and gender. This work provides evidence that EEG, deployed on freely behaving subjects, can detect selective signal flow in neural networks, identify significant differences between subject groups, and report with greater-than-chance accuracy the complexity of a subject's visual percept of aesthetically pleasing art. Our approach, which allows acquisition of neural activity “in action and context,” could lead to understanding of how the brain integrates sensory input and its ongoing internal state to produce the phenomenon which we term aesthetic experience.

A Novel Experimental and Analytical Approach to the Multimodal Neural Decoding of Intent During Social Interaction in Freely-behaving Human Infants

Understanding typical and atypical development remains one of the fundamental questions in developmental human neuroscience. Traditionally, experimental paradigms and analysis tools have been limited to constrained laboratory tasks and contexts due to technical limitations imposed by the available set of measuring and analysis techniques and the age of the subjects. These limitations severely limit the study of developmental neural dynamics and associated neural networks engaged in cognition, perception and action in infants performing “in action and in context”. This protocol presents a novel approach to study infants and young children as they freely organize their own behavior, and its consequences in a complex, partly unpredictable and highly dynamic environment. The proposed methodology integrates synchronized high-density active scalp electroencephalography (EEG), inertial measurement units (IMUs), video recording and behavioral analysis to capture brain activity and movement non-invasively in freely-behaving infants. This setup allows for the study of neural network dynamics in the developing brain, in action and context, as these networks are recruited during goal-oriented, exploration and social interaction tasks.

Detecting movement intent from scalp EEG in a novel upper limb robotic rehabilitation system for stroke

Stroke can be a source of significant upper extremity dysfunction and affect the quality of life (QoL) in survivors. In this context, novel rehabilitation approaches employing robotic rehabilitation devices combined with brain-machine interfaces can greatly help in expediting functional recovery in these individuals by actively engaging the user during therapy. However, optimal training conditions and parameters for these novel therapeutic systems are still unknown. Here, we present preliminary findings demonstrating successful movement intent detection from scalp electroencephalography (EEG) during robotic rehabilitation using the MAHI Exo-II in an individual with hemiparesis following stroke. These findings have strong clinical implications for the development of closed-loop brain-machine interfaces to robotic rehabilitation systems.

Decoding of intentional actions from scalp electroencephalography (EEG) in freely-behaving infants

The mirror neuron system (MNS) in humans is thought to enable an individual's understanding of the meaning of actions performed by others and the potential imitation and learning of those actions. In humans, electroencephalographic (EEG) changes in sensorimotor a-band at central electrodes, which desynchronizes both during execution and observation of goal-directed actions (i.e., μ suppression), have been considered an analog to MNS function. However, methodological and developmental issues, as well as the nature of generalized μ suppression to imagined, observed, and performed actions, have yet to provide a mechanistic relationship between EEG μ-rhythm and MNS function, and the extent to which EEG can be used to infer intent during MNS tasks remains unknown. In this study we present a novel methodology using active EEG and inertial sensors to record brain activity and behavioral actions from freely-behaving infants during exploration, imitation, attentive rest, pointing, reaching and grasping, and interaction with an actor. We used 5-band (1-4Hz) EEG as input to a dimensionality reduction algorithm (locality-preserving Fisher's discriminant analysis, LFDA) followed by a neural classifier (Gaussian mixture models, GMMs) to decode the each MNS task performed by freely-behaving 6-24 month old infants during interaction with an adult actor. Here, we present results from a 20-month male infant to illustrate our approach and show the feasibility of EEG-based classification of freely occurring MNS behaviors displayed by an infant. These results, which provide an alternative to the μ-rhythm theory of MNS function, indicate the informative nature of EEG in relation to intentionality (goal) for MNS tasks which may support action-understanding and thus bear implications for advancing the understanding of MNS function.

Classification of stand-to-sit and sit-to-stand movement from low frequency EEG with locality preserving dimensionality reduction

Recent studies have demonstrated decoding of lower extremity limb kinematics from noninvasive electroencephalography (EEG), showing feasibility for development of an EEG-based brain-machine interface (BMI) to restore mobility following paralysis. Here, we present a new technique that preserves the statistical richness of EEG data to classify movement state from time-embedded low frequency EEG signals. We tested this new classifier, using cross-validation procedures, during sit-to-stand and stand-to-sit activity in 10 subjects and found decoding accuracy of greater than 95% on average. These results suggest that this classification technique could be used in a BMI system that, when combined with a robotic exoskeleton, can restore functional movement to individuals with paralysis.