Hybrid EEG/EOG-based brain/neural hand exoskeleton restores fully independent daily living activities after quadriplegia

Real-world exoskeletons are better than those in the movie Atlas

The recent movie Atlas misses fundamental robotics advances in self-stabilization and human-robot interaction.

Hand assistive device with suction cup (HADS) technology for poststroke patients

A stroke is a neurological disease that primarily causes paralysis. Besides paraplegia, all other types of paralysis affect the upper extremity. Advanced technologies, such as wearable devices and rehabilitation regimens, are also being developed to enhance the functional ability of a stroke person to grasp and release daily living objects. In this research, we developed a rehabilitation functional assist device combining a flexion and extension mechanism with suction cup technology (hybrid technology) to help post-stroke patients improve their hand grip strength in day-to-day grasping activities. Ten poststroke hemiplegia patients were studied to test the functional ability of the impaired hand by wearing and not wearing the device. The outcomes were validated by three standard clinical tests, such as the Toronto Rehabilitation Institute - Hand Functional Test (TRI-HFT), the Chedoke Arm Hand Activity Inventory (CAHAI-9), and the Fugl-Meyer Assessment (FMA) with overall score improvements of 14.5 ± 3.8-25 ± 2.2 (p = 0.005), 5.4 ± 2.8-10 ± 1.6 (p = 0.008), and 9.6 ± 2.6-17 ± 2.4 (p = 0.005) respectively. The p-value for each of the three evaluations was less than 0.05, indicating significantly improved results and the average feedback score of the participants was 3.8 out of 5. The proposed device significantly increased impaired hand functionality in post-stroke patients. The subjects could complete some of the grasping tasks that they could not grasp without the device.Clinical trial registrationThe Clinical Trial Registry of India approved the work CTRI/2022/02/040495 described in this manuscript.

Advanced Electrode Technologies for Noninvasive Brain-Computer Interfaces

Brain-computer interfaces (BCIs) have garnered significant attention in recent years due to their potential applications in medical, assistive, and communication technologies. Building on this, noninvasive BCIs stand out as they provide a safe and user-friendly method for interacting with the human brain. In this work, we provide a comprehensive overview of the latest developments and advancements in material, design, and application of noninvasive BCIs electrode technology. We also explore the challenges and limitations currently faced by noninvasive BCI electrode technology and sketch out the technological roadmap from three dimensions: Materials and Design; Performances; Mode and Function. We aim to unite research efforts within the field of noninvasive BCI electrode technology, focusing on the consolidation of shared goals and fostering integrated development strategies among a diverse array of multidisciplinary researchers.

An Upper-Limb Rehabilitation Exoskeleton System Controlled by MI Recognition Model With Deep Emphasized Informative Features in a VR Scene

The prevalence of stroke continues to increase with the global aging. Based on the motor imagery (MI) brain-computer interface (BCI) paradigm and virtual reality (VR) technology, we designed and developed an upper-limb rehabilitation exoskeleton system (VR-ULE) in the VR scenes for stroke patients. The VR-ULE system makes use of the MI electroencephalogram (EEG) recognition model with a convolutional neural network and squeeze-and-excitation (SE) blocks to obtain the patient's motion intentions and control the exoskeleton to move during rehabilitation training movement. Due to the individual differences in EEG, the frequency bands with optimal MI EEG features for each patient are different. Therefore, the weight of different feature channels is learned by combining SE blocks to emphasize the useful information frequency band features. The MI cues in the VR-based virtual scenes can improve the interhemispheric balance and the neuroplasticity of patients. It also makes up for the disadvantages of the current MI-BCIs, such as single usage scenarios, poor individual adaptability, and many interfering factors. We designed the offline training experiment to evaluate the feasibility of the EEG recognition strategy, and designed the online control experiment to verify the effectiveness of the VR-ULE system. The results showed that the MI classification method with MI cues in the VR scenes improved the accuracy of MI classification (86.49% ± 3.02%); all subjects performed two types of rehabilitation training tasks under their own models trained in the offline training experiment, with the highest average completion rates of 86.82% ± 4.66% and 88.48% ± 5.84%. The VR-ULE system can efficiently help stroke patients with hemiplegia complete upper-limb rehabilitation training tasks, and provide the new methods and strategies for BCI-based rehabilitation devices.

Neural Correlation of EEG and Eye Movement in Natural Grasping Intention Estimation

Decoding the user's natural grasp intent enhances the application of wearable robots, improving the daily lives of individuals with disabilities. Electroencephalogram (EEG) and eye movements are two natural representations when users generate grasp intent in their minds, with current studies decoding human intent by fusing EEG and eye movement signals. However, the neural correlation between these two signals remains unclear. Thus, this paper aims to explore the consistency between EEG and eye movement in natural grasping intention estimation. Specifically, six grasp intent pairs are decoded by combining feature vectors and utilizing the optimal classifier. Extensive experimental results indicate that the coupling between the EEG and eye movements intent patterns remains intact when the user generates a natural grasp intent, and concurrently, the EEG pattern is consistent with the eye movements pattern across the task pairs. Moreover, the findings reveal a solid connection between EEG and eye movements even when taking into account cortical EEG (originating from the visual cortex or motor cortex) and the presence of a suboptimal classifier. Overall, this work uncovers the coupling correlation between EEG and eye movements and provides a reference for intention estimation.

Decoding Upper-Limb Movement Intention Through Adaptive Dynamic Movement Primitives: A Proof-of-Concept Study with a Shoulder-Elbow Exoskeleton

This work presents an intention decoding algorithm that can be used to control a 4 degrees-of-freedom shoulder-elbow exoskeleton in reaching tasks. The algorithm was designed to assist the movement of users with upper-limb impairments who can initiate the movement by themselves. It relies on the observation of the initial part of the user's movement through joint angle measures and aims to estimate in real-time the phase of the movement and predict the goal position of the hand in the reaching task. The algorithm is based on adaptive Dynamic Movement Primitives and Gaussian Mixture Models. The performance of the algorithm was verified in robot-assisted planar reaching movements performed by one healthy subject wearing the exoskeleton. Tests included movements of different amplitudes and orientations. Results showed that the algorithm could predict the hand's final position with an error lower than 5 cm after 0.25 s from the movement onset, and that the final position reached during the tests was on average less than 4 cm far from the target position. Finally, the effects of the assistance were observed in a reduction of the activation of the Biceps Brachii and of the time to execute the reaching tasks.

Hybrid brain/neural interface and autonomous vision-guided whole-arm exoskeleton control to perform activities of daily living (ADLs)

BACKGROUND

The aging of the population and the progressive increase of life expectancy in developed countries is leading to a high incidence of age-related cerebrovascular diseases, which affect people's motor and cognitive capabilities and might result in the loss of arm and hand functions. Such conditions have a detrimental impact on people's quality of life. Assistive robots have been developed to help people with motor or cognitive disabilities to perform activities of daily living (ADLs) independently. Most of the robotic systems for assisting on ADLs proposed in the state of the art are mainly external manipulators and exoskeletal devices. The main objective of this study is to compare the performance of an hybrid EEG/EOG interface to perform ADLs when the user is controlling an exoskeleton rather than using an external manipulator.

METHODS

Ten impaired participants (5 males and 5 females, mean age 52 ± 16 years) were instructed to use both systems to perform a drinking task and a pouring task comprising multiple subtasks. For each device, two modes of operation were studied: synchronous mode (the user received a visual cue indicating the sub-tasks to be performed at each time) and asynchronous mode (the user started and finished each of the sub-tasks independently). Fluent control was assumed when the time for successful initializations ranged below 3 s and a reliable control in case it remained below 5 s. NASA-TLX questionnaire was used to evaluate the task workload. For the trials involving the use of the exoskeleton, a custom Likert-Scale questionnaire was used to evaluate the user's experience in terms of perceived comfort, safety, and reliability.

RESULTS

All participants were able to control both systems fluently and reliably. However, results suggest better performances of the exoskeleton over the external manipulator (75% successful initializations remain below 3 s in case of the exoskeleton and bellow 5s in case of the external manipulator).

CONCLUSIONS

Although the results of our study in terms of fluency and reliability of EEG control suggest better performances of the exoskeleton over the external manipulator, such results cannot be considered conclusive, due to the heterogeneity of the population under test and the relatively limited number of participants.

The Berlin Bimanual Test for Tetraplegia (BeBiTT): development, psychometric properties, and sensitivity to change in assistive hand exoskeleton application

BACKGROUND

Assistive hand exoskeletons are promising tools to restore hand function after cervical spinal cord injury (SCI) but assessing their specific impact on bimanual hand and arm function is limited due to lack of reliable and valid clinical tests. Here, we introduce the Berlin Bimanual Test for Tetraplegia (BeBiTT) and demonstrate its psychometric properties and sensitivity to assistive hand exoskeleton-related improvements in bimanual task performance.

METHODS

Fourteen study participants with subacute cervical SCI performed the BeBiTT unassisted (baseline). Thereafter, participants repeated the BeBiTT while wearing a brain/neural hand exoskeleton (B/NHE) (intervention). Online control of the B/NHE was established via a hybrid sensorimotor rhythm-based brain-computer interface (BCI) translating electroencephalographic (EEG) and electrooculographic (EOG) signals into open/close commands. For reliability assessment, BeBiTT scores were obtained by four independent observers. Besides internal consistency analysis, construct validity was assessed by correlating baseline BeBiTT scores with the Spinal Cord Independence Measure III (SCIM III) and Quadriplegia Index of Function (QIF). Sensitivity to differences in bimanual task performance was assessed with a bootstrapped paired t-test.

RESULTS

The BeBiTT showed excellent interrater reliability (intraclass correlation coefficients > 0.9) and internal consistency (α = 0.91). Validity of the BeBiTT was evidenced by strong correlations between BeBiTT scores and SCIM III as well as QIF. Wearing a B/NHE (intervention) improved the BeBiTT score significantly (p < 0.05) with high effect size (d = 1.063), documenting high sensitivity to intervention-related differences in bimanual task performance.

CONCLUSION

The BeBiTT is a reliable and valid test for evaluating bimanual task performance in persons with tetraplegia, suitable to assess the impact of assistive hand exoskeletons on bimanual function.

Passive somatosensory training enhances piano skill in adolescent and adult pianists: A preliminary study

Sensory afferent information, such as auditory and somatosensory feedback while moving, plays a crucial role in both control and learning of motor performance across the lifespan. Music performance requires skillful integration of multimodal sensory information for the production of dexterous movements. However, it has not been understood what roles somatosensory afferent information plays in the acquisition and sophistication of specialized motor skills of musicians across different stages of development. In the present preliminary study, we addressed this issue by using a novel technique with a hand exoskeleton robot that can externally move the fingers of pianists. Short-term exposure to fast and complex finger movements generated by the exoskeleton (i.e., passive movements) increased the maximum rate of repetitive piano keystrokes by the pianists. This indicates that somatosensory inputs derived from the externally generated motions enhanced the quickness of the sequential finger movements in piano performance, even though the pianists did not voluntarily move the fingers. The enhancement of motor skill through passive somatosensory training using the exoskeleton was more pronounced in adolescent pianists than adult pianists. These preliminary results implicate a sensitive period of neuroplasticity of the somatosensory-motor system of trained pianists, which emphasizes the importance of somatosensory-motor training in professional music education during adolescence.

Noninvasive neuroimaging and spatial filter transform enable ultra low delay motor imagery EEG decoding

Objective.The brain-computer interface (BCI) system based on sensorimotor rhythm can convert the human spirit into instructions for machine control, and it is a new human-computer interaction system with broad applications. However, the spatial resolution of scalp electroencephalogram (EEG) is limited due to the presence of volume conduction effects. Therefore, it is very meaningful to explore intracranial activities in a noninvasive way and improve the spatial resolution of EEG. Meanwhile, low-delay decoding is an essential factor for the development of a real-time BCI system.Approach.In this paper, EEG conduction is modeled by using public head anatomical templates, and cortical EEG is obtained using dynamic parameter statistical mapping. To solve the problem of a large amount of computation caused by the increase in the number of channels, the filter bank common spatial pattern method is used to obtain a spatial filter kernel, which reduces the computational cost of feature extraction to a linear level. And the feature classification and selection of important features are completed using a neural network containing band-spatial-time domain self-attention mechanisms.Main results.The results show that the method proposed in this paper achieves high accuracy for the four types of motor imagery EEG classification tasks, with fairly low latency and high physiological interpretability.Significance.The proposed decoding framework facilitates the realization of low-latency human-computer interaction systems.

A Review of Brain Activity and EEG-Based Brain-Computer Interfaces for Rehabilitation Application

Patients with severe CNS injuries struggle primarily with their sensorimotor function and communication with the outside world. There is an urgent need for advanced neural rehabilitation and intelligent interaction technology to provide help for patients with nerve injuries. Recent studies have established the brain-computer interface (BCI) in order to provide patients with appropriate interaction methods or more intelligent rehabilitation training. This paper reviews the most recent research on brain-computer-interface-based non-invasive rehabilitation systems. Various endogenous and exogenous methods, advantages, limitations, and challenges are discussed and proposed. In addition, the paper discusses the communication between the various brain-computer interface modes used between severely paralyzed and locked patients and the surrounding environment, particularly the brain-computer interaction system utilizing exogenous (induced) EEG signals (such as P300 and SSVEP). This discussion reveals with an examination of the interface for collecting EEG signals, EEG components, and signal postprocessing. Furthermore, the paper describes the development of natural interaction strategies, with a focus on signal acquisition, data processing, pattern recognition algorithms, and control techniques.

Brain-Computer Interface-Controlled Exoskeletons in Clinical Neurorehabilitation: Ready or Not?

The development of brain-computer interface-controlled exoskeletons promises new treatment strategies for neurorehabilitation after stroke or spinal cord injury. By converting brain/neural activity into control signals of wearable actuators, brain/neural exoskeletons (B/NEs) enable the execution of movements despite impaired motor function. Beyond the use as assistive devices, it was shown that-upon repeated use over several weeks-B/NEs can trigger motor recovery, even in chronic paralysis. Recent development of lightweight robotic actuators, comfortable and portable real-world brain recordings, as well as reliable brain/neural control strategies have paved the way for B/NEs to enter clinical care. Although B/NEs are now technically ready for broader clinical use, their promotion will critically depend on early adopters, for example, research-oriented physiotherapists or clinicians who are open for innovation. Data collected by early adopters will further elucidate the underlying mechanisms of B/NE-triggered motor recovery and play a key role in increasing efficacy of personalized treatment strategies. Moreover, early adopters will provide indispensable feedback to the manufacturers necessary to further improve robustness, applicability, and adoption of B/NEs into existing therapy plans.

Feasibility of home hand rehabilitation using musicglove after chronic spinal cord injury

STUDY DESIGN

Randomized, controlled single-blind cross over study. This study was registered on ClinicalTrials.gov (NCT02473614).

OBJECTIVES

Examine usership patterns and feasibility of MusicGlove for at home hand rehabilitation therapy following chronic spinal cord injury.

SETTING

Homes of participants.

METHODS

Ten participants with chronic spinal cord injury completed two baseline assessments of hand function. After a stable baseline was determined all participants were randomized into two groups: Experimental and Control. Each group was given a recommended therapy dosage. Following this participants switched interventions.

RESULTS

On average participants had higher levels of compliance (6.1 ± 3.5 h.), and completed more grips (15,760 ± 9,590 grips) compared to participants in previous stroke studies using the same device. Participants modulated game parameters in a manner consistent with optimal challenge principles from motor learning theory. Participants in the experimental group increased their prehension ability (1 ± 1.4 MusicGlove, 0.2 ± 0.5 Control) and performance (1.4 ± 2.2 MusicGlove, 0.4 ± 0.55 Control) on the Graded and Redefined Assessment of Strength, Sensibility, and Prehension subtests. Increases in performance on the Box and Blocks Test also favored the experimental group compared to the conventional group at the end of therapy (4.2 ± 5.9, -1.0 ± 3.4 respectively).

CONCLUSIONS

MusicGlove is a feasible option for hand therapy in the home-setting for individuals with chronic SCI. Participants completed nearly twice as many gripping movements compared to individuals from the sub-acute and chronic stroke populations, and a number far greater than the number of movements typically achieved during traditional rehabilitation.

User Based Development and Test of the EXOTIC Exoskeleton: Empowering Individuals with Tetraplegia Using a Compact, Versatile, 5-DoF Upper Limb Exoskeleton Controlled through Intelligent Semi-Automated Shared Tongue Control

This paper presents the EXOTIC- a novel assistive upper limb exoskeleton for individuals with complete functional tetraplegia that provides an unprecedented level of versatility and control. The current literature on exoskeletons mainly focuses on the basic technical aspects of exoskeleton design and control while the context in which these exoskeletons should function is less or not prioritized even though it poses important technical requirements. We considered all sources of design requirements, from the basic technical functions to the real-world practical application. The EXOTIC features: (1) a compact, safe, wheelchair-mountable, easy to don and doff exoskeleton capable of facilitating multiple highly desired activities of daily living for individuals with tetraplegia; (2) a semi-automated computer vision guidance system that can be enabled by the user when relevant; (3) a tongue control interface allowing for full, volitional, and continuous control over all possible motions of the exoskeleton. The EXOTIC was tested on ten able-bodied individuals and three users with tetraplegia caused by spinal cord injury. During the tests the EXOTIC succeeded in fully assisting tasks such as drinking and picking up snacks, even for users with complete functional tetraplegia and the need for a ventilator. The users confirmed the usability of the EXOTIC.

Drawn-on-Skin Sensors from Fully Biocompatible Inks toward High-Quality Electrophysiology

The need to develop wearable devices for personal health monitoring, diagnostics, and therapy has inspired the production of innovative on-demand, customizable technologies. Several of these technologies enable printing of raw electronic materials directly onto biological organs and tissues. However, few of them have been thoroughly investigated for biocompatibility of the raw materials on the cellular, tissue, and organ levels or with different cell types. In addition, highly accurate multiday in vivo monitoring using such on-demand, in situ fabricated devices has yet to be done. Presented herein is the first fully biocompatible, on-skin fabricated electronics for multiple cell types and tissues that can capture electrophysiological signals with high fidelity. While also demonstrating improved mechanical and electrical properties, the drawn-on-skin ink retains its properties under various writing conditions, which minimizes the variation in electrical performance. Furthermore, the drawn-on-skin ink shows excellent biocompatibility with cardiomyocytes, neurons, mice skin tissue, and human skin. The high signal-to-noise ratios of the electrophysiological signals recorded with the DoS sensor over multiple days demonstrate its potential for personalized, long-term, and accurate electrophysiological health monitoring.

A Comprehensive Review of Endogenous EEG-Based BCIs for Dynamic Device Control

Electroencephalogram (EEG)-based brain–computer interfaces (BCIs) provide a novel approach for controlling external devices. BCI technologies can be important enabling technologies for people with severe mobility impairment. Endogenous paradigms, which depend on user-generated commands and do not need external stimuli, can provide intuitive control of external devices. This paper discusses BCIs to control various physical devices such as exoskeletons, wheelchairs, mobile robots, and robotic arms. These technologies must be able to navigate complex environments or execute fine motor movements. Brain control of these devices presents an intricate research problem that merges signal processing and classification techniques with control theory. In particular, obtaining strong classification performance for endogenous BCIs is challenging, and EEG decoder output signals can be unstable. These issues present myriad research questions that are discussed in this review paper. This review covers papers published until the end of 2021 that presented BCI-controlled dynamic devices. It discusses the devices controlled, EEG paradigms, shared control, stabilization of the EEG signal, traditional machine learning and deep learning techniques, and user experience. The paper concludes with a discussion of open questions and avenues for future work.

Current State of Robotics in Hand Rehabilitation after Stroke: A Systematic Review

Among the methods of hand function rehabilitation after stroke, robot-assisted rehabilitation is widely used, and the use of hand rehabilitation robots can provide functional training of the hand or assist the paralyzed hand with activities of daily living. However, patients with hand disorders consistently report that the needs of some users are not being met. The purpose of this review is to understand the reasons why these user needs are not being adequately addressed, to explore research on hand rehabilitation robots, to review their current state of research in recent years, and to summarize future trends in the hope that it will be useful to researchers in this research area. This review summarizes the techniques in this paper in a systematic way. We first provide a comprehensive review of research institutions, commercial products, and literature. Thus, the state of the art and deficiencies of functional hand rehabilitation robots are sought and guide the development of subsequent hand rehabilitation robots. This review focuses specifically on the actuation and control of hand functional rehabilitation robots, as user needs are primarily focused on actuation and control strategies. We also review hand detection technologies and compare them with patient needs. The results show that the trends in recent years are more inclined to pursue new lightweight materials to improve hand adaptability, investigating intelligent control methods for human-robot interaction in hand functional rehabilitation robots to improve control robustness and accuracy, and VR virtual task positioning to improve the effectiveness of active rehabilitation training.

Highly Conformal Polymers for Ambulatory Electrophysiological Sensing

Stable ambulatory electrophysiological sensing is widely utilized for smart e-healthcare monitoring, clinical diagnosis of cardiovascular diseases, treatment of neurological diseases, and intelligent human-machine interaction. As the favorable signal interaction platform of electrophysiological sensing, the conformal property of on-skin electrodes is an extremely crucial factor that can affect the stability of long-term ambulatory electrophysiological sensing. From the perspective of materials, to realize conformal contact between electrodes and skin for stable sensing, highly conformal polymers are strongly demanding and attracting ever-growing attention. In this review, we focused on the recent progress of highly conformal polymers for ambulatory electrophysiological sensing, including their synthetic methods, conformal property, and potential applications. Specifically, three main types of highly conformal polymers for stable long-term electrophysiological signals monitoring were proposed, including nature silk fibroin based conformal polymers, marine mussels bio-inspired conformal polymers, and other conformal polymers such as zwitterionic polymers and polyacrylamide. Furthermore, the future challenges and opportunities of preparing highly conformal polymers for on-skin electrodes were also highlighted. This article is protected by copyright. All rights reserved.

Flexible Electronics and Devices as Human-Machine Interfaces for Medical Robotics

Medical robots are invaluable players in non-pharmaceutical treatment of disabilities. Particularly, using prosthetic and rehabilitation devices with human-machine interfaces can greatly improve the quality of life for impaired patients. In recent years, flexible electronic interfaces and soft robotics have attracted tremendous attention in this field due to their high biocompatibility, functionality, conformability, and low-cost. Flexible human-machine interfaces on soft robotics will make a promising alternative to conventional rigid devices, which can potentially revolutionize the paradigm and future direction of medical robotics in terms of rehabilitation feedback and user experience. In this review, the fundamental components of the materials, structures, and mechanisms in flexible human-machine interfaces are summarized by recent and renowned applications in five primary areas: physical and chemical sensing, physiological recording, information processing and communication, soft robotic actuation, and feedback stimulation. This review further concludes by discussing the outlook and current challenges of these technologies as a human-machine interface in medical robotics.

Effects of Gaze Fixation on the Performance of a Motor Imagery-Based Brain-Computer Interface

Motor imagery-based brain-computer interfaces (BCIs) have been studied without controlling subjects’ gaze fixation position previously. The effect of gaze fixation and covert attention on the behavioral performance of BCI is still unknown. This study designed a gaze fixation controlled experiment. Subjects were required to conduct a secondary task of gaze fixation when performing the primary task of motor imagination. Subjects’ performance was analyzed according to the relationship between motor imagery target and the gaze fixation position, resulting in three BCI control conditions, i.e., congruent, incongruent, and center cross trials. A group of fourteen subjects was recruited. The average group performances of three different conditions did not show statistically significant differences in terms of BCI control accuracy, feedback duration, and trajectory length. Further analysis of gaze shift response time revealed a significantly shorter response time for congruent trials compared to incongruent trials. Meanwhile, the parietal occipital cortex also showed active neural activities for congruent and incongruent trials, and this was revealed by a contrast analysis of R-square values and lateralization index. However, the lateralization index computed from the parietal and occipital areas was not correlated with the BCI behavioral performance. Subjects’ BCI behavioral performance was not affected by the position of gaze fixation and covert attention. This indicated that motor imagery-based BCI could be used freely in robotic arm control without sacrificing performance.

Motor Imagery Classification Using Inter-Task Transfer Learning via A Channel-Wise Variational Autoencoder-based Convolutional Neural Network

Highly sophisticated control based on a brain-computer interface (BCI) requires decoding kinematic information from brain signals. The forearm is a region of the upper limb that is often used in everyday life, but intuitive movements within the same limb have rarely been investigated in previous BCI studies. In this study, we focused on various forearm movement decoding from electroencephalography (EEG) signals using a small number of samples. Ten healthy participants took part in an experiment and performed motor execution (ME) and motor imagery (MI) of the intuitive movement tasks (Dataset I). We propose a convolutional neural network using a channel-wise variational autoencoder (CVNet) based on inter-task transfer learning. We approached that training the reconstructed ME-EEG signals together will also achieve more sufficient classification performance with only a small amount of MI-EEG signals. The proposed CVNet was validated on our own Dataset I and a public dataset, BNCI Horizon 2020 (Dataset II). The classification accuracies of various movements are confirmed to be 0.83 (±0.04) and 0.69 (±0.04) for Dataset I and II, respectively. The results show that the proposed method exhibits performance increases of approximately 0.09~0.27 and 0.08~0.24 compared with the conventional models for Dataset I and II, respectively. The outcomes suggest that the training model for decoding imagined movements can be performed using data from ME and a small number of data samples from MI. Hence, it is presented the feasibility of BCI learning strategies that can sufficiently learn deep learning with a few amount of calibration dataset and time only, with stable performance.

Motion intensity modeling and trajectory control of upper limb rehabilitation exoskeleton robot based on multi-modal information

The motion intensity of patient is significant for the trajectory control of exoskeleton robot during rehabilitation, as it may have important influence on training effect and human–robot interaction. To design rehabilitation training task according to situation of patients, a novel control method of rehabilitation exoskeleton robot is designed based on motion intensity perception model. The motion signal of robot and the heart rate signal of patient are collected and fused into multi-modal information as the input layer vector of deep learning framework, which is used for the human–robot interaction model of control system. A 6-degree of freedom (DOF) upper limb rehabilitation exoskeleton robot is designed previously to implement the test. The parameters of the model are iteratively optimized by grouping the experimental data, and identification effect of the model is analyzed and compared. The average recognition accuracy of the proposed model can reach up to 99.0% in the training data set and 95.7% in the test data set, respectively. The experimental results show that the proposed motion intensity perception model based on deep neural network (DNN) and the trajectory control method can improve the performance of human–robot interaction, and it is possible to further improve the effect of rehabilitation training.

Eyes-Free Tongue Gesture and Tongue Joystick Control of a Five DOF Upper-Limb Exoskeleton for Severely Disabled Individuals

Spinal cord injury can leave the affected individual severely disabled with a low level of independence and quality of life. Assistive upper-limb exoskeletons are one of the solutions that can enable an individual with tetraplegia (paralysis in both arms and legs) to perform simple activities of daily living by mobilizing the arm. Providing an efficient user interface that can provide full continuous control of such a device—safely and intuitively—with multiple degrees of freedom (DOFs) still remains a challenge. In this study, a control interface for an assistive upper-limb exoskeleton with five DOFs based on an intraoral tongue-computer interface (ITCI) for individuals with tetraplegia was proposed. Furthermore, we evaluated eyes-free use of the ITCI for the first time and compared two tongue-operated control methods, one based on tongue gestures and the other based on dynamic virtual buttons and a joystick-like control. Ten able-bodied participants tongue controlled the exoskeleton for a drinking task with and without visual feedback on a screen in three experimental sessions. As a baseline, the participants performed the drinking task with a standard gamepad. The results showed that it was possible to control the exoskeleton with the tongue even without visual feedback and to perform the drinking task at 65.1% of the speed of the gamepad. In a clinical case study, an individual with tetraplegia further succeeded to fully control the exoskeleton and perform the drinking task only 5.6% slower than the able-bodied group. This study demonstrated the first single-modal control interface that can enable individuals with complete tetraplegia to fully and continuously control a five-DOF upper limb exoskeleton and perform a drinking task after only 2 h of training. The interface was used both with and without visual feedback.

Adaptation Strategies for Personalized Gait Neuroprosthetics

Personalization of gait neuroprosthetics is paramount to ensure their efficacy for users, who experience severe limitations in mobility without an assistive device. Our goal is to develop assistive devices that collaborate with and are tailored to their users, while allowing them to use as much of their existing capabilities as possible. Currently, personalization of devices is challenging, and technological advances are required to achieve this goal. Therefore, this paper presents an overview of challenges and research directions regarding an interface with the peripheral nervous system, an interface with the central nervous system, and the requirements of interface computing architectures. The interface should be modular and adaptable, such that it can provide assistance where it is needed. Novel data processing technology should be developed to allow for real-time processing while accounting for signal variations in the human. Personalized biomechanical models and simulation techniques should be developed to predict assisted walking motions and interactions between the user and the device. Furthermore, the advantages of interfacing with both the brain and the spinal cord or the periphery should be further explored. Technological advances of interface computing architecture should focus on learning on the chip to achieve further personalization. Furthermore, energy consumption should be low to allow for longer use of the neuroprosthesis. In-memory processing combined with resistive random access memory is a promising technology for both. This paper discusses the aforementioned aspects to highlight new directions for future research in gait neuroprosthetics.

Design and Evaluation of a Hands-Free Video Game Controller for Individuals With Motor Impairments

Over the past few decades, video gaming has evolved at a tremendous rate although game input methods have been slower to change. Game input methods continue to rely on two-handed control of the joystick and D-pad or the keyboard and mouse for simultaneously controlling player movement and camera actions. Bi-manual input poses a significant play impediment to those with severe motor impairments. In this work, we propose and evaluate a hands-free game input control method that uses real-time facial expression recognition. Through our novel input method, our goal is to enable and empower individuals with neurological and neuromuscular diseases, who may lack hand muscle control, to be able to independently play video games. To evaluate the usability and acceptance of our system, we conducted a remote user study with eight severely motor-impaired individuals. Our results indicate high user satisfaction and greater preference for our input system with participants rating the input system as easy to learn. With this work, we aim to highlight that facial expression recognition can be a valuable input method.

Variation Trends of Fractal Dimension in Epileptic EEG Signals

Epileptic diseases take EEG as an important basis for clinical judgment, and fractal algorithms were often used to analyze electroencephalography (EEG) signals. However, the variation trends of fractal dimension (D) were opposite in the literature, i.e., both D decreasing and increasing were reported in previous studies during seizure status relative to the normal status, undermining the feasibility of fractal algorithms for EEG analysis to detect epileptic seizures. In this study, two algorithms with high accuracy in the D calculation, Higuchi and roughness scaling extraction (RSE), were used to study D variation of EEG signals with seizures. It was found that the denoising operation had an important influence on D variation trend. Moreover, the D variation obtained by RSE algorithm was larger than that by Higuchi algorithm, because the non-fractal nature of EEG signals during normal status could be detected and quantified by RSE algorithm. The above findings in this study could be promising to make more understandings of the nonlinear nature and scaling behaviors of EEG signals.

Flexible Temperature Sensors

Temperature reflects the balance between production and dissipate of heat. Flexible temperature sensors are primary sensors used for temperature monitoring. To obtain real-time and accurate information of temperature, different flexible temperature sensors are developed according to the principle of flexible resistance temperature detector (FRTC), flexible thermocouple, flexible thermistor and flexible thermochromic, showing great potential in energy conversion and storage. In order to obtain high integration and multifunction, various flexible temperature sensors are studied and optimized, including active-matrix flexible temperature sensor, self-powered flexible temperature sensor, self-healing flexible temperature sensor and self-cleaning flexible temperature sensor. This review focuses on the structure, material, fabrication and performance of flexible temperature sensors. Also, some typical applications of flexible temperature sensors are discussed and summarized.

A Review on Signal Processing Approaches to Reduce Calibration Time in EEG-Based Brain-Computer Interface

In an electroencephalogram- (EEG-) based brain–computer interface (BCI), a subject can directly communicate with an electronic device using his EEG signals in a safe and convenient way. However, the sensitivity to noise/artifact and the non-stationarity of EEG signals result in high inter-subject/session variability. Therefore, each subject usually spends long and tedious calibration time in building a subject-specific classifier. To solve this problem, we review existing signal processing approaches, including transfer learning (TL), semi-supervised learning (SSL), and a combination of TL and SSL. Cross-subject TL can transfer amounts of labeled samples from different source subjects for the target subject. Moreover, Cross-session/task/device TL can reduce the calibration time of the subject for the target session, task, or device by importing the labeled samples from the source sessions, tasks, or devices. SSL simultaneously utilizes the labeled and unlabeled samples from the target subject. The combination of TL and SSL can take advantage of each other. For each kind of signal processing approaches, we introduce their concepts and representative methods. The experimental results show that TL, SSL, and their combination can obtain good classification performance by effectively utilizing the samples available. In the end, we draw a conclusion and point to research directions in the future.

A Modular Mobile Robotic Platform to Assist People with Different Degrees of Disability

Robotics to support elderly people in living independently and to assist disabled people in carrying out the activities of daily living independently have demonstrated good results. Basically, there are two approaches: one of them is based on mobile robot assistants, such as Care-O-bot, PR2, and Tiago, among others; the other one is the use of an external robotic arm or a robotic exoskeleton fixed or mounted on a wheelchair. In this paper, a modular mobile robotic platform to assist moderately and severely impaired people based on an upper limb robotic exoskeleton mounted on a robotized wheel chair is presented. This mobile robotic platform can be customized for each user’s needs by exploiting its modularity. Finally, experimental results in a simulated home environment with a living room and a kitchen area, in order to simulate the interaction of the user with different elements of a home, are presented. In this experiment, a subject suffering from multiple sclerosis performed different activities of daily living (ADLs) using the platform in front of a group of clinicians composed of nurses, doctors, and occupational therapists. After that, the subject and the clinicians replied to a usability questionnaire. The results were quite good, but two key factors arose that need to be improved: the complexity and the cumbersome aspect of the platform.

Heart rate variability predicts decline in sensorimotor rhythm control

Objective.Voluntary control of sensorimotor rhythms (SMRs, 8-12 Hz) can be used for brain-computer interface (BCI)-based operation of an assistive hand exoskeleton, e.g. in finger paralysis after stroke. To gain SMR control, stroke survivors are usually instructed to engage in motor imagery (MI) or to attempt moving the paralyzed fingers resulting in task- or event-related desynchronization (ERD) of SMR (SMR-ERD). However, as these tasks are cognitively demanding, especially for stroke survivors suffering from cognitive impairments, BCI control performance can deteriorate considerably over time. Therefore, it would be important to identify biomarkers that predict decline in BCI control performance within an ongoing session in order to optimize the man-machine interaction scheme.Approach.Here we determine the link between BCI control performance over time and heart rate variability (HRV). Specifically, we investigated whether HRV can be used as a biomarker to predict decline of SMR-ERD control across 17 healthy participants using Granger causality. SMR-ERD was visually displayed on a screen. Participants were instructed to engage in MI-based SMR-ERD control over two consecutive runs of 8.5 min each. During the 2nd run, task difficulty was gradually increased.Main results.While control performance (p= .18) and HRV (p= .16) remained unchanged across participants during the 1st run, during the 2nd run, both measures declined over time at high correlation (performance: -0.61%/10 s,p= 0; HRV: -0.007 ms/10 s,p< .001). We found that HRV exhibited predictive characteristics with regard to within-session BCI control performance on an individual participant level (p< .001).Significance.These results suggest that HRV can predict decline in BCI performance paving the way for adaptive BCI control paradigms, e.g. to individualize and optimize assistive BCI systems in stroke.

Advancing sensory neuroprosthetics using artificial brain networks

Implementation of effective brain or neural stimulation protocols for restoration of complex sensory perception, e.g., in the visual domain, is an unresolved challenge. By leveraging the capacity of deep learning to model the brain's visual system, optic nerve stimulation patterns could be derived that are predictive of neural responses of higher-level cortical visual areas in silico. This novel approach could be generalized to optimize different types of neuroprosthetics or bidirectional brain-computer interfaces (BCIs).

Challenges and Opportunities for the Future of Brain-Computer Interface in Neurorehabilitation

Brain-computer interfaces (BCIs) provide a unique technological solution to circumvent the damaged motor system. For neurorehabilitation, the BCI can be used to translate neural signals associated with movement intentions into tangible feedback for the patient, when they are unable to generate functional movement themselves. Clinical interest in BCI is growing rapidly, as it would facilitate rehabilitation to commence earlier following brain damage and provides options for patients who are unable to partake in traditional physical therapy. However, substantial challenges with existing BCI implementations have prevented its widespread adoption. Recent advances in knowledge and technology provide opportunities to facilitate a change, provided that researchers and clinicians using BCI agree on standardisation of guidelines for protocols and shared efforts to uncover mechanisms. We propose that addressing the speed and effectiveness of learning BCI control are priorities for the field, which may be improved by multimodal or multi-stage approaches harnessing more sensitive neuroimaging technologies in the early learning stages, before transitioning to more practical, mobile implementations. Clarification of the neural mechanisms that give rise to improvement in motor function is an essential next step towards justifying clinical use of BCI. In particular, quantifying the unknown contribution of non-motor mechanisms to motor recovery calls for more stringent control conditions in experimental work. Here we provide a contemporary viewpoint on the factors impeding the scalability of BCI. Further, we provide a future outlook for optimal design of the technology to best exploit its unique potential, and best practices for research and reporting of findings.

Literature review on assistive devices available for quadriplegic people: Indian context

PURPOSE

This literature review aims to find the current state of the art in self-help devices (SHD) available for people with quadriplegia.

MATERIALS AND METHODS

We searched original articles, technical and case studies, conference articles, and literature reviews published between 2014 to 2019 with the keywords ("Self-help devices" OR "Assistive Devices" OR "Assistive Product" OR "Assistive Technology") AND "Quadriplegia" in Science Direct, Pubmed, IEEE Xplore digital library and Web of Science.

RESULTS

Total 222 articles were found. After removing duplicates and screening these articles based on their title and abstracts 80 articles remained. After this, we reviewed the full text, and articles unrelated to SHD development or about the patients who require mechanical ventilation or where the upper limb is functional (C2 or above and T2 or below injuries) were discarded. After the exclusion of articles using the above-mentioned criterion 75 articles were used for further review.

CONCLUSION

The abandonment rate of SHD currently available in the literature is very high. The major requirement of the people was independence and improved quality of life. The situation in India is very bad as compared to the developed countries. The people with spinal cord injury in India are uneducated and very poor, with an average income of 3000 ₹ (41$). They require SHDs and training specially designed for them, keeping their needs in mind.Implications for rehabilitationPeople with quadriplegia are totally dependent on caregivers. Assistive devices not only help these people to do day-to-day tasks but also provides them self-confidence.Even though there are a lot of self-help devices currently available, still they are not able to fulfil the requirements of people with quadriplegia, hence there is a very high abandonment rate of such devices.This study provides an evidence that developing devices after understanding the functional and non-functional requirements of these subjects will decrease the abandonment rate and increase the effectiveness of the device.The results of this study can be used for planning and developing assistive devices which are more focussed on fulfilling the requirements of people with quadriplegia.

Adding Tactile Feedback and Changing ISI to Improve BCI Systems' Robustness: An Error-Related Potential Study

Nowadays, the brain-computer interface (BCI) systems attract much more attention than before, yet they have not found their ways into our lives since their accuracy is not satisfying. Error Related Potential (ErRP) is a potential that occurs in human brain signals when an unintended event happens, against ones' will and thoughts. An example is the occurrence of an error in BCI systems. Investigation of the ErRP could enable researchers to increase the accuracy of BCI systems by detecting instances of inaccuracy in the system. In this research the effects of two parameters on the ErRP are studied: (1) The Motor Imagery Time, also known as Inter-Stimulus Interval (ISI) and (2) different types of feedback (Visual and Tactile). The statistical analysis of the ErRP characteristics showed that feedback type meaningfully affects the ErRP in a cue-paced BCI system and it will affect the time of occurrence of this potential. To validate the proposed idea, different feature extraction, and classification techniques were used for the classification of the BCI system responses. It was shown that by proper selection of the parameters and features, the accuracy of the system could be improved. Tactile feedback together with higher ISI could increase the accuracy of finding erroneous trials up to 90%. The proposed method's accuracy was significantly higher (p-value < 0.05) compared to other methods of feature extraction.

A set of electroencephalographic (EEG) data recorded during amplitude-modulated transcranial alternating current stimulation (AM-tACS) targeting 10-Hz steady-state visually evoked potentials (SSVEP)

Transcranial alternating current stimulation (tACS) can affect perception, learning and cognition, but the underlying mechanisms are not well understood. A promising strategy to elucidate these mechanisms aims at applying tACS while electric or magnetic brain oscillations targeted by stimulation are recorded. However, reconstructing brain oscillations targeted by tACS remains a challenging problem due to stimulation artifacts. Besides lack of an established strategy to effectively supress such stimulation artifacts, there are also no resources available that allow for the development and testing of new and effective tACS artefact suppression algorithms, such as adaptive spatial filtering using beamforming or signal-space projection. Here, we provide a full dataset comprising encephalographic (EEG) recordings across six healthy human volunteers who underwent 10-Hz amplitude-modulated tACS (AM-tACS) during a 10-Hz steady-state visually evoked potential (SSVEP) paradigm. Moreover, data and scripts are provided related to the validation of a novel stimulation artefact suppression strategy, Stimulation Artifact Source Separation (SASS), removing EEG signal components that are maximally different in the presence versus absence of stimulation. Besides including EEG single-trial data and comparisons of 10-Hz brain oscillatory phase and amplitude recorded across three conditions (condition 1: no stimulation, condition 2: stimulation with SASS, condition 3: stimulation without SASS), also power spectra and topographies of SSVEP amplitudes across all three conditions are presented. Moreover, data is provided for assessing nonlinear modulations of the stimulation artifact in both time and frequency domains due to heartbeats. Finally, the dataset includes eigenvalue spectra and spatial patterns of signal components that were identified and removed by SASS for stimulation artefact suppression at the target frequency. Besides providing a valuable resource to assess properties of AM-tACS artifacts in the EEG, this dataset allows for testing different artifact rejection methods and offers in-depth insights into the workings of SASS.

DESIGN, DYNAMIC MODELING AND CONTROL OF WEARABLE FINGER ORTHOSIS

Human hands and fingers are of significant importance in people’s capacity to perform daily tasks (touching, feeling, holding, gripping, writing). However, about 1.5 million people around the world are suffering from injuries, muscle and neurological disorders, a loss of hand function, or a few fingers due to stroke. This paper focuses on newly developed finger orthotics, which is thin, adaptable to the length of each finger and low energy costs. The aim of the study is to design and control a new robotic orthosis using for daily rehabilitation therapy. Kinematic and dynamic analysis of orthosis was calculated and the joint regulation of orthosis was obtained. The Lagrange method was used to obtain dynamics, and the Denavit–Hartenberg (D–H) method was used for kinematic analysis of hand. In order to understand its behavior, the robotic finger orthotics model was simulated in MatLab/Simulink. The simulation results show that the efficiency and robustness of proportional integral derivative (PID) controller are appropriate for the use of robotic finger orthotics.

Optical brain imaging and its application to neurofeedback

Besides passive recording of brain electric or magnetic activity, also non-ionizing electromagnetic or optical radiation can be used for real-time brain imaging. Here, changes in the radiation's absorption or scattering allow for continuous in vivo assessment of regional neurometabolic and neurovascular activity. Besides magnetic resonance imaging (MRI), over the last years, also functional near-infrared spectroscopy (fNIRS) was successfully established in real-time metabolic brain imaging. In contrast to MRI, fNIRS is portable and can be applied at bedside or in everyday life environments, e.g., to restore communication and movement. Here we provide a comprehensive overview of the history and state-of-the-art of real-time optical brain imaging with a special emphasis on its clinical use towards neurofeedback and brain-computer interface (BCI) applications. Besides pointing to the most critical challenges in clinical use, also novel approaches that combine real-time optical neuroimaging with other recording modalities (e.g. electro- or magnetoencephalography) are described, and their use in the context of neuroergonomics, neuroenhancement or neuroadaptive systems discussed.

Eyes are faster than hands: A soft wearable robot learns user intention from the egocentric view

To perceive user intentions for wearable robots, we present a learning-based intention detection methodology using a first-person-view camera.

Improvement of hand functions of spinal cord injury patients with electromyography-driven hand exoskeleton: A feasibility study

We have developed a one-of-a-kind hand exoskeleton, called Maestro, which can power finger movements of those surviving severe disabilities to complete daily tasks using compliant joints. In this paper, we present results from an electromyography (EMG) control strategy conducted with spinal cord injury (SCI) patients (C5, C6, and C7) in which the subjects completed daily tasks controlling Maestro with EMG signals from their forearm muscles. With its compliant actuation and its degrees of freedom that match the natural finger movements, Maestro is capable of helping the subjects grasp and manipulate a variety of daily objects (more than 15 from a standardized set). To generate control commands for Maestro, an artificial neural network algorithm was implemented along with a probabilistic control approach to classify and deliver four hand poses robustly with three EMG signals measured from the forearm and palm. Increase in the scores of a standardized test, called the Sollerman hand function test, and enhancement in different aspects of grasping such as strength shows feasibility that Maestro can be capable of improving the hand function of SCI subjects.

Materials, Devices, and Systems of On-Skin Electrodes for Electrophysiological Monitoring and Human-Machine Interfaces

On-skin electrodes function as an ideal platform for collecting high-quality electrophysiological (EP) signals due to their unique characteristics, such as stretchability, conformal interfaces with skin, biocompatibility, and wearable comfort. The past decade has witnessed great advancements in performance optimization and function extension of on-skin electrodes. With continuous development and great promise for practical applications, on-skin electrodes are playing an increasingly important role in EP monitoring and human-machine interfaces (HMI). In this review, the latest progress in the development of on-skin electrodes and their integrated system is summarized. Desirable features of on-skin electrodes are briefly discussed from the perspective of performances. Then, recent advances in the development of electrode materials, followed by the analysis of strategies and methods to enhance adhesion and breathability of on-skin electrodes are examined. In addition, representative integrated electrode systems and practical applications of on-skin electrodes in healthcare monitoring and HMI are introduced in detail. It is concluded with the discussion of key challenges and opportunities for on-skin electrodes and their integrated systems.

3D Multi-Material Printing of an Anthropomorphic, Personalized Replacement Hand for Use in Neuroprosthetics Using 3D Scanning and Computer-Aided Design: First Proof-of-Technical-Concept Study

Background: This paper presents a novel approach for a hand prosthesis consisting of a flexible, anthropomorphic, 3D-printed replacement hand combined with a commercially available motorized orthosis that allows gripping. Methods: A 3D light scanner was used to produce a personalized replacement hand. The wrist of the replacement hand was printed of rigid material; the rest of the hand was printed of flexible material. A standard arm liner was used to enable the user’s arm stump to be connected to the replacement hand. With computer-aided design, two different concepts were developed for the scanned hand model: In the first concept, the replacement hand was attached to the arm liner with a screw. The second concept involved attaching with a commercially available fastening system; furthermore, a skeleton was designed that was located within the flexible part of the replacement hand. Results: 3D-multi-material printing of the two different hands was unproblematic and inexpensive. The printed hands had approximately the weight of the real hand. When testing the replacement hands with the orthosis it was possible to prove a convincing everyday functionality. For example, it was possible to grip and lift a 1-L water bottle. In addition, a pen could be held, making writing possible. Conclusions: This first proof-of-concept study encourages further testing with users.

Myoelectric untethered robotic glove enhances hand function and performance on daily living tasks after stroke

Introduction

Wearable robots controlled using electromyography could motivate greater use of the affected upper extremity after stroke and enable bimanual activities of daily living to be completed independently.

Methods

We have developed a myoelectric untethered robotic glove (My-HERO) that provides five-finger extension and grip assistance.

Results

The myoelectric controller detected the grip and release intents of the 9 participants after stroke with 84.7% accuracy. While using My-HERO, all 9 participants performed better on the Fugl-Meyer Assessment-Hand (8.4 point increase, scale out of 14, p < 0.01) and the Chedoke Arm and Hand Activity Inventory (8.2 point increase, scale out of 91, p < 0.01). Established criteria for clinically meaningful important differences were surpassed for both the hand function and daily living task assessments. The majority of participants provided satisfaction and usability questionnaire scores above 70%. Seven participants desired to use My-HERO in the clinic and at home during their therapy and daily routines.

Conclusions

People with hand impairment after stroke value that myoelectric untethered robotic gloves enhance their motion and bimanual task performance and motivate them to use their muscles during engaging activities of daily living. They desire to use these gloves daily to enable greater independence and investigate the effects on neuromuscular recovery.

Feasibility and Safety of Bilateral Hybrid EEG/EOG Brain/Neural-Machine Interaction

Cervical spinal cord injuries (SCIs) often lead to loss of motor function in both hands and legs, limiting autonomy and quality of life. While it was shown that unilateral hand function can be restored after SCI using a hybrid electroencephalography/electrooculography (EEG/EOG) brain/neural hand exoskeleton (B/NHE), it remained unclear whether such hybrid paradigm also could be used for operating two hand exoskeletons, e.g., in the context of bimanual tasks such as eating with fork and knife. To test whether EEG/EOG signals allow for fluent and reliable as well as safe and user-friendly bilateral B/NHE control, eight healthy participants (six females, mean age 24.1 ± 3.2 years) as well as four chronic tetraplegics (four males, mean age 51.8 ± 15.2 years) performed a complex sequence of EEG-controlled bilateral grasping and EOG-controlled releasing motions of two exoskeletons visually presented on a screen. A novel EOG command performed by prolonged horizontal eye movements (>1 s) to the left or right was introduced as a reliable switch to activate either the left or right exoskeleton. Fluent EEG control was defined as average “time to initialize” (TTI) grasping motions below 3 s. Reliable EEG control was assumed when classification accuracy exceeded 80%. Safety was defined as “time to stop” (TTS) all unintended grasping motions within 2 s. After the experiment, tetraplegics were asked to rate the user-friendliness of bilateral B/NHE control using Likert scales. Average TTI and accuracy of EEG-controlled operations ranged at 2.14 ± 0.66 s and 85.89 ± 15.81% across healthy participants and at 1.90 ± 0.97 s and 81.25 ± 16.99% across tetraplegics. Except for one tetraplegic, all participants met the safety requirements. With 88 ± 11% of the maximum achievable score, tetraplegics rated the control paradigm as user-friendly and reliable. These results suggest that hybrid EEG/EOG B/NHE control of two assistive devices is feasible and safe, paving the way to test this paradigm in larger clinical trials performing bimanual tasks in everyday life environments.

Scalp electroencephalograms over ipsilateral sensorimotor cortex reflect contraction patterns of unilateral finger muscles

A variety of neural substrates are implicated in the initiation, coordination, and stabilization of voluntary movements underpinned by adaptive contraction and relaxation of agonist and antagonist muscles. To achieve such flexible and purposeful control of the human body, brain systems exhibit extensive modulation during the transition from resting state to motor execution and to maintain proper joint impedance. However, the neural structures contributing to such sensorimotor control under unconstrained and naturalistic conditions are not fully characterized. To elucidate which brain regions are implicated in generating and coordinating voluntary movements, we employed a physiologically inspired, two-stage method to decode relaxation and three patterns of contraction in unilateral finger muscles (i.e., extension, flexion, and co-contraction) from high-density scalp electroencephalograms (EEG). The decoder consisted of two parts employed in series. The first discriminated between relaxation and contraction. If the EEG data were discriminated as contraction, the second stage then discriminated among the three contraction patterns. Despite the difficulty in dissociating detailed contraction patterns of muscles within a limb from scalp EEG signals, the decoder performance was higher than chance-level by 2-fold in the four-class classification. Moreover, weighted features in the trained decoders revealed EEG features differentially contributing to decoding performance. During the first stage, consistent with previous reports, weighted features were localized around sensorimotor cortex (SM1) contralateral to the activated fingers, while those during the second stage were localized around ipsilateral SM1. The loci of these weighted features suggested that the coordination of unilateral finger muscles induced different signaling patterns in ipsilateral SM1 contributing to motor control. Weighted EEG features enabled a deeper understanding of human sensorimotor processing as well as of a more naturalistic control of brain-computer interfaces.

Upregulating excitability of corticospinal pathways in stroke patients using TMS neurofeedback; A pilot study

Upper limb weakness following a stroke affects 80% of survivors and is a key factor in preventing their return to independence. State-of-the art approaches to rehabilitation often require that the patient can generate some activity in the paretic limb, which is not possible for many patients in the early period following stroke. Approaches that enable more patients to engage with upper limb therapy earlier are urgently needed. Motor imagery has shown promise as a potential means to maintain activity in the brain's motor network, when the patient is incapable of generating functional movement. However, as imagery is a hidden mental process, it is impossible for individuals to gauge what impact this is having upon their neural activity. Here we used a novel brain-computer interface (BCI) approach allowing patients to gain an insight into the effect of motor imagery on their brain-muscle pathways, in real-time. Seven patients 2-26 weeks post stroke were provided with neurofeedback (NF) of their corticospinal excitability measured by the size of motor evoked potentials (MEP) in response to transcranial magnetic stimulation (TMS). The aim was to train patients to use motor imagery to increase the size of MEPs, using the BCI with a computer game displaying neurofeedback. Patients training finger muscles learned to elevate MEP amplitudes above their resting baseline values for the first dorsal interosseous (FDI) and abductor digiti minimi (ADM) muscles. By day 3 for ADM and day 4 for FDI, MEP amplitudes were sustained above baseline in all three NF blocks. Here we have described the first clinical implementation of TMS NF in a population of sub-acute stroke patients. The results show that in the context of severe upper limb paralysis, patients are capable of using neurofeedback to elevate corticospinal excitability in the affected muscles. This may provide a new training modality for early intervention following stroke.

Electrode-free visual prosthesis/exoskeleton control using augmented reality glasses in a first proof-of-technical-concept study

In the field of neuroprosthetics, the current state-of-the-art method involves controlling the prosthesis with electromyography (EMG) or electrooculography/electroencephalography (EOG/EEG). However, these systems are both expensive and time consuming to calibrate, susceptible to interference, and require a lengthy learning phase by the patient. Therefore, it is an open challenge to design more robust systems that are suitable for everyday use and meet the needs of patients. In this paper, we present a new concept of complete visual control for a prosthesis, an exoskeleton or another end effector using augmented reality (AR) glasses presented for the first time in a proof-of-concept study. By using AR glasses equipped with a monocular camera, a marker attached to the prosthesis is tracked. Minimal relative movements of the head with respect to the prosthesis are registered by tracking and used for control. Two possible control mechanisms including visual feedback are presented and implemented for both a motorized hand orthosis and a motorized hand prosthesis. Since the grasping process is mainly controlled by vision, the proposed approach appears to be natural and intuitive.