Hybrid EEG/EOG-based brain/neural hand exoskeleton restores fully independent daily living activities after quadriplegia

User-Driven Functional Movement Training With a Wearable Hand Robot After Stroke

We studied the performance of a robotic orthosis designed to assist the paretic hand after stroke. It is wearable and fully user-controlled, serving two possible roles: as a therapeutic tool that facilitates device-mediated hand exercises to recover neuromuscular function or as an assistive device for use in everyday activities to aid functional use of the hand. We present the clinical outcomes of a pilot study designed as a feasibility test for these hypotheses. 11 chronic stroke (>2 years) patients with moderate muscle tone (Modified Ashworth Scale ≤ 2 in upper extremity) engaged in a month-long training protocol using the orthosis. Individuals were evaluated using standardized outcome measures, both with and without orthosis assistance. Fugl-Meyer post intervention scores without robotic assistance showed improvement focused specifically at the distal joints of the upper limb, suggesting the use of the orthosis as a rehabilitative device for the hand. Action Research Arm Test scores post intervention with robotic assistance showed that the device may serve an assistive role in grasping tasks. These results highlight the potential for wearable and user-driven robotic hand orthoses to extend the use and training of the affected upper limb after stroke.

Continuous Finite-Time Torque Control for Flexible Assistance Exoskeleton with Delay Variation Input

Accurate torque control is a critical issue in the compliant human–robot interaction scenario, which is, however, challenging due to the ever-changing human intentions, input delay, and various disturbances. Even worse, the performances of existing control strategies are limited on account of the compromise between precision and stability. To this end, this paper presents a novel high-performance torque control scheme without compromise. In this scheme, a new nonlinear disturbance observer incorporated with equivalent control concept is proposed, where the faster convergence and stronger anti-noise capability can be obtained simultaneously. Meanwhile, a continuous fractional power control law is designed with an iteration method to address the matched/unmatched disturbance rejection and global finite-time convergence. Moreover, the finite-time stability proof and prescribed control performance are guaranteed using constructed Lyapunov function with adding power integrator technique. Both the simulation and experiments demonstrate enhanced control accuracy, faster convergence rate, perfect disturbance rejection capability, and stronger robustness of the proposed control scheme. Furthermore, the evaluated assistance effects present improved gait patterns and reduced muscle efforts during walking and upstair activity.

A Survey on the Use of Haptic Feedback for Brain-Computer Interfaces and Neurofeedback

Neurofeedback (NF) and brain-computer interface (BCI) applications rely on the registration and real-time feedback of individual patterns of brain activity with the aim of achieving self-regulation of specific neural substrates or control of external devices. These approaches have historically employed visual stimuli. However, in some cases vision is unsuitable or inadequately engaging. Other sensory modalities, such as auditory or haptic feedback have been explored, and multisensory stimulation is expected to improve the quality of the interaction loop. Moreover, for motor imagery tasks, closing the sensorimotor loop through haptic feedback may be relevant for motor rehabilitation applications, as it can promote plasticity mechanisms. This survey reviews the various haptic technologies and describes their application to BCIs and NF. We identify major trends in the use of haptic interfaces for BCI and NF systems and discuss crucial aspects that could motivate further studies.

Is an artificial limb embodied as a hand? Brain decoding in prosthetic limb users

The potential ability of the human brain to represent an artificial limb as a body part (embodiment) has been inspiring engineers, clinicians, and scientists as a means to optimise human-machine interfaces. Using functional MRI (fMRI), we studied whether neural embodiment actually occurs in prosthesis users' occipitotemporal cortex (OTC). Compared with controls, different prostheses types were visually represented more similarly to each other, relative to hands and tools, indicating the emergence of a dissociated prosthesis categorisation. Greater daily life prosthesis usage correlated positively with greater prosthesis categorisation. Moreover, when comparing prosthesis users' representation of their own prosthesis to controls' representation of a similar looking prosthesis, prosthesis users represented their own prosthesis more dissimilarly to hands, challenging current views of visual prosthesis embodiment. Our results reveal a use-dependent neural correlate for wearable technology adoption, demonstrating adaptive use-related plasticity within the OTC. Because these neural correlates were independent of the prostheses' appearance and control, our findings offer new opportunities for prosthesis design by lifting restrictions imposed by the embodiment theory for artificial limbs.

General principles of machine learning for brain-computer interfacing

Brain-computer interfaces (BCIs) are systems that translate brain activity patterns into commands that can be executed by an artificial device. This enables the possibility of controlling devices such as a prosthetic arm or exoskeleton, a wheelchair, typewriting applications, or games directly by modulating our brain activity. For this purpose, BCI systems rely on signal processing and machine learning algorithms to decode the brain activity. This chapter provides an overview of the main steps required to do such a process, including signal preprocessing, feature extraction and selection, and decoding. Given the large amount of possible methods that can be used for these processes, a comprehensive review of them is beyond the scope of this chapter, and it is focused instead on the general principles that should be taken into account, as well as discussing good practices on how these methods should be applied and evaluated for proper design of reliable BCI systems.

Home-based rehabilitation using a soft robotic hand glove device leads to improvement in hand function in people with chronic spinal cord injury:a pilot study

BACKGROUND

Loss of hand function following high level spinal cord injury (SCI) is perceived as a high priority area for rehabilitation. Following discharge, it is often impractical for the specialist care centre to provide ongoing therapy for people living with chronic SCI at home, which can lead to further deterioration of hand function and a direct impact on an individual's capability to perform essential activities of daily living (ADL).

OBJECTIVE

This pilot study investigated the therapeutic effect of a self-administered home-based hand rehabilitation programme for people with cervical SCI using the soft extra muscle (SEM) Glove by Bioservo Technologies AB.

METHODS

Fifteen participants with chronic cervical motor incomplete (AIS C and D) SCI were recruited and provided with the glove device to use at home to complete a set task and perform their usual ADL for a minimum of 4 h a day for 12 weeks. Assessment was made at Week 0 (Initial), 6, 12 and 18 (6-week follow-up). The primary outcome measure was the Toronto Rehabilitation Institute hand function test (TRI-HFT), with secondary outcome measures including pinch dynamometry and the modified Ashworth scale.

RESULTS

The TRI-HFT demonstrated improvement in hand function at Week 6 of the therapy including improvement in object manipulation (58.3 ±3.2 to 66.9 ±1.8, p ≈ 0.01), and palmar grasp assessed as the length of the wooden bar that can be held using a pronated palmar grip (29.1 ±6.0 cm to 45.8 ±6.8 cm, p <0.01). A significant improvement in pinch strength, with reduced thumb muscle hypertonia was also detected. Improvements in function were present during the Week 12 assessment and also during the follow-up.

CONCLUSIONS

Self-administered rehabilitation using the SEM Glove is effective for improving and retaining gross and fine hand motor function for people living with chronic spinal cord injury at home. Retention of improved hand function suggests that an intensive activity-based rehabilitation programme in specific individuals is sufficient to improve long-term neuromuscular activity. Future studies should characterise the neuromuscular mechanism of action and the minimal rehabilitation programme necessary with the assistive device to improve ADL tasks following chronic cervical SCI.

TRIAL REGISTRATION NUMBER

Trial registration: ISRCTN, ISRCTN98677526, Registered 01/June/2017 - Retrospectively registered, http://www.isrctn.com/ISRCTN98677526.

Hand Extension Robot Orthosis (HERO) Grip Glove: enabling independence amongst persons with severe hand impairments after stroke

BACKGROUND

The Hand Extension Robot Orthosis (HERO) Grip Glove was iteratively designed to meet requests from therapists and persons after a stroke who have severe hand impairment to create a device that extends all five fingers, enhances grip strength and is portable, lightweight, easy to put on, comfortable and affordable.

METHODS

Eleven persons who have minimal or no active finger extension (Chedoke McMaster Stage of Hand 1-4) post-stroke were recruited to evaluate how well they could perform activities of daily living and finger function assessments with and without wearing the HERO Grip Glove.

RESULTS

The 11 participants showed statistically significant improvements (p < 0.01), while wearing the HERO Grip Glove, in the water bottle grasp and manipulation task (increase of 2.3 points, SD 1.2, scored using the Chedoke Hand and Arm Inventory scale from 1 to 7) and in index finger extension (increase of 147o, SD 44) and range of motion (increase of 145o, SD 36). The HERO Grip Glove provided 12.7 N (SD 8.9 N) of grip force and 11.0 N (SD 4.8) of pinch force to their affected hands, which enabled those without grip strength to grasp and manipulate blocks, a fork and a water bottle, as well as write with a pen. The participants were 'more or less satisfied' with the HERO Grip Glove as an assistive device (average of 3.3 out of 5 on the Quebec User Evaluation of Satisfaction with Assistive Technology 2.0 Scale). The highest satisfaction scores were given for safety and security (4.6) and ease of use (3.8) and the lowest satisfaction scores were given for ease of donning (2.3), which required under 5 min with assistance. The most common requests were for greater grip strength and a smaller glove size for small hands.

CONCLUSIONS

The HERO Grip Glove is a safe and effective tool for enabling persons with a stroke that have severe hand impairment to incorporate their affected hand into activities of daily living, which may motivate greater use of the affected upper extremity in daily life to stimulate neuromuscular recovery.

Non-contact visual control of personalized hand prostheses/exoskeletons by tracking using augmented reality glasses

A new concept for robust non-invasive optical activation of motorized hand prostheses by simple and non-contact commands is presented. In addition, a novel approach for aiding hand amputees is shown, outlining significant progress in thinking worth testing. In this, personalized 3D-printed artificial flexible hands are combined with commercially available motorized exoskeletons, as they are used e.g. in tetraplegics.

Motor Imagery Based Continuous Teleoperation Robot Control with Tactile Feedback

Brain computer interface (BCI) adopts human brain signals to control external devices directly without using normal neural pathway. Recent study has explored many applications, such as controlling a teleoperation robot by electroencephalography (EEG) signals. However, utilizing the motor imagery EEG-based BCI to perform teleoperation for reach and grasp task still has many difficulties, especially in continuous multidimensional control of robot and tactile feedback. In this research, a motor imagery EEG-based continuous teleoperation robot control system with tactile feedback was proposed. Firstly, mental imagination of different hand movements was translated into continuous command to control the remote robotic arm to reach the hover area of the target through a wireless local area network (LAN). Then, the robotic arm automatically completed the task of grasping the target. Meanwhile, the tactile information of remote robotic gripper was detected and converted to the feedback command. Finally, the vibrotactile stimulus was supplied to users to improve their telepresence. Experimental results demonstrate the feasibility of using the motor imagery EEG acquired by wireless portable equipment to realize the continuous teleoperation robot control system to finish the reach and grasp task. The average two-dimensional continuous control success rates for online Task 1 and Task 2 of the six subjects were 78.0% ± 6.1% and 66.2% ± 6.0%, respectively. Furthermore, compared with the traditional EEG triggered robot control using the predefined trajectory, the continuous fully two-dimensional control can not only improve the teleoperation robot system’s efficiency but also give the subject a more natural control which is critical to human–machine interaction (HMI). In addition, vibrotactile stimulus can improve the operator’s telepresence and task performance.

Development of 3D-printed myoelectric hand orthosis for patients with spinal cord injury

BACKGROUND

Spinal cord injury (SCI) is a severe medical condition affecting the hand and locomotor function. New medical technologies, including various wearable devices, as well as rehabilitation treatments are being developed to enhance hand function in patients with SCI. As three-dimensional (3D) printing has the advantage of being able to produce low-cost personalized devices, there is a growing appeal to apply this technology to rehabilitation equipment in conjunction with scientific advances. In this study, we proposed a novel 3D-printed hand orthosis that is controlled by electromyography (EMG) signals. The orthosis was designed to aid the grasping function for patients with cervical SCI. We applied this hand exoskeleton system to individuals with tetraplegia due to SCI and validated its effectiveness.

METHODS

The 3D architecture of the device was designed using computer-aided design software and printed with a polylactic acid filament. The dynamic hand orthosis enhanced the tenodesis grip to provide sufficient grasping function. The root mean square of the EMG signal was used as the input for controlling the device. Ten subjects with hand weakness due to chronic cervical SCI were enrolled in this study, and their hand function was assessed before and after wearing the orthosis. The Toronto Rehabilitation Institute Hand Function Test (TRI-HFT) was used as the primary outcome measure. Furthermore, improvements in functional independence in daily living and device usability were evaluated.

RESULTS

The newly developed orthosis improved hand function of subjects, as determined using the TRI-HFT (p < 0.05). Furthermore, participants obtained immediate functionality on eating after wearing the orthosis. Moreover, most participants were satisfied with the device as determined by the usability test. There were no side effects associated with the experiment.

CONCLUSIONS

The 3D-printed myoelectric hand orthosis was intuitive, easy to use, and showed positive effects in its ability to handle objects encountered in daily life. This study proved that combining simple EMG-based control strategies and 3D printing techniques was feasible and promising in rehabilitation engineering.

TRIAL REGISTRATION

Clinical Research Information Service (CRiS), Republic of Korea. KCT0003995. Registered 2 May 2019 - Retrospectively registered.

Myoelectric Control of a Soft Hand Exoskeleton Using Kinematic Synergies

Soft hand exoskeletons offer a lightweight, low-profile alternative to rigid rehabilitative robotic systems, enabling their use to restore activities of daily living (ADL) in those with hand paresis due to stroke or other conditions. The hand exoskeleton with embedded synergies (HEXOES) is a soft cable-driven hand exoskeleton capable of independently actuating and sensing 10 degrees of freedom (DoF) of the hand. Control of the 10 DoF exoskeleton is dimensionally reduced using three manually defined synergies in software corresponding to thumb, index, and 3-finger flexion and extension. In this paper, five healthy subjects control HEXOES using a neural network which decodes synergy weights from contralateral electromyography (EMG) activity. The three synergies are manipulated in real time to grasp and lift 15 ADL objects of various sizes and weights. The neural network's training and validation mean squared error, object grasp time, and grasp success rate were measured for five healthy subjects. The final training error of the neural network was 4.8 ± 1.8% averaged across subjects and tasks, with 8.3 ± 3.4% validation error. The time to reach, grasp, and lift an object was 11.15 ± 4.35 s on average, with an average success rate of 66.7% across all objects. The complete system demonstrates real time use of biosignals and machine learning to allow subjects to operate kinematic synergies to grasp objects using a wearable hand exoskeleton. Future work and applications are further discussed, including possible design improvements and enrollment of individuals with stroke.

Physiological Responses During Hybrid BNCI Control of an Upper-Limb Exoskeleton

When combined with assistive robotic devices, such as wearable robotics, brain/neural-computer interfaces (BNCI) have the potential to restore the capabilities of handicapped people to carry out activities of daily living. To improve applicability of such systems, workload and stress should be reduced to a minimal level. Here, we investigated the user's physiological reactions during the exhaustive use of the interfaces of a hybrid control interface. Eleven BNCI-naive healthy volunteers participated in the experiments. All participants sat in a comfortable chair in front of a desk and wore a whole-arm exoskeleton as well as wearable devices for monitoring physiological, electroencephalographic (EEG) and electrooculographic (EoG) signals. The experimental protocol consisted of three phases: (i) Set-up, calibration and BNCI training; (ii) Familiarization phase; and (iii) Experimental phase during which each subject had to perform EEG and EoG tasks. After completing each task, the NASA-TLX questionnaire and self-assessment manikin (SAM) were completed by the user. We found significant differences (p-value < 0.05) in heart rate variability (HRV) and skin conductance level (SCL) between participants during the use of the two different biosignal modalities (EEG, EoG) of the BNCI. This indicates that EEG control is associated with a higher level of stress (associated with a decrease in HRV) and mental work load (associated with a higher level of SCL) when compared to EoG control. In addition, HRV and SCL modulations correlated with the subject's workload perception and emotional responses assessed through NASA-TLX questionnaires and SAM.

Noninvasive neuroimaging enhances continuous neural tracking for robotic device control

Brain-computer interfaces (BCIs) utilizing signals acquired with intracortical implants have achieved successful high-dimensional robotic device control useful for completing daily tasks. However, the substantial amount of medical and surgical expertise required to correctly implant and operate these systems significantly limits their use beyond a few clinical cases. A noninvasive counterpart requiring less intervention that can provide high-quality control would profoundly impact the integration of BCIs into the clinical and home setting. Here, we present and validate a noninvasive framework utilizing electroencephalography (EEG) to achieve the neural control of a robotic device for continuous random target tracking. This framework addresses and improves upon both the "brain" and "computer" components by respectively increasing user engagement through a continuous pursuit task and associated training paradigm, and the spatial resolution of noninvasive neural data through EEG source imaging. In all, our unique framework enhanced BCI learning by nearly 60% for traditional center-out tasks and by over 500% in the more realistic continuous pursuit task. We further demonstrated an additional enhancement in BCI control of almost 10% by using online noninvasive neuroimaging. Finally, this framework was deployed in a physical task, demonstrating a near seamless transition from the control of an unconstrained virtual cursor to the real-time control of a robotic arm. Such combined advances in the quality of neural decoding and the practical utility of noninvasive robotic arm control will have major implications on the eventual development and implementation of neurorobotics by means of noninvasive BCI.

Synchronization of Slow Cortical Rhythms During Motor Imagery-Based Brain–Machine Interface Control

Modulation of sensorimotor rhythm (SMR) power, a rhythmic brain oscillation physiologically linked to motor imagery, is a popular Brain–Machine Interface (BMI) paradigm, but its interplay with slower cortical rhythms, also involved in movement preparation and cognitive processing, is not entirely understood. In this study, we evaluated the changes in phase and power of slow cortical activity in delta and theta bands, during a motor imagery task controlled by an SMR-based BMI system. In Experiment I, EEG of 20 right-handed healthy volunteers was recorded performing a motor-imagery task using an SMR-based BMI controlling a visual animation, and during task-free intervals. In Experiment II, 10 subjects were evaluated along five daily sessions, while BMI-controlling same visual animation, a buzzer, and a robotic hand exoskeleton. In both experiments, feedback received from the controlled device was proportional to SMR power (11–14[Formula: see text]Hz) detected by a real-time EEG-based system. Synchronization of slow EEG frequencies along the trials was evaluated using inter-trial-phase coherence (ITPC). Results: cortical oscillations of EEG in delta and theta frequencies synchronized at the onset and at the end of both active and task-free trials; ITPC was significantly modulated by feedback sensory modality received during the tasks; and ITPC synchronization progressively increased along the training. These findings suggest that phase-locking of slow rhythms and resetting by sensory afferences might be a functionally relevant mechanism in cortical control of motor function. We propose that analysis of phase synchronization of slow cortical rhythms might also improve identification of temporal edges in BMI tasks and might help to develop physiological markers for identification of context task switching and practice-related changes in brain function, with potentially important implications for design and monitoring of motor imagery-based BMI systems, an emerging tool in neurorehabilitation of stroke.

Brain-Machine Interfaces: Powerful Tools for Clinical Treatment and Neuroscientific Investigations

Brain-machine interfaces (BMIs) have exploded in popularity in the past decade. BMIs, also called brain-computer interfaces, provide a direct link between the brain and a computer, usually to control an external device. BMIs have a wide array of potential clinical applications, ranging from restoring communication to people unable to speak due to amyotrophic lateral sclerosis or a stroke, to restoring movement to people with paralysis from spinal cord injury or motor neuron disease, to restoring memory to people with cognitive impairment. Because BMIs are controlled directly by the activity of prespecified neurons or cortical areas, they also provide a powerful paradigm with which to investigate fundamental questions about brain physiology, including neuronal behavior, learning, and the role of oscillations. This article reviews the clinical and neuroscientific applications of BMIs, with a primary focus on motor BMIs.

Detection of movement onset using EMG signals for upper-limb exoskeletons in reaching tasks

BACKGROUND

To assist people with disabilities, exoskeletons must be provided with human-robot interfaces and smart algorithms capable to identify the user's movement intentions. Surface electromyographic (sEMG) signals could be suitable for this purpose, but their applicability in shared control schemes for real-time operation of assistive devices in daily-life activities is limited due to high inter-subject variability, which requires custom calibrations and training. Here, we developed a machine-learning-based algorithm for detecting the user's motion intention based on electromyographic signals, and discussed its applicability for controlling an upper-limb exoskeleton for people with severe arm disabilities.

METHODS

Ten healthy participants, sitting in front of a screen while wearing the exoskeleton, were asked to perform several reaching movements toward three LEDs, presented in a random order. EMG signals from seven upper-limb muscles were recorded. Data were analyzed offline and used to develop an algorithm that identifies the onset of the movement across two different events: moving from a resting position toward the LED (Go-forward), and going back to resting position (Go-backward). A set of subject-independent time-domain EMG features was selected according to information theory and their probability distributions corresponding to rest and movement phases were modeled by means of a two-component Gaussian Mixture Model (GMM). The detection of movement onset by two types of detectors was tested: the first type based on features extracted from single muscles, whereas the second from multiple muscles. Their performances in terms of sensitivity, specificity and latency were assessed for the two events with a leave one-subject out test method.

RESULTS

The onset of movement was detected with a maximum sensitivity of 89.3% for Go-forward and 60.9% for Go-backward events. Best performances in terms of specificity were 96.2 and 94.3% respectively. For both events the algorithm was able to detect the onset before the actual movement, while computational load was compatible with real-time applications.

CONCLUSIONS

The detection performances and the low computational load make the proposed algorithm promising for the control of upper-limb exoskeletons in real-time applications. Fast initial calibration makes it also suitable for helping people with severe arm disabilities in performing assisted functional tasks.

In-home and remote use of robotic body surrogates by people with profound motor deficits

By controlling robots comparable to the human body, people with profound motor deficits could potentially perform a variety of physical tasks for themselves, improving their quality of life. The extent to which this is achievable has been unclear due to the lack of suitable interfaces by which to control robotic body surrogates and a dearth of studies involving substantial numbers of people with profound motor deficits. We developed a novel, web-based augmented reality interface that enables people with profound motor deficits to remotely control a PR2 mobile manipulator from Willow Garage, which is a human-scale, wheeled robot with two arms. We then conducted two studies to investigate the use of robotic body surrogates. In the first study, 15 novice users with profound motor deficits from across the United States controlled a PR2 in Atlanta, GA to perform a modified Action Research Arm Test (ARAT) and a simulated self-care task. Participants achieved clinically meaningful improvements on the ARAT and 12 of 15 participants (80%) successfully completed the simulated self-care task. Participants agreed that the robotic system was easy to use, was useful, and would provide a meaningful improvement in their lives. In the second study, one expert user with profound motor deficits had free use of a PR2 in his home for seven days. He performed a variety of self-care and household tasks, and also used the robot in novel ways. Taking both studies together, our results suggest that people with profound motor deficits can improve their quality of life using robotic body surrogates, and that they can gain benefit with only low-level robot autonomy and without invasive interfaces. However, methods to reduce the rate of errors and increase operational speed merit further investigation.

User activity recognition system to improve the performance of environmental control interfaces: a pilot study with patients

BACKGROUND

Assistive technologies aim to increase quality of life, reduce dependence on care giver and on the long term care system. Several studies have demonstrated the effectiveness in the use of assistive technology for environment control and communication systems. The progress of brain-computer interfaces (BCI) research together with exoskeleton enable a person with motor impairment to interact with new elements in the environment. This paper aims to evaluate the environment control interface (ECI) developed under the AIDE project conditions, a multimodal interface able to analyze and extract relevant information from the environments as well as from the identification of residual abilities, behaviors, and intentions of the user.

METHODS

This study evaluated the ECI in a simulated scenario using a two screen layout: one with the ECI and the other with a simulated home environment, developed for this purpose. The sensorimotor rhythms and the horizontal oculoversion, acquired through BCI2000, a multipurpose standard BCI platform, were used to online control the ECI after the user training and system calibration. Eight subjects with different neurological diseases and spinal cord injury participated in this study. The subjects performed simulated activities of daily living (ADLs), i.e. actions in the simulated environment as drink, switch on a lamp or raise the bed head, during ten minutes in two different modes, AIDE mode, using a prediction model, to recognize the user intention facilitating the scan, and Manual mode, without a prediction model.

RESULTS

The results show that the mean task time spent in the AIDE mode was less than in the Manual, i.e the users were able to perform more tasks in the AIDE mode during the same time. The results showed a statistically significant differences with p<0.001. Regarding the steps, i.e the number of abstraction levels crossed in the ECI to perform an ADL, the users performed one step in the 90% of the tasks using the AIDE mode and three steps, at least, were necessary in the Manual mode. The user's intention prediction was performed through conditional random fields (CRF), with a global accuracy about 87%.

CONCLUSIONS

The environment analysis and the identification of the user's behaviors can be used to predict the user intention opening a new paradigm in the design of the ECIs. Although the developed ECI was tested only in a simulated home environment, it can be easily adapted to a real environment increasing the user independence at home.

Three-Dimensional Brain-Computer Interface Control Through Simultaneous Overt Spatial Attentional and Motor Imagery Tasks

OBJECTIVE

While noninvasive electroenceph-alography (EEG) based brain-computer interfacing (BCI) has been successfully demonstrated in two-dimensional (2-D) control tasks, little work has been published regarding its extension to practical three-dimensional (3-D) control.

METHODS

In this study, we developed a new BCI approach for 3-D control by combining a novel form of endogenous visuospatial attentional modulation, defined as overt spatial attention (OSA), and motor imagery (MI).

RESULTS

OSA modulation was shown to provide comparable control to conventional MI modulation in both 1-D and 2-D tasks. Furthermore, this paper provides evidence for the functional independence of traditional MI and OSA, as well as an investigation into the simultaneous use of both. Using this newly proposed BCI paradigm, 16 participants successfully completed a 3-D eight-target control task. Nine of these subjects further demonstrated robust 3-D control in a 12-target task, significantly outperforming the information transfer rate achieved in the 1-D and 2-D control tasks (29.7 ± 1.6 b/min).

CONCLUSION

These results strongly support the hypothesis that noninvasive EEG-based BCI can provide robust 3-D control through endogenous neural modulation in broader populations with limited training.

SIGNIFICANCE

Through the combination of the two strategies (MI and OSA), a substantial portion of the recruited subjects were capable of robustly controlling a virtual cursor in 3-D space. The proposed novel approach could broaden the dimensionality of BCI control and shorten the training time.

A large electroencephalographic motor imagery dataset for electroencephalographic brain computer interfaces

Recent advancements in brain computer interfaces (BCI) have demonstrated control of robotic systems by mental processes alone. Together with invasive BCI, electroencephalographic (EEG) BCI represent an important direction in the development of BCI systems. In the context of EEG BCI, the processing of EEG data is the key challenge. Unfortunately, advances in that direction have been complicated by a lack of large and uniform datasets that could be used to design and evaluate different data processing approaches. In this work, we release a large set of EEG BCI data collected during the development of a slow cortical potentials-based EEG BCI. The dataset contains 60 h of EEG recordings, 13 participants, 75 recording sessions, 201 individual EEG BCI interaction session-segments, and over 60 000 examples of motor imageries in 4 interaction paradigms. The current dataset presents one of the largest EEG BCI datasets publically available to date.

Feasibility and safety of shared EEG/EOG and vision-guided autonomous whole-arm exoskeleton control to perform activities of daily living

Arm and finger paralysis, e.g. due to brain stem stroke, often results in the inability to perform activities of daily living (ADLs) such as eating and drinking. Recently, it was shown that a hybrid electroencephalography/electrooculography (EEG/EOG) brain/neural hand exoskeleton can restore hand function to quadriplegics, but it was unknown whether such control paradigm can be also used for fluent, reliable and safe operation of a semi-autonomous whole-arm exoskeleton restoring ADLs. To test this, seven abled-bodied participants (seven right-handed males, mean age 30 ± 8 years) were instructed to use an EEG/EOG-controlled whole-arm exoskeleton attached to their right arm to perform a drinking task comprising multiple sub-tasks (reaching, grasping, drinking, moving back and releasing a cup). Fluent and reliable control was defined as average 'time to initialize' (TTI) execution of each sub-task below 3 s with successful initializations of at least 75% of sub-tasks within 5 s. During use of the system, no undesired side effects were reported. All participants were able to fluently and reliably control the vision-guided autonomous whole-arm exoskeleton (average TTI 2.12 ± 0.78 s across modalities with 75% successful initializations reached at 1.9 s for EOG and 4.1 s for EEG control) paving the way for restoring ADLs in severe arm and hand paralysis.

Assisting hand function after spinal cord injury with a fabric-based soft robotic glove

BACKGROUND

Spinal cord injury is a devastating condition that can dramatically impact hand motor function. Passive and active assistive devices are becoming more commonly used to enhance lost hand strength and dexterity. Soft robotics is an emerging discipline that combines the classical principles of robotics with soft materials and could provide a new class of active assistive devices. Soft robotic assistive devices enable a human-robot interaction facilitated by compliant and light-weight structures. The scope of this work was to demonstrate that a fabric-based soft robotic glove can effectively assist participants affected by spinal cord injury in manipulating objects encountered in daily living.

METHODS

The Toronto Rehabilitation Institute Hand Function Test was administered to 9 participants with C4-C7 spinal cord injuries to assess the functionality of the soft robotic glove. The test included object manipulation tasks commonly encountered during activities of daily living (ADL) and lift force measurements. The test was administered to each participant twice; once without the assistive glove to provide baseline data and once while wearing the assistive glove. The object manipulation subtests were evaluated using a linear mixed model, including interaction effects of variables such as time since injury. The lift force measures were separately evaluated using the Wilcoxon signed-rank test.

RESULTS

The soft robotic glove improved object manipulation in ADL tasks. The difference in mean scores between baseline and assisted conditions was significant across all participants and for all manipulated objects. An improvement of 33.42 ± 15.43% relative to the maximal test score indicates that the glove sufficiently enhances hand function during ADL tasks. Moreover, lift force also increased when using the assistive soft robotic glove, further demonstrating the effectiveness of the device in assisting hand function.

CONCLUSIONS

The results gathered in this study validate our fabric-based soft robotic glove as an effective device to assist hand function in individuals who have suffered upper limb paralysis following a spinal cord injury.

Brain-computer interfaces for post-stroke motor rehabilitation: a meta-analysis

Brain‐computer interfaces (BCIs) can provide sensory feedback of ongoing brain oscillations, enabling stroke survivors to modulate their sensorimotor rhythms purposefully. A number of recent clinical studies indicate that repeated use of such BCIs might trigger neurological recovery and hence improvement in motor function. Here, we provide a first meta‐analysis evaluating the clinical effectiveness of BCI‐based post‐stroke motor rehabilitation. Trials were identified using MEDLINE, CENTRAL, PEDro and by inspection of references in several review articles. We selected randomized controlled trials that used BCIs for post‐stroke motor rehabilitation and provided motor impairment scores before and after the intervention. A random‐effects inverse variance method was used to calculate the summary effect size. We initially identified 524 articles and, after removing duplicates, we screened titles and abstracts of 473 articles. We found 26 articles corresponding to BCI clinical trials, of these, there were nine studies that involved a total of 235 post‐stroke survivors that fulfilled the inclusion criterion (randomized controlled trials that examined motor performance as an outcome measure) for the meta‐analysis. Motor improvements, mostly quantified by the upper limb Fugl‐Meyer Assessment (FMA‐UE), exceeded the minimal clinically important difference (MCID=5.25) in six BCI studies, while such improvement was reached only in three control groups. Overall, the BCI training was associated with a standardized mean difference of 0.79 (95% CI: 0.37 to 1.20) in FMA‐UE compared to control conditions, which is in the range of medium to large summary effect size. In addition, several studies indicated BCI‐induced functional and structural neuroplasticity at a subclinical level. This suggests that BCI technology could be an effective intervention for post‐stroke upper limb rehabilitation. However, more studies with larger sample size are required to increase the reliability of these results.

Sensitive Manipulation: Manipulation Through Tactile Feedback

Object grasping and manipulation in robotics has been largely approached using visual feedback. Human studies on the other hand have demonstrated the importance of tactile and force feedback to guide the interaction between the fingers and the objects. Inspired by these observations, we propose an approach that consists in guiding a robot’s actions mainly by tactile feedback, with remote sensing such as vision, used only as a complement. Directly sensing the interaction forces between the object, the environment, and the robot’s hand enables it to obtain information relevant to the task that can be used to perform it more reliably. This approach (that we call sensitive manipulation) requires important changes in the hardware and in the way the robot is programmed. At the hardware level, we exploit compliant actuators and novel sensors that allow to safely interact and detect the environment. We developed strategies to perform manipulation tasks that take advantage of these new sensing and actuation capabilities. In this paper, we demonstrate that using these strategies the humanoid robot Obrero can safely find, reach and grab unknown objects that are neither held in place by a fixture nor placed in a specific orientation. The robot can also make insertions by “feeling” the hole without specialized mechanisms such as a remote center of compliance (RCC).

Closed-Loop Hybrid Gaze Brain-Machine Interface Based Robotic Arm Control with Augmented Reality Feedback

Brain-machine interface (BMI) can be used to control the robotic arm to assist paralysis people for performing activities of daily living. However, it is still a complex task for the BMI users to control the process of objects grasping and lifting with the robotic arm. It is hard to achieve high efficiency and accuracy even after extensive trainings. One important reason is lacking of sufficient feedback information for the user to perform the closed-loop control. In this study, we proposed a method of augmented reality (AR) guiding assistance to provide the enhanced visual feedback to the user for a closed-loop control with a hybrid Gaze-BMI, which combines the electroencephalography (EEG) signals based BMI and the eye tracking for an intuitive and effective control of the robotic arm. Experiments for the objects manipulation tasks while avoiding the obstacle in the workspace are designed to evaluate the performance of our method for controlling the robotic arm. According to the experimental results obtained from eight subjects, the advantages of the proposed closed-loop system (with AR feedback) over the open-loop system (with visual inspection only) have been verified. The number of trigger commands used for controlling the robotic arm to grasp and lift the objects with AR feedback has reduced significantly and the height gaps of the gripper in the lifting process have decreased more than 50% compared to those trials with normal visual inspection only. The results reveal that the hybrid Gaze-BMI user can benefit from the information provided by the AR interface, improving the efficiency and reducing the cognitive load during the grasping and lifting processes.

A Hybrid FPGA-Based System for EEG- and EMG-Based Online Movement Prediction

A current trend in the development of assistive devices for rehabilitation, for example exoskeletons or active orthoses, is to utilize physiological data to enhance their functionality and usability, for example by predicting the patient's upcoming movements using electroencephalography (EEG) or electromyography (EMG). However, these modalities have different temporal properties and classification accuracies, which results in specific advantages and disadvantages. To use physiological data analysis in rehabilitation devices, the processing should be performed in real-time, guarantee close to natural movement onset support, provide high mobility, and should be performed by miniaturized systems that can be embedded into the rehabilitation device. We present a novel Field Programmable Gate Array (FPGA) -based system for real-time movement prediction using physiological data. Its parallel processing capabilities allows the combination of movement predictions based on EEG and EMG and additionally a P300 detection, which is likely evoked by instructions of the therapist. The system is evaluated in an offline and an online study with twelve healthy subjects in total. We show that it provides a high computational performance and significantly lower power consumption in comparison to a standard PC. Furthermore, despite the usage of fixed-point computations, the proposed system achieves a classification accuracy similar to systems with double precision floating-point precision.