Body-Machine Interface Enables People With Cervical Spinal Cord Injury to Control Devices With Available Body Movements: Proof of Concept

Learning to Control Complex Robots Using High-Dimensional Body-Machine Interfaces

When individuals are paralyzed from injury or damage to the brain, upper body movement and function can be compromised. While the use of body motions to interface with machines has shown to be an effective noninvasive strategy to provide movement assistance and to promote physical rehabilitation, learning to use such interfaces to control complex machines is not well understood. In a five session study, we demonstrate that a subset of an uninjured population is able to learn and improve their ability to use a high-dimensional Body-Machine Interface (BoMI), to control a robotic arm. We use a sensor net of four inertial measurement units, placed bilaterally on the upper body, and a BoMI with the capacity to directly control a robot in six dimensions. We consider whether the way in which the robot control space is mapped from human inputs has any impact on learning. Our results suggest that the space of robot control does play a role in the evolution of human learning: specifically, though robot control in joint space appears to be more intuitive initially, control in task space is found to have a greater capacity for longer-term improvement and learning. Our results further suggest that there is an inverse relationship between control dimension couplings and task performance.

A novel virtual robotic platform for controlling six degrees of freedom assistive devices with body-machine interfaces

Body-machine interfaces (BoMIs)-systems that control assistive devices (e.g., a robotic manipulator) with a person's movements-offer a robust and non-invasive alternative to brain-machine interfaces for individuals with neurological injuries. However, commercially-available assistive devices offer more degrees of freedom (DOFs) than can be efficiently controlled with a user's residual motor function. Therefore, BoMIs often rely on nonintuitive mappings between body and device movements. Learning these mappings requires considerable practice time in a lab/clinic, which can be challenging. Virtual environments can potentially address this challenge, but there are limited options for high-DOF assistive devices, and it is unclear if learning with a virtual device is similar to learning with its physical counterpart. We developed a novel virtual robotic platform that replicated a commercially-available 6-DOF robotic manipulator. Participants controlled the physical and virtual robots using four wireless inertial measurement units (IMUs) fixed to the upper torso. Forty-three neurologically unimpaired adults practiced a target-matching task using either the physical (sample size n = 25) or virtual device (sample size n = 18) involving pre-, mid-, and post-tests separated by four training blocks. We found that both groups made similar improvements from pre-test in movement time at mid-test (Δvirtual: 9.9 ± 9.5 s; Δphysical: 11.1 ± 9.9 s) and post-test (Δvirtual: 11.1 ± 9.1 s; Δphysical: 11.8 ± 10.5 s) and in path length at mid-test (Δvirtual: 6.1 ± 6.3 m/m; Δphysical: 3.3 ± 3.5 m/m) and post-test (Δvirtual: 6.6 ± 6.2 m/m; Δphysical: 3.5 ± 4.0 m/m). Our results indicate the feasibility of using virtual environments for learning to control assistive devices. Future work should determine how these findings generalize to clinical populations.

Effectiveness of the Combined Use of a Brain-Machine Interface System and Virtual Reality as a Therapeutic Approach in Patients with Spinal Cord Injury: A Systematic Review

Spinal cord injury has a major impact on both the individual and society. This damage can cause permanent loss of sensorimotor functions, leading to structural and functional changes in somatotopic regions of the spinal cord. The combined use of a brain-machine interface and virtual reality offers a therapeutic alternative to be considered in the treatment of this pathology. This systematic review aimed to evaluate the effectiveness of the combined use of virtual reality and the brain-machine interface in the treatment of spinal cord injuries. A search was performed in PubMed, Web of Science, PEDro, Cochrane Central Register of Controlled Trials, CINAHL, Scopus, and Medline, including articles published from the beginning of each database until January 2023. Articles were selected based on strict inclusion and exclusion criteria. The Cochrane Collaboration's tool was used to assess the risk of bias and the PEDro scale and SCIRE systems were used to evaluate the methodological quality of the studies. Eleven articles were selected from a total of eighty-two. Statistically significant changes were found in the upper limb, involving improvements in shoulder and upper arm mobility, and weaker muscles were strengthened. In conclusion, most of the articles analyzed used the electroencephalogram as a measurement instrument for the assessment of various parameters, and most studies have shown improvements. Nonetheless, further research is needed with a larger sample size and long-term follow-up to establish conclusive results regarding the effect size of these interventions.

A Computer Interface Controlled by Upper Limb Muscles: Effects of a Two Weeks Training on Younger and Older Adults

As the population worldwide ages, there is a growing need for assistive technology and effective human-machine interfaces to address the wider range of motor disabilities that older adults may experience. Motor disabilities can make it difficult for individuals to perform basic daily tasks, such as getting dressed, preparing meals, or using a computer. The goal of this study was to investigate the effect of two weeks of training with a myoelectric computer interface (MCI) on motor functions in younger and older adults. Twenty people were recruited in the study: thirteen younger (range: 22-35 years old) and seven older (range: 61-78 years old) adults. Participants completed six training sessions of about 2 hours each, during which the activity of right and left biceps and trapezius were mapped into a control signal for the cursor of a computer. Results highlighted significant improvements in cursor control, and therefore in muscle coordination, in both groups. All participants with training became faster and more accurate, although people in different age range learned with a different dynamic. Results of the questionnaire on system usability and quality highlighted a general consensus about easiness of use and intuitiveness. These findings suggest that the proposed MCI training can be a powerful tool in the framework of assistive technologies for both younger and older adults. Further research is needed to determine the optimal duration and intensity of MCI training for different age groups and to investigate long-term effects of training on physical and cognitive function.

Autoencoder-based myoelectric controller for prosthetic hands

In the past, linear dimensionality-reduction techniques, such as Principal Component Analysis, have been used to simplify the myoelectric control of high-dimensional prosthetic hands. Nonetheless, their nonlinear counterparts, such as Autoencoders, have been shown to be more effective at compressing and reconstructing complex hand kinematics data. As a result, they have a potential of being a more accurate tool for prosthetic hand control. Here, we present a novel Autoencoder-based controller, in which the user is able to control a high-dimensional (17D) virtual hand via a low-dimensional (2D) space. We assess the efficacy of the controller via a validation experiment with four unimpaired participants. All the participants were able to significantly decrease the time it took for them to match a target gesture with a virtual hand to an average of 6.9 s and three out of four participants significantly improved path efficiency. Our results suggest that the Autoencoder-based controller has the potential to be used to manipulate high-dimensional hand systems via a myoelectric interface with a higher accuracy than PCA; however, more exploration needs to be done on the most effective ways of learning such a controller.

Learning to operate a high-dimensional hand via a low-dimensional controller

Dimensionality reduction techniques have proven useful in simplifying complex hand kinematics. They may allow for a low-dimensional kinematic or myoelectric interface to be used to control a high-dimensional hand. Controlling a high-dimensional hand, however, is difficult to learn since the relationship between the low-dimensional controls and the high-dimensional system can be hard to perceive. In this manuscript, we explore how training practices that make this relationship more explicit can aid learning. We outline three studies that explore different factors which affect learning of an autoencoder-based controller, in which a user is able to operate a high-dimensional virtual hand via a low-dimensional control space. We compare computer mouse and myoelectric control as one factor contributing to learning difficulty. We also compare training paradigms in which the dimensionality of the training task matched or did not match the true dimensionality of the low-dimensional controller (both 2D). The training paradigms were a) a full-dimensional task, in which the user was unaware of the underlying controller dimensionality, b) an implicit 2D training, which allowed the user to practice on a simple 2D reaching task before attempting the full-dimensional one, without establishing an explicit connection between the two, and c) an explicit 2D training, during which the user was able to observe the relationship between their 2D movements and the higher-dimensional hand. We found that operating a myoelectric interface did not pose a big challenge to learning the low-dimensional controller and was not the main reason for the poor performance. Implicit 2D training was found to be as good, but not better, as training directly on the high-dimensional hand. What truly aided the user's ability to learn the controller was the 2D training that established an explicit connection between the low-dimensional control space and the high-dimensional hand movements.

Restoration of sensory information via bionic hands

Individuals who have lost the use of their hands because of amputation or spinal cord injury can use prosthetic hands to restore their independence. A dexterous prosthesis requires the acquisition of control signals that drive the movements of the robotic hand, and the transmission of sensory signals to convey information to the user about the consequences of these movements. In this Review, we describe non-invasive and invasive technologies for conveying artificial sensory feedback through bionic hands, and evaluate the technologies' long-term prospects.

A Residual Movement Classification Based User Interface for Control of Assistive Devices by Persons with Complete Tetraplegia

OBJECTIVE

Complete tetraplegia can deprive a person of hand function. Assistive technologies may improve autonomy but needs for ergonomic interfaces for the user to pilot these devices still persist. Despite the paralysis of their arms, people with tetraplegia may retain residual shoulder movements. In this work we explored these movements as a mean to control assistive devices.

METHODS

We captured shoulder movement with a single inertial sensor and, by training a support vector machine based classifier, we decode such information into user intent.

RESULTS

The setup and training process take only a few minutes and so the classifiers can be user specific. We tested the algorithm with 10 able body and 2 spinal cord injury participants. The average classification accuracy was 80% and 84%, respectively.

CONCLUSION

The proposed algorithm is easy to set up, its operation is fully automated, and achieved results are on par with state-of-the-art systems.

SIGNIFICANCE

Assistive devices for persons without hand function present limitations in their user interfaces. Our work present a novel method to overcome some of these limitations by classifying user movement and decoding it into user intent, all with simple setup and training and no need for manual tuning. We demonstrate its feasibility with experiments with end users, including persons with complete tetraplegia without hand function.

Recovery of Distal Arm Movements in Spinal Cord Injured Patients with a Body-Machine Interface: A Proof-of-Concept Study

BACKGROUND

The recovery of upper limb mobility and functions is essential for people with cervical spinal cord injuries (cSCI) to maximize independence in daily activities and ensure a successful return to normality. The rehabilitative path should include a thorough neuromotor evaluation and personalized treatments aimed at recovering motor functions. Body-machine interfaces (BoMI) have been proven to be capable of harnessing residual joint motions to control objects like computer cursors and virtual or physical wheelchairs and to promote motor recovery. However, their therapeutic application has still been limited to shoulder movements. Here, we expanded the use of BoMI to promote the whole arm's mobility, with a special focus on elbow movements. We also developed an instrumented evaluation test and a set of kinematic indicators for assessing residual abilities and recovery.

METHODS

Five inpatient cSCI subjects (four acute, one chronic) participated in a BoMI treatment complementary to their standard rehabilitative routine. The subjects wore a BoMI with sensors placed on both proximal and distal arm districts and practiced for 5 weeks. The BoMI was programmed to promote symmetry between right and left arms use and the forearms' mobility while playing games. To evaluate the effectiveness of the treatment, the subjects' kinematics were recorded while performing an evaluation test that involved functional bilateral arms movements, before, at the end, and three months after training.

RESULTS

At the end of the training, all subjects learned to efficiently use the interface despite being compelled by it to engage their most impaired movements. The subjects completed the training with bilateral symmetry in body recruitment, already present at the end of the familiarization, and they increased the forearm activity. The instrumental evaluation confirmed this. The elbow motion's angular amplitude improved for all subjects, and other kinematic parameters showed a trend towards the normality range.

CONCLUSION

The outcomes are preliminary evidence supporting the efficacy of the proposed BoMI as a rehabilitation tool to be considered for clinical practice. It also suggests an instrumental evaluation protocol and a set of indicators to assess and evaluate motor impairment and recovery in cSCI.

Building an adaptive interface via unsupervised tracking of latent manifolds

In human-machine interfaces, decoder calibration is critical to enable an effective and seamless interaction with the machine. However, recalibration is often necessary as the decoder off-line predictive power does not generally imply ease-of-use, due to closed loop dynamics and user adaptation that cannot be accounted for during the calibration procedure. Here, we propose an adaptive interface that makes use of a non-linear autoencoder trained iteratively to perform online manifold identification and tracking, with the dual goal of reducing the need for interface recalibration and enhancing human-machine joint performance. Importantly, the proposed approach avoids interrupting the operation of the device and it neither relies on information about the state of the task, nor on the existence of a stable neural or movement manifold, allowing it to be applied in the earliest stages of interface operation, when the formation of new neural strategies is still on-going. In order to more directly test the performance of our algorithm, we defined the autoencoder latent space as the control space of a body-machine interface. After an initial offline parameter tuning, we evaluated the performance of the adaptive interface versus that of a static decoder in approximating the evolving low-dimensional manifold of users simultaneously learning to perform reaching movements within the latent space. Results show that the adaptive approach increased the representational efficiency of the interface decoder. Concurrently, it significantly improved users' task-related performance, indicating that the development of a more accurate internal model is encouraged by the online co-adaptation process.

Controlling a robotic arm for functional tasks using a wireless head-joystick: A case study of a child with congenital absence of upper and lower limbs

Children with movement impairments needing assistive devices for activities of daily living often require novel methods for controlling these devices. Body-machine interfaces, which rely on body movements, are particularly well-suited for children as they are non-invasive and have high signal-to-noise ratios. Here, we examined the use of a head-joystick to enable a child with congenital absence of all four limbs to control a seven degree-of-freedom robotic arm. Head movements were measured with a wireless inertial measurement unit and used to control a robotic arm to perform two functional tasks-a drinking task and a block stacking task. The child practiced these tasks over multiple sessions; a control participant performed the same tasks with a manual joystick. Our results showed that the child was able to successfully perform both tasks, with movement times decreasing by ~40-50% over 6-8 sessions of training. The child's performance with the head-joystick was also comparable to the control participant using a manual joystick. These results demonstrate the potential of using head movements for the control of high degree-of-freedom tasks in children with limited movement repertoire.

A hybrid Body-Machine Interface integrating signals from muscles and motions

OBJECTIVE

Body-Machine Interfaces (BoMIs) establish a way to operate a variety of devices, allowing their users to extend the limits of their motor abilities by exploiting the redundancy of muscles and motions that remain available after spinal cord injury or stroke. Here, we considered the integration of two types of signals, motion signals derived from inertial measurement units (IMUs) and muscle activities recorded with electromyography (EMG), both contributing to the operation of the BoMI.

APPROACH

A direct combination of IMU and EMG signals might result in inefficient control due to the differences in their nature. Accordingly, we used a nonlinear-regression-based approach to predict IMU from EMG signals, after which the predicted and actual IMU signals were combined into a hybrid control signal. The goal of this approach was to provide users with the possibility to switch seamlessly between movement and EMG control, using the BoMI as a tool for promoting the engagement of selected muscles. We tested the interface in three control modalities, EMG-only, IMU-only and hybrid, in a cohort of 15 unimpaired participants. Participants practiced reaching movements by guiding a computer cursor over a set of targets.

MAIN RESULTS

We found that the proposed hybrid control led to comparable performance to IMU-based control and significantly outperformed the EMG-only control. Results also indicated that hybrid cursor control was predominantly influenced by EMG signals.

SIGNIFICANCE

We concluded that combining EMG with IMU signals could be an efficient way to target muscle activations while overcoming the limitations of an EMG-only control.

Guiding functional reorganization of motor redundancy using a body-machine interface

Background

Body-machine interfaces map movements onto commands to external devices. Redundant motion signals derived from inertial sensors are mapped onto lower-dimensional device commands. Then, the device users face two problems, a) the structural problem of understanding the operation of the interface and b) the performance problem of controlling the external device with high efficiency. We hypothesize that these problems, while being distinct are connected in that aligning the space of body movements with the space encoded by the interface, i.e. solving the structural problem, facilitates redundancy resolution towards increasing efficiency, i.e. solving the performance problem.

Methods

Twenty unimpaired volunteers practiced controlling the movement of a computer cursor by moving their arms. Eight signals from four inertial sensors were mapped onto the two cursor’s coordinates on a screen. The mapping matrix was initialized by asking each user to perform free-form spontaneous upper-limb motions and deriving the two main principal components of the motion signals. Participants engaged in a reaching task for 18 min, followed by a tracking task. One group of 10 participants practiced with the same mapping throughout the experiment, while the other 10 with an adaptive mapping that was iteratively updated by recalculating the principal components based on ongoing movements.

Results

Participants quickly reduced reaching time while also learning to distribute most movement variance over two dimensions. Participants with the fixed mapping distributed movement variance over a subspace that did not match the potent subspace defined by the interface map. In contrast, participant with the adaptive map reduced the difference between the two subspaces, resulting in a smaller amount of arm motions distributed over the null space of the interface map. This, in turn, enhanced movement efficiency without impairing generalization from reaching to tracking.

Conclusions

Aligning the potent subspace encoded by the interface map to the user’s movement subspace guides redundancy resolution towards increasing movement efficiency, with implications for controlling assistive devices. In contrast, in the pursuit of rehabilitative goals, results would suggest that the interface must change to drive the statistics of user’s motions away from the established pattern and toward the engagement of movements to be recovered.

Trial registration

ClinicalTrials.gov, NCT01608438, Registered 16 April 2012.

Providing low-dimensional feedback of a high-dimensional movement allows for improved performance of a skilled walking task

Learning a skilled movement often requires changing multiple dimensions of movement in a coordinated manner. Serial training is one common approach to learning a new movement pattern, where each feature is learned in isolation from the others. Once one feature is learned, we move on to the next. However, when learning a complex movement pattern, serial training is not only laborious but can also be ineffective. Often, movement features are linked such that they cannot simply be added together as we progress through training. Thus, the ability to learn multiple features in parallel could make training faster and more effective. When using visual feedback as the tool for changing movement, however, such parallel training may increase the attentional load of training and impair performance. Here, we developed a novel visual feedback system that uses principal component analysis to weight four features of movement to create a simple one-dimensional 'summary' of performance. We used this feedback to teach healthy, young participants a modified walking pattern and compared their performance to those who received four concurrent streams of visual information to learn the same goal walking pattern. We demonstrated that those who used the principal component-based visual feedback improved their performance faster and to a greater extent compared to those who received concurrent feedback of all features. These results suggest that our novel principal component-based visual feedback provides a method for altering multiple features of movement toward a prescribed goal in an intuitive, low-dimensional manner.

Smoothness: an Unexplored Window into Coordinated Running Proficiency

Over the expanse of evolutionary history, humans, and predecessor Homo species, ran to survive. This legacy is reflected in many deeply and irrevocably embedded neurological and biological design features, features which shape how we run, yet were themselves shaped by running.

Smoothness is a widely recognised feature of healthy, proficient movement. Nevertheless, although the term ‘smoothness’ is commonly used to describe skilled athletic movement within practical sporting contexts, it is rarely specifically defined, is rarely quantified and remains barely explored experimentally. Elsewhere, however, within various health-related and neuro-physiological domains, many manifestations of movement smoothness have been extensively investigated. Within this literature, smoothness is considered a reflection of a healthy central nervous system (CNS) and is implicitly associated with practiced coordinated proficiency; ‘non-smooth’ movement, in contrast, is considered a consequence of pathological, un-practiced or otherwise inhibited motor control.

Despite the ubiquity of running across human cultures, however, and the apparent importance of smoothness as a fundamental feature of healthy movement control, to date, no theoretical framework linking the phenomenon of movement smoothness to running proficiency has been proposed. Such a framework could, however, provide a novel lens through which to contextualise the deep underlying nature of coordinated running control. Here, we consider the relevant evidence and suggest how running smoothness may integrate with other related concepts such as complexity, entropy and variability. Finally, we suggest that these insights may provide new means of coherently conceptualising running coordination, may guide future research directions, and may productively inform practical coaching philosophies.

Age-dependent differences in learning to control a robot arm using a body-machine interface

Body-machine interfaces, i.e. interfaces that rely on body movements to control external assistive devices, have been proposed as a safe and robust means of achieving movement and mobility; however, how children learn these novel interfaces is poorly understood. Here we characterized the learning of a body-machine interface in young unimpaired adults, two groups of typically developing children (9-year and 12-year olds), and one child with congenital limb deficiency. Participants had to control the end-effector of a robot arm in 2D using movements of the shoulder and torso. Results showed a striking effect of age - children had much greater difficulty in learning the task compared to adults, with a majority of the 9-year old group unable to even complete the task. The 12-year olds also showed poorer task performance compared to adults (as measured by longer movement times and greater path lengths), which were associated with less effective search strategies. The child with congenital limb deficiency showed superior task performance compared to age-matched children, but had qualitatively distinct coordination strategies from the adults. Taken together, these results imply that children have difficulty learning non-intuitive interfaces and that the design of body-machine interfaces should account for these differences in pediatric populations.

Development of an EMG-Controlled Serious Game for Rehabilitation

A majority of the seven million stroke survivors in the U.S. have hand impairments, adversely affecting performance of a variety of activities of daily living, because of the fundamental role of the hand in performing functional tasks. Disability in stroke survivors is largely attributable to damaged neuronal pathways, which result in inappropriate activation of muscles, a condition prevalent in distal upper extremity muscles following stroke. While conventional rehabilitation methods focus on the amplification of existing muscle activation, the effectiveness of therapy targeting the reorganization of pathological activation patterns is often unexplored. To encourage modulation of activation level and exploration of the activation workspace, we developed a novel platform for playing a serious game through electromyographic control. This system was evaluated by a group of neurologically intact subjects over multiple sessions held on different days. Subjects were assigned to one of two groups, training either with their non-dominant hand only (unilateral) or with both hands (bilateral). Both groups of subjects displayed improved performance in controlling the cursor with their non-dominant hand, with retention from one session to the next. The system holds promise for rehabilitation of control of muscle activation patterns.

Learning new movements after paralysis: Results from a home-based study

Body-machine interfaces (BMIs) decode upper-body motion for operating devices, such as computers and wheelchairs. We developed a low-cost portable BMI for survivors of cervical spinal cord injury and investigated it as a means to support personalized assistance and therapy within the home environment. Depending on the specific impairment of each participant, we modified the interface gains to restore a higher level of upper body mobility. The use of the BMI over one month led to increased range of motion and force at the shoulders in chronic survivors. Concurrently, subjects learned to reorganize their body motions as they practiced the control of a computer cursor to perform different tasks and games. The BMI allowed subjects to generate any movement of the cursor with different motions of their body. Through practice subjects demonstrated a tendency to increase the similarity between the body motions used to control the cursor in distinct tasks. Nevertheless, by the end of learning, some significant and persistent differences appeared to persist. This suggests the ability of the central nervous system to concurrently learn operating the BMI while exploiting the possibility to adapt the available mobility to the specific spatio-temporal requirements of each task.