Learning new movements after paralysis: Results from a home-based study

An Exploratory Multi-Session Study of Learning High-Dimensional Body-Machine Interfacing for Assistive Robot Control

Individuals who suffer from severe paralysis often lose the capacity to perform fundamental body movements and everyday activities. Empowering these individuals with the ability to operate robotic arms, in high degrees-of-freedom (DoFs), can help to maximize both functional utility and independence. However, robot teleoperation in high DoFs currently lacks accessibility due to the challenge in capturing high-dimensional control signals from the human, especially in the face of motor impairments. Body-machine interfacing is a viable option that offers the necessary high-dimensional motion capture, and it moreover is noninvasive, affordable, and promotes movement and motor recovery. Nevertheless, to what extent body-machine interfacing is able to scale to high-DoF robot control, and whether it is feasible for humans to learn, remains an open question. In this exploratory multi-session study, we demonstrate the feasibility of human learning to operate a body-machine interface to control a complex, assistive robotic arm. We use a sensor net of four inertial measurement unit sensors, bilaterally placed on the scapulae and humeri. Ten uninjured participants are familiarized, trained, and evaluated in reaching and Activities of Daily Living tasks, using the body- machine interface. Our results suggest the manner of control space mapping (joint-space control versus task-space control), from interface to robot, plays a critical role in the evolution of human learning. Though joint-space control shows to be more intuitive initially, task-space control is found to have a greater capacity for longer-term improvement and learning.

Motor improvements enabled by spinal cord stimulation combined with physical training after spinal cord injury: review of experimental evidence in animals and humans

Electrical spinal cord stimulation (SCS) has been gaining momentum as a potential therapy for motor paralysis in consequence of spinal cord injury (SCI). Specifically, recent studies combining SCS with activity-based training have reported unprecedented improvements in motor function in people with chronic SCI that persist even without stimulation. In this work, we first provide an overview of the critical scientific advancements that have led to the current uses of SCS in neurorehabilitation: e.g. the understanding that SCS activates dormant spinal circuits below the lesion by recruiting large-to-medium diameter sensory afferents within the posterior roots. We discuss how this led to the standardization of implant position which resulted in consistent observations by independent clinical studies that SCS in combination with physical training promotes improvements in motor performance and neurorecovery. While all reported participants were able to move previously paralyzed limbs from day 1, recovery of more complex motor functions was gradual, and the timeframe for first observations was proportional to the task complexity. Interestingly, individuals with SCI classified as AIS B and C regained motor function in paralyzed joints even without stimulation, but not individuals with motor and sensory complete SCI (AIS A). Experiments in animal models of SCI investigating the potential mechanisms underpinning this neurorecovery suggest a synaptic reorganization of cortico-reticulo-spinal circuits that correlate with improvements in voluntary motor control. Future experiments in humans and animal models of paralysis will be critical to understand the potential and limits for functional improvements in people with different types, levels, timeframes, and severities of SCI.

Recovery of Distal Arm Movements in Spinal Cord Injured Patients with a Body-Machine Interface: A Proof-of-Concept Study

BACKGROUND

The recovery of upper limb mobility and functions is essential for people with cervical spinal cord injuries (cSCI) to maximize independence in daily activities and ensure a successful return to normality. The rehabilitative path should include a thorough neuromotor evaluation and personalized treatments aimed at recovering motor functions. Body-machine interfaces (BoMI) have been proven to be capable of harnessing residual joint motions to control objects like computer cursors and virtual or physical wheelchairs and to promote motor recovery. However, their therapeutic application has still been limited to shoulder movements. Here, we expanded the use of BoMI to promote the whole arm's mobility, with a special focus on elbow movements. We also developed an instrumented evaluation test and a set of kinematic indicators for assessing residual abilities and recovery.

METHODS

Five inpatient cSCI subjects (four acute, one chronic) participated in a BoMI treatment complementary to their standard rehabilitative routine. The subjects wore a BoMI with sensors placed on both proximal and distal arm districts and practiced for 5 weeks. The BoMI was programmed to promote symmetry between right and left arms use and the forearms' mobility while playing games. To evaluate the effectiveness of the treatment, the subjects' kinematics were recorded while performing an evaluation test that involved functional bilateral arms movements, before, at the end, and three months after training.

RESULTS

At the end of the training, all subjects learned to efficiently use the interface despite being compelled by it to engage their most impaired movements. The subjects completed the training with bilateral symmetry in body recruitment, already present at the end of the familiarization, and they increased the forearm activity. The instrumental evaluation confirmed this. The elbow motion's angular amplitude improved for all subjects, and other kinematic parameters showed a trend towards the normality range.

CONCLUSION

The outcomes are preliminary evidence supporting the efficacy of the proposed BoMI as a rehabilitation tool to be considered for clinical practice. It also suggests an instrumental evaluation protocol and a set of indicators to assess and evaluate motor impairment and recovery in cSCI.

Building an adaptive interface via unsupervised tracking of latent manifolds

In human-machine interfaces, decoder calibration is critical to enable an effective and seamless interaction with the machine. However, recalibration is often necessary as the decoder off-line predictive power does not generally imply ease-of-use, due to closed loop dynamics and user adaptation that cannot be accounted for during the calibration procedure. Here, we propose an adaptive interface that makes use of a non-linear autoencoder trained iteratively to perform online manifold identification and tracking, with the dual goal of reducing the need for interface recalibration and enhancing human-machine joint performance. Importantly, the proposed approach avoids interrupting the operation of the device and it neither relies on information about the state of the task, nor on the existence of a stable neural or movement manifold, allowing it to be applied in the earliest stages of interface operation, when the formation of new neural strategies is still on-going. In order to more directly test the performance of our algorithm, we defined the autoencoder latent space as the control space of a body-machine interface. After an initial offline parameter tuning, we evaluated the performance of the adaptive interface versus that of a static decoder in approximating the evolving low-dimensional manifold of users simultaneously learning to perform reaching movements within the latent space. Results show that the adaptive approach increased the representational efficiency of the interface decoder. Concurrently, it significantly improved users' task-related performance, indicating that the development of a more accurate internal model is encouraged by the online co-adaptation process.

Guiding functional reorganization of motor redundancy using a body-machine interface

Background

Body-machine interfaces map movements onto commands to external devices. Redundant motion signals derived from inertial sensors are mapped onto lower-dimensional device commands. Then, the device users face two problems, a) the structural problem of understanding the operation of the interface and b) the performance problem of controlling the external device with high efficiency. We hypothesize that these problems, while being distinct are connected in that aligning the space of body movements with the space encoded by the interface, i.e. solving the structural problem, facilitates redundancy resolution towards increasing efficiency, i.e. solving the performance problem.

Methods

Twenty unimpaired volunteers practiced controlling the movement of a computer cursor by moving their arms. Eight signals from four inertial sensors were mapped onto the two cursor’s coordinates on a screen. The mapping matrix was initialized by asking each user to perform free-form spontaneous upper-limb motions and deriving the two main principal components of the motion signals. Participants engaged in a reaching task for 18 min, followed by a tracking task. One group of 10 participants practiced with the same mapping throughout the experiment, while the other 10 with an adaptive mapping that was iteratively updated by recalculating the principal components based on ongoing movements.

Results

Participants quickly reduced reaching time while also learning to distribute most movement variance over two dimensions. Participants with the fixed mapping distributed movement variance over a subspace that did not match the potent subspace defined by the interface map. In contrast, participant with the adaptive map reduced the difference between the two subspaces, resulting in a smaller amount of arm motions distributed over the null space of the interface map. This, in turn, enhanced movement efficiency without impairing generalization from reaching to tracking.

Conclusions

Aligning the potent subspace encoded by the interface map to the user’s movement subspace guides redundancy resolution towards increasing movement efficiency, with implications for controlling assistive devices. In contrast, in the pursuit of rehabilitative goals, results would suggest that the interface must change to drive the statistics of user’s motions away from the established pattern and toward the engagement of movements to be recovered.

Trial registration

ClinicalTrials.gov, NCT01608438, Registered 16 April 2012.

Development of SaraHome: A novel, well-accepted, technology-based assessment tool for patients with ataxia

BACKGROUND AND OBJECTIVE

Early onset ataxias (EOAs) are a heterogeneous group of neurological conditions, responsible for severe motor disability in paediatric age, which still lack reliable outcome measures. Available scales to assess ataxia, such as the Scale for Assessment and Rating of Ataxia (SARA), are based on subjective assessment of specific motor and language tasks by an examiner, and therefore is age dependent and lacks accuracy in detecting small variations in disease severity. In last years, novel technologies, including computer interfaces and videogames, have emerged for clinical applications and the advent of Internet of Medical Things and of Information Communication Technology have allowed the remote control of such technologies. This pilot study describes a newly developed tool (SaraHome) for the assessment at home of EOA evaluating its feasibility and acceptability on a small sample of children.

METHODS

Ten EOA children and ten caregivers have been enrolled for a preliminary outpatient evaluation. The Microsoft Kinect 2.0 and Leap Motion Controller (LMC) connected to a personal computer with an ad hoc software have been set-up, for the acquisition of standardized motor tasks performed by the patients with the caregivers' assistance. Acceptance and practicability have been tested by QUEST 2.0 and IMI questionnaires in caregivers and patients respectively.

RESULTS

The SaraHome software was developed, based on a collection of services provided by a complex architecture that consists of a Restful interface, which enables to access a series of plugins for the execution of different tasks. A graphical user interface allows the acquisition of the patient movements while performing a motor task. A protocol of standard tasks inspired by SARA was established, and a system of video-assisted instruction provided. The set-up for the optimal acquisition of such protocol by Kinect and LMC has been defined. Both patients and caregivers accomplished the SaraHome assessment with good feedback at the technology acceptance questionnaires.

CONCLUSIONS

SaraHome represents a newly developed tool for the assessment of ataxia in patients, resulting from the integration of low-cost and easy-accessible technologies. This pilot application highlighted the feasibility and the acceptability of the system, suggesting the potential use in clinical practice.

The dynamics of motor learning through the formation of internal models

A medical student learning to perform a laparoscopic procedure or a recently paralyzed user of a powered wheelchair must learn to operate machinery via interfaces that translate their actions into commands for an external device. Since the user's actions are selected from a number of alternatives that would result in the same effect in the control space of the external device, learning to use such interfaces involves dealing with redundancy. Subjects need to learn an externally chosen many-to-one map that transforms their actions into device commands. Mathematically, we describe this type of learning as a deterministic dynamical process, whose state is the evolving forward and inverse internal models of the interface. The forward model predicts the outcomes of actions, while the inverse model generates actions designed to attain desired outcomes. Both the mathematical analysis of the proposed model of learning dynamics and the learning performance observed in a group of subjects demonstrate a first-order exponential convergence of the learning process toward a particular state that depends only on the initial state of the inverse and forward models and on the sequence of targets supplied to the users. Noise is not only present but necessary for the convergence of learning through the minimization of the difference between actual and predicted outcomes.

Age-dependent differences in learning to control a robot arm using a body-machine interface

Body-machine interfaces, i.e. interfaces that rely on body movements to control external assistive devices, have been proposed as a safe and robust means of achieving movement and mobility; however, how children learn these novel interfaces is poorly understood. Here we characterized the learning of a body-machine interface in young unimpaired adults, two groups of typically developing children (9-year and 12-year olds), and one child with congenital limb deficiency. Participants had to control the end-effector of a robot arm in 2D using movements of the shoulder and torso. Results showed a striking effect of age - children had much greater difficulty in learning the task compared to adults, with a majority of the 9-year old group unable to even complete the task. The 12-year olds also showed poorer task performance compared to adults (as measured by longer movement times and greater path lengths), which were associated with less effective search strategies. The child with congenital limb deficiency showed superior task performance compared to age-matched children, but had qualitatively distinct coordination strategies from the adults. Taken together, these results imply that children have difficulty learning non-intuitive interfaces and that the design of body-machine interfaces should account for these differences in pediatric populations.