Do Cost Functions for Tracking Error Generalize across Tasks with Different Noise Levels?

A Review of Sensory Feedback in Upper-Limb Prostheses From the Perspective of Human Motor Control

This manuscript reviews historical and recent studies that focus on supplementary sensory feedback for use in upper limb prostheses. It shows that the inability of many studies to speak to the issue of meaningful performance improvements in real-life scenarios is caused by the complexity of the interactions of supplementary sensory feedback with other types of feedback along with other portions of the motor control process. To do this, the present manuscript frames the question of supplementary feedback from the perspective of computational motor control, providing a brief review of the main advances in that field over the last 20 years. It then separates the studies on the closed-loop prosthesis control into distinct categories, which are defined by relating the impact of feedback to the relevant components of the motor control framework, and reviews the work that has been done over the last 50+ years in each of those categories. It ends with a discussion of the studies, along with suggestions for experimental construction and connections with other areas of research, such as machine learning.

Dissociating error-based and reinforcement-based loss functions during sensorimotor learning

It has been proposed that the sensorimotor system uses a loss (cost) function to evaluate potential movements in the presence of random noise. Here we test this idea in the context of both error-based and reinforcement-based learning. In a reaching task, we laterally shifted a cursor relative to true hand position using a skewed probability distribution. This skewed probability distribution had its mean and mode separated, allowing us to dissociate the optimal predictions of an error-based loss function (corresponding to the mean of the lateral shifts) and a reinforcement-based loss function (corresponding to the mode). We then examined how the sensorimotor system uses error feedback and reinforcement feedback, in isolation and combination, when deciding where to aim the hand during a reach. We found that participants compensated differently to the same skewed lateral shift distribution depending on the form of feedback they received. When provided with error feedback, participants compensated based on the mean of the skewed noise. When provided with reinforcement feedback, participants compensated based on the mode. Participants receiving both error and reinforcement feedback continued to compensate based on the mean while repeatedly missing the target, despite receiving auditory, visual and monetary reinforcement feedback that rewarded hitting the target. Our work shows that reinforcement-based and error-based learning are separable and can occur independently. Further, when error and reinforcement feedback are in conflict, the sensorimotor system heavily weights error feedback over reinforcement feedback.

Whether serving a tennis ball on a gusty day or walking over an unpredictable surface, the human nervous system has a remarkable ability to account for uncertainty when performing goal-directed actions. Here we address how different types of feedback, error and reinforcement, are used to guide such behavior during sensorimotor learning. Using a task that dissociates the optimal predictions of error-based and reinforcement-based learning, we show that the human sensorimotor system uses two distinct loss functions when deciding where to aim the hand during a reach—one that minimizes error and another that maximizes success. Interestingly, when both of these forms of feedback are available our nervous system heavily weights error feedback over reinforcement feedback.

Adaptation to random and systematic errors: Comparison of amputee and non-amputee control interfaces with varying levels of process noise

The objective of this study was to understand how people adapt to errors when using a myoelectric control interface. We compared adaptation across 1) non-amputee subjects using joint angle, joint torque, and myoelectric control interfaces, and 2) amputee subjects using myoelectric control interfaces with residual and intact limbs (five total control interface conditions). We measured trial-by-trial adaptation to self-generated errors and random perturbations during a virtual, single degree-of-freedom task with two levels of feedback uncertainty, and evaluated adaptation by fitting a hierarchical Kalman filter model. We have two main results. First, adaptation to random perturbations was similar across all control interfaces, whereas adaptation to self-generated errors differed. These patterns matched predictions of our model, which was fit to each control interface by changing the process noise parameter that represented system variability. Second, in amputee subjects, we found similar adaptation rates and error levels between residual and intact limbs. These results link prosthesis control to broader areas of motor learning and adaptation and provide a useful model of adaptation with myoelectric control. The model of adaptation will help us understand and solve prosthesis control challenges, such as providing additional sensory feedback.