Improving Fine Control of Grasping Force during Hand-Object Interactions for a Soft Synergy-Inspired Myoelectric Prosthetic Hand

A multifaceted suite of metrics for comparative myoelectric prosthesis controller research

Upper limb robotic (myoelectric) prostheses are technologically advanced, but challenging to use. In response, substantial research is being done to develop person-specific prosthesis controllers that can predict a user's intended movements. Most studies that test and compare new controllers rely on simple assessment measures such as task scores (e.g., number of objects moved across a barrier) or duration-based measures (e.g., overall task completion time). These assessment measures, however, fail to capture valuable details about: the quality of device arm movements; whether these movements match users' intentions; the timing of specific wrist and hand control functions; and users' opinions regarding overall device reliability and controller training requirements. In this work, we present a comprehensive and novel suite of myoelectric prosthesis control evaluation metrics that better facilitates analysis of device movement details-spanning measures of task performance, control characteristics, and user experience. As a case example of their use and research viability, we applied these metrics in real-time control experimentation. Here, eight participants without upper limb impairment compared device control offered by a deep learning-based controller (recurrent convolutional neural network-based classification with transfer learning, or RCNN-TL) to that of a commonly used controller (linear discriminant analysis, or LDA). The participants wore a simulated prosthesis and performed complex functional tasks across multiple limb positions. Analysis resulting from our suite of metrics identified 16 instances of a user-facing problem known as the "limb position effect". We determined that RCNN-TL performed the same as or significantly better than LDA in four such problem instances. We also confirmed that transfer learning can minimize user training burden. Overall, this study contributes a multifaceted new suite of control evaluation metrics, along with a guide to their application, for use in research and testing of myoelectric controllers today, and potentially for use in broader rehabilitation technologies of the future.

An Improved Approach for Grasp Force Sensing and Control of Upper Limb Soft Robotic Prosthetics

The following research proposes a closed loop force control system, which is implemented on a soft robotic prosthetic hand. The proposed system uses a force sensing approach that does not require any sensing elements to be embedded in the prosthetic's fingers, therefore maintaining their monolithic structural integrity, and subsequently decreasing the cost and manufacturing complexity. This is achieved by embedding an aluminum test specimen with a full bridge strain gauge circuit directly inside the actuator's housing rather than in the finger. The location of the test specimen is precisely at the location of the critical section of the bending moment on the actuator housing due to the tension in the driving tendon. Therefore, the resulting loadcell can acquire a signal proportional to the prosthetic's grasping force. A PI controller is implemented and tested using this force sensing approach. The experiment design includes a flexible test object, which serves to visually demonstrate the force controller's performance through the deformation that the test object experiences. Setpoints corresponding to "light", "medium", and "hard" grasps were tested with pinch, tripod, and full grasps and the results of these tests are documented in this manuscript. The developed controller was found to have an accuracy of ±2%. Additionally, the deformation of the test object increased proportionally with the given grasp force setpoint, with almost no deformation during the light grasp test, slight deformation during the medium grasp test, and relatively large deformation of the test object during the hard grasp test.

EMG feedback outperforms force feedback in the presence of prosthesis control disturbance

Closing the prosthesis control loop by providing artificial somatosensory feedback can improve utility and user experience. Additionally, closed-loop control should be more robust with respect to disturbance, but this might depend on the type of feedback provided. Thus, the present study investigates and compares the performance of EMG and force feedback in the presence of control disturbances. Twenty able-bodied subjects and one transradial amputee performed delicate and power grasps with a prosthesis in a functional task, while the control signal gain was temporarily increased (high-gain disturbance) or decreased (low-gain disturbance) without their knowledge. Three outcome measures were considered: the percentage of trials successful in the first attempt (reaction to disturbance), the average number of attempts in trials where the wrong force was initially applied (adaptation to disturbance), and the average completion time of the last attempt in every trial. EMG feedback was shown to offer significantly better performance compared to force feedback during power grasping in terms of reaction to disturbance and completion time. During power grasping with high-gain disturbance, the median first-attempt success rate was significantly higher with EMG feedback (73.3%) compared to that achieved with force feedback (60%). Moreover, the median completion time for power grasps with low-gain disturbance was significantly longer with force feedback than with EMG feedback (3.64 against 2.48 s, an increase of 32%). Contrary to our expectations, there was no significant difference between feedback types with regards to adaptation to disturbances and the two feedback types performed similarly in delicate grasps. The results indicated that EMG feedback displayed better performance than force feedback in the presence of control disturbances, further demonstrating the potential of this approach to provide a reliable prosthesis-user interaction.

Multichannel haptic feedback unlocks prosthetic hand dexterity

Loss of tactile sensations is a major roadblock preventing upper limb-absent people from multitasking or using the full dexterity of their prosthetic hands. With current myoelectric prosthetic hands, limb-absent people can only control one grasp function at a time even though modern artificial hands are mechanically capable of individual control of all five digits. In this paper, we investigated whether people could precisely control the grip forces applied to two different objects grasped simultaneously with a dexterous artificial hand. Toward that end, we developed a novel multichannel wearable soft robotic armband to convey artificial sensations of touch to the robotic hand users. Multiple channels of haptic feedback enabled subjects to successfully grasp and transport two objects simultaneously with the dexterous artificial hand without breaking or dropping them, even when their vision of both objects was obstructed. Simultaneous transport of the objects provided a significant time savings to perform the deliveries in comparison to a one-at-a-time approach. This paper demonstrated that subjects were able to integrate multiple channels of haptic feedback into their motor control strategies to perform a complex simultaneous object grasp control task with an artificial limb, which could serve as a paradigm shift in the way prosthetic hands are operated.

The "Federica" Hand

Hand prostheses partially restore hand appearance and functionalities. In particular, 3D printers have provided great opportunities by simplifying the manufacturing process and reducing costs. The "Federica" hand is 3D-printed and equipped with a single servomotor, which synergically actuates its five fingers by inextensible tendons; no springs are used for hand opening. A differential mechanical system simultaneously distributes the motor force on each finger in predefined portions. The proportional control of hand closure/opening is achieved by monitoring muscle contraction by means of a thin force sensor, as an alternative to EMG. The electrical current of the servomotor is monitored to provide sensory feedback of the grip force, through a small vibration motor. A simple Arduino board was adopted as the processing unit. A closed-chain, differential mechanism guarantees efficient transfer of mechanical energy and a secure grasp of any object, regardless of its shape and deformability. The force sensor offers some advantages over the EMG: it does not require any electrical contact or signal processing to monitor muscle contraction intensity. The activation speed (about half a second) is high enough to allow the user to grab objects on the fly. The cost of the device is less then 100 USD. The "Federica" hand has proved to be a lightweight, low-cost and extremely efficient prosthesis. It is now available as an open-source project (CAD files and software can be downloaded from a public repository), thus allowing everyone to use the "Federica" hand and customize or improve it.

Evaluation of Grip Force and Energy Efficiency of the “Federica” Hand

The actual grip force provided by a hand prosthesis is an important parameter to evaluate its efficiency. To this end, a split cylindrical handlebar embedding a single-axis load cell was designed, 3D printed and assembled. Various measurements were made to evaluate the performances of the “Federica” hand, a simple low-cost hand prosthesis. The handlebar was placed at different angular positions with respect to the hand palm, and the experimental data were processed to estimate the overall grip force. In addition, piezoresistive force sensors were applied on selected phalanxes of the prosthesis, in order to map the distribution of the grasping forces between them. The electrical current supplied to the single servomotor that actuates all the five fingers, was monitored to estimate the force exerted on the main actuator tendon, while tendon displacement was evaluated by a rotary potentiometer fixed to the servomotor shaft. The force transfer ratio of the whole system was about 12.85 %, and the mean dissipated energy for a complete cycle of closing-opening was 106.80 Nmm, resulting lower than that of many commercial prostheses. The mean grip force of the “Federica” hand was 8.80 N, that is enough to support the user in many actions of daily life, also considering the adaptive wrapping capability of the prosthesis. On average, the middle phalanges exerted the greatest grip force (2.65 N) on the handlebar, while the distal phalanges a force of 1.66 N.

Exploring Stiffness Modulation in Prosthetic Hands and Its Perceived Function in Manipulation and Social Interaction

To physically interact with a rich variety of environments and to match situation-dependent requirements, humans adapt both the force and stiffness of their limbs. Reflecting this behavior in prostheses may promote a more natural and intuitive control and, consequently, improve prostheses acceptance in everyday life. This pilot study proposes a method to control a prosthetic robot hand and its impedance, and explores the utility of variable stiffness when performing activities of daily living and physical social interactions. The proposed method is capable of a simultaneous and proportional decoding of position and stiffness intentions from two surface electro-myographic sensors placed over a pair of antagonistic muscles. The feasibility of our approach is validated and compared to existing control modalities in a preliminary study involving one prosthesis user. The algorithm is implemented in a soft under-actuated prosthetic hand (SoftHand Pro). Then, we evaluate the usability of the proposed approach while executing a variety of tasks. Among these tasks, the user interacts with other 12 able-bodied subjects, whose experiences were also assessed. Several statistically significant aspects from the System Usability Scale indicate user's preference of variable stiffness control over low or high constant stiffness due to its reactivity and adaptability. Feedback reported by able-bodied subjects reveal a general tendency to favor soft interaction, i.e., low stiffness, which is perceived more human-like and comfortable. These combined results suggest the use of variable stiffness as a viable compromise between firm control and safe interaction which is worth investigating further.

Skin Stretch Haptic Feedback to Convey Closure Information in Anthropomorphic, Under-Actuated Upper Limb Soft Prostheses

Restoring hand function in individuals with upper limb loss is a challenging task, made difficult by the complexity of human hands from both a functional and sensory point of view. Users of commercial prostheses, even sophisticated devices, must visually attend to the hand to know its state, since in most cases they are not provided with any direct sensory information. Among the different types of haptic feedback that can be delivered, particularly information on hand opening is likely to reduce the requirement of constant visual attention. In recent years, there has been a trend of using underactuated, compliant multi-fingered hands as upper limb prostheses, in part due to their simplicity and ease of use attributed to low degree-of-freedom (d.o.f.) actuation. The trend toward underactuation encourages the design of one d.o.f. haptic devices to provide intuitive sensory feedback from the prosthesis. However, mapping the closure of a multi-d.o.f. prosthetic hand to a simple and intuitive haptic cue is not a trivial task. In this paper, we explore the use of a one d.o.f. skin stretch haptic device, the rice haptic rocker, to provide intuitive proprioceptive feedback indicating overall hand closure of an underactuated prosthesis. The benefits and challenges of the system are assessed in multi-tasking and reduced vision scenarios for an object-size discrimination task, in an effort to simulate challenges in daily life, and are compared against the haptic resolution of the device using the just noticeable difference. Finally, an evaluation done with a prosthesis user, in the form of a truncated version of the Activities Measure for Upper Limb Amputees (AM-ULA), shows possible benefits of the addition of haptic feedback in tasks with reduced visual attention.

A Physics-based Virtual Reality Environment to Quantify Functional Performance of Upper-limb Prostheses

Usability of upper-limb prostheses remains to be a challenge due to the complexity of hand-object interactions in activities of daily living. Functional evaluation is critical for the optimization of prosthesis performance during device design and parameter tuning phase. Therefore, we implemented a low-cost physics-based virtual reality environment (VRE) capable of simulating wide range of object grasping and manipulation tasks to enable human-in-the-loop optimization. Importantly, our novel VRE can assess user performance quantitatively using movement kinematics and interaction forces. We present a preliminary experiment to validate our VRE. Four able-bodied subjects performed object transfer tasks with a simulated myoelectric one DoF soft-synergy prosthetic hand, while wearing braces to restrain different levels of wrist motion. We found that the task completion time was similar across conditions, however limited wrist pronation led to more shoulder compensatory motion whereas challenging object orientation caused more torso compensatory motion.

Inter-Limb Transfer of Grasp Force Perception With Closed-Loop Hand Prosthesis

Sensory feedback of grasp forces provides important information about physical interactions between the hand and objects, enabling both reactive and anticipatory neural control mechanisms. The numerous studies have shown artificial sensory feedback of various forms improves force control during grasping tasks by prosthetic hand users through a closed-feedback loop. However, little is known about how perceptual information is transferred between an intact limb and a closed-loop prosthetic limb, and the extent to which training inter-limb transfer may improve myoelectric prosthetic control. We addressed these gaps by using a contralateral force-matching task in which able-bodied participants were asked to generate grasp forces with their native hand, and then match it using the contralateral hand or a soft-synergy prosthetic hand worn on the contralateral arm that was coupled with a mechanotactile feedback device. We found that absolute matching error and matching time were greater when using the prosthetic system than the native hand. However, with contralateral specific training, subjects were able to produce similar relative matching error with the prosthetic system and the native hand, especially at the untrained force level. These findings suggest that an association can be established between the perception produced by the prosthetic limb and the contralateral intact limb, and provide novel insights about potential applications to training and design of the closed-loop prosthesis.

Robotic Finger Force Sensor Fabrication and Evaluation Through a Glove

This force-feedback approach compares the effect on the sensing ability through a worn glove of the force application of an i-Limb Ultra robotic hand for several experimental scenarios. A Takktile sensor was integrated into a fabricated fingertip to measure the applied force of the i-Limb Ultra. A controller was then designed using MATLAB/Simulink to manipulate the finger motion of the i-Limb to apply force to an external load cell. Testing was performed to check the force measurements and sensing ability/quality for two cases: hand with no glove and hand with a nitrile glove. Each of these scenarios were tested by applying fingertip force in 3 different modes: open/close with no contact, continuous tapping and constant force.