Tactile Feedback in Closed-Loop Control of Myoelectric Hand Grasping: Conveying Information of Multiple Sensors Simultaneously via a Single Feedback Channel

Encoding force modulation in two electrotactile feedback parameters strengthens sensory integration according to maximum likelihood estimation

Bidirectional human-machine interfaces involve commands from the central nervous system to an external device and feedback characterizing device state. Such feedback may be elicited by electrical stimulation of somatosensory nerves, where a task-relevant variable is encoded in stimulation amplitude or frequency. Recently, concurrent modulation in amplitude and frequency (multimodal encoding) was proposed. We hypothesized that feedback with multimodal encoding may effectively be processed by the central nervous system as two independent inputs encoded in amplitude and frequency, respectively, thereby increasing state estimate quality in accordance with maximum-likelihood estimation. Using an adaptation paradigm, we tested this hypothesis during a grasp force matching task where subjects received electrotactile feedback encoding instantaneous force in amplitude, frequency, or both, in addition to their natural force feedback. The results showed that adaptations in grasp force with multimodal encoding could be accurately predicted as the integration of three independent inputs according to maximum-likelihood estimation: amplitude modulated electrotactile feedback, frequency modulated electrotactile feedback, and natural force feedback (r² = 0.73). These findings show that multimodal electrotactile feedback carries an intrinsic advantage for state estimation accuracy with respect to single-variable modulation and suggest that this scheme should be the preferred strategy for bidirectional human-machine interfaces with electrotactile feedback.

Temporal and spatial characteristics of bone conduction as non-invasive haptic sensory feedback for upper-limb prosthesis

Bone conduction is a promising haptic feedback modality for upper-limb prosthesis users, however, its potential and characteristics as a non-invasive feedback modality have not been thoroughly investigated. This study aimed to establish the temporal and spatial characteristics of non-invasive bone conduction as a sensory feedback interface for upper-limb prostheses. Psychometric human-subject experiments were conducted on three bony landmarks of the elbow, with a vibrotactile transducer affixed to each to provide the stimulus. The study characterized the temporal domain by testing perception threshold and resolution in amplitude and frequency. The spatial domain was evaluated by assessing the ability of subjects to detect the number of simultaneous active stimulation sites. The experiment was conducted with ten able-bodied subjects and compared to two subjects with trans-radial amputation. The psychometric evaluation of the proposed non-invasive bone conduction feedback showed results comparable to invasive methods. The experimental results demonstrated similar amplitude and frequency resolution of the interface for all three stimulation sites for both able-bodied subjects and subjects with trans-radial amputation, highlighting its potential as a non-invasive feedback modality for upper-limb prostheses.

Object stiffness recognition and vibratory feedback without ad-hoc sensing on the Hannes prosthesis: A machine learning approach

Introduction

In recent years, hand prostheses achieved relevant improvements in term of both motor and functional recovery. However, the rate of devices abandonment, also due to their poor embodiment, is still high. The embodiment defines the integration of an external object - in this case a prosthetic device - into the body scheme of an individual. One of the limiting factors causing lack of embodiment is the absence of a direct interaction between user and environment. Many studies focused on the extraction of tactile information via custom electronic skin technologies coupled with dedicated haptic feedback, though increasing the complexity of the prosthetic system. Contrary wise, this paper stems from the authors' preliminary works on multi-body prosthetic hand modeling and the identification of possible intrinsic information to assess object stiffness during interaction.

Methods

Based on these initial findings, this work presents the design, implementation and clinical validation of a novel real-time stiffness detection strategy, without ad-hoc sensing, based on a Non-linear Logistic Regression (NLR) classifier. This exploits the minimum grasp information available from an under-sensorized and under-actuated myoelectric prosthetic hand, Hannes. The NLR algorithm takes as input motor-side current, encoder position, and reference position of the hand and provides as output a classification of the grasped object (no-object, rigid object, and soft object). This information is then transmitted to the user via vibratory feedback to close the loop between user control and prosthesis interaction. This implementation was validated through a user study conducted both on able bodied subjects and amputees.

Results

The classifier achieved excellent performance in terms of F1Score (94.93%). Further, the able-bodied subjects and amputees were able to successfully detect the objects' stiffness with a F1Score of 94.08% and 86.41%, respectively, by using our proposed feedback strategy. This strategy allowed amputees to quickly recognize the objects' stiffness (response time of 2.82 s), indicating high intuitiveness, and it was overall appreciated as demonstrated by the questionnaire. Furthermore, an embodiment improvement was also obtained as highlighted by the proprioceptive drift toward the prosthesis (0.7 cm).

Compound motion decoding based on sEMG consisting of gestures, wrist angles, and strength

This study aimed to highlight the demand for upper limb compound motion decoding to provide a more diversified and flexible operation for the electromyographic hand. In total, 60 compound motions were selected, which were combined with four gestures, five wrist angles, and three strength levels. Both deep learning methods and machine learning classifiers were compared to analyze the decoding performance. For deep learning, three structures and two ways of label encoding were assessed for their training processes and accuracies; for machine learning, 24 classifiers, seven features, and a combination of classifier chains were analyzed. Results show that for this relatively small sample multi-target surface electromyography (sEMG) classification, feature combination (mean absolute value, root mean square, variance, 4th-autoregressive coefficient, wavelength, zero crossings, and slope signal change) with Support Vector Machine (quadric kernel) outstood because of its high accuracy, short training process, less computation cost, and stability (p < 0.05). The decoding result achieved an average test accuracy of 98.42 ± 1.71% with 150 ms sEMG. The average accuracy for separate gestures, wrist angles, and strength levels were 99.35 ± 0.67%, 99.34 ± 0.88%, and 99.04 ± 1.16%. Among all 60 motions, 58 showed a test accuracy greater than 95%, and one part was equal to 100%.

Separability of Input Features and the Resulting Accuracy in Classifying Target Poses for Active Transhumeral Prosthetic Interfaces

In active prostheses, it is desired to achieve target poses for a given family of tasks, for example, in the task of forward reaching using a transhumeral prosthesis with coordinated joint movements. To do so, it is necessary to distinguish these target poses accurately using the input features (e.g. kinematic and sEMG) obtained from the human users. However, the input features have conventionally been selected through human observations and influenced heavily by the availability of sensors in this context, which may not always yield the most relevant information to differentiate the target poses in the given task. In order to better select from a pool of available input features, those most appropriate for a given set of target poses, a measure that correlates well with the resulting classification accuracy is required so that it can inform the interface design process. In this paper, a scatter-matrix based class separability measure is adopted to quantitatively evaluate the separability of the target poses from their corresponding input features. A human experiment was performed on ten able-bodied subjects. Subjects were asked to perform forward-reaching movements with their arms on nine target poses in a virtual reality (VR) platform and the corresponding kinematic information of their arm movement and muscle activities were recorded. The accuracy of the prosthetic interface in determining the intended target poses of the human user during forward reaching is evaluated for different combinations of input features, selected from the kinematic and sEMG sensors worn by the users. The results demonstrate that employing input features that yield a high separability measure between target poses results in a high accuracy in identifying the intended target poses in the execution of the task.

On the Efficiency of Haptic Based Object Identification: Determining Where to Grasp to Get the Most Distinguishing Information

Haptic perception is one of the key modalities in obtaining physical information of objects and in object identification. Most existing literature focused on improving the accuracy of identification algorithms with less attention paid to the efficiency. This work aims to investigate the efficiency of haptic object identification to reduce the number of grasps required to correctly identify an object out of a given object set. Thus, in a case where multiple grasps are required to characterise an object, the proposed algorithm seeks to determine where the next grasp should be on the object to obtain the most amount of distinguishing information. As such, the paper proposes the construction of the object description that preserves the association of the spatial information and the haptic information on the object. A clustering technique is employed both to construct the description of the object in a data set and for the identification process. An information gain (IG) based method is then employed to determine which pose would yield the most distinguishing information among the remaining possible candidates in the object set to improve the efficiency of the identification process. This proposed algorithm is validated experimentally. A Reflex TakkTile robotic hand with integrated joint displacement and tactile sensors is used to perform both the data collection for the dataset and the object identification procedure. The proposed IG approach was found to require a significantly lower number of grasps to identify the objects compared to a baseline approach where the decision was made by random choice of grasps.

The Need to Work Arm in Arm: Calling for Collaboration in Delivering Neuroprosthetic Limb Replacements

Over the last few decades there has been a push to enhance the use of advanced prosthetics within the fields of biomedical engineering, neuroscience, and surgery. Through the development of peripheral neural interfaces and invasive electrodes, an individual's own nervous system can be used to control a prosthesis. With novel improvements in neural recording and signal decoding, this intimate communication has paved the way for bidirectional and intuitive control of prostheses. While various collaborations between engineers and surgeons have led to considerable success with motor control and pain management, it has been significantly more challenging to restore sensation. Many of the existing peripheral neural interfaces have demonstrated success in one of these modalities; however, none are currently able to fully restore limb function. Though this is in part due to the complexity of the human somatosensory system and stability of bioelectronics, the fragmentary and as-yet uncoordinated nature of the neuroprosthetic industry further complicates this advancement. In this review, we provide a comprehensive overview of the current field of neuroprosthetics and explore potential strategies to address its unique challenges. These include exploration of electrodes, surgical techniques, control methods, and prosthetic technology. Additionally, we propose a new approach to optimizing prosthetic limb function and facilitating clinical application by capitalizing on available resources. It is incumbent upon academia and industry to encourage collaboration and utilization of different peripheral neural interfaces in combination with each other to create versatile limbs that not only improve function but quality of life. Despite the rapidly evolving technology, if the field continues to work in divided "silos," we will delay achieving the critical, valuable outcome: creating a prosthetic limb that is right for the patient and positively affects their life.