1
|
Gholinezhad S, Farina D, Dosen S, Dideriksen J. Encoding force modulation in two electrotactile feedback parameters strengthens sensory integration according to maximum likelihood estimation. Sci Rep 2023; 13:12461. [PMID: 37528160 PMCID: PMC10393971 DOI: 10.1038/s41598-023-38753-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/06/2023] [Accepted: 07/14/2023] [Indexed: 08/03/2023] Open
Abstract
Bidirectional human-machine interfaces involve commands from the central nervous system to an external device and feedback characterizing device state. Such feedback may be elicited by electrical stimulation of somatosensory nerves, where a task-relevant variable is encoded in stimulation amplitude or frequency. Recently, concurrent modulation in amplitude and frequency (multimodal encoding) was proposed. We hypothesized that feedback with multimodal encoding may effectively be processed by the central nervous system as two independent inputs encoded in amplitude and frequency, respectively, thereby increasing state estimate quality in accordance with maximum-likelihood estimation. Using an adaptation paradigm, we tested this hypothesis during a grasp force matching task where subjects received electrotactile feedback encoding instantaneous force in amplitude, frequency, or both, in addition to their natural force feedback. The results showed that adaptations in grasp force with multimodal encoding could be accurately predicted as the integration of three independent inputs according to maximum-likelihood estimation: amplitude modulated electrotactile feedback, frequency modulated electrotactile feedback, and natural force feedback (r2 = 0.73). These findings show that multimodal electrotactile feedback carries an intrinsic advantage for state estimation accuracy with respect to single-variable modulation and suggest that this scheme should be the preferred strategy for bidirectional human-machine interfaces with electrotactile feedback.
Collapse
Affiliation(s)
- Shima Gholinezhad
- Department of Health Science and Technology, Aalborg University, Aalborg, Denmark
| | - Dario Farina
- Department of Bioengineering, Faculty of Engineering, Imperial College London, London, SW7 2AZ, UK
| | - Strahinja Dosen
- Department of Health Science and Technology, Aalborg University, Aalborg, Denmark
| | - Jakob Dideriksen
- Department of Health Science and Technology, Aalborg University, Aalborg, Denmark.
| |
Collapse
|
2
|
Mayer RM, Mohammadi A, Tan Y, Alici G, Choong P, Oetomo D. Temporal and spatial characteristics of bone conduction as non-invasive haptic sensory feedback for upper-limb prosthesis. Front Neurosci 2023; 17:1113009. [PMID: 37056306 PMCID: PMC10086128 DOI: 10.3389/fnins.2023.1113009] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/30/2022] [Accepted: 02/24/2023] [Indexed: 03/30/2023] Open
Abstract
Bone conduction is a promising haptic feedback modality for upper-limb prosthesis users, however, its potential and characteristics as a non-invasive feedback modality have not been thoroughly investigated. This study aimed to establish the temporal and spatial characteristics of non-invasive bone conduction as a sensory feedback interface for upper-limb prostheses. Psychometric human-subject experiments were conducted on three bony landmarks of the elbow, with a vibrotactile transducer affixed to each to provide the stimulus. The study characterized the temporal domain by testing perception threshold and resolution in amplitude and frequency. The spatial domain was evaluated by assessing the ability of subjects to detect the number of simultaneous active stimulation sites. The experiment was conducted with ten able-bodied subjects and compared to two subjects with trans-radial amputation. The psychometric evaluation of the proposed non-invasive bone conduction feedback showed results comparable to invasive methods. The experimental results demonstrated similar amplitude and frequency resolution of the interface for all three stimulation sites for both able-bodied subjects and subjects with trans-radial amputation, highlighting its potential as a non-invasive feedback modality for upper-limb prostheses.
Collapse
Affiliation(s)
- Raphael M. Mayer
- Department of Mechanical Engineering, The University of Melbourne, Parkville, VIC, Australia
- *Correspondence: Raphael M. Mayer
| | - Alireza Mohammadi
- Department of Mechanical Engineering, The University of Melbourne, Parkville, VIC, Australia
| | - Ying Tan
- Department of Mechanical Engineering, The University of Melbourne, Parkville, VIC, Australia
| | - Gursel Alici
- School of Mechanical, Materials, Mechatronic and Biomedical Engineering, University of Wollongong, Wollongong, NSW, Australia
- ARC Centre of Excellence for Electromaterials Science, Wollongong, NSW, Australia
| | - Peter Choong
- ARC Centre of Excellence for Electromaterials Science, Wollongong, NSW, Australia
- Department of Surgery, St Vincent's Hospital, The University of Melbourne, Parkville, VIC, Australia
| | - Denny Oetomo
- Department of Mechanical Engineering, The University of Melbourne, Parkville, VIC, Australia
- ARC Centre of Excellence for Electromaterials Science, Wollongong, NSW, Australia
| |
Collapse
|
3
|
Bruni G, Marinelli A, Bucchieri A, Boccardo N, Caserta G, Di Domenico D, Barresi G, Florio A, Canepa M, Tessari F, Laffranchi M, De Michieli L. Object stiffness recognition and vibratory feedback without ad-hoc sensing on the Hannes prosthesis: A machine learning approach. Front Neurosci 2023; 17:1078846. [PMID: 36875662 PMCID: PMC9978002 DOI: 10.3389/fnins.2023.1078846] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/24/2022] [Accepted: 01/24/2023] [Indexed: 02/18/2023] Open
Abstract
Introduction In recent years, hand prostheses achieved relevant improvements in term of both motor and functional recovery. However, the rate of devices abandonment, also due to their poor embodiment, is still high. The embodiment defines the integration of an external object - in this case a prosthetic device - into the body scheme of an individual. One of the limiting factors causing lack of embodiment is the absence of a direct interaction between user and environment. Many studies focused on the extraction of tactile information via custom electronic skin technologies coupled with dedicated haptic feedback, though increasing the complexity of the prosthetic system. Contrary wise, this paper stems from the authors' preliminary works on multi-body prosthetic hand modeling and the identification of possible intrinsic information to assess object stiffness during interaction. Methods Based on these initial findings, this work presents the design, implementation and clinical validation of a novel real-time stiffness detection strategy, without ad-hoc sensing, based on a Non-linear Logistic Regression (NLR) classifier. This exploits the minimum grasp information available from an under-sensorized and under-actuated myoelectric prosthetic hand, Hannes. The NLR algorithm takes as input motor-side current, encoder position, and reference position of the hand and provides as output a classification of the grasped object (no-object, rigid object, and soft object). This information is then transmitted to the user via vibratory feedback to close the loop between user control and prosthesis interaction. This implementation was validated through a user study conducted both on able bodied subjects and amputees. Results The classifier achieved excellent performance in terms of F1Score (94.93%). Further, the able-bodied subjects and amputees were able to successfully detect the objects' stiffness with a F1Score of 94.08% and 86.41%, respectively, by using our proposed feedback strategy. This strategy allowed amputees to quickly recognize the objects' stiffness (response time of 2.82 s), indicating high intuitiveness, and it was overall appreciated as demonstrated by the questionnaire. Furthermore, an embodiment improvement was also obtained as highlighted by the proprioceptive drift toward the prosthesis (0.7 cm).
Collapse
Affiliation(s)
- Giulia Bruni
- Rehab Technologies, Istituto Italiano di Tecnologia, Genoa, Italy
| | - Andrea Marinelli
- Rehab Technologies, Istituto Italiano di Tecnologia, Genoa, Italy.,Department of Informatics, Bioengineering, Robotics System Engineering (DIBRIS), University of Genova, Genoa, Italy
| | - Anna Bucchieri
- Rehab Technologies, Istituto Italiano di Tecnologia, Genoa, Italy.,Department of Electronics, Information and Bioengineering (NearLab), Politecnico of Milan, Milan, Italy
| | - Nicolò Boccardo
- Rehab Technologies, Istituto Italiano di Tecnologia, Genoa, Italy.,The Open University Affiliated Research Centre at Istituto Italiano di Tecnologia (ARC@IIT), Genoa, Italy
| | - Giulia Caserta
- Rehab Technologies, Istituto Italiano di Tecnologia, Genoa, Italy
| | - Dario Di Domenico
- Rehab Technologies, Istituto Italiano di Tecnologia, Genoa, Italy.,Department of Electronics and Telecommunications, Politecnico of Torino, Turin, Italy
| | - Giacinto Barresi
- Rehab Technologies, Istituto Italiano di Tecnologia, Genoa, Italy
| | - Astrid Florio
- Rehab Technologies, Istituto Italiano di Tecnologia, Genoa, Italy
| | - Michele Canepa
- Rehab Technologies, Istituto Italiano di Tecnologia, Genoa, Italy.,The Open University Affiliated Research Centre at Istituto Italiano di Tecnologia (ARC@IIT), Genoa, Italy
| | - Federico Tessari
- Newman Laboratory, Massachusetts Institute of Technology, Boston, MA, United States
| | | | | |
Collapse
|
4
|
Zhang X, Lu Z, Fan C, Wang Y, Zhang T, Li H, Tao Q. Compound motion decoding based on sEMG consisting of gestures, wrist angles, and strength. Front Neurorobot 2022; 16:979949. [DOI: 10.3389/fnbot.2022.979949] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/28/2022] [Accepted: 10/03/2022] [Indexed: 11/11/2022] Open
Abstract
This study aimed to highlight the demand for upper limb compound motion decoding to provide a more diversified and flexible operation for the electromyographic hand. In total, 60 compound motions were selected, which were combined with four gestures, five wrist angles, and three strength levels. Both deep learning methods and machine learning classifiers were compared to analyze the decoding performance. For deep learning, three structures and two ways of label encoding were assessed for their training processes and accuracies; for machine learning, 24 classifiers, seven features, and a combination of classifier chains were analyzed. Results show that for this relatively small sample multi-target surface electromyography (sEMG) classification, feature combination (mean absolute value, root mean square, variance, 4th-autoregressive coefficient, wavelength, zero crossings, and slope signal change) with Support Vector Machine (quadric kernel) outstood because of its high accuracy, short training process, less computation cost, and stability (p < 0.05). The decoding result achieved an average test accuracy of 98.42 ± 1.71% with 150 ms sEMG. The average accuracy for separate gestures, wrist angles, and strength levels were 99.35 ± 0.67%, 99.34 ± 0.88%, and 99.04 ± 1.16%. Among all 60 motions, 58 showed a test accuracy greater than 95%, and one part was equal to 100%.
Collapse
|
5
|
Yu T, Garcia-Rosas R, Mohammadi A, Tan Y, Choong P, Oetomo D. Separability of Input Features and the Resulting Accuracy in Classifying Target Poses for Active Transhumeral Prosthetic Interfaces. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2021; 2021:4615-4618. [PMID: 34892242 DOI: 10.1109/embc46164.2021.9630041] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
In active prostheses, it is desired to achieve target poses for a given family of tasks, for example, in the task of forward reaching using a transhumeral prosthesis with coordinated joint movements. To do so, it is necessary to distinguish these target poses accurately using the input features (e.g. kinematic and sEMG) obtained from the human users. However, the input features have conventionally been selected through human observations and influenced heavily by the availability of sensors in this context, which may not always yield the most relevant information to differentiate the target poses in the given task. In order to better select from a pool of available input features, those most appropriate for a given set of target poses, a measure that correlates well with the resulting classification accuracy is required so that it can inform the interface design process. In this paper, a scatter-matrix based class separability measure is adopted to quantitatively evaluate the separability of the target poses from their corresponding input features. A human experiment was performed on ten able-bodied subjects. Subjects were asked to perform forward-reaching movements with their arms on nine target poses in a virtual reality (VR) platform and the corresponding kinematic information of their arm movement and muscle activities were recorded. The accuracy of the prosthetic interface in determining the intended target poses of the human user during forward reaching is evaluated for different combinations of input features, selected from the kinematic and sEMG sensors worn by the users. The results demonstrate that employing input features that yield a high separability measure between target poses results in a high accuracy in identifying the intended target poses in the execution of the task.
Collapse
|
6
|
Xia Y, Mohammadi A, Tan Y, Chen B, Choong P, Oetomo D. On the Efficiency of Haptic Based Object Identification: Determining Where to Grasp to Get the Most Distinguishing Information. Front Robot AI 2021; 8:686490. [PMID: 34395537 PMCID: PMC8358325 DOI: 10.3389/frobt.2021.686490] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/26/2021] [Accepted: 07/08/2021] [Indexed: 01/07/2023] Open
Abstract
Haptic perception is one of the key modalities in obtaining physical information of objects and in object identification. Most existing literature focused on improving the accuracy of identification algorithms with less attention paid to the efficiency. This work aims to investigate the efficiency of haptic object identification to reduce the number of grasps required to correctly identify an object out of a given object set. Thus, in a case where multiple grasps are required to characterise an object, the proposed algorithm seeks to determine where the next grasp should be on the object to obtain the most amount of distinguishing information. As such, the paper proposes the construction of the object description that preserves the association of the spatial information and the haptic information on the object. A clustering technique is employed both to construct the description of the object in a data set and for the identification process. An information gain (IG) based method is then employed to determine which pose would yield the most distinguishing information among the remaining possible candidates in the object set to improve the efficiency of the identification process. This proposed algorithm is validated experimentally. A Reflex TakkTile robotic hand with integrated joint displacement and tactile sensors is used to perform both the data collection for the dataset and the object identification procedure. The proposed IG approach was found to require a significantly lower number of grasps to identify the objects compared to a baseline approach where the decision was made by random choice of grasps.
Collapse
Affiliation(s)
- Yu Xia
- Human Robotics Laboratory, Department of Mechanical Engineering, The University of Melbourne, Parkville, VIC, Australia
| | - Alireza Mohammadi
- Human Robotics Laboratory, Department of Mechanical Engineering, The University of Melbourne, Parkville, VIC, Australia
| | - Ying Tan
- Human Robotics Laboratory, Department of Mechanical Engineering, The University of Melbourne, Parkville, VIC, Australia
| | - Bernard Chen
- Human Robotics Laboratory, Department of Mechanical Engineering, The University of Melbourne, Parkville, VIC, Australia
| | - Peter Choong
- Department of Surgery, St. Vincent's Hospital, The University of Melbourne, Parkville, VIC, Australia.,Aikenhead Centre for Medical Discovery (ACMD), St. Vincent's Hospital, Parkville, VIC, Australia
| | - Denny Oetomo
- Human Robotics Laboratory, Department of Mechanical Engineering, The University of Melbourne, Parkville, VIC, Australia.,Aikenhead Centre for Medical Discovery (ACMD), St. Vincent's Hospital, Parkville, VIC, Australia
| |
Collapse
|
7
|
Karczewski AM, Dingle AM, Poore SO. The Need to Work Arm in Arm: Calling for Collaboration in Delivering Neuroprosthetic Limb Replacements. Front Neurorobot 2021; 15:711028. [PMID: 34366820 PMCID: PMC8334559 DOI: 10.3389/fnbot.2021.711028] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/17/2021] [Accepted: 06/22/2021] [Indexed: 11/21/2022] Open
Abstract
Over the last few decades there has been a push to enhance the use of advanced prosthetics within the fields of biomedical engineering, neuroscience, and surgery. Through the development of peripheral neural interfaces and invasive electrodes, an individual's own nervous system can be used to control a prosthesis. With novel improvements in neural recording and signal decoding, this intimate communication has paved the way for bidirectional and intuitive control of prostheses. While various collaborations between engineers and surgeons have led to considerable success with motor control and pain management, it has been significantly more challenging to restore sensation. Many of the existing peripheral neural interfaces have demonstrated success in one of these modalities; however, none are currently able to fully restore limb function. Though this is in part due to the complexity of the human somatosensory system and stability of bioelectronics, the fragmentary and as-yet uncoordinated nature of the neuroprosthetic industry further complicates this advancement. In this review, we provide a comprehensive overview of the current field of neuroprosthetics and explore potential strategies to address its unique challenges. These include exploration of electrodes, surgical techniques, control methods, and prosthetic technology. Additionally, we propose a new approach to optimizing prosthetic limb function and facilitating clinical application by capitalizing on available resources. It is incumbent upon academia and industry to encourage collaboration and utilization of different peripheral neural interfaces in combination with each other to create versatile limbs that not only improve function but quality of life. Despite the rapidly evolving technology, if the field continues to work in divided "silos," we will delay achieving the critical, valuable outcome: creating a prosthetic limb that is right for the patient and positively affects their life.
Collapse
Affiliation(s)
| | - Aaron M. Dingle
- Division of Plastic Surgery, Department of Surgery, University of Wisconsin–Madison, Madison, WI, United States
| | | |
Collapse
|