1
|
Guo Y, Xu W, Ben-Tzvi P. Vision-Based Human-Machine Interface for an Assistive Robotic Exoskeleton Glove. RESEARCH SQUARE 2023:rs.3.rs-3300722. [PMID: 37693405 PMCID: PMC10491327 DOI: 10.21203/rs.3.rs-3300722/v1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 09/12/2023]
Abstract
This paper presents a vision-based Human-Machine Interface (HMI) for an assistive exoskeleton glove, designed to incorporate force planning capabilities. While Electroencephalogram (EEG) and Electromyography (EMG)-based HMIs allow direct grasp force planning via user signals, voice and vision-based HMIs face limitations. In particular, two primary force planning methods encounter issues in these HMIs. First, traditional force optimization struggles with unfamiliar objects due to lack of object information. Second, the slip-grasp method faces a high failure rate due to inadequate initial grasp force. To address these challenges, this paper introduces a vision-based HMI to estimate the initial grasp forces of the target object. The initial grasp force estimation is performed based on the size and surface material of the target object. The experimental results demonstrate a grasp success rate of 87. 5%, marking significant improvements over the slip-grasp method (71.9%).
Collapse
Affiliation(s)
- Yunfei Guo
- Electrical and Computer Engineering Department, Virginia Tech, Blacksburg, VA, USA
| | - Wenda Xu
- Mechanical Engineering Department, Virginia Tech, Blacksburg, VA, USA
| | - Pinhas Ben-Tzvi
- Electrical and Computer Engineering Department, Virginia Tech, Blacksburg, VA, USA
- Mechanical Engineering Department, Virginia Tech, Blacksburg, VA, USA
| |
Collapse
|
2
|
Lu J, Guo K, Yang H. Dynamic Analysis and Experimental Study of Lasso Transmission for Hand Rehabilitation Robot. MICROMACHINES 2023; 14:858. [PMCID: PMC10146587 DOI: 10.3390/mi14040858] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/07/2023] [Revised: 04/06/2023] [Accepted: 04/13/2023] [Indexed: 06/01/2023]
Abstract
Lasso transmission is a method for realizing long-distance flexible transmission and lightweight robots. However, there are transmission characteristic losses of velocity, force, and displacement during the motion of lasso transmission. Therefore, the analysis of transmission characteristic losses of lasso transmission has become the focus of research. For this study, at first, we developed a new flexible hand rehabilitation robot with a lasso transmission method. Second, the theoretical analysis and simulation analysis of the dynamics of the lasso transmission in the flexible hand rehabilitation robot were carried out to calculate the force, velocity, and displacement losses of the lasso transmission. Finally, the mechanism and transmission models were established for experimental studies to measure the effects of different curvatures and speeds on the lasso transmission torque. The experimental data and image analysis results show torque loss in the process of lasso transmission and an increase in torque loss with the increase in the lasso curvature radius and transmission speed. The study of the lasso transmission characteristics is important for the design and control of hand functional rehabilitation robots, providing an important reference for the design of flexible rehabilitation robots and also guiding the research on the lasso regarding the compensation method for transmission losses.
Collapse
Affiliation(s)
- Jingxin Lu
- Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou 215163, China
- School of Mechanical and Electrical Engineering, Changchun University of Science and Technology, Changchun 130022, China
| | - Kai Guo
- Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou 215163, China
- School of Biomedical Engineering (Suzhou), Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei 230026, China
| | - Hongbo Yang
- Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou 215163, China
- School of Mechanical and Electrical Engineering, Changchun University of Science and Technology, Changchun 130022, China
- School of Biomedical Engineering (Suzhou), Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei 230026, China
| |
Collapse
|
3
|
Xu W, Guo Y, Bravo C, Ben-Tzvi P. Design, Control, and Experimental Evaluation of A Novel Robotic Glove System for Patients with Brachial Plexus Injuries. IEEE T ROBOT 2023; 39:1637-1652. [PMID: 37035529 PMCID: PMC10079272 DOI: 10.1109/tro.2022.3220973] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/25/2022]
Abstract
This paper presents the development of an exoskeleton glove system for people who suffer from brachial plexus injuries, aiming to assist their lost grasping functionality. The robotic system consists of a portable glove system and an embedded controller. The glove system consists of Linear Series Elastic Actuators (LSEA), Rotary Series Elastic Actuators (RSEA), and optimized finger linkages to provide imitated human motion to each finger and a coupled motion of the hand. The design principles and optimization strategies were investigated to balance functionality, portability, and stability. The model-based force control strategy compensated with a backlash model and model-free force control strategy are presented and compared. Results show that our proposed model-free control method achieves the goal of accurate force control. Finally, experiments were conducted with the prototype of the developed integrated exoskeleton glove system. Results from 3 subjects with 150 trials show that our proposed exoskeleton glove system has the potential to be used as a rehabilitation device for patients.
Collapse
Affiliation(s)
- Wenda Xu
- Mechanical Engineering department in Virginia Tech
| | - Yunfei Guo
- Electrical and Computer Engineering department in Virginia Tech
| | - Cesar Bravo
- Carilion Clinic Institute of Orthopaedics and Neurosciences, Virginia Tech Carilion School of Medicine
| | - Pinhas Ben-Tzvi
- Electrical and Mechanical Engineering departments in Virginia Tech
| |
Collapse
|
4
|
Xu W, Guo Y, Bravo C, Ben-Tzvi P. Development and Experimental Evaluation of a Novel Portable Haptic Robotic Exoskeleton Glove System for Patients with Brachial Plexus Injuries. PROCEEDINGS OF THE ... IEEE/RSJ INTERNATIONAL CONFERENCE ON INTELLIGENT ROBOTS AND SYSTEMS. IEEE/RSJ INTERNATIONAL CONFERENCE ON INTELLIGENT ROBOTS AND SYSTEMS 2022; 2022:11115-11120. [PMID: 37303849 PMCID: PMC10256117 DOI: 10.1109/iros47612.2022.9981468] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
This paper presents the development and experimental evaluation of a portable haptic exoskeleton glove system designed for people who suffer from brachial plexus injuries to restore their lost grasping functionality. The proposed glove system involves force perception, linkage-driven finger mechanism, and personalized voice control to achieve various grasping functionality requirements. The fully integrated system provides our wearable device with lightweight, portable, and comfortable characterization for grasping objects used in daily activities. Rigid articulated linkages powered by Series Elastic Actuators (SEAs) with slip detection on the fingertips provide stable and robust grasp for multiple objects. The passive abduction-adduction motion of each finger is also considered to provide better grasping flexibility for the user. The continuous voice control with bio-authentication also provides a hands-free user interface. The experiments with different objects verify the functionalities and capabilities of the proposed exoskeleton glove system in grasping objects with various shapes and weights used in activities of daily living (ADLs).
Collapse
Affiliation(s)
- Wenda Xu
- Department of Mechanical Engineering, Virginia Tech, Blacksburg, VA, 24061 USA
| | - Yunfei Guo
- Department of Electrical and Computer Engineering, Virginia Tech, Blacksburg, VA 24061 USA
| | - Cesar Bravo
- Carilion Clinic Institute of Orthopaedics and Neurosciences, Virginia Tech Carilion School of Medicine, Roanoke, VA 24016 USA
| | - Pinhas Ben-Tzvi
- Departments of Mechanical Engineering and Electrical and Computer Engineering, Virginia Tech, Blacksburg, VA, 24061 USA
| |
Collapse
|
5
|
Guo Y, Xu W, Pradhan S, Bravo C, Ben-Tzvi P. Personalized Voice Activated Grasping System for a Robotic Exoskeleton Glove ★. MECHATRONICS : THE SCIENCE OF INTELLIGENT MACHINES 2022; 83:102745. [PMID: 35241876 PMCID: PMC8887884 DOI: 10.1016/j.mechatronics.2022.102745] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/03/2023]
Abstract
This paper proposes a novel human machine interface (HMI) and electronics system design to control a rehabilitation robotic exoskeleton glove. Such system can be activated with biometric authentication using the user's voice, take voice commands as input, recognize the command and perform biometric authentication in real-time with limited computing power, and execute the command on the exoskeleton. The electronics design is a stand-alone plug-and-play modulated design independent of the exoskeleton design. This personalized voice activated grasping system achieves better wearability, lower latency, and improved security than any existing exoskeleton glove control system.
Collapse
Affiliation(s)
- Yunfei Guo
- Department of Electric and Computer Engineering, Virginia Tech
| | - Wenda Xu
- Department of Mechanical Engineering, Virginia Tech
| | | | - Cesar Bravo
- Carilion Clinic Institute of Orthopaedics and Neurosciences, Virginia Tech Carilion School of Medicine
| | - Pinhas Ben-Tzvi
- Department of Electric and Computer Engineering, Virginia Tech
- Department of Mechanical Engineering, Virginia Tech
| |
Collapse
|
6
|
The Effect of the Degree of Freedom and Weight of the Hand Exoskeleton on Joint Mobility Function. ROBOTICS 2022. [DOI: 10.3390/robotics11020053] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022] Open
Abstract
This study aims to investigate the effects of the degree of freedom (DOF) and weight of the hand exoskeleton (HE) on hand joint mobility function (ease of movement, movement range) in fine hand use activities. A three-digit passive HE prototype was built to fit each of the 12 participants. Two DOF setups (three DOF, two DOF), two digits’ weight levels (70 g, 140 g), and barehand conditions were tested. A productivity task (performed with Standardized-Nine Hole Peg Test) and motion tasks, both performing the tip pinch and tripod pinch, were conducted to measure the task completion time and the range of motion (ROM) of the digit joints, respectively, using a motion capture system. The perceived ease rating was also measured. The results showed that DOF reduction and weight addition caused a significant task completion time increase and rating drop (p < 0.05). Meanwhile, the DOF reduction increased the ROM reduction of the proximal interphalangeal joints; however, the weight addition caused a correction of the ROM reduction of several joints (p < 0.05) at the tripod pinch. In conclusion, wearing an HE reduces hand joint mobility, especially in lower DOF. However, a certain weight addition may improve joint mobility in terms of the fingers’ movement range.
Collapse
|
7
|
Li K, Li Z, Zeng H, Wei N. Control of Newly-Designed Wearable Robotic Hand Exoskeleton Based on Surface Electromyographic Signals. Front Neurorobot 2021; 15:711047. [PMID: 34603003 PMCID: PMC8480391 DOI: 10.3389/fnbot.2021.711047] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/17/2021] [Accepted: 08/11/2021] [Indexed: 11/23/2022] Open
Abstract
The human hand plays a role in a variety of daily activities. This intricate instrument is vulnerable to trauma or neuromuscular disorders. Wearable robotic exoskeletons are an advanced technology with the potential to remarkably promote the recovery of hand function. However, the still face persistent challenges in mechanical and functional integration, with real-time control of the multiactuators in accordance with the motion intentions of the user being a particular sticking point. In this study, we demonstrated a newly-designed wearable robotic hand exoskeleton with multijoints, more degrees of freedom (DOFs), and a larger range of motion (ROM). The exoskeleton hand comprises six linear actuators (two for the thumb and the other four for the fingers) and can realize both independent movements of each digit and coordinative movement involving multiple fingers for grasp and pinch. The kinematic parameters of the hand exoskeleton were analyzed by a motion capture system. The exoskeleton showed higher ROM of the proximal interphalangeal and distal interphalangeal joints compared with the other exoskeletons. Five classifiers including support vector machine (SVM), K-near neighbor (KNN), decision tree (DT), multilayer perceptron (MLP), and multichannel convolutional neural networks (multichannel CNN) were compared for the offline classification. The SVM and KNN had a higher accuracy than the others, reaching up to 99%. For the online classification, three out of the five subjects showed an accuracy of about 80%, and one subject showed an accuracy over 90%. These results suggest that the new wearable exoskeleton could facilitate hand rehabilitation for a larger ROM and higher dexterity and could be controlled according to the motion intention of the subjects.
Collapse
Affiliation(s)
- Ke Li
- Laboratory of Rehabilitation Engineering, Research Center of Intelligent Medical Engineering, School of Control Science and Engineering, Shandong University, Jinan, China
| | - Zhengzhen Li
- Laboratory of Rehabilitation Engineering, Research Center of Intelligent Medical Engineering, School of Control Science and Engineering, Shandong University, Jinan, China
| | - Haibin Zeng
- Department of Radiotherapy, Suzhou Dushu Lake Hospital, Suzhou, China
| | - Na Wei
- Department of Geriatrics, Qilu Hospital, Shandong University, Jinan, China
| |
Collapse
|
8
|
Guo Y, Xu W, Pradhan S, Bravo C, Ben-Tzvi P. Data Driven Calibration and Control of Compact Lightweight Series Elastic Actuators for Robotic Exoskeleton Gloves. IEEE SENSORS JOURNAL 2021; 21:21120-21130. [PMID: 35422676 PMCID: PMC9004480 DOI: 10.1109/jsen.2021.3101143] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
The working principle of a SEA is based on using an elastic material connected serially to the mechanical power source to simulate the dynamic behavior of a human muscle. Due to weight and size limitations of a wearable robotic exoskeleton, the hardware design of the SEA is limited. Compact and lightweight SEAs usually have noisy signal output, and can easily be deformed. This paper uses a compact lightweight SEA designed for exoskeleton gloves to demonstrate immeasurable strain and friction force which can cause an average of 34.31% and maximum of 44.7% difference in force measurement on such SEAs. This paper proposes two data driven machine learning methods to accurately calibrate and control SEAs. The multi-layer perception (MLP) method can reduce the average force measurement error to 10.18% and maximum error to 29.13%. The surface fitting method (SF) method can reduce the average force measurement error to 8.06% and maximum error to 35.72%. In control experiments, the weighted MLP method achieves an average of 0.21N force control difference, and the SF method achieves an average of 0.29N force control difference on the finger tips of the exoskeleton glove.
Collapse
Affiliation(s)
- Yunfei Guo
- Electrical and Computer Engineering department in Virginia Tech
| | - Wenda Xu
- Mechanical Engineering department in Virginia Tech
| | | | - Cesar Bravo
- Carilion Clinic Institute of Orthopaedics and Neurosciences, Virginia Tech Carilion School of Medicine
| | - Pinhas Ben-Tzvi
- Electrical and Mechanical Engineering departments in Virginia Tech
| |
Collapse
|
9
|
Ceccarelli M, Morales-Cruz C. A prototype characterization of ExoFinger, a finger exoskeleton. INT J ADV ROBOT SYST 2021. [DOI: 10.1177/17298814211024880] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022] Open
Abstract
This article presents an experimental characterization of ExoFinger, a finger exoskeleton for finger motion assistance. The exoskeletal device is analyzed in experimental lab activities that have been conducted with different users to characterize the operation performance and to demonstrate the adaptability of the proposed device. The behavior of this device is characterized in detail using sensors to measure finger motion and power consumption. Sensor measures also demonstrate the given motion assistance performance in terms of an electrical finger response and finger temperature by resulting in an efficient solution with a large motion range of a finger in assistance of recovering finger motion.
Collapse
Affiliation(s)
- Marco Ceccarelli
- LARM2: Laboratory of Robot Mechatronics, Department of Industrial Engnineering, University of Rome Tor Vergata, Roma, Italy
| | - Cuauhtemoc Morales-Cruz
- Centro de Innovacion y desarrollo tecnologico en Computo, Instituto Politecnico Nacional, Mexico City, Mexico
| |
Collapse
|
10
|
Dynamic Modeling and Control of Antagonistic Variable Stiffness Joint Actuator. ACTUATORS 2021. [DOI: 10.3390/act10060116] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
This study aims to develop a novel decoupling method for the independent control of the position and stiffness of a variable stiffness joint actuator (VSJA), which has been proven to be able to vary its stiffness in a larger range than other variable stiffness actuators. Using static analysis and the Jacobian matrix, we obtained the model of the stiffness of the robot joint actuator and dynamics. Based on the hybrid dynamic model of position and stiffness, it is possible to compensate for the torque of the variable stiffness joint actuator (VSJA) to enhance position control. Finally, after describing the actuator prototype, the established compliance control method is verified using simulation and experimental analysis.
Collapse
|
11
|
Guo Y, Xu W, Pradhan S, Bravo C, Ben-Tzvi P. INTEGRATED AND CONFIGURABLE VOICE ACTIVATION AND SPEAKER VERIFICATION SYSTEM FOR A ROBOTIC EXOSKELETON GLOVE. PROCEEDINGS OF THE ... ASME DESIGN ENGINEERING TECHNICAL CONFERENCES. ASME DESIGN ENGINEERING TECHNICAL CONFERENCES 2020; 10:V010T10A043. [PMID: 36478444 PMCID: PMC9726174 DOI: 10.1115/detc2020-22365] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/14/2022]
Abstract
Efficient human-machine interface (HMI) for exoskeletons remains an active research topic, where sample methods have been proposed including using computer vision, EEG (electroencephalogram), and voice recognition. However, some of these methods lack sufficient accuracy, security, and portability. This paper proposes a HMI referred as integrated trigger-word configurable voice activation and speaker verification system (CVASV). The CVASV system is designed for embedded systems with limited computing power that can be applied to any exoskeleton platform. The CVASV system consists of two main sections, including an API based voice activation section and a deep learning based text-independent voice verification section. These two sections are combined into a system that allows the user to configure the activation trigger-word and verify the user's command in real-time.
Collapse
Affiliation(s)
- Yunfei Guo
- Robotics and Mechatronics Lab, Department of Electrical and Computer Engineering, Virginia Tech, Blacksburg, Virginia 24061
| | - Wenda Xu
- Robotics and Mechatronics Lab, Department of Mechanical Engineering, Virginia Tech, Blacksburg, Virginia 24061
| | - Sarthak Pradhan
- Robotics and Mechatronics Lab, Department of Mechanical Engineering, Virginia Tech, Blacksburg, Virginia 24061
| | - Cesar Bravo
- Carilion Clinic Institute of Orthopaedics and Neurosciences, Virginia Tech Carilion School of Medicine, Roanoke, VA 24014
| | | |
Collapse
|
12
|
Xu W, Pradhan S, Guo Y, Bravo C, Ben-Tzvi P. A NOVEL DESIGN OF A ROBOTIC GLOVE SYSTEM FOR PATIENTS WITH BRACHIAL PLEXUS INJURIES. PROCEEDINGS OF THE ... ASME DESIGN ENGINEERING TECHNICAL CONFERENCES. ASME DESIGN ENGINEERING TECHNICAL CONFERENCES 2020; 10:V010T10A042. [PMID: 36479635 PMCID: PMC9726175 DOI: 10.1115/detc2020-22348] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/18/2023]
Abstract
This paper presents the design of an exoskeleton glove system for people who suffer from the brachial plexus injuries in an effort to restore their lost grasping functionality. The robotic system consists of an embedded controller and a portable glove system. The glove system consists of Linear Series Elastic Actuators (SEA), Rotary SEA and optimized finger linkages to provide motion to each finger and a coupled motion of the hand and the wrist. The design is based on various functionality requirements such as being lightweight and portable for activities of daily living, especially for grasping. The contact force at each fingertip and bending angle of each finger are measured for future implementation of intelligent control algorithms for autonomous grasping. To provide better flexibility and comfort for the users, abduction and adduction of each finger as well as flexion of the thumb were taken into consideration in the design. The glove system is adjustable for different hand sizes. The micro-controllers and batteries are integrated on the forearm in order to provide a completely portable design solution.
Collapse
Affiliation(s)
| | | | - Yunfei Guo
- Robotics and Mechatronics Lab, Department of Electrical and Computer Engineering, Virginia Tech, Blacksburg, Virginia 24060
| | - Cesar Bravo
- Carilion Clinic Institute of Orthopaedics and Neurosciences, Virginia Tech Carilion School of Medicine, Roanoke, VA 24014
| | | |
Collapse
|