1
|
Zhou Z, Lu Y, Tortós PE, Qin R, Kokubu S, Matsunaga F, Xie Q, Yu W. Addressing data imbalance in Sim2Real: ImbalSim2Real scheme and its application in finger joint stiffness self-sensing for soft robot-assisted rehabilitation. Front Bioeng Biotechnol 2024; 12:1334643. [PMID: 38948382 PMCID: PMC11212110 DOI: 10.3389/fbioe.2024.1334643] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/07/2023] [Accepted: 05/10/2024] [Indexed: 07/02/2024] Open
Abstract
The simulation-to-reality (sim2real) problem is a common issue when deploying simulation-trained models to real-world scenarios, especially given the extremely high imbalance between simulation and real-world data (scarce real-world data). Although the cycle-consistent generative adversarial network (CycleGAN) has demonstrated promise in addressing some sim2real issues, it encounters limitations in situations of data imbalance due to the lower capacity of the discriminator and the indeterminacy of learned sim2real mapping. To overcome such problems, we proposed the imbalanced Sim2Real scheme (ImbalSim2Real). Differing from CycleGAN, the ImbalSim2Real scheme segments the dataset into paired and unpaired data for two-fold training. The unpaired data incorporated discriminator-enhanced samples to further squash the solution space of the discriminator, for enhancing the discriminator's ability. For paired data, a term targeted regression loss was integrated to ensure specific and quantitative mapping and further minimize the solution space of the generator. The ImbalSim2Real scheme was validated through numerical experiments, demonstrating its superiority over conventional sim2real methods. In addition, as an application of the proposed ImbalSim2Real scheme, we designed a finger joint stiffness self-sensing framework, where the validation loss for estimating real-world finger joint stiffness was reduced by roughly 41% compared to the supervised learning method that was trained with scarce real-world data and by 56% relative to the CycleGAN trained with the imbalanced dataset. Our proposed scheme and framework have potential applicability to bio-signal estimation when facing an imbalanced sim2real problem.
Collapse
Affiliation(s)
- Zhongchao Zhou
- Department of Medical System Engineering, Chiba University, Chiba, Japan
| | - Yuxi Lu
- Department of Medical System Engineering, Chiba University, Chiba, Japan
| | | | - Ruian Qin
- Department of Medical System Engineering, Chiba University, Chiba, Japan
| | - Shota Kokubu
- Department of Medical System Engineering, Chiba University, Chiba, Japan
| | - Fuko Matsunaga
- Department of Medical System Engineering, Chiba University, Chiba, Japan
| | - Qiaolian Xie
- Department of Medical System Engineering, Chiba University, Chiba, Japan
- Institute of Rehabilitation Engineering and Technology, University of Shanghai for Science and Technology, Shanghai, China
| | - Wenwei Yu
- Department of Medical System Engineering, Chiba University, Chiba, Japan
- Center for Frontier Medical Engineering, Chiba University, Chiba, Japan
| |
Collapse
|
2
|
Lu Y, Zhou Z, Kokubu S, Qin R, Tortós Vinocour PE, Yu W. Neural Network-Based Active Load-Sensing Scheme and Stiffness Adjustment for Pneumatic Soft Actuators for Minimally Invasive Surgery Support. SENSORS (BASEL, SWITZERLAND) 2023; 23:833. [PMID: 36679629 PMCID: PMC9861017 DOI: 10.3390/s23020833] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 12/09/2022] [Revised: 01/05/2023] [Accepted: 01/09/2023] [Indexed: 06/17/2023]
Abstract
To provide a stable surgical view in Minimally Invasive Surgery (MIS), it is necessary for a flexible endoscope applied in MIS to have adjustable stiffness to resist different external loads from surrounding organs and tissues. Pneumatic soft actuators are expected to fulfill this role, since they could feed the endoscope with an internal access channel and adjust their stiffness via an antagonistic mechanism. For that purpose, it is essential to estimate the external load. In this study, we proposed a neural network (NN)-based active load-sensing scheme and stiffness adjustment for a soft actuator for MIS support with antagonistic chambers for three degrees of freedom (DoFs) of control. To deal with the influence of the nonlinearity of the soft actuating system and uncertainty of the interaction between the soft actuator and its environment, an environment exploration strategy was studied for improving the robustness of sensing. Moreover, a NN-based inverse dynamics model for controlling the stiffness of the soft actuator with different flexible endoscopes was proposed too. The results showed that the exploration strategy with different sequence lengths improved the estimation accuracy of external loads in different conditions. The proposed method for external load exploration and inverse dynamics model could be used for in-depth studies of stiffness control of soft actuators for MIS support.
Collapse
Affiliation(s)
- Yuxi Lu
- Department of Medical Engineering, Graduate School of Engineering, Chiba University, Chiba 263-8522, Japan
| | - Zhongchao Zhou
- Department of Medical Engineering, Graduate School of Engineering, Chiba University, Chiba 263-8522, Japan
| | - Shota Kokubu
- Department of Medical Engineering, Graduate School of Engineering, Chiba University, Chiba 263-8522, Japan
| | - Ruian Qin
- Department of Medical Engineering, Graduate School of Engineering, Chiba University, Chiba 263-8522, Japan
| | - Pablo E. Tortós Vinocour
- Department of Medical Engineering, Graduate School of Engineering, Chiba University, Chiba 263-8522, Japan
| | - Wenwei Yu
- Department of Medical Engineering, Graduate School of Engineering, Chiba University, Chiba 263-8522, Japan
- Center for Frontier Medical Engineering, Chiba University, Chiba 263-8522, Japan
| |
Collapse
|
3
|
Evaluation of Fiber-Reinforced Modular Soft Actuators for Individualized Soft Rehabilitation Gloves. ACTUATORS 2022. [DOI: 10.3390/act11030084] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
Applying soft actuators to hand motion assist for rehabilitation has been receiving increasing interest in recent years. Pioneering research efforts have shown the feasibility of soft rehabilitation gloves (SRGs). However, one important and practical issue, the effects of users’ individual differences in finger size and joint stiffness on both bending performance (e.g., Range of motion (ROM) and torque) and the mechanical loads applied to finger joints when the actuators are placed on a patient’s hand, has not been well investigated. Moreover, the design considerations of SRGs for individual users, considering individual differences, have not been addressed. These, along with the inherent safety of soft actuators, should be investigated carefully before the practical use of SRGs. This work aimed to clarify the effects of individual differences on the actuator’s performance through a series of experiments using dummy fingers designed with individualized parameters. Two types of fiber-reinforced soft actuators, the modular type for assisting each joint and conventional (whole-finger assist) type, were designed and compared. It was found that the modular soft actuators respond better to individual differences set in the experiment and exhibit a superior performance to the conventional ones. By suitable connectors and air pressure, the modular soft actuators could cope with the individual differences with minimal effort. The effects of the individualized parameters are discussed, and design considerations are extracted and summarized. This study will play an important role in pushing forward the SRGs to real rehabilitation practice.
Collapse
|
4
|
Wang Y, Kokubu S, Zhou Z, Guo X, Hsueh YH, Yu W. Designing Soft Pneumatic Actuators for Thumb Movements. IEEE Robot Autom Lett 2021. [DOI: 10.1109/lra.2021.3105799] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
|
5
|
Mechanical Design and Analysis of the End-Effector Finger Rehabilitation Robot (EFRR) for Stroke Patients. MACHINES 2021. [DOI: 10.3390/machines9060110] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
Most existing finger rehabilitation robots are structurally complex and cannot be adapted to multiple work conditions, such as clinical and home. In addition, there is a lack of attention to active adduction/abduction (A/A) movement, which prevents stroke patients from opening the joint in time and affects the rehabilitation process. In this paper, an end-effector finger rehabilitation robot (EFRR) with active A/A motion that can be applied to a variety of applications is proposed. First, the natural movement curve of the finger is analyzed, which is the basis of the mechanism design. Based on the working principle of the cam mechanism, the flexion/extension (F/E) movement module is designed and the details used to ensure the safety and reliability of the device are introduced. Then, a novel A/A movement module is proposed, using the components that can easily individualized design to achieve active A/A motion only by one single motor, which makes up for the shortcomings of the existing devices. As for the control system, a fuzzy proportional-derivative (PD) adaptive impedance control strategy based on the position information is proposed, which can make the device more compliant, avoid secondary injuries caused by excessive muscle tension, and protect the fingers effectively. Finally, some preliminary experiments of the prototype are reported, and the results shows that the EFRR has good performance, which lays the foundation for future work.
Collapse
|
6
|
Kokubu S, Yu W. Developing a hybrid soft mechanism for assisting individualized flexion and extension of finger joints. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2020; 2020:4873-4877. [PMID: 33019081 DOI: 10.1109/embc44109.2020.9176061] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
Various robotic devices have been developed for home rehabilitation and support of therapists. Special attention has been focused on soft actuators due to their high viscoelasticity and flexibility, which can contribute to the safety and affinity with the users. However, most of them aimed at the assist of finger flexion, and few have been designed to support extension actively. Moreover, for most soft actuator based mechanisms, the individual-adaptability have not been considered nor appropriately evaluated. Consequently, the effect of individual difference on the assistance using soft robotic devices is unclear, and for the purpose of dealing with the individual difference, the whole mechanism should be designed and fabricated for each individual user, which is ineffective for the rehabilitation support. In this study, we proposed a hybrid soft mechanism with modularized fiber-reinforced elastomer actuators for joint-dependent flexion support and McKibben actuators for finger extension support. Without further changing the design of the elastomer actuator, the hybrid mechanism could be adapted to individual hand difference: proportions of hand segments, range of motion (ROM) and torque characteristics of the joints. A prototype of the mechanism was fabricated and evaluated. The results showed that the mechanism could meet the requirement of finger function assist. Moreover, the mechanism could be fine-tuned towards the individual hand by changing the fiber-reinforcement and adjusting the fittings of the actuators.
Collapse
|
7
|
Functional connectivity of brain associated with passive range of motion exercise: Proprioceptive input promoting motor activation? Neuroimage 2019; 202:116023. [DOI: 10.1016/j.neuroimage.2019.116023] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/15/2018] [Revised: 06/07/2019] [Accepted: 07/15/2019] [Indexed: 11/24/2022] Open
|