1
|
Selim M, Dresscher D, Abayazid M. Virtual Needle Insertion with Enhanced Haptic Feedback for Guidance and Needle-Tissue Interaction Forces. SENSORS (BASEL, SWITZERLAND) 2024; 24:5560. [PMID: 39275470 PMCID: PMC11397964 DOI: 10.3390/s24175560] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/28/2024] [Revised: 08/17/2024] [Accepted: 08/19/2024] [Indexed: 09/16/2024]
Abstract
Interventional radiologists mainly rely on visual feedback via imaging modalities to steer a needle toward a tumor during biopsy and ablation procedures. In the case of CT-guided procedures, there is a risk of exposure to hazardous X-ray-based ionizing radiation. Therefore, CT scans are usually not used continuously, which increases the chances of a misplacement of the needle and the need for reinsertion, leading to more tissue trauma. Interventionalists also encounter haptic feedback via needle-tissue interaction forces while steering a needle. These forces are useful but insufficient to clearly perceive and identify deep-tissue structures such as tumors. The objective of this paper was to investigate the effect of enhanced force feedback for sensing interaction forces and guiding the needle when applied individually and simultaneously during a virtual CT-guided needle insertion task. We also compared the enhanced haptic feedback to enhanced visual feedback. We hypothesized that enhancing the haptic feedback limits the time needed to reach the target accurately and reduces the number of CT scans, as the interventionalist depends more on real-time enhanced haptic feedback. To test the hypothesis, a simulation environment was developed to virtually steer a needle in five degrees of freedom (DoF) to reach a tumor target embedded in a liver model. Twelve participants performed in the experiment with different feedback conditions where we measured their performance in terms of the following: targeting accuracy, trajectory tracking, number of CT scans required, and the time needed to finish the task. The results suggest that the combination of enhanced haptic feedback for guidance and sensing needle-tissue interaction forces significantly reduce the number of scans and the duration required to finish the task by 32.1% and 46.9%, respectively, when compared to nonenhanced haptic feedback. The other feedback modalities significantly reduced the duration to finish the task by around 30% compared to nonenhanced haptic feedback.
Collapse
Affiliation(s)
- Mostafa Selim
- Robotics and Mechatronics Research Group, Faculty of Electrical Engineering, Mathematics and Computer Science, University of Twente, 7500 AE Enschede, The Netherlands
| | - Douwe Dresscher
- Robotics and Mechatronics Research Group, Faculty of Electrical Engineering, Mathematics and Computer Science, University of Twente, 7500 AE Enschede, The Netherlands
| | - Momen Abayazid
- Robotics and Mechatronics Research Group, Faculty of Electrical Engineering, Mathematics and Computer Science, University of Twente, 7500 AE Enschede, The Netherlands
| |
Collapse
|
2
|
Wang C, Guo L, Zhu J, Zhu L, Li C, Zhu H, Song A, Lu L, Teng GJ, Navab N, Jiang Z. Review of robotic systems for thoracoabdominal puncture interventional surgery. APL Bioeng 2024; 8:021501. [PMID: 38572313 PMCID: PMC10987197 DOI: 10.1063/5.0180494] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/10/2023] [Accepted: 03/11/2024] [Indexed: 04/05/2024] Open
Abstract
Cancer, with high morbidity and high mortality, is one of the major burdens threatening human health globally. Intervention procedures via percutaneous puncture have been widely used by physicians due to its minimally invasive surgical approach. However, traditional manual puncture intervention depends on personal experience and faces challenges in terms of precisely puncture, learning-curve, safety and efficacy. The development of puncture interventional surgery robotic (PISR) systems could alleviate the aforementioned problems to a certain extent. This paper attempts to review the current status and prospective of PISR systems for thoracic and abdominal application. In this review, the key technologies related to the robotics, including spatial registration, positioning navigation, puncture guidance feedback, respiratory motion compensation, and motion control, are discussed in detail.
Collapse
Affiliation(s)
- Cheng Wang
- Hanglok-Tech Co. Ltd., Hengqin 519000, People's Republic of China
| | - Li Guo
- Hanglok-Tech Co. Ltd., Hengqin 519000, People's Republic of China
| | | | - Lifeng Zhu
- State Key Laboratory of Digital Medical Engineering, Jiangsu Key Lab of Remote Measurement and Control, School of Instrument Science and Engineering, Southeast University, Nanjing 210096, People's Republic of China
| | - Chichi Li
- School of Computer Science and Engineering, Macau University of Science and Technology, Macau, 999078, People's Republic of China
| | - Haidong Zhu
- Center of Interventional Radiology and Vascular Surgery, Department of Radiology, Zhongda Hospital, Medical School, Southeast University, Nanjing 210009, People's Republic of China
| | - Aiguo Song
- State Key Laboratory of Digital Medical Engineering, Jiangsu Key Lab of Remote Measurement and Control, School of Instrument Science and Engineering, Southeast University, Nanjing 210096, People's Republic of China
| | | | - Gao-Jun Teng
- Center of Interventional Radiology and Vascular Surgery, Department of Radiology, Zhongda Hospital, Medical School, Southeast University, Nanjing 210009, People's Republic of China
| | | | - Zhongliang Jiang
- Computer Aided Medical Procedures, Technical University of Munich, Munich 80333, Germany
| |
Collapse
|
3
|
Su K, Liu J, Ren X, Huo Y, Du G, Zhao W, Wang X, Liang B, Li D, Liu PX. A fully autonomous robotic ultrasound system for thyroid scanning. Nat Commun 2024; 15:4004. [PMID: 38734697 DOI: 10.1038/s41467-024-48421-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/08/2023] [Accepted: 04/23/2024] [Indexed: 05/13/2024] Open
Abstract
The current thyroid ultrasound relies heavily on the experience and skills of the sonographer and the expertise of the radiologist, and the process is physically and cognitively exhausting. In this paper, we report a fully autonomous robotic ultrasound system, which is able to scan thyroid regions without human assistance and identify malignant nod- ules. In this system, human skeleton point recognition, reinforcement learning, and force feedback are used to deal with the difficulties in locating thyroid targets. The orientation of the ultrasound probe is adjusted dynamically via Bayesian optimization. Experimental results on human participants demonstrated that this system can perform high-quality ultrasound scans, close to manual scans obtained by clinicians. Additionally, it has the potential to detect thyroid nodules and provide data on nodule characteristics for American College of Radiology Thyroid Imaging Reporting and Data System (ACR TI-RADS) calculation.
Collapse
Affiliation(s)
- Kang Su
- School of Computer Science and Engineering, South China University of Technology, Guangzhou, 510006, China
| | - Jingwei Liu
- School of Computer Science and Engineering, South China University of Technology, Guangzhou, 510006, China
| | - Xiaoqi Ren
- School of Future Technology, South China University of Technology, Guangzhou, 511442, China
- Peng Cheng Laboratory, Shenzhen, 518000, China
| | - Yingxiang Huo
- School of Future Technology, South China University of Technology, Guangzhou, 511442, China
- Peng Cheng Laboratory, Shenzhen, 518000, China
| | - Guanglong Du
- School of Computer Science and Engineering, South China University of Technology, Guangzhou, 510006, China.
| | - Wei Zhao
- Division of Vascular and Interventional Radiology, Nanfang Hospital Southern Medical University, Guangzhou, 510515, China
| | - Xueqian Wang
- Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, 518055, China.
| | - Bin Liang
- Department of Automation, Tsinghua University, 100854, Beijing, China.
| | - Di Li
- School of Mechanical and Automotive Engineering, South China University of Technology, Guangzhou, 510641, China
| | - Peter Xiaoping Liu
- Department of Systems and Computer Engineering, Carleton University, Ottawa, ON, K1S 5B6, Canada.
| |
Collapse
|
4
|
Huang Y, Yu L, Zhang F. A survey on puncture models and path planning algorithms of bevel-tipped flexible needles. Heliyon 2024; 10:e25002. [PMID: 38322890 PMCID: PMC10844025 DOI: 10.1016/j.heliyon.2024.e25002] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/10/2023] [Revised: 12/02/2023] [Accepted: 01/18/2024] [Indexed: 02/08/2024] Open
Abstract
Percutaneous needle insertion is a minimally invasive surgery with broad medical application prospects, such as biopsy and brachytherapy. However, the currently adopted rigid needles have limitations, as they cannot bypass obstacles or correct puncture deviations and can only travel along a straight path. Bevel-tip flexible needles are increasingly being adopted to address these issues, owing to their needle body's ease of deformation and bending. Successful puncture of flexible needles relies on accurate models and path planning, ensuring the needle reaches the target while avoiding vital tissues. This review investigates puncture models and path-planning algorithms by reviewing recent literature, focusing on the path-planning part. According to the literature, puncture models can be divided into three types: mechanical, finite element method (FEM), and kinematic models, while path-planning algorithms are categorized and discussed following the division used for mobile robots, which differs from the conventional approach for flexible needles-an innovation in this review. This review systematically summarizes the following categories: graph theory search, sampling-based, intelligent search, local obstacle avoidance, and other algorithms, including their implementation, advantages, and disadvantages, to further explore the potential to overcome obstacles in path planning for minimally invasive puncture needles. Finally, this study proposes future development trends in path-planning algorithms, providing possible directions for subsequent research for bevel-tipped flexible needles. This research aims to provide a resource for researchers to quickly learn about common path-planning algorithms, their backgrounds, and puncture models.
Collapse
Affiliation(s)
- Ye Huang
- Department of Medical Information Engineering, School of Biomedical Engineering Anhui Medical University, China
| | - Longfeng Yu
- Department of Medical Information Engineering, School of Biomedical Engineering Anhui Medical University, China
| | - Feifan Zhang
- Department of Medical Information Engineering, School of Biomedical Engineering Anhui Medical University, China
| |
Collapse
|
5
|
Bergholz M, Ferle M, Weber BM. The benefits of haptic feedback in robot assisted surgery and their moderators: a meta-analysis. Sci Rep 2023; 13:19215. [PMID: 37932393 PMCID: PMC10628231 DOI: 10.1038/s41598-023-46641-8] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/03/2023] [Accepted: 11/03/2023] [Indexed: 11/08/2023] Open
Abstract
Robot assisted surgery (RAS) provides medical practitioners with valuable tools, decreasing strain during surgery and leading to better patient outcomes. While the loss of haptic sensation is a commonly cited disadvantage of RAS, new systems aim to address this problem by providing artificial haptic feedback. N = 56 papers that compared robotic surgery systems with and without haptic feedback were analyzed to quantify the performance benefits of restoring the haptic modality. Additionally, this study identifies factors moderating the effect of restoring haptic sensation. Overall results showed haptic feedback was effective in reducing average forces (Hedges' g = 0.83) and peak forces (Hedges' g = 0.69) applied during surgery, as well as reducing the completion time (Hedges' g = 0.83). Haptic feedback has also been found to lead to higher accuracy (Hedges' g = 1.50) and success rates (Hedges' g = 0.80) during surgical tasks. Effect sizes on several measures varied between tasks, the type of provided feedback, and the subjects' levels of surgical expertise, with higher levels of expertise generally associated with smaller effect sizes. No significant differences were found between virtual fixtures and rendering contact forces. Implications for future research are discussed.
Collapse
Affiliation(s)
- Max Bergholz
- Department of Ergonomics, Technical University of Munich, 85748, Garching, Germany
- Institute of Robotics and Mechatronics, German Aerospace Center, 82234, Wessling, Germany
| | - Manuel Ferle
- Department of Ergonomics, Technical University of Munich, 85748, Garching, Germany.
| | - Bernhard M Weber
- Institute of Robotics and Mechatronics, German Aerospace Center, 82234, Wessling, Germany
| |
Collapse
|
6
|
Wang T, Li H, Pu T, Yang L. Microsurgery Robots: Applications, Design, and Development. SENSORS (BASEL, SWITZERLAND) 2023; 23:8503. [PMID: 37896597 PMCID: PMC10611418 DOI: 10.3390/s23208503] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/24/2023] [Revised: 10/07/2023] [Accepted: 10/09/2023] [Indexed: 10/29/2023]
Abstract
Microsurgical techniques have been widely utilized in various surgical specialties, such as ophthalmology, neurosurgery, and otolaryngology, which require intricate and precise surgical tool manipulation on a small scale. In microsurgery, operations on delicate vessels or tissues require high standards in surgeons' skills. This exceptionally high requirement in skills leads to a steep learning curve and lengthy training before the surgeons can perform microsurgical procedures with quality outcomes. The microsurgery robot (MSR), which can improve surgeons' operation skills through various functions, has received extensive research attention in the past three decades. There have been many review papers summarizing the research on MSR for specific surgical specialties. However, an in-depth review of the relevant technologies used in MSR systems is limited in the literature. This review details the technical challenges in microsurgery, and systematically summarizes the key technologies in MSR with a developmental perspective from the basic structural mechanism design, to the perception and human-machine interaction methods, and further to the ability in achieving a certain level of autonomy. By presenting and comparing the methods and technologies in this cutting-edge research, this paper aims to provide readers with a comprehensive understanding of the current state of MSR research and identify potential directions for future development in MSR.
Collapse
Affiliation(s)
- Tiexin Wang
- ZJU-UIUC Institute, International Campus, Zhejiang University, Haining 314400, China; (T.W.); (H.L.); (T.P.)
- School of Mechanical Engineering, Zhejiang University, Hangzhou 310058, China
| | - Haoyu Li
- ZJU-UIUC Institute, International Campus, Zhejiang University, Haining 314400, China; (T.W.); (H.L.); (T.P.)
| | - Tanhong Pu
- ZJU-UIUC Institute, International Campus, Zhejiang University, Haining 314400, China; (T.W.); (H.L.); (T.P.)
| | - Liangjing Yang
- ZJU-UIUC Institute, International Campus, Zhejiang University, Haining 314400, China; (T.W.); (H.L.); (T.P.)
- School of Mechanical Engineering, Zhejiang University, Hangzhou 310058, China
- Department of Mechanical Engineering, University of Illinois Urbana-Champaign, Urbana, IL 61801, USA
| |
Collapse
|
7
|
Lu M, Zhang Y, Lim CM, Ren H. Flexible Needle Steering with Tethered and Untethered Actuation: Current States, Targeting Errors, Challenges and Opportunities. Ann Biomed Eng 2023; 51:905-924. [PMID: 36943414 DOI: 10.1007/s10439-023-03163-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/25/2022] [Accepted: 02/05/2023] [Indexed: 03/23/2023]
Abstract
Accurate needle targeting is critical for many clinical procedures, such as transcutaneous biopsy or radiofrequency ablation of tumors. However, targeting errors may arise, limiting the widespread adoption of these procedures. Advances in flexible needle (FN) steering are emerging to mitigate these errors. This review summarizes the state-of-the-art developments of FNs and addresses possible targeting errors that can be overcome with steering actuation techniques. FN steering techniques can be classified as passive and active. Passive steering directly results from the needle-tissue interaction forces, whereas active steering requires additional forces to be applied at the needle tip, which enhances needle steerability. Therefore, the corresponding targeting errors of most passive FNs and active FNs are between 1 and 2 mm, and less than 1 mm, respectively. However, the diameters of active FNs range from 1.42 to 12 mm, which is larger than the passive steering needle varying from 0.5 to 1.4 mm. Therefore, the development of active FNs is an area of active research. These active FNs can be steered using tethered internal direct actuation or untethered external actuation. Examples of tethered internal direct actuation include tendon-driven, longitudinal segment transmission and concentric tube transmission. Tendon-driven FNs have various structures, and longitudinal segment transmission needles could be adapted to reduce tissue damage. Additionally, concentric tube needles have immediate advantages and clinical applications in natural orifice surgery. Magnetic actuation enables active FN steering with untethered external actuation and facilitates miniaturization. The challenges faced in the fabrication, sensing, and actuation methods of FN are analyzed. Finally, bio-inspired FNs may offer solutions to address the challenges faced in FN active steering mechanisms.
Collapse
Affiliation(s)
- Mingyue Lu
- The Key Laboratory of Advanced Manufacturing and Intelligent Technology, Harbin University of Science and Technology, Harbin, China
- Duke-NUS Graduate Medical School, Singapore, Singapore
- The Department of Biomedical Engineering, National University of Singapore, Singapore, Singapore
| | - Yongde Zhang
- The Key Laboratory of Advanced Manufacturing and Intelligent Technology, Harbin University of Science and Technology, Harbin, China
| | - Chwee Ming Lim
- The Department of Otolaryngology-Head and Neck Surgery, Singapore General Hospital, Singapore, Singapore
- Duke-NUS Graduate Medical School, Singapore, Singapore
| | - Hongliang Ren
- The Department of Electronic Engineering and the Shun Hing Institute of Advanced Engineering, The Chinese University of Hong Kong, Hong Kong, China.
- The Department of Biomedical Engineering, National University of Singapore, Singapore, Singapore.
| |
Collapse
|
8
|
Optical force estimation for interactions between tool and soft tissues. Sci Rep 2023; 13:506. [PMID: 36627354 PMCID: PMC9831996 DOI: 10.1038/s41598-022-27036-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/25/2022] [Accepted: 12/23/2022] [Indexed: 01/11/2023] Open
Abstract
Robotic assistance in minimally invasive surgery offers numerous advantages for both patient and surgeon. However, the lack of force feedback in robotic surgery is a major limitation, and accurately estimating tool-tissue interaction forces remains a challenge. Image-based force estimation offers a promising solution without the need to integrate sensors into surgical tools. In this indirect approach, interaction forces are derived from the observed deformation, with learning-based methods improving accuracy and real-time capability. However, the relationship between deformation and force is determined by the stiffness of the tissue. Consequently, both deformation and local tissue properties must be observed for an approach applicable to heterogeneous tissue. In this work, we use optical coherence tomography, which can combine the detection of tissue deformation with shear wave elastography in a single modality. We present a multi-input deep learning network for processing of local elasticity estimates and volumetric image data. Our results demonstrate that accounting for elastic properties is critical for accurate image-based force estimation across different tissue types and properties. Joint processing of local elasticity information yields the best performance throughout our phantom study. Furthermore, we test our approach on soft tissue samples that were not present during training and show that generalization to other tissue properties is possible.
Collapse
|
9
|
3D Curvature-Based Tip Load Estimation for Continuum Robots. IEEE Robot Autom Lett 2022. [DOI: 10.1109/lra.2022.3194680] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
|