1
|
Benavides D, Cisnal A, Fontúrbel C, de la Fuente E, Fraile JC. Real-Time Tool Localization for Laparoscopic Surgery Using Convolutional Neural Network. SENSORS (BASEL, SWITZERLAND) 2024; 24:4191. [PMID: 39000974 PMCID: PMC11243864 DOI: 10.3390/s24134191] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 05/19/2024] [Revised: 06/17/2024] [Accepted: 06/25/2024] [Indexed: 07/16/2024]
Abstract
Partially automated robotic systems, such as camera holders, represent a pivotal step towards enhancing efficiency and precision in surgical procedures. Therefore, this paper introduces an approach for real-time tool localization in laparoscopy surgery using convolutional neural networks. The proposed model, based on two Hourglass modules in series, can localize up to two surgical tools simultaneously. This study utilized three datasets: the ITAP dataset, alongside two publicly available datasets, namely Atlas Dione and EndoVis Challenge. Three variations of the Hourglass-based models were proposed, with the best model achieving high accuracy (92.86%) and frame rates (27.64 FPS), suitable for integration into robotic systems. An evaluation on an independent test set yielded slightly lower accuracy, indicating limited generalizability. The model was further analyzed using the Grad-CAM technique to gain insights into its functionality. Overall, this work presents a promising solution for automating aspects of laparoscopic surgery, potentially enhancing surgical efficiency by reducing the need for manual endoscope manipulation.
Collapse
Affiliation(s)
| | - Ana Cisnal
- Instituto de las Tecnologías Avanzadas de la Producción (ITAP), Escuela de Ingenierías Industriales, Universidad de Valladolid, Paseo Prado de la Magdalena 3-5, 47011 Valladolid, Spain; (D.B.); (C.F.); (E.d.l.F.); (J.C.F.)
| | | | | | | |
Collapse
|
2
|
Sun Y, Zong C, Pancheri F, Chen T, Lueth TC. Design of topology optimized compliant legs for bio-inspired quadruped robots. Sci Rep 2023; 13:4875. [PMID: 36966220 PMCID: PMC10039952 DOI: 10.1038/s41598-023-32106-5] [Citation(s) in RCA: 8] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/02/2022] [Accepted: 03/21/2023] [Indexed: 03/27/2023] Open
Abstract
Robotic legs are an important component of the quadruped robot for achieving different motion gaits. Although the conventional rigid-link-based legs can generally perform robust motions, they still have the issues with poor sealing when operating in complex and liquid terrains. To cope with this problem, fully compliant legs with monolithic structure have been introduced in recent years to improve the system compactness and structural compliance of quadruped robots. In this article, we present a topology-optimization-based method to achieve efficient design of compliant robotic legs. In order to balance the structural stiffness and bending flexibility of the realized leg, a multi-objective optimization algorithm is utilized. A series of design cases are presented to illustrate the design principle and analytical procedure of the proposed method. In addition, experimental evaluation is also performed, and the results have demonstrated that, a quadruped robot with the optimized legs can successfully achieve stable and continuous straight-line walking motions.
Collapse
Affiliation(s)
- Yilun Sun
- Institute of Micro Technology and Medical Device Technology, Technical University of Munich, 85748, Garching, Germany.
| | - Chujun Zong
- Institute of Energy Efficient and Sustainable Design and Building, Technical University of Munich, 80333, Munich, Germany
| | - Felix Pancheri
- Institute of Micro Technology and Medical Device Technology, Technical University of Munich, 85748, Garching, Germany
| | - Tong Chen
- Institute of Micro Technology and Medical Device Technology, Technical University of Munich, 85748, Garching, Germany
- Department of Precision Engineering, School of Engineering, The University of Tokyo, Tokyo, 113-8656, Japan
| | - Tim C Lueth
- Institute of Micro Technology and Medical Device Technology, Technical University of Munich, 85748, Garching, Germany
| |
Collapse
|
3
|
2.5-mm articulated endoluminal forceps using a compliant mechanism. Int J Comput Assist Radiol Surg 2023; 18:1-8. [PMID: 35922706 DOI: 10.1007/s11548-022-02726-9] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/01/2022] [Accepted: 07/22/2022] [Indexed: 02/01/2023]
Abstract
PURPOSE Gastrointestinal cancer can be treated using a flexible endoscope through a natural orifice. However, treatment instruments with limited degrees of freedom (DOFs) require a highly skilled operator. Articulated devices useful for endoluminal procedures, such as endoscopic submucosal dissection and biopsy, have been developed. These devices enable dexterous operation in a narrow lumen; however, they suffer from limitations such as large size and high cost. To overcome these limitations, we developed a 2.5-mm articulated forceps that can be inserted into a standard endoscope channel based on a compliant mechanism. METHODS The compliant mechanism allows the device to be compact and affordable, which is possible due to its monolithic structure. The proposed mechanism consists of two segments, 1-DOF grasping and 2-DOF bending, that are actuated by tendon-sheath mechanisms. A prototype was designed based on finite element analysis results. RESULTS To confirm the effectiveness of the proposed mechanism, we fabricated the prototype using a 3D printer. A series of mechanical performance tests on the prototype revealed that it achieved the following specifications: (1) DOF: 1-DOF grasping + 2-DOF bending, (2) outer diameter: 2.5 mm, (3) length of the bending segment: 30 mm, and (4) range of motion: [Formula: see text] to [Formula: see text] (grasping) and [Formula: see text] to [Formula: see text] (bending). Finally, we performed a tissue manipulation test on an excised porcine colon and found that a piece of mucous membrane tissue was successfully resected using an electric knife while being lifted with the developed forceps. CONCLUSION The results of the evaluation experiment demonstrated a positive feasibility of the proposed mechanism, which has a simpler structure compared to those of other conventional mechanisms; furthermore, it is potentially more cost-effective and is disposable. The mechanical design, prototype implementation, and evaluations are reported in this paper.
Collapse
|
4
|
Sun Y, Liu Y, Lueth TC. Optimization of Stress Distribution in Tendon-Driven Continuum Robots Using Fish-Tail-Inspired Method. IEEE Robot Autom Lett 2022. [DOI: 10.1109/lra.2022.3147456] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/25/2023]
|
5
|
Stress Dispersion Design in Continuum Compliant Structure toward Multi-DOF Endoluminal Forceps. APPLIED SCIENCES-BASEL 2022. [DOI: 10.3390/app12052480] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/04/2023]
Abstract
Gastrointestinal cancer, when detected early, is treated by accessing the lesion through the natural orifice using flexible endoscopes. However, the limited degree-of-freedom (DOF) of conventional treatment devices and the narrow surgical view through the endoscope demand advanced techniques. In contrast, multi-DOF forceps systems are an excellent alternative; however, these systems often involve high fabrication costs because they require a large number of micro-parts. To solve this problem, we designed compact multi-DOF endoluminal forceps with a monolithic structure comprising compliant hinges. To allow an efficient stress dispersion at the base end when the hinge bends, we proposed a novel design method to obtain the hinge parameters using the beam of uniform strength theory. This method does not involve a high computational cost. The results show that the improved design with a variable hinge thickness can reduce the maximum bending stress, dispersing the stress in a larger area than that of the previous design considering a constant thickness of the hinge. Moreover, the experiments conducted in a prototype confirm that the radius of the curvature was significantly improved. The proposed method could aid in designing other continuum robots relying on compliant hinges.
Collapse
|
6
|
A Fully 3D-Printed Steerable Instrument for Minimally Invasive Surgery. MATERIALS 2021; 14:ma14247910. [PMID: 34947503 PMCID: PMC8703879 DOI: 10.3390/ma14247910] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 11/11/2021] [Revised: 12/10/2021] [Accepted: 12/17/2021] [Indexed: 12/02/2022]
Abstract
In the field of medical instruments, additive manufacturing allows for a drastic reduction in the number of components while improving the functionalities of the final design. In addition, modifications for users’ needs or specific procedures become possible by enabling the production of single customized items. In this work, we present the design of a new fully 3D-printed handheld steerable instrument for laparoscopic surgery, which was mechanically actuated using cables. The pistol-grip handle is based on ergonomic principles and allows for single-hand control of both grasping and omnidirectional steering, while compliant joints and snap-fit connectors enable fast assembly and minimal part count. Additive manufacturing allows for personalization of the handle to each surgeon’s needs by adjusting specific dimensions in the CAD model, which increases the user’s comfort during surgery. Testing showed that the forces on the instrument handle required for steering and grasping were below 15 N, while the grasping force efficiency was calculated to be 10–30%. The instrument combines the advantages of additive manufacturing with regard to personalization and simplified assembly, illustrating a new approach to the design of advanced surgical instruments where the customization for a single procedure or user’s need is a central aspect.
Collapse
|
7
|
Sun Y, Lueth TC. Cruciate-Ligament-Inspired Compliant Joints: Application to 3D-Printed Continuum Surgical Robots. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2021; 2021:4645-4648. [PMID: 34892249 DOI: 10.1109/embc46164.2021.9630239] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
The rapid development of additive manufacturing technology makes it possible to fabricate a patient-specific surgical robot in a short time. To simplify the assembly process of the printed robotic system, compliant-joint-based monolithic structures are often used as substitutes for rigid-link mechanisms to realize flexible bending. In this paper, we introduce a cruciate-ligament-inspired compliant joint (CLCJ) to improve the bending stability of the 3D-printed continuum surgical robots. The basic structure of the tendon-driven CLCJ mechanism and its kinematic model were described in detail. The bending performance of CLCJ was also successfully evaluated by FEM simulation and experimental tests. Besides, a prototype of CLCJ-based surgical robotic system was presented to demonstrate its application in 3D-printed continuum surgical robots.
Collapse
|
8
|
Sun Y, Lueth TC. Design of Bionic Prosthetic Fingers Using 3D Topology Optimization. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2021; 2021:4505-4508. [PMID: 34892219 DOI: 10.1109/embc46164.2021.9630390] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
Compliant mechanisms are frequently used in the design of prosthetic fingers since their monolithic structure and flexible movement are quite similar to the biological human fingers. However, the design of compliant prosthetic fingers is not easy, as the conventional rigid-link-based mechanism theory cannot be directly applied. In this paper, we introduce a 3D topology optimization based design framework to simplify the synthesis process of bionic compliant prosthetic fingers. The proposed framework is implemented in the software MATLAB and the realized fingers can be quickly fabricated using selective laser sintering (SLS) technology. To illustrate the design process of the proposed framework, a design example was presented. The bending performance of the realized finger was successfully verified by the FEM-based simulation and the payload test. In future work, the optimized fingers have the potential to be integrated into prosthetic hands to realize sophisticated grasping movements.
Collapse
|
9
|
Zhang D, Sun Y, Lueth TC. Design of a novel tendon-driven manipulator structure based on monolithic compliant rolling-contact joint for minimally invasive surgery. Int J Comput Assist Radiol Surg 2021; 16:1615-1625. [PMID: 34235629 PMCID: PMC8354933 DOI: 10.1007/s11548-021-02442-w] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/18/2021] [Accepted: 06/28/2021] [Indexed: 11/26/2022]
Abstract
Purpose Compliant mechanisms are commonly used in the design of manipulator and surgical robotic tools for minimally invasive surgery (MIS) thanks to their compactness, ability of miniaturization and lower part count. However, conventional compliant joint has higher internal stiffness, which limits the bending radius. To overcome this problem, a novel tendon-driven manipulator structure based on monolithic compliant rolling-contact joint (CRCJ) is proposed. Methods The proposed rolling-contact mechanism is used to prevent cable slack during actuation, which occurs in conventional compliant joint design. By means of selective laser sintering (SLS) technique, the CRCJ can be fabricated in a monolithic structure, thus granting the CRCJ both the advantages of compliant joints and rolling-contact mechanism. Simulations with nonlinear finite element analysis (FEA) and experiments were conducted to evaluate and compare the mechanical properties of the proposed CRCJ with conventional leaf-type compliant joint including the bending and compliant motion. Results Experimental results showed that the CRCJ has lower bending stiffness, higher maximum bending angle (over \documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$$180^{\circ }$$\end{document}180∘) and a higher compliance compared to conventional compliant hinges, which allows a larger workspace and reduces the possibility of tissue injury. Agreement was also found between the nonlinear FEA and experiments regarding the relation between actuation force and bending angle. A primary prototype of a 3-DOF handheld laparoscopic manipulator with a diameter of 7 mm was further developed. Conclusion A dexterous tendon-driven monolithic manipulator structure based on CRCJ for MIS is proposed. A preliminary prototype of a handheld laparoscopic manipulator demonstrates the capability of the CRCJ for steerable medical devices. However, design improvements based on FEA and application-orientated prototypes considering anatomical requirements still show room for improvements. Supplementary Information The online version supplementary material available at 10.1007/s11548-021-02442-w.
Collapse
Affiliation(s)
- Dingzhi Zhang
- Institute of Micro Technology and Medical Device Technology, Technical University of Munich, Boltzmannstraße 15, 85748 Garching, Germany
| | - Yilun Sun
- Institute of Micro Technology and Medical Device Technology, Technical University of Munich, Boltzmannstraße 15, 85748 Garching, Germany
| | - Tim C. Lueth
- Institute of Micro Technology and Medical Device Technology, Technical University of Munich, Boltzmannstraße 15, 85748 Garching, Germany
| |
Collapse
|
10
|
Manoj A, Bhuyan M, Raj Banik S, Ravi Sankar M. 3D printing of nasopharyngeal swabs for COVID-19 diagnose: Past and current trends. MATERIALS TODAY. PROCEEDINGS 2020; 44:1361-1368. [PMID: 33262931 PMCID: PMC7687488 DOI: 10.1016/j.matpr.2020.11.505] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 11/05/2020] [Accepted: 11/17/2020] [Indexed: 12/15/2022]
Abstract
The current technological advancements in emerging 3D printing technologies are indeed propitious. To date, ground-breaking 3D printing technologies are used in automobile, aerospace, clothing, pharma, and biomedical industries by creating pre-requisite engineered and tailored end-user products reaching standard sets. 3D printing is also becoming a crucial technology in support of enhanced health care and general emergency response since the beginning of the COVID-19 pandemic. As the world is facing a significant lack of medicinal supplies, manufactures are struggling to fulfill demands due to the ongoing COVID-19 pandemic. The decline in the diagnostic testing kits supply chained to increased interest in 3D printed Nasopharyngeal (NP) swabs. This article has reviewed and studied the sensitivity of the NP swabs and various NP swab designs. The process of different 3D printing technologies that are employed to address the swab shortages during COVID-19 is explained in detail. The paper ends with the conclusions drawn from the literature review.
Collapse
Affiliation(s)
- Aluri Manoj
- Department of Mechanical Engineering, Rajiv Gandhi University of Knowledge Technologies-Basar, Nirmal 504107, India
| | - Monami Bhuyan
- Department of Mechanical Engineering, Assam Engineering College, Guwahati 781013, Assam, India
| | - Swarup Raj Banik
- Department of Mechanical Engineering, National Institute of Technology, Silchar 788010, Silchar, India
| | - Mamilla Ravi Sankar
- Department of Mechanical Engineering, Indian Institute of Technology, Tirupati 517506, India
| |
Collapse
|
11
|
Sun Y, Zhang D, Lueth TC. Bionic Design of a Disposable Compliant Surgical Forceps With Optimized Clamping Performance. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2020; 2020:4704-4707. [PMID: 33019042 DOI: 10.1109/embc44109.2020.9176027] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
Disposable forceps are frequently used in different surgical procedures to prevent infections caused by poorly sterilized reusable metal forceps. Compared to traditional rigid-joint mechanisms, compliant mechanisms are much easier to sterilize due to their monolithic structure, hence they are widely used for designing disposable surgical forceps. However, the clamping performance of plastic compliant forceps is generally less robust than metal forceps, which has greatly limited their use in medical applications. To cope with this problem, a novel 3D-printed plastic compliant forceps with optimized clamping performance was developed in this paper for open surgery and physical nursing applications. Bio-inspired topology optimization techniques were employed to synthesize the forceps. The clamping capability of the proposed forceps was evaluated by finite element analysis and loading tests. Results showed that the proposed forceps can generate greater and more stable clamping forces than the previous model of disposable compliant forceps. The proposed bionic optimization method also has potential for synthesizing compliant devices for robotic surgery.
Collapse
|