1
|
Xu X, Tang R, Gong L, Chen B, Zuo S. Two Dimensional Position-Based Visual Servoing for Soft Tissue Endomicroscopy. IEEE Robot Autom Lett 2021. [DOI: 10.1109/lra.2021.3084885] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/20/2022]
|
2
|
Li Z, Shahbazi M, Patel N, O' Sullivan E, Zhang H, Vyas K, Chalasani P, Gehlbach PL, Iordachita I, Yang GZ, Taylor RH. A Novel Semi-Autonomous Control Framework for Retina Confocal Endomicroscopy Scanning. PROCEEDINGS OF THE ... IEEE/RSJ INTERNATIONAL CONFERENCE ON INTELLIGENT ROBOTS AND SYSTEMS. IEEE/RSJ INTERNATIONAL CONFERENCE ON INTELLIGENT ROBOTS AND SYSTEMS 2019; 2019:7083-7090. [PMID: 33643680 DOI: 10.1109/iros40897.2019.8967751] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
In this paper, a novel semi-autonomous control framework is presented for enabling probe-based confocal laser endomicroscopy (pCLE) scan of the retinal tissue. With pCLE, retinal layers such as nerve fiber layer (NFL) and retinal ganglion cell (RGC) can be scanned and characterized in real-time for an improved diagnosis and surgical outcome prediction. However, the limited field of view of the pCLE system and the micron-scale optimal focus distance of the probe, which are in the order of physiological hand tremor, act as barriers to successful manual scan of retinal tissue. Therefore, a novel sensorless framework is proposed for real-time semi-autonomous endomicroscopy scanning during retinal surgery. The framework consists of the Steady-Hand Eye Robot (SHER) integrated with a pCLE system, where the motion of the probe is controlled semi-autonomously. Through a hybrid motion control strategy, the system autonomously controls the confocal probe to optimize the sharpness and quality of the pCLE images, while providing the surgeon with the ability to scan the tissue in a tremor-free manner. Effectiveness of the proposed architecture is validated through experimental evaluations as well as a user study involving 9 participants. It is shown through statistical analyses that the proposed framework can reduce the work load experienced by the users in a statistically-significant manner, while also enhancing their performance in retaining pCLE images with optimized quality.
Collapse
Affiliation(s)
- Zhaoshuo Li
- Laboratory for Computational Sensing and Robotics, Johns Hopkins University, Baltimore, Maryland 21218, USA
| | - Mahya Shahbazi
- Laboratory for Computational Sensing and Robotics, Johns Hopkins University, Baltimore, Maryland 21218, USA
| | - Niravkumar Patel
- Laboratory for Computational Sensing and Robotics, Johns Hopkins University, Baltimore, Maryland 21218, USA
| | - Eimear O' Sullivan
- Hamlyn Centre for Robotic Surgery, Imperial College London, SW7 2AZ, London, UK
| | - Haojie Zhang
- Hamlyn Centre for Robotic Surgery, Imperial College London, SW7 2AZ, London, UK
| | - Khushi Vyas
- Hamlyn Centre for Robotic Surgery, Imperial College London, SW7 2AZ, London, UK
| | - Preetham Chalasani
- Laboratory for Computational Sensing and Robotics, Johns Hopkins University, Baltimore, Maryland 21218, USA
| | - Peter L Gehlbach
- Johns Hopkins Wilmer Eye Institute, Johns Hopkins Hospital, 600 N. Wolfe Street, Maryland 21287, USA
| | - Iulian Iordachita
- Laboratory for Computational Sensing and Robotics, Johns Hopkins University, Baltimore, Maryland 21218, USA
| | - Guang-Zhong Yang
- Hamlyn Centre for Robotic Surgery, Imperial College London, SW7 2AZ, London, UK
| | - Russell H Taylor
- Laboratory for Computational Sensing and Robotics, Johns Hopkins University, Baltimore, Maryland 21218, USA
| |
Collapse
|
3
|
Troccaz J, Dagnino G, Yang GZ. Frontiers of Medical Robotics: From Concept to Systems to Clinical Translation. Annu Rev Biomed Eng 2019; 21:193-218. [DOI: 10.1146/annurev-bioeng-060418-052502] [Citation(s) in RCA: 54] [Impact Index Per Article: 10.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
Medical robotics is poised to transform all aspects of medicine—from surgical intervention to targeted therapy, rehabilitation, and hospital automation. A key area is the development of robots for minimally invasive interventions. This review provides a detailed analysis of the evolution of interventional robots and discusses how the integration of imaging, sensing, and robotics can influence the patient care pathway toward precision intervention and patient-specific treatment. It outlines how closer coupling of perception, decision, and action can lead to enhanced dexterity, greater precision, and reduced invasiveness. It provides a critical analysis of some of the key interventional robot platforms developed over the years and their relative merit and intrinsic limitations. The review also presents a future outlook for robotic interventions and emerging trends in making them easier to use, lightweight, ergonomic, and intelligent, and thus smarter, safer, and more accessible for clinical use.
Collapse
Affiliation(s)
- Jocelyne Troccaz
- Université Grenoble Alpes, CNRS, Grenoble INP, TIMC-IMAG, F-38000 Grenoble, France
| | - Giulio Dagnino
- The Hamlyn Centre for Robotic Surgery, Imperial College London, London SW7 2AZ, United Kingdom;,
| | - Guang-Zhong Yang
- The Hamlyn Centre for Robotic Surgery, Imperial College London, London SW7 2AZ, United Kingdom;,
| |
Collapse
|
4
|
Giataganas P, Hughes M, Payne CJ, Wisanuvej P, Temelkuran B, Yang GZ. Intraoperative Robotic-Assisted Large-Area High-Speed Microscopic Imaging and Intervention. IEEE Trans Biomed Eng 2018; 66:208-216. [PMID: 29993497 DOI: 10.1109/tbme.2018.2837058] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
OBJECTIVE Probe-based confocal endomicroscopy is an emerging high-magnification optical imaging technique that provides in vivo and in situ cellular-level imaging for real-time assessment of tissue pathology. Endomicroscopy could potentially be used for intraoperative surgical guidance, but it is challenging to assess a surgical site using individual microscopic images due to the limited field-of-view and difficulties associated with manually manipulating the probe. METHODS In this paper, a novel robotic device for large-area endomicroscopy imaging is proposed, demonstrating a rapid, but highly accurate, scanning mechanism with image-based motion control, which is able to generate histology-like endomicroscopy mosaics. The device also includes, for the first time in robotic-assisted endomicroscopy, the capability to ablate tissue without the need for an additional tool. RESULTS The device achieves preprogrammed trajectories with positioning accuracy of less than 30 [Formula: see text], while the image-based approach demonstrated that it can suppress random motion disturbances up to [Formula: see text]. Mosaics are presented from a range of ex vivo human and animal tissues, over areas of more than [Formula: see text], scanned in approximate [Formula: see text]. CONCLUSION This paper demonstrates the potential of the proposed instrument to generate large-area, high-resolution microscopic images for intraoperative tissue identification and margin assessment. SIGNIFICANCE This approach presents an important alternative to current histology techniques, significantly reducing the tissue assessment time, while simultaneously providing the capability to mark and ablate suspicious areas intraoperatively.
Collapse
|
5
|
Abstract
Endomicroscopy is a new technique that allows human tissue to be characterized in vivo and in situ, circumventing the need for conventional biopsy and histology. Despite increased application and growing research interests in this area, the clinical application of endomicroscopy, however, is limited by difficulties in ergonomic control, consistent probe-tissue contact, large area surveillance, and retargeting. Recently, advances in high-speed imaging, mosaicing, and robotics have aimed to address these difficulties. The development of robot-assisted devices in particular has shown great promises in extending the clinical potential of endomicroscopy. Issues related to miniaturization, adaptation to tissue deformation, control stability, force and position compensation, cost, and sterility are being pursued by both research and commercial communities. In this review, recent clinical and technical developments in different aspects of computer and robotic assisted endomicroscopy interventions including instrumentation, multiscale integration, and high-speed imaging techniques are presented. We further address emerging trends and new research opportunities toward more widespread clinical acceptance of robotically assisted endomicroscopy technologies.
Collapse
|
6
|
Feasibility and reliability of pancreatic cancer staging using a new confocal non-fluorescent microscopy probe: a double-blind study in rats. Surg Endosc 2016; 31:995-1003. [PMID: 27352785 DOI: 10.1007/s00464-016-5062-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/04/2015] [Accepted: 06/16/2016] [Indexed: 10/21/2022]
Abstract
BACKGROUND Surgical management of pancreatic cancer depends on tumor resectability and staging. Lymph node (LN) metastases represent an important decision-making factor when it comes to surgical treatment. AIMS To evaluate a new in vivo, endoscopic confocal microscopy (CM) system not requiring fluorescence markers, for detection and staging of pancreatic cancer in rats. METHODS A confocal system consisting of a confocal scanning laser operating in reflection mode and a dedicated rigid Hopkins rod-lens endoscope were used for in vivo imaging in a rat model of pancreatic ductal adenocarcinoma. A double-blind study compared CM to standard histology in (1) the detection of tumors in rat bearing cancer (n = 11) and controls (n = 6), and (2) in the detection of local nodal involvement at 3 and 6 weeks after tumor induction. RESULTS CM detected all pancreatic tumors with 100 % sensitivity and specificity and identified 15 metastatic LNs with an average adenocarcinoma nodule diameter of 2.3 mm (range from 1 to 4.2 mm) out of the 66 examined. CM demonstrated a sensitivity of 87.5 % and a specificity of 98 % in LN detection. The Spearman's rank correlation/rho calculator was of 0.87. CM demonstrated a negative predictive value of 96.1 % and a positive predictive value of 93.3 % in the detection of metastatic LNs. CONCLUSIONS Interpretation of confocal images has a high concurrence rate with histopathology examination for primary tumor and lymphatic involvement detection making it a promising technique for in vivo real-time detection and staging of pancreatic cancer. Larger studies are warranted to confirm these preliminary results.
Collapse
|
7
|
Abbaci M, Dartigues P, De Leeuw F, Soufan R, Fabre M, Laplace-Builhé C. Patent blue V and indocyanine green for fluorescence microimaging of human peritoneal carcinomatosis using probe-based confocal laser endomicroscopy. Surg Endosc 2016; 30:5255-5265. [DOI: 10.1007/s00464-016-4873-2] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/19/2016] [Accepted: 03/12/2016] [Indexed: 02/07/2023]
|
8
|
|
9
|
Hara H, Takahashi T, Nakatsuka R, Higashi S, Naka T, Sumiyama K, Miyazaki Y, Makino T, Kurokawa Y, Yamasaki M, Takiguchi S, Mori M, Doki Y, Nakajima K. A novel approach of optical biopsy using probe-based confocal laser endomicroscopy for peritoneal metastasis. Surg Endosc 2015; 30:3437-46. [DOI: 10.1007/s00464-015-4626-7] [Citation(s) in RCA: 8] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/03/2015] [Accepted: 10/16/2015] [Indexed: 12/15/2022]
|
10
|
Ye M, Giannarou S, Meining A, Yang GZ. Online tracking and retargeting with applications to optical biopsy in gastrointestinal endoscopic examinations. Med Image Anal 2015; 30:144-157. [PMID: 26970592 DOI: 10.1016/j.media.2015.10.003] [Citation(s) in RCA: 15] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/28/2015] [Revised: 09/30/2015] [Accepted: 10/02/2015] [Indexed: 10/22/2022]
Abstract
With recent advances in biophotonics, techniques such as narrow band imaging, confocal laser endomicroscopy, fluorescence spectroscopy, and optical coherence tomography, can be combined with normal white-light endoscopes to provide in vivo microscopic tissue characterisation, potentially avoiding the need for offline histological analysis. Despite the advantages of these techniques to provide online optical biopsy in situ, it is challenging for gastroenterologists to retarget the optical biopsy sites during endoscopic examinations. This is because optical biopsy does not leave any mark on the tissue. Furthermore, typical endoscopic cameras only have a limited field-of-view and the biopsy sites often enter or exit the camera view as the endoscope moves. In this paper, a framework for online tracking and retargeting is proposed based on the concept of tracking-by-detection. An online detection cascade is proposed where a random binary descriptor using Haar-like features is included as a random forest classifier. For robust retargeting, we have also proposed a RANSAC-based location verification component that incorporates shape context. The proposed detection cascade can be readily integrated with other temporal trackers. Detailed performance evaluation on in vivo gastrointestinal video sequences demonstrates the performance advantage of the proposed method over the current state-of-the-art.
Collapse
Affiliation(s)
- Menglong Ye
- The Hamlyn Centre for Robotic Surgery, Imperial College London, United Kingdom.
| | - Stamatia Giannarou
- The Hamlyn Centre for Robotic Surgery, Imperial College London, United Kingdom
| | | | - Guang-Zhong Yang
- The Hamlyn Centre for Robotic Surgery, Imperial College London, United Kingdom
| |
Collapse
|
11
|
Giataganas P, Hughes M, Yang GZ. Force adaptive robotically assisted endomicroscopy for intraoperative tumour identification. Int J Comput Assist Radiol Surg 2015; 10:825-32. [PMID: 25900339 DOI: 10.1007/s11548-015-1179-0] [Citation(s) in RCA: 17] [Impact Index Per Article: 1.9] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/20/2015] [Accepted: 03/13/2015] [Indexed: 02/02/2023]
Abstract
PURPOSE For effective tumour margin definition for cancer surgery, there is an increasing demand for the development of real-time intraoperative tissue biopsy techniques. Recent advances in miniaturized biophotonics probes have permitted the development of endomicroscopy techniques that are clinically attractive. With these approaches, cellular-level imaging can be achieved through millimetre-scale flexible probes and be performed in real-time, in vivo and in situ. Due to the limited field of view and flexibility of these probes, however, large area tissue coverage for acquiring histology-like images over complex three-dimensional surfaces is challenging. This is particularly the case because current surgical robots, such as the Da Vinci ®, lack haptic feedback, making it difficult to maintain optimum tissue contact when these probes are deployed in vivo. METHODS This paper proposes a simple force-controlled pick-up probe that can be integrated with the Da Vinci instruments for intraoperative endomicroscopy imaging. The device uses a new low-friction air bearing with adaptive axial force control to maintain constant contact between the tissue and the imaging probe, facilitating microscopy scans over complex surfaces. Detailed ex vivo user experiments have been conducted to demonstrate the effectiveness of the technique. RESULTS The adaptive probe mount could achieve consistent low-magnitude probe-sample contact forces compared with a rigid mount. In the user study, the adaptive probe combined with a high frame rate endomicroscopy system allowed larger mosaics to be generated over curved surfaces. CONCLUSIONS The device can improve the performance of large area mosaicking over complex 3D surfaces with improved handling and intraoperative control.
Collapse
Affiliation(s)
- Petros Giataganas
- Hamlyn Centre for Robotic Surgery, Institute of Global Health Innovation, Imperial College London, London, SW7 2AZ, UK,
| | | | | |
Collapse
|
12
|
Bergeles C, Yang GZ. From passive tool holders to microsurgeons: safer, smaller, smarter surgical robots. IEEE Trans Biomed Eng 2013; 61:1565-76. [PMID: 24723622 DOI: 10.1109/tbme.2013.2293815] [Citation(s) in RCA: 69] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/11/2022]
Abstract
Within only a few decades from its initial introduction, the field of surgical robotics has evolved into a dynamic and rapidly growing research area with increasing clinical uptake worldwide. Initially introduced for stereotaxic neurosurgery, surgical robots are now involved in an increasing number of procedures, demonstrating their practical clinical potential while propelling further advances in surgical innovations. Emerging platforms are also able to perform complex interventions through only a single-entry incision, and navigate through natural anatomical pathways in a tethered or wireless fashion. New devices facilitate superhuman dexterity and enable the performance of surgical steps that are otherwise impossible. They also allow seamless integration of microimaging techniques at the cellular level, significantly expanding the capabilities of surgeons. This paper provides an overview of the significant achievements in surgical robotics and identifies the current trends and future research directions of the field in making surgical robots safer, smaller, and smarter.
Collapse
|
13
|
Vitiello V, Lee SL, Cundy TP, Yang GZ. Emerging robotic platforms for minimally invasive surgery. IEEE Rev Biomed Eng 2012; 6:111-26. [PMID: 23288354 DOI: 10.1109/rbme.2012.2236311] [Citation(s) in RCA: 124] [Impact Index Per Article: 10.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
Recent technological advances in surgery have resulted in the development of a range of new techniques that have reduced patient trauma, shortened hospitalization, and improved diagnostic accuracy and therapeutic outcome. Despite the many appreciated benefits of minimally invasive surgery (MIS) compared to traditional approaches, there are still significant drawbacks associated with conventional MIS including poor instrument control and ergonomics caused by rigid instrumentation and its associated fulcrum effect. The use of robot assistance has helped to realize the full potential of MIS with improved consistency, safety and accuracy. The development of articulated, precision tools to enhance the surgeon's dexterity has evolved in parallel with advances in imaging and human-robot interaction. This has improved hand-eye coordination and manual precision down to micron scales, with the capability of navigating through complex anatomical pathways. In this review paper, clinical requirements and technical challenges related to the design of robotic platforms for flexible access surgery are discussed. Allied technical approaches and engineering challenges related to instrument design, intraoperative guidance, and intelligent human-robot interaction are reviewed. We also highlight emerging designs and research opportunities in the field by assessing the current limitations and open technical challenges for the wider clinical uptake of robotic platforms in MIS.
Collapse
|