1
|
Cannon PC, Setia SA, Klein-Gardner S, Kavoussi NL, Webster RJ, Herrell SD. Are 3D Image Guidance Systems Ready for Use? A Comparative Analysis of 3D Image Guidance Implementations in Minimally Invasive Partial Nephrectomy. J Endourol 2024; 38:395-407. [PMID: 38251637 PMCID: PMC10979686 DOI: 10.1089/end.2023.0059] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/23/2024] Open
Abstract
Introduction: Three-dimensional image-guided surgical (3D-IGS) systems for minimally invasive partial nephrectomy (MIPN) can potentially improve the efficiency and accuracy of intraoperative anatomical localization and tumor resection. This review seeks to analyze the current state of research regarding 3D-IGS, including the evaluation of clinical outcomes, system functionality, and qualitative insights regarding 3D-IGS's impact on surgical procedures. Methods: We have systematically reviewed the clinical literature pertaining to 3D-IGS deployed for MIPN. For inclusion, studies must produce a patient-specific 3D anatomical model from two-dimensional imaging. Data extracted from the studies include clinical results, registration (alignment of the 3D model to the surgical scene) method used, limitations, and data types reported. A subset of studies was qualitatively analyzed through an inductive coding approach to identify major themes and subthemes across the studies. Results: Twenty-five studies were included in the review. Eight (32%) studies reported clinical results that point to 3D-IGS improving multiple surgical outcomes. Manual registration was the most utilized (48%). Soft tissue deformation was the most cited limitation among the included studies. Many studies reported qualitative statements regarding surgeon accuracy improvement, but quantitative surgeon accuracy data were not reported. During the qualitative analysis, six major themes emerged across the nine applicable studies. They are as follows: 3D-IGS is necessary, 3D-IGS improved surgical outcomes, researcher/surgeon confidence in 3D-IGS system, enhanced surgeon ability/accuracy, anatomical explanation for qualitative assessment, and claims without data or reference to support. Conclusions: Currently, clinical outcomes are the main source of quantitative data available to point to 3D-IGS's efficacy. However, the literature qualitatively suggests the benefit of accurate 3D-IGS for robotic partial nephrectomy.
Collapse
Affiliation(s)
- Piper C. Cannon
- Department of Mechanical Engineering, Vanderbilt University, Nashville, Tennessee, USA
| | - Shaan A. Setia
- Department of Urologic Surgery, Vanderbilt University Medical Center, Nashville, Tennessee, USA
| | - Stacy Klein-Gardner
- Department of Mechanical Engineering, Vanderbilt University, Nashville, Tennessee, USA
| | - Nicholas L. Kavoussi
- Department of Urologic Surgery, Vanderbilt University Medical Center, Nashville, Tennessee, USA
| | - Robert J. Webster
- Department of Mechanical Engineering, Vanderbilt University, Nashville, Tennessee, USA
| | - S. Duke Herrell
- Department of Urologic Surgery, Vanderbilt University Medical Center, Nashville, Tennessee, USA
| |
Collapse
|
2
|
Cannon PC, Ferguson JM, Pitt EB, Shrand JA, Setia SA, Nimmagadda N, Barth EJ, Kavoussi NL, Galloway RL, Herrell SD, Webster RJ. A Safe Framework for Quantitative In Vivo Human Evaluation of Image Guidance. IEEE OPEN JOURNAL OF ENGINEERING IN MEDICINE AND BIOLOGY 2023; 5:133-139. [PMID: 38487093 PMCID: PMC10939321 DOI: 10.1109/ojemb.2023.3271853] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/30/2022] [Revised: 02/16/2023] [Accepted: 03/27/2023] [Indexed: 03/17/2024] Open
Abstract
Goal: We present a new framework for in vivo image guidance evaluation and provide a case study on robotic partial nephrectomy. Methods: This framework (called the "bystander protocol") involves two surgeons, one who solely performs the therapeutic process without image guidance, and another who solely periodically collects data to evaluate image guidance. This isolates the evaluation from the therapy, so that in-development image guidance systems can be tested without risk of negatively impacting the standard of care. We provide a case study applying this protocol in clinical cases during robotic partial nephrectomy surgery. Results: The bystander protocol was performed successfully in 6 patient cases. We find average lesion centroid localization error with our IGS system to be 6.5 mm in vivo compared to our prior result of 3.0 mm in phantoms. Conclusions: The bystander protocol is a safe, effective method for testing in-development image guidance systems in human subjects.
Collapse
Affiliation(s)
| | | | | | | | | | - Naren Nimmagadda
- Vanderbilt University Medical CenterNashvilleTN37232USA
- The Johns Hopkins University School of MedicineBaltimoreMD21287USA
| | | | | | | | | | | |
Collapse
|
3
|
A survey of augmented reality methods to guide minimally invasive partial nephrectomy. World J Urol 2023; 41:335-343. [PMID: 35776173 DOI: 10.1007/s00345-022-04078-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/15/2021] [Accepted: 05/21/2022] [Indexed: 10/17/2022] Open
Abstract
INTRODUCTION Minimally invasive partial nephrectomy (MIPN) has become the standard of care for localized kidney tumors over the past decade. The characteristics of each tumor, in particular its size and relationship with the excretory tract and vessels, allow one to judge its complexity and to attempt predicting the risk of complications. The recent development of virtual 3D model reconstruction and computer vision has opened the way to image-guided surgery and augmented reality (AR). OBJECTIVE Our objective was to perform a systematic review to list and describe the different AR techniques proposed to support PN. MATERIALS AND METHODS The systematic review of the literature was performed on 12/04/22, using the keywords "nephrectomy" and "augmented reality" on Embase and Medline. Articles were considered if they reported surgical outcomes when using AR with virtual image overlay on real vision, during ex vivo or in vivo MIPN. We classified them according to the registration technique they use. RESULTS We found 16 articles describing an AR technique during MIPN procedures that met the eligibility criteria. A moderate to high risk of bias was recorded for all the studies. We classified registration methods into three main families, of which the most promising one seems to be surface-based registration. CONCLUSION Despite promising results, there do not exist studies showing an improvement in clinical outcomes using AR. The ideal AR technique is probably yet to be established, as several designs are still being actively explored. More clinical data will be required to establish the potential contribution of this technology to MIPN.
Collapse
|
4
|
Chandelon K, Sharifian R, Marchand S, Khaddad A, Bourdel N, Mottet N, Bernhard JC, Bartoli A. Kidney tracking for live augmented reality in stereoscopic mini-invasive partial nephrectomy. COMPUTER METHODS IN BIOMECHANICS AND BIOMEDICAL ENGINEERING: IMAGING & VISUALIZATION 2022. [DOI: 10.1080/21681163.2022.2157750] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/23/2022]
Affiliation(s)
- Kilian Chandelon
- Institut Pascal, Clermont-Ferrand University Hospital, Clermont-Ferrand, France
- SurgAR - Surgical Augmented Reality, Clermont-Ferrand, France
| | - Rasoul Sharifian
- Institut Pascal, Clermont-Ferrand University Hospital, Clermont-Ferrand, France
| | - Salomé Marchand
- Department of Urology, Hôpital Nord, Saint-Etienne University Hospital, Saint-Etienne, France
| | - Abderrahmane Khaddad
- Department of Urology, Hôpital Pellegrin, Bordeaux University Hospital, Bordeaux, France
| | - Nicolas Bourdel
- Institut Pascal, Clermont-Ferrand University Hospital, Clermont-Ferrand, France
- SurgAR - Surgical Augmented Reality, Clermont-Ferrand, France
- Department of Obstetrics and Gynecology, Clermont-Ferrand University Hospital, Clermont-Ferrand, France
| | - Nicolas Mottet
- Department of Urology, Hôpital Nord, Saint-Etienne University Hospital, Saint-Etienne, France
| | | | - Adrien Bartoli
- Institut Pascal, Clermont-Ferrand University Hospital, Clermont-Ferrand, France
- SurgAR - Surgical Augmented Reality, Clermont-Ferrand, France
- Department of Clinical Research and Innovation, Clermont-Ferrand University Hospital, Clermont-Ferrand, France
| |
Collapse
|
5
|
Kokko MA, Van Citters DW, Seigne JD, Halter RJ. A particle filter approach to dynamic kidney pose estimation in robotic surgical exposure. Int J Comput Assist Radiol Surg 2022; 17:1079-1089. [PMID: 35511394 DOI: 10.1007/s11548-022-02638-8] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/14/2021] [Accepted: 04/08/2022] [Indexed: 11/27/2022]
Abstract
PURPOSE Traditional soft tissue registration methods require direct intraoperative visualization of a significant portion of the target anatomy in order to produce acceptable surface alignment. Image guidance is therefore generally not available during the robotic exposure of structures like the kidneys which are not immediately visualized upon entry into the abdomen. This paper proposes guiding surgical exposure with an iterative state estimator that assimilates small visual cues into an a priori anatomical model as exposure progresses, thereby evolving pose estimates for the occluded structures of interest. METHODS Intraoperative surface observations of a right kidney are simulated using endoscope tracking and preoperative tomography from a representative robotic partial nephrectomy case. Clinically relevant random perturbations of the true kidney pose are corrected using this sequence of observations in a particle filter framework to estimate an optimal similarity transform for fitting a patient-specific kidney model at each step. The temporal response of registration error is compared against that of serial rigid coherent point drift (CPD) in both static and simulated dynamic surgical fields, and for varying levels of observation persistence. RESULTS In the static case, both particle filtering and persistent CPD achieved sub-5 mm accuracy, with CPD processing observations 75% faster. Particle filtering outperformed CPD in the dynamic case under equivalent computation times due to the former requiring only minimal persistence. CONCLUSION This proof-of-concept simulation study suggests that Bayesian state estimation may provide a viable pathway to image guidance for surgical exposure in the abdomen, especially in the presence of dynamic intraoperative tissue displacement and deformation.
Collapse
Affiliation(s)
- Michael A Kokko
- Thayer School of Engineering, Dartmouth College, 15 Thayer Drive, Hanover, NH, 03755, USA.
| | - Douglas W Van Citters
- Thayer School of Engineering, Dartmouth College, 15 Thayer Drive, Hanover, NH, 03755, USA
| | - John D Seigne
- Dartmouth-Hitchcock Medical Center, Section of Urology, 1 Medical Center Drive, Lebanon, NH, 03756, USA.,Geisel School of Medicine, Dartmouth College, 1 Rope Ferry Road, Hanover, NH, 03755, USA
| | - Ryan J Halter
- Thayer School of Engineering, Dartmouth College, 15 Thayer Drive, Hanover, NH, 03755, USA.,Geisel School of Medicine, Dartmouth College, 1 Rope Ferry Road, Hanover, NH, 03755, USA
| |
Collapse
|
6
|
Laparoscopic augmented reality registration for oncological resection site repair. Int J Comput Assist Radiol Surg 2021; 16:1577-1586. [PMID: 33797689 PMCID: PMC8354909 DOI: 10.1007/s11548-021-02336-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/29/2020] [Accepted: 02/25/2021] [Indexed: 10/31/2022]
Abstract
PURPOSE Resection site repair during laparoscopic oncological surgery (e.g. laparoscopic partial nephrectomy) poses some unique challenges and opportunities for augmented reality (AR) navigation support. This work introduces an AR registration workflow that addresses the time pressure that is present during resection site repair. METHODS We propose a two-step registration process: the AR content is registered as accurately as possible prior to the tumour resection (the primary registration). This accurate registration is used to apply artificial fiducials to the physical organ and the virtual model. After the resection, these fiducials can be used for rapid re-registration (the secondary registration). We tested this pipeline in a simulated-use study with [Formula: see text] participants. We compared the registration accuracy and speed for our method and for landmark-based registration as a reference. RESULTS Acquisition of and, thereby, registration with the artificial fiducials were significantly faster than the initial use of anatomical landmarks. Our method also had a trend to be more accurate in cases in which the primary registration was successful. The accuracy loss between the elaborate primary registration and the rapid secondary registration could be quantified with a mean target registration error increase of 2.35 mm. CONCLUSION This work introduces a registration pipeline for AR navigation support during laparoscopic resection site repair and provides a successful proof-of-concept evaluation thereof. Our results indicate that the concept is better suited than landmark-based registration during this phase, but further work is required to demonstrate clinical suitability and applicability.
Collapse
|
7
|
Kavoussi NL, Pitt B, Ferguson JM, Granna J, Remirez A, Nimmagadda N, Melnyk R, Ghazi A, Barth EJ, Webster RJ, Herrell SD. Accuracy of Touch-Based Registration During Robotic Image-Guided Partial Nephrectomy Before and After Tumor Resection in Validated Phantoms. J Endourol 2021; 35:362-368. [PMID: 33040602 PMCID: PMC7987368 DOI: 10.1089/end.2020.0363] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/12/2022] Open
Abstract
Aim: Image-guided surgery (IGS) allows for accurate, real-time localization of subsurface critical structures during surgery. No prior IGS systems have described a feasible method of intraoperative reregistration after manipulation of the kidney during robotic partial nephrectomy (PN). We present a method for seamless reregistration during IGS and evaluate accuracy before and after tumor resection in two validated kidney phantoms. Materials and Methods: We performed robotic PN on two validated kidney phantoms-one with an endophytic tumor and one with an exophytic tumor-with our IGS system utilizing the da Vinci Xi robot. Intraoperatively, the kidney phantoms' surfaces were digitized with the da Vinci robotic manipulator via a touch-based method and registered to a three-dimensional segmented model created from cross-sectional CT imaging of the phantoms. Fiducial points were marked with a surgical marking pen and identified after the initial registration using the robotic manipulator. Segmented images were displayed via picture-in-picture in the surgeon console as tumor resection was performed. After resection, reregistration was performed by reidentifying the fiducial points. The accuracy of the initial registration and reregistration was compared. Results: The root mean square (RMS) averages of target registration error (TRE) were 2.53 and 4.88 mm for the endophytic and exophytic phantoms, respectively. IGS enabled resection along preplanned contours. Specifically, the RMS averages of the normal TRE over the entire resection surface were 0.75 and 2.15 mm for the endophytic and exophytic phantoms, respectively. Both tumors were resected with grossly negative margins. Point-based reregistration enabled instantaneous reregistration with minimal impact on RMS TRE compared with the initial registration (from 1.34 to 1.70 mm preresection and from 1.60 to 2.10 mm postresection). Conclusions: We present a novel and accurate registration and reregistration framework for use during IGS for PN with the da Vinci Xi surgical system. The technology is easily integrated into the surgical workflow and does not require additional hardware.
Collapse
Affiliation(s)
- Nicholas L. Kavoussi
- Department of Urology, Vanderbilt University Medical Center, Nashville, Tennessee, USA
| | - Bryn Pitt
- Department of Mechanical Engineering, School of Engineering, Vanderbilt University, Nashville, Tennessee, USA
| | - James M. Ferguson
- Department of Mechanical Engineering, School of Engineering, Vanderbilt University, Nashville, Tennessee, USA
| | - Josephine Granna
- Department of Mechanical Engineering, School of Engineering, Vanderbilt University, Nashville, Tennessee, USA
| | - Andria Remirez
- Department of Mechanical Engineering, School of Engineering, Vanderbilt University, Nashville, Tennessee, USA
| | - Naren Nimmagadda
- Department of Urology, Vanderbilt University Medical Center, Nashville, Tennessee, USA
| | - Rachel Melnyk
- Simulation Innovation Laboratory, Department of Urology, University of Rochester, Rochester, New York, USA
| | - Ahmed Ghazi
- Department of Urology, University of Rochester Medical Center, Rochester, New York, USA
| | - Eric J. Barth
- Department of Mechanical Engineering, School of Engineering, Vanderbilt University, Nashville, Tennessee, USA
| | - Robert J. Webster
- Department of Mechanical Engineering, School of Engineering, Vanderbilt University, Nashville, Tennessee, USA
| | - Stanley Duke Herrell
- Department of Urology, Vanderbilt University Medical Center, Nashville, Tennessee, USA
| |
Collapse
|
8
|
Collins T, Pizarro D, Gasparini S, Bourdel N, Chauvet P, Canis M, Calvet L, Bartoli A. Augmented Reality Guided Laparoscopic Surgery of the Uterus. IEEE TRANSACTIONS ON MEDICAL IMAGING 2021; 40:371-380. [PMID: 32986548 DOI: 10.1109/tmi.2020.3027442] [Citation(s) in RCA: 24] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/02/2023]
Abstract
A major research area in Computer Assisted Intervention (CAI) is to aid laparoscopic surgery teams with Augmented Reality (AR) guidance. This involves registering data from other modalities such as MR and fusing it with the laparoscopic video in real-time, to reveal the location of hidden critical structures. We present the first system for AR guided laparoscopic surgery of the uterus. This works with pre-operative MR or CT data and monocular laparoscopes, without requiring any additional interventional hardware such as optical trackers. We present novel and robust solutions to two main sub-problems: the initial registration, which is solved using a short exploratory video, and update registration, which is solved with real-time tracking-by-detection. These problems are challenging for the uterus because it is a weakly-textured, highly mobile organ that moves independently of surrounding structures. In the broader context, our system is the first that has successfully performed markerless real-time registration and AR of a mobile human organ with monocular laparoscopes in the OR.
Collapse
|
9
|
Schiavina R, Bianchi L, Chessa F, Barbaresi U, Cercenelli L, Lodi S, Gaudiano C, Bortolani B, Angiolini A, Bianchi FM, Ercolino A, Casablanca C, Molinaroli E, Porreca A, Golfieri R, Diciotti S, Marcelli E, Brunocilla E. Augmented Reality to Guide Selective Clamping and Tumor Dissection During Robot-assisted Partial Nephrectomy: A Preliminary Experience. Clin Genitourin Cancer 2020; 19:e149-e155. [PMID: 33060033 DOI: 10.1016/j.clgc.2020.09.005] [Citation(s) in RCA: 21] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/21/2020] [Revised: 08/31/2020] [Accepted: 09/07/2020] [Indexed: 12/11/2022]
Affiliation(s)
- Riccardo Schiavina
- Department of Urology, University of Bologna, Azienda Ospedaliero, Universitaria di Bologna, Bologna, Italia; Department of Experimental, Diagnostic, and Specialty Medicine (DIMES), Cardio-Nephro-Thoracic Sciences Doctorate, University of Bologna, Bologna, Italy
| | - Lorenzo Bianchi
- Department of Urology, University of Bologna, Azienda Ospedaliero, Universitaria di Bologna, Bologna, Italia; Department of Experimental, Diagnostic, and Specialty Medicine (DIMES), Cardio-Nephro-Thoracic Sciences Doctorate, University of Bologna, Bologna, Italy.
| | - Francesco Chessa
- Department of Urology, University of Bologna, Azienda Ospedaliero, Universitaria di Bologna, Bologna, Italia; Department of Experimental, Diagnostic, and Specialty Medicine (DIMES), Cardio-Nephro-Thoracic Sciences Doctorate, University of Bologna, Bologna, Italy
| | - Umberto Barbaresi
- Department of Urology, University of Bologna, Azienda Ospedaliero, Universitaria di Bologna, Bologna, Italia
| | - Laura Cercenelli
- Department of Experimental, Diagnostic and Specialty Medicine (DIMES), Laboratory of Bioengineering, University of Bologna, Bologna, Italy; Department of Biomedical and Neuromotor Sciences (DIBINEM), University of Bologna, Bologna, Italy
| | - Simone Lodi
- Department of Electrical, Electronic, and Information Engineering, "Guglielmo Marconi," University of Bologna, Bologna, Italy
| | - Caterina Gaudiano
- Radiology Unit, Department of Diagnostic Medicine and Prevention, Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italia
| | - Barbara Bortolani
- Department of Experimental, Diagnostic and Specialty Medicine (DIMES), Laboratory of Bioengineering, University of Bologna, Bologna, Italy
| | - Andrea Angiolini
- Department of Urology, University of Bologna, Azienda Ospedaliero, Universitaria di Bologna, Bologna, Italia
| | - Federico Mineo Bianchi
- Department of Urology, University of Bologna, Azienda Ospedaliero, Universitaria di Bologna, Bologna, Italia
| | - Amelio Ercolino
- Department of Urology, University of Bologna, Azienda Ospedaliero, Universitaria di Bologna, Bologna, Italia
| | - Carlo Casablanca
- Department of Urology, University of Bologna, Azienda Ospedaliero, Universitaria di Bologna, Bologna, Italia
| | - Enrico Molinaroli
- Department of Urology, University of Bologna, Azienda Ospedaliero, Universitaria di Bologna, Bologna, Italia
| | - Angelo Porreca
- Department of Urology, Abano Terme Hospital, Padua, Italy
| | - Rita Golfieri
- Radiology Unit, Department of Diagnostic Medicine and Prevention, Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italia
| | - Stefano Diciotti
- Department of Electrical, Electronic, and Information Engineering, "Guglielmo Marconi," University of Bologna, Bologna, Italy
| | - Emanuela Marcelli
- Department of Experimental, Diagnostic and Specialty Medicine (DIMES), Laboratory of Bioengineering, University of Bologna, Bologna, Italy
| | - Eugenio Brunocilla
- Department of Urology, University of Bologna, Azienda Ospedaliero, Universitaria di Bologna, Bologna, Italia; Department of Experimental, Diagnostic, and Specialty Medicine (DIMES), Cardio-Nephro-Thoracic Sciences Doctorate, University of Bologna, Bologna, Italy
| |
Collapse
|
10
|
Ferguson JM, Pitt B, Kuntz A, Granna J, Kavoussi NL, Nimmagadda N, Barth EJ, Herrell SD, Webster RJ. Comparing the accuracy of the da Vinci Xi and da Vinci Si for image guidance and automation. Int J Med Robot 2020; 16:1-10. [DOI: 10.1002/rcs.2149] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/07/2020] [Revised: 08/06/2020] [Accepted: 08/11/2020] [Indexed: 12/19/2022]
Affiliation(s)
- James M. Ferguson
- Department of Mechanical Engineering Vanderbilt University Nashville Tennessee USA
- Vanderbilt Institute for Surgery and Engineering (VISE) Nashville Tennessee USA
| | - Bryn Pitt
- Department of Mechanical Engineering Vanderbilt University Nashville Tennessee USA
- Vanderbilt Institute for Surgery and Engineering (VISE) Nashville Tennessee USA
| | - Alan Kuntz
- Robotics Center and School of Computing, University of Utah Salt Lake City Utah USA
| | - Josephine Granna
- Department of Mechanical Engineering Vanderbilt University Nashville Tennessee USA
- Vanderbilt Institute for Surgery and Engineering (VISE) Nashville Tennessee USA
| | - Nicholas L. Kavoussi
- Vanderbilt Institute for Surgery and Engineering (VISE) Nashville Tennessee USA
- Vanderbilt University Medical Center Nashville Tennessee USA
| | - Naren Nimmagadda
- Vanderbilt Institute for Surgery and Engineering (VISE) Nashville Tennessee USA
- Vanderbilt University Medical Center Nashville Tennessee USA
| | - Eric J. Barth
- Department of Mechanical Engineering Vanderbilt University Nashville Tennessee USA
- Vanderbilt Institute for Surgery and Engineering (VISE) Nashville Tennessee USA
| | - Stanley Duke Herrell
- Vanderbilt Institute for Surgery and Engineering (VISE) Nashville Tennessee USA
- Vanderbilt University Medical Center Nashville Tennessee USA
| | - Robert J. Webster
- Department of Mechanical Engineering Vanderbilt University Nashville Tennessee USA
- Vanderbilt Institute for Surgery and Engineering (VISE) Nashville Tennessee USA
- Vanderbilt University Medical Center Nashville Tennessee USA
| |
Collapse
|
11
|
Gribaudo M, Piazzolla P, Porpiglia F, Vezzetti E, Violante MG. 3D augmentation of the surgical video stream: Toward a modular approach. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2020; 191:105505. [PMID: 32387863 DOI: 10.1016/j.cmpb.2020.105505] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/04/2019] [Revised: 03/29/2020] [Accepted: 04/08/2020] [Indexed: 06/11/2023]
Abstract
BACKGROUND AND OBJECTIVE We present an original approach to the development of augmented reality (AR) real-time solutions for robotic surgery navigation. The surgeon operating the robotic system through a console and a visor experiences reduced awareness of the operatory scene. In order to improve the surgeon's spatial perception during robot-assisted minimally invasive procedures, we provide him/her with a solid automatic software system to position, rotate and scale in real-time the 3D virtual model of a patient's organ aligned over its image captured by the endoscope. METHODS We observed that the surgeon may benefit differently from the 3D augmentation during each stage of the surgical procedure; moreover, each stage may present different visual elements that provide specific challenges and opportunities to exploit for organ detection strategies implementation. Hence we integrate different solutions, each dedicated to a specific stage of the surgical procedure, into a single software system. RESULTS We present a formal model that generalizes our approach, describing a system composed of integrated solutions for AR in robot-assisted surgery. Following the proposed framework, and application has been developed which is currently used during in vivo surgery, for extensive testing, by the Urology unity of the San Luigi Hospital, in Orbassano (To), Italy. CONCLUSIONS The main contribution of this paper is in presenting a modular approach to the tracking problem during in-vivo robotic surgery, whose efficacy from a medical point of view has been assessed in cited works. The segmentation of the whole procedure in a set of stages allows associating the best tracking strategy to each of them, as well as to re-utilize implemented software mechanisms in stages with similar features.
Collapse
Affiliation(s)
- Marco Gribaudo
- Dept. of Electronics, Information and Bioengineering, Politecnico di Milano, Milano, Italy
| | - Pietro Piazzolla
- Dept. of Management and Production Engineering, Politecnico di Torino, Torino, Italy.
| | - Francesco Porpiglia
- Division of Urology, Department of Oncology, School of Medicine, University of Turin, Italy
| | - Enrico Vezzetti
- Dept. of Management and Production Engineering, Politecnico di Torino, Torino, Italy
| | - Maria Grazia Violante
- Dept. of Management and Production Engineering, Politecnico di Torino, Torino, Italy
| |
Collapse
|
12
|
Ferguson JM, Pitt EB, Remirez AA, Siebold MA, Kuntz A, Kavoussi NL, Barth EJ, Herrell SD, Webster RJ. Toward Practical and Accurate Touch-Based Image Guidance for Robotic Partial Nephrectomy. ACTA ACUST UNITED AC 2020; 2:196-205. [DOI: 10.1109/tmrb.2020.2989661] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
|
13
|
Stewart S, Darr M, Gomer H, Smith A, Samiei A, Post JC, Miller RJ, Lyne J, Cohen J, Treado PJ. Visible near infrared reflectance molecular chemical imaging of human ex vivo carcinomas and murine in vivo carcinomas. JOURNAL OF BIOMEDICAL OPTICS 2020; 25:1-18. [PMID: 32096369 PMCID: PMC7039338 DOI: 10.1117/1.jbo.25.2.026003] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/30/2019] [Accepted: 01/27/2020] [Indexed: 05/23/2023]
Abstract
SIGNIFICANCE A key risk faced by oncological surgeons continues to be complete removal of tumor. Currently, there is no intraoperative imaging device to detect kidney tumors during excision. AIM We are evaluating molecular chemical imaging (MCI) as a technology for real-time tumor detection and margin assessment during tumor removal surgeries. APPROACH In exploratory studies, we evaluate visible near infrared (Vis-NIR) MCI for differentiating tumor from adjacent tissue in ex vivo human kidney specimens, and in anaesthetized mice with breast or lung tumor xenografts. Differentiation of tumor from nontumor tissues is made possible with diffuse reflectance spectroscopic signatures and hyperspectral imaging technology. Tumor detection is achieved by score image generation to localize the tumor, followed by application of computer vision algorithms to define tumor border. RESULTS Performance of a partial least squares discriminant analysis (PLS-DA) model for kidney tumor in a 22-patient study is 0.96 for area under the receiver operating characteristic curve. A PLS-DA model for in vivo breast and lung tumor xenografts performs with 100% sensitivity, 83% specificity, and 89% accuracy. CONCLUSION Detection of cancer in surgically resected human kidney tissues is demonstrated ex vivo with Vis-NIR MCI, and in vivo on mice with breast or lung xenografts.
Collapse
Affiliation(s)
- Shona Stewart
- ChemImage Corporation, Pittsburgh, Pennsylvania, United States
| | - Marlena Darr
- ChemImage Corporation, Pittsburgh, Pennsylvania, United States
| | - Heather Gomer
- ChemImage Corporation, Pittsburgh, Pennsylvania, United States
| | - Aaron Smith
- ChemImage Corporation, Pittsburgh, Pennsylvania, United States
| | - Arash Samiei
- ChemImage Corporation, Pittsburgh, Pennsylvania, United States
- Allegheny General Hospital, Pittsburgh, Pennsylvania, United States
| | | | - Ralph J. Miller
- Allegheny General Hospital, Pittsburgh, Pennsylvania, United States
| | - John Lyne
- Allegheny General Hospital, Pittsburgh, Pennsylvania, United States
| | - Jeffrey Cohen
- ChemImage Corporation, Pittsburgh, Pennsylvania, United States
- Allegheny General Hospital, Pittsburgh, Pennsylvania, United States
| | | |
Collapse
|
14
|
Joeres F, Schindele D, Luz M, Blaschke S, Russwinkel N, Schostak M, Hansen C. How well do software assistants for minimally invasive partial nephrectomy meet surgeon information needs? A cognitive task analysis and literature review study. PLoS One 2019; 14:e0219920. [PMID: 31318919 PMCID: PMC6638947 DOI: 10.1371/journal.pone.0219920] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/18/2019] [Accepted: 07/04/2019] [Indexed: 12/30/2022] Open
Abstract
INTRODUCTION Intraoperative software assistance is gaining increasing importance in laparoscopic and robot-assisted surgery. Within the user-centred development process of such systems, the first question to be asked is: What information does the surgeon need and when does he or she need it? In this article, we present an approach to investigate these surgeon information needs for minimally invasive partial nephrectomy and compare these needs to the relevant surgical computer assistance literature. MATERIALS AND METHODS First, we conducted a literature-based hierarchical task analysis of the surgical procedure. This task analysis was taken as a basis for a qualitative in-depth interview study with nine experienced surgical urologists. The study employed a cognitive task analysis method to elicit surgeons' information needs during minimally invasive partial nephrectomy. Finally, a systematic literature search was conducted to review proposed software assistance solutions for minimally invasive partial nephrectomy. The review focused on what information the solutions present to the surgeon and what phase of the surgery they aim to support. RESULTS The task analysis yielded a workflow description for minimally invasive partial nephrectomy. During the subsequent interview study, we identified three challenging phases of the procedure, which may particularly benefit from software assistance. These phases are I. Hilar and vascular management, II. Tumour excision, and III. Repair of the renal defects. Between these phases, 25 individual challenges were found which define the surgeon information needs. The literature review identified 34 relevant publications, all of which aim to support the surgeon in hilar and vascular management (phase I) or tumour excision (phase II). CONCLUSION The work presented in this article identified unmet surgeon information needs in minimally invasive partial nephrectomy. Namely, our results suggest that future solutions should address the repair of renal defects (phase III) or put more focus on the renal collecting system as a critical anatomical structure.
Collapse
Affiliation(s)
- Fabian Joeres
- Department of Simulation and Graphics, Faculty of Computer Science, Otto von Guericke University Magdeburg, Magdeburg, Germany
| | - Daniel Schindele
- Clinic of Urology and Paediatric Urology, University Hospital of Magdeburg, Magdeburg, Germany
| | - Maria Luz
- Department of Simulation and Graphics, Faculty of Computer Science, Otto von Guericke University Magdeburg, Magdeburg, Germany
| | - Simon Blaschke
- Clinic of Urology and Paediatric Urology, University Hospital of Magdeburg, Magdeburg, Germany
| | - Nele Russwinkel
- Department of Cognitive Modelling in Dynamic Human-Machine Systems, Technische Universität Berlin, Berlin, Germany
| | - Martin Schostak
- Clinic of Urology and Paediatric Urology, University Hospital of Magdeburg, Magdeburg, Germany
| | - Christian Hansen
- Department of Simulation and Graphics, Faculty of Computer Science, Otto von Guericke University Magdeburg, Magdeburg, Germany
| |
Collapse
|
15
|
Kahng PW, Wu X, Ramesh NP, Pastel DA, Halter RJ, Paydarfar JA. Improving target localization during trans-oral surgery with use of intraoperative imaging. Int J Comput Assist Radiol Surg 2019; 14:885-893. [DOI: 10.1007/s11548-018-01907-9] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/03/2018] [Accepted: 12/26/2018] [Indexed: 10/27/2022]
|
16
|
Intraoperative Imaging Techniques to Support Complete Tumor Resection in Partial Nephrectomy. Eur Urol Focus 2018; 4:960-968. [DOI: 10.1016/j.euf.2017.04.008] [Citation(s) in RCA: 44] [Impact Index Per Article: 7.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/03/2017] [Accepted: 04/29/2017] [Indexed: 12/22/2022]
|
17
|
Griesenauer RH, Weis JA, Arlinghaus LR, Meszoely IM, Miga MI. Toward quantitative quasistatic elastography with a gravity-induced deformation source for image-guided breast surgery. J Med Imaging (Bellingham) 2018; 5:015003. [PMID: 29430479 DOI: 10.1117/1.jmi.5.1.015003] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/27/2017] [Accepted: 01/15/2018] [Indexed: 11/14/2022] Open
Abstract
Biomechanical breast models have been employed for applications in image registration and diagnostic analysis, breast augmentation simulation, and for surgical and biopsy guidance. Accurate applications of stress-strain relationships of tissue within the breast can improve the accuracy of biomechanical models that attempt to simulate breast deformations. Reported stiffness values for adipose, glandular, and cancerous tissue types vary greatly. Variations in reported stiffness properties have been attributed to differences in testing methodologies and assumptions, measurement errors, and natural interpatient differences in tissue elasticity. Therefore, the ability to determine patient-specific in vivo breast tissue properties would be advantageous for these procedural applications. While some in vivo elastography methods are not quantitative and others do not measure material properties under deformation conditions that are appropriate to the application of concern, in this study, we developed an elasticity estimation method that is performed using deformations representative of supine therapeutic procedures. More specifically, reconstruction of mechanical properties appropriate for the standard-of-care supine lumpectomy was performed by iteratively fitting two anatomical images before and after deformations taking place in the supine breast configuration. The method proposed is workflow-friendly, quantitative, and uses a noncontact, gravity-induced deformation source.
Collapse
Affiliation(s)
- Rebekah H Griesenauer
- Vanderbilt University, Department of Biomedical Engineering, Nashville, Tennessee, United States
| | - Jared A Weis
- Wake Forest School of Medicine, Department of Biomedical Engineering, Winston Salem, North Carolina, United States
| | - Lori R Arlinghaus
- Vanderbilt University Medical Center, Vanderbilt University Institute of Imaging Science, Nashville, Tennessee, United States
| | - Ingrid M Meszoely
- Vanderbilt University Medical Center, Department of Surgery, Nashville, Tennessee, United States
| | - Michael I Miga
- Vanderbilt University, Department of Biomedical Engineering, Nashville, Tennessee, United States.,Vanderbilt University Medical Center, Vanderbilt University Institute of Imaging Science, Nashville, Tennessee, United States.,Vanderbilt University, Vanderbilt Institute for Surgery and Engineering, Nashville, Tennessee, United States.,Vanderbilt University Medical Center, Department of Radiology and Radiological Sciences, Nashville, Tennessee, United States.,Vanderbilt University Medical Center, Department of Neurological Surgery, Nashville, Tennessee, United States
| |
Collapse
|
18
|
Detmer FJ, Hettig J, Schindele D, Schostak M, Hansen C. Virtual and Augmented Reality Systems for Renal Interventions: A Systematic Review. IEEE Rev Biomed Eng 2017; 10:78-94. [PMID: 28885161 DOI: 10.1109/rbme.2017.2749527] [Citation(s) in RCA: 28] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/05/2022]
Abstract
PURPOSE Many virtual and augmented reality systems have been proposed to support renal interventions. This paper reviews such systems employed in the treatment of renal cell carcinoma and renal stones. METHODS A systematic literature search was performed. Inclusion criteria were virtual and augmented reality systems for radical or partial nephrectomy and renal stone treatment, excluding systems solely developed or evaluated for training purposes. RESULTS In total, 52 research papers were identified and analyzed. Most of the identified literature (87%) deals with systems for renal cell carcinoma treatment. About 44% of the systems have already been employed in clinical practice, but only 20% in studies with ten or more patients. Main challenges remaining for future research include the consideration of organ movement and deformation, human factor issues, and the conduction of large clinical studies. CONCLUSION Augmented and virtual reality systems have the potential to improve safety and outcomes of renal interventions. In the last ten years, many technical advances have led to more sophisticated systems, which are already applied in clinical practice. Further research is required to cope with current limitations of virtual and augmented reality assistance in clinical environments.
Collapse
|
19
|
Augmented reality in a tumor resection model. Surg Endosc 2017; 32:1192-1201. [DOI: 10.1007/s00464-017-5791-7] [Citation(s) in RCA: 13] [Impact Index Per Article: 1.9] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/31/2017] [Accepted: 07/28/2017] [Indexed: 01/20/2023]
|
20
|
The status of augmented reality in laparoscopic surgery as of 2016. Med Image Anal 2017; 37:66-90. [DOI: 10.1016/j.media.2017.01.007] [Citation(s) in RCA: 183] [Impact Index Per Article: 26.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/10/2016] [Revised: 01/16/2017] [Accepted: 01/23/2017] [Indexed: 12/27/2022]
|
21
|
Abstract
The last two decades witnessed the inception and exponential implementation of key technological advancements in laparoscopic urology. While some of these technologies thrived and became part of daily practice, others are still hindered by major challenges. This review was conducted through a comprehensive literature search in order to highlight some of the most promising technologies in laparoscopic visualization, augmented reality, and insufflation. Additionally, this review will provide an update regarding the current status of single-site and natural orifice surgery in urology.
Collapse
|
22
|
Kassab GS, An G, Sander EA, Miga MI, Guccione JM, Ji S, Vodovotz Y. Augmenting Surgery via Multi-scale Modeling and Translational Systems Biology in the Era of Precision Medicine: A Multidisciplinary Perspective. Ann Biomed Eng 2016; 44:2611-25. [PMID: 27015816 DOI: 10.1007/s10439-016-1596-4] [Citation(s) in RCA: 14] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/01/2015] [Accepted: 03/18/2016] [Indexed: 12/18/2022]
Abstract
In this era of tremendous technological capabilities and increased focus on improving clinical outcomes, decreasing costs, and increasing precision, there is a need for a more quantitative approach to the field of surgery. Multiscale computational modeling has the potential to bridge the gap to the emerging paradigms of Precision Medicine and Translational Systems Biology, in which quantitative metrics and data guide patient care through improved stratification, diagnosis, and therapy. Achievements by multiple groups have demonstrated the potential for (1) multiscale computational modeling, at a biological level, of diseases treated with surgery and the surgical procedure process at the level of the individual and the population; along with (2) patient-specific, computationally-enabled surgical planning, delivery, and guidance and robotically-augmented manipulation. In this perspective article, we discuss these concepts, and cite emerging examples from the fields of trauma, wound healing, and cardiac surgery.
Collapse
Affiliation(s)
- Ghassan S Kassab
- California Medical Innovations Institute, San Diego, CA, 92121, USA
| | - Gary An
- Department of Surgery, University of Chicago, Chicago, IL, 60637, USA
| | - Edward A Sander
- Department of Biomedical Engineering, University of Iowa, Iowa City, IA, 52242, USA
| | - Michael I Miga
- Department of Biomedical Engineering, Vanderbilt University, Nashville, TN, 37235, USA
| | - Julius M Guccione
- Department of Surgery, University of California, San Francisco, CA, 94143, USA
| | - Songbai Ji
- Thayer School of Engineering, Dartmouth College, Hanover, NH, 03755, USA.,Department of Surgery and of Orthopaedic Surgery, Geisel School of Medicine, Dartmouth College, Hanover, NH, 03755, USA
| | - Yoram Vodovotz
- Department of Surgery, University of Pittsburgh, W944 Starzl Biomedical Sciences Tower, 200 Lothrop St., Pittsburgh, PA, 15213, USA. .,Center for Inflammation and Regenerative Modeling, McGowan Institute for Regenerative Medicine, University of Pittsburgh, Pittsburgh, PA, 15219, USA.
| |
Collapse
|
23
|
Computational Modeling for Enhancing Soft Tissue Image Guided Surgery: An Application in Neurosurgery. Ann Biomed Eng 2015; 44:128-38. [PMID: 26354118 DOI: 10.1007/s10439-015-1433-1] [Citation(s) in RCA: 15] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/01/2015] [Accepted: 08/18/2015] [Indexed: 01/14/2023]
Abstract
With the recent advances in computing, the opportunities to translate computational models to more integrated roles in patient treatment are expanding at an exciting rate. One area of considerable development has been directed towards correcting soft tissue deformation within image guided neurosurgery applications. This review captures the efforts that have been undertaken towards enhancing neuronavigation by the integration of soft tissue biomechanical models, imaging and sensing technologies, and algorithmic developments. In addition, the review speaks to the evolving role of modeling frameworks within surgery and concludes with some future directions beyond neurosurgical applications.
Collapse
|
24
|
Hughes-Hallett A, Pratt P, Mayer E, Clark M, Vale J, Darzi A. Using preoperative imaging for intraoperative guidance: a case of mistaken identity. Int J Med Robot 2015; 12:262-7. [PMID: 25891963 DOI: 10.1002/rcs.1654] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/21/2014] [Revised: 01/13/2015] [Accepted: 03/02/2015] [Indexed: 12/14/2022]
Abstract
BACKGROUND Surgical image guidance systems to date have tended to rely on reconstructions of preoperative datasets. This paper assesses the accuracy of these reconstructions to establish whether they are appropriate for use in image guidance platforms. METHODS Nine raters (two experts in image interpretation and preparation, three in image interpretation, and four in neither interpretation nor preparation) were asked to perform a segmentation of ten renal tumours (four cystic and six solid tumours). These segmentations were compared with a gold standard consensus segmentation generated using a previously validated algorithm. RESULTS Average sensitivity and positive predictive value (PPV) were 0.902 and 0.891, respectively. When assessing for variability between raters, significant differences were seen in the PPV, sensitivity and incursions and excursions from consensus tumour boundary. CONCLUSIONS This paper has demonstrated that the interpretation required for the segmentation of preoperative imaging of renal tumours introduces significant inconsistency and inaccuracy. Copyright © 2015 John Wiley & Sons, Ltd.
Collapse
Affiliation(s)
| | - Philip Pratt
- Hamlyn Centre, Institute of Global Heath Innovation, Imperial College, London, UK
| | - Erik Mayer
- Department of Surgery and Cancer, Imperial College, London, UK
| | - Martin Clark
- Department of Radiology, Imperial College NHS Trust, London, UK
| | - Justin Vale
- Department of Surgery and Cancer, Imperial College, London, UK
| | - Ara Darzi
- Department of Surgery and Cancer, Imperial College, London, UK
- Hamlyn Centre, Institute of Global Heath Innovation, Imperial College, London, UK
| |
Collapse
|
25
|
|
26
|
Image Guidance for All—TilePro Display of 3-Dimensionally Reconstructed Images in Robotic Partial Nephrectomy. Urology 2014; 84:237-42. [DOI: 10.1016/j.urology.2014.02.051] [Citation(s) in RCA: 31] [Impact Index Per Article: 3.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/06/2013] [Revised: 01/02/2014] [Accepted: 02/08/2014] [Indexed: 11/23/2022]
|
27
|
Hughes-Hallett A, Mayer EK, Pratt P, Mottrie A, Darzi A, Vale J. The current and future use of imaging in urological robotic surgery: a survey of the European Association of Robotic Urological Surgeons. Int J Med Robot 2014; 11:8-14. [DOI: 10.1002/rcs.1596] [Citation(s) in RCA: 19] [Impact Index Per Article: 1.9] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 03/22/2014] [Indexed: 12/13/2022]
Affiliation(s)
| | - Erik K. Mayer
- Department of Surgery and Cancer; Imperial College; London UK
| | - Philip Pratt
- The Hamlyn Centre for Robotic Surgery; Imperial College; London UK
| | - Alex Mottrie
- Department of Urology; OLV Clinic; Aalst Belgium
- O.L.V. Vattikuti Robotic Surgery Institute; Aalst Belgium
| | - Ara Darzi
- Department of Surgery and Cancer; Imperial College; London UK
- The Hamlyn Centre for Robotic Surgery; Imperial College; London UK
| | - Justin Vale
- Department of Surgery and Cancer; Imperial College; London UK
| |
Collapse
|
28
|
Hughes-Hallett A, Mayer EK, Marcus HJ, Cundy TP, Pratt PJ, Darzi AW, Vale JA. Augmented reality partial nephrectomy: examining the current status and future perspectives. Urology 2013; 83:266-73. [PMID: 24149104 DOI: 10.1016/j.urology.2013.08.049] [Citation(s) in RCA: 55] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/10/2013] [Revised: 06/28/2013] [Accepted: 08/24/2013] [Indexed: 10/26/2022]
Abstract
A minimal access approach to partial nephrectomy has historically been under-utilized, but is now becoming more popular with the growth of robot-assisted laparoscopy. One of the criticisms of minimal access partial nephrectomy is the loss of haptic feedback. Augmented reality operating environments are forecast to play a major enabling role in the future of minimal access partial nephrectomy by integrating enhanced visual information to supplement this loss of haptic sensation. In this article, we systematically examine the current status of augmented reality in partial nephrectomy by identifying existing research challenges and exploring future agendas for this technology to achieve wider clinical translation.
Collapse
Affiliation(s)
| | - Erik K Mayer
- Department of Surgery and Cancer, Imperial College London, United Kingdom.
| | - Hani J Marcus
- The Hamlyn Centre, Institute of Global Health Innovation, Imperial College London, United Kingdom
| | - Thomas P Cundy
- The Hamlyn Centre, Institute of Global Health Innovation, Imperial College London, United Kingdom
| | - Philip J Pratt
- The Hamlyn Centre, Institute of Global Health Innovation, Imperial College London, United Kingdom
| | - Ara W Darzi
- Department of Surgery and Cancer, Imperial College London, United Kingdom; The Hamlyn Centre, Institute of Global Health Innovation, Imperial College London, United Kingdom
| | - Justin A Vale
- Department of Surgery and Cancer, Imperial College London, United Kingdom
| |
Collapse
|
29
|
Schneider C, Nguan C, Longpre M, Rohling R, Salcudean S. Motion of the Kidney Between Preoperative and Intraoperative Positioning. IEEE Trans Biomed Eng 2013; 60:1619-27. [DOI: 10.1109/tbme.2013.2239644] [Citation(s) in RCA: 15] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
|
30
|
Tobis S, Knopf JK, Silvers C, Messing E, Yao J, Rashid H, Wu G, Golijanin D. Robot-Assisted and Laparoscopic Partial Nephrectomy with Near Infrared Fluorescence Imaging. J Endourol 2012; 26:797-802. [DOI: 10.1089/end.2011.0604] [Citation(s) in RCA: 53] [Impact Index Per Article: 4.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/13/2022] Open
Affiliation(s)
- Scott Tobis
- Department of Urology at the University of Rochester Medical Center, Rochester, New York
| | - Joy K. Knopf
- Department of Urology at the University of Rochester Medical Center, Rochester, New York
| | - Christopher Silvers
- Department of Urology at the University of Rochester Medical Center, Rochester, New York
| | - Edward Messing
- Department of Urology at the University of Rochester Medical Center, Rochester, New York
| | - Jorge Yao
- Department of Pathology at the University of Rochester Medical Center, Rochester, New York
| | - Hani Rashid
- Department of Urology at the University of Rochester Medical Center, Rochester, New York
| | - Guan Wu
- Department of Urology at the University of Rochester Medical Center, Rochester, New York
| | - Dragan Golijanin
- Department of Urology at the University of Rochester Medical Center, Rochester, New York
| |
Collapse
|
31
|
Galloway RL, Herrell SD, Miga MI. Image-Guided Abdominal Surgery and Therapy Delivery. JOURNAL OF HEALTHCARE ENGINEERING 2012; 3:203-228. [PMID: 25077012 PMCID: PMC4112601 DOI: 10.1260/2040-2295.3.2.203] [Citation(s) in RCA: 13] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/01/2011] [Accepted: 07/01/2011] [Indexed: 01/31/2023]
Abstract
Image-Guided Surgery has become the standard of care in intracranial neurosurgery providing more exact resections while minimizing damage to healthy tissue. Moving that process to abdominal organs presents additional challenges in the form of image segmentation, image to physical space registration, organ motion and deformation. In this paper, we present methodologies and results for addressing these challenges in two specific organs: the liver and the kidney.
Collapse
Affiliation(s)
- Robert L. Galloway
- Department of Biomedical Engineering
- Department of Neurosurgery
- Department of Surgery
| | | | - Michael I. Miga
- Department of Biomedical Engineering
- Department of Neurosurgery
- Department of Radiology and Radiological Sciences Vanderbilt University
| |
Collapse
|
32
|
Abstract
PURPOSE OF REVIEW Advancements in surgery are progressing at a rapid rate; however, there are still limitations, including the ability to accurately visualize the target organ, in particular during laparoscopic surgery. Augmented reality visualization is a novel technique that has been developed to allow the fusion of three-dimensional medical images, such as those from transrectal ultrasound or computed tomography/MRI, with live camera images in real-time. In this review, we describe the current advancements and future directions of augmented reality and its application to laparoscopic surgery. RECENT FINDINGS Geometrically-correct superimposed images can be generated by tracking of the laparoscope and registration of the target organ. The fused image between the live laparoscopic images and the reconstructed three-dimensional organ model aides the surgeon in his or her understanding of anatomical structures. Laparoscopic and robot-assisted surgeries in both general surgery and urology have been performed with technical success to date. The primary limitation of the current augmented reality systems is its infancy in dynamic tracking of organ motion or deformation. Recently, augmented reality systems with organ tracking based on real-time image analysis were developed. Further improvement and/or development of such new technologies would resolve these issues. SUMMARY Augmented reality visualization is a significant advancement, improving the precision of laparoscopic/endoscopic surgery. New technologies to improve the dynamic tracking of organ motion or deformation are currently under investigation.
Collapse
|
33
|
Image-guided robotic surgery: update on research and potential applications in urologic surgery. Curr Opin Urol 2012; 22:47-54. [PMID: 22080871 DOI: 10.1097/mou.0b013e32834d4ce5] [Citation(s) in RCA: 22] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/25/2022]
Abstract
PURPOSE OF REVIEW New methods of imaging and image-guidance technology have the potential to provide surgeons with spatially accurate three-dimensional information about the location and anatomical relationships of critical subsurface structures and instrument position updated and displayed during the performance of surgery. Robotic platforms and technology in various forms continues to revolutionize surgery and will soon incorporate image guidance. RECENT RESEARCH Image-guided surgery (IGS) for abdominal and urologic interventions presents complex engineering and surgical challenges along with potential benefits to surgeons and patients. Key concepts such as registration, localization, accuracy, and targeting error are necessary for surgeons to understand and utilize the potential of IGS. Standard robotic surgeries, such as partial nephrectomy and radical prostatectomy may soon incorporate IGS. SUMMARY Research continues to explore the potential for combining image guidance and robotics to augment and improve a variety of surgical interventions.
Collapse
|
34
|
An effective visualisation and registration system for image-guided robotic partial nephrectomy. J Robot Surg 2012; 6:23-31. [DOI: 10.1007/s11701-011-0334-z] [Citation(s) in RCA: 25] [Impact Index Per Article: 2.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/20/2011] [Accepted: 12/15/2011] [Indexed: 10/14/2022]
|