1
|
Cannon PC, Setia SA, Klein-Gardner S, Kavoussi NL, Webster RJ, Herrell SD. Are 3D Image Guidance Systems Ready for Use? A Comparative Analysis of 3D Image Guidance Implementations in Minimally Invasive Partial Nephrectomy. J Endourol 2024; 38:395-407. [PMID: 38251637 PMCID: PMC10979686 DOI: 10.1089/end.2023.0059] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/23/2024] Open
Abstract
Introduction: Three-dimensional image-guided surgical (3D-IGS) systems for minimally invasive partial nephrectomy (MIPN) can potentially improve the efficiency and accuracy of intraoperative anatomical localization and tumor resection. This review seeks to analyze the current state of research regarding 3D-IGS, including the evaluation of clinical outcomes, system functionality, and qualitative insights regarding 3D-IGS's impact on surgical procedures. Methods: We have systematically reviewed the clinical literature pertaining to 3D-IGS deployed for MIPN. For inclusion, studies must produce a patient-specific 3D anatomical model from two-dimensional imaging. Data extracted from the studies include clinical results, registration (alignment of the 3D model to the surgical scene) method used, limitations, and data types reported. A subset of studies was qualitatively analyzed through an inductive coding approach to identify major themes and subthemes across the studies. Results: Twenty-five studies were included in the review. Eight (32%) studies reported clinical results that point to 3D-IGS improving multiple surgical outcomes. Manual registration was the most utilized (48%). Soft tissue deformation was the most cited limitation among the included studies. Many studies reported qualitative statements regarding surgeon accuracy improvement, but quantitative surgeon accuracy data were not reported. During the qualitative analysis, six major themes emerged across the nine applicable studies. They are as follows: 3D-IGS is necessary, 3D-IGS improved surgical outcomes, researcher/surgeon confidence in 3D-IGS system, enhanced surgeon ability/accuracy, anatomical explanation for qualitative assessment, and claims without data or reference to support. Conclusions: Currently, clinical outcomes are the main source of quantitative data available to point to 3D-IGS's efficacy. However, the literature qualitatively suggests the benefit of accurate 3D-IGS for robotic partial nephrectomy.
Collapse
Affiliation(s)
- Piper C. Cannon
- Department of Mechanical Engineering, Vanderbilt University, Nashville, Tennessee, USA
| | - Shaan A. Setia
- Department of Urologic Surgery, Vanderbilt University Medical Center, Nashville, Tennessee, USA
| | - Stacy Klein-Gardner
- Department of Mechanical Engineering, Vanderbilt University, Nashville, Tennessee, USA
| | - Nicholas L. Kavoussi
- Department of Urologic Surgery, Vanderbilt University Medical Center, Nashville, Tennessee, USA
| | - Robert J. Webster
- Department of Mechanical Engineering, Vanderbilt University, Nashville, Tennessee, USA
| | - S. Duke Herrell
- Department of Urologic Surgery, Vanderbilt University Medical Center, Nashville, Tennessee, USA
| |
Collapse
|
2
|
Cannon PC, Ferguson JM, Pitt EB, Shrand JA, Setia SA, Nimmagadda N, Barth EJ, Kavoussi NL, Galloway RL, Herrell SD, Webster RJ. A Safe Framework for Quantitative In Vivo Human Evaluation of Image Guidance. IEEE OPEN JOURNAL OF ENGINEERING IN MEDICINE AND BIOLOGY 2023; 5:133-139. [PMID: 38487093 PMCID: PMC10939321 DOI: 10.1109/ojemb.2023.3271853] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/30/2022] [Revised: 02/16/2023] [Accepted: 03/27/2023] [Indexed: 03/17/2024] Open
Abstract
Goal: We present a new framework for in vivo image guidance evaluation and provide a case study on robotic partial nephrectomy. Methods: This framework (called the "bystander protocol") involves two surgeons, one who solely performs the therapeutic process without image guidance, and another who solely periodically collects data to evaluate image guidance. This isolates the evaluation from the therapy, so that in-development image guidance systems can be tested without risk of negatively impacting the standard of care. We provide a case study applying this protocol in clinical cases during robotic partial nephrectomy surgery. Results: The bystander protocol was performed successfully in 6 patient cases. We find average lesion centroid localization error with our IGS system to be 6.5 mm in vivo compared to our prior result of 3.0 mm in phantoms. Conclusions: The bystander protocol is a safe, effective method for testing in-development image guidance systems in human subjects.
Collapse
Affiliation(s)
| | | | | | | | | | - Naren Nimmagadda
- Vanderbilt University Medical CenterNashvilleTN37232USA
- The Johns Hopkins University School of MedicineBaltimoreMD21287USA
| | | | | | | | | | | |
Collapse
|
3
|
A 3D Image Registration Method for Laparoscopic Liver Surgery Navigation. ELECTRONICS 2022. [DOI: 10.3390/electronics11111670] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/01/2023]
Abstract
At present, laparoscopic augmented reality (AR) navigation has been applied to minimally invasive abdominal surgery, which can help doctors to see the location of blood vessels and tumors in organs, so as to perform precise surgery operations. Image registration is the process of optimally mapping one or more images to the target image, and it is also the core of laparoscopic AR navigation. The key is how to shorten the registration time and optimize the registration accuracy. We have studied the three-dimensional (3D) image registration technology in laparoscopic liver surgery navigation and proposed a new registration method combining rough registration and fine registration. First, the adaptive fireworks algorithm (AFWA) is applied to rough registration, and then the optimized iterative closest point (ICP) algorithm is applied to fine registration. We proposed a method that is validated by the computed tomography (CT) dataset 3D-IRCADb-01. Experimental results show that our method is superior to other registration methods based on stochastic optimization algorithms in terms of registration time and accuracy.
Collapse
|
4
|
Kokko MA, Van Citters DW, Seigne JD, Halter RJ. A particle filter approach to dynamic kidney pose estimation in robotic surgical exposure. Int J Comput Assist Radiol Surg 2022; 17:1079-1089. [PMID: 35511394 DOI: 10.1007/s11548-022-02638-8] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/14/2021] [Accepted: 04/08/2022] [Indexed: 11/27/2022]
Abstract
PURPOSE Traditional soft tissue registration methods require direct intraoperative visualization of a significant portion of the target anatomy in order to produce acceptable surface alignment. Image guidance is therefore generally not available during the robotic exposure of structures like the kidneys which are not immediately visualized upon entry into the abdomen. This paper proposes guiding surgical exposure with an iterative state estimator that assimilates small visual cues into an a priori anatomical model as exposure progresses, thereby evolving pose estimates for the occluded structures of interest. METHODS Intraoperative surface observations of a right kidney are simulated using endoscope tracking and preoperative tomography from a representative robotic partial nephrectomy case. Clinically relevant random perturbations of the true kidney pose are corrected using this sequence of observations in a particle filter framework to estimate an optimal similarity transform for fitting a patient-specific kidney model at each step. The temporal response of registration error is compared against that of serial rigid coherent point drift (CPD) in both static and simulated dynamic surgical fields, and for varying levels of observation persistence. RESULTS In the static case, both particle filtering and persistent CPD achieved sub-5 mm accuracy, with CPD processing observations 75% faster. Particle filtering outperformed CPD in the dynamic case under equivalent computation times due to the former requiring only minimal persistence. CONCLUSION This proof-of-concept simulation study suggests that Bayesian state estimation may provide a viable pathway to image guidance for surgical exposure in the abdomen, especially in the presence of dynamic intraoperative tissue displacement and deformation.
Collapse
Affiliation(s)
- Michael A Kokko
- Thayer School of Engineering, Dartmouth College, 15 Thayer Drive, Hanover, NH, 03755, USA.
| | - Douglas W Van Citters
- Thayer School of Engineering, Dartmouth College, 15 Thayer Drive, Hanover, NH, 03755, USA
| | - John D Seigne
- Dartmouth-Hitchcock Medical Center, Section of Urology, 1 Medical Center Drive, Lebanon, NH, 03756, USA.,Geisel School of Medicine, Dartmouth College, 1 Rope Ferry Road, Hanover, NH, 03755, USA
| | - Ryan J Halter
- Thayer School of Engineering, Dartmouth College, 15 Thayer Drive, Hanover, NH, 03755, USA.,Geisel School of Medicine, Dartmouth College, 1 Rope Ferry Road, Hanover, NH, 03755, USA
| |
Collapse
|
5
|
Ferguson JM, Pitt EB, Remirez AA, Siebold MA, Kuntz A, Kavoussi NL, Barth EJ, Herrell SD, Webster RJ. Toward Practical and Accurate Touch-Based Image Guidance for Robotic Partial Nephrectomy. ACTA ACUST UNITED AC 2020; 2:196-205. [DOI: 10.1109/tmrb.2020.2989661] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
|
6
|
Joeres F, Schindele D, Luz M, Blaschke S, Russwinkel N, Schostak M, Hansen C. How well do software assistants for minimally invasive partial nephrectomy meet surgeon information needs? A cognitive task analysis and literature review study. PLoS One 2019; 14:e0219920. [PMID: 31318919 PMCID: PMC6638947 DOI: 10.1371/journal.pone.0219920] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/18/2019] [Accepted: 07/04/2019] [Indexed: 12/30/2022] Open
Abstract
INTRODUCTION Intraoperative software assistance is gaining increasing importance in laparoscopic and robot-assisted surgery. Within the user-centred development process of such systems, the first question to be asked is: What information does the surgeon need and when does he or she need it? In this article, we present an approach to investigate these surgeon information needs for minimally invasive partial nephrectomy and compare these needs to the relevant surgical computer assistance literature. MATERIALS AND METHODS First, we conducted a literature-based hierarchical task analysis of the surgical procedure. This task analysis was taken as a basis for a qualitative in-depth interview study with nine experienced surgical urologists. The study employed a cognitive task analysis method to elicit surgeons' information needs during minimally invasive partial nephrectomy. Finally, a systematic literature search was conducted to review proposed software assistance solutions for minimally invasive partial nephrectomy. The review focused on what information the solutions present to the surgeon and what phase of the surgery they aim to support. RESULTS The task analysis yielded a workflow description for minimally invasive partial nephrectomy. During the subsequent interview study, we identified three challenging phases of the procedure, which may particularly benefit from software assistance. These phases are I. Hilar and vascular management, II. Tumour excision, and III. Repair of the renal defects. Between these phases, 25 individual challenges were found which define the surgeon information needs. The literature review identified 34 relevant publications, all of which aim to support the surgeon in hilar and vascular management (phase I) or tumour excision (phase II). CONCLUSION The work presented in this article identified unmet surgeon information needs in minimally invasive partial nephrectomy. Namely, our results suggest that future solutions should address the repair of renal defects (phase III) or put more focus on the renal collecting system as a critical anatomical structure.
Collapse
Affiliation(s)
- Fabian Joeres
- Department of Simulation and Graphics, Faculty of Computer Science, Otto von Guericke University Magdeburg, Magdeburg, Germany
| | - Daniel Schindele
- Clinic of Urology and Paediatric Urology, University Hospital of Magdeburg, Magdeburg, Germany
| | - Maria Luz
- Department of Simulation and Graphics, Faculty of Computer Science, Otto von Guericke University Magdeburg, Magdeburg, Germany
| | - Simon Blaschke
- Clinic of Urology and Paediatric Urology, University Hospital of Magdeburg, Magdeburg, Germany
| | - Nele Russwinkel
- Department of Cognitive Modelling in Dynamic Human-Machine Systems, Technische Universität Berlin, Berlin, Germany
| | - Martin Schostak
- Clinic of Urology and Paediatric Urology, University Hospital of Magdeburg, Magdeburg, Germany
| | - Christian Hansen
- Department of Simulation and Graphics, Faculty of Computer Science, Otto von Guericke University Magdeburg, Magdeburg, Germany
| |
Collapse
|
7
|
Singla R, Edgcumbe P, Pratt P, Nguan C, Rohling R. Intra-operative ultrasound-based augmented reality guidance for laparoscopic surgery. Healthc Technol Lett 2017; 4:204-209. [PMID: 29184666 PMCID: PMC5683195 DOI: 10.1049/htl.2017.0063] [Citation(s) in RCA: 20] [Impact Index Per Article: 2.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/24/2017] [Accepted: 07/28/2017] [Indexed: 01/20/2023] Open
Abstract
In laparoscopic surgery, the surgeon must operate with a limited field of view and reduced depth perception. This makes spatial understanding of critical structures difficult, such as an endophytic tumour in a partial nephrectomy. Such tumours yield a high complication rate of 47%, and excising them increases the risk of cutting into the kidney's collecting system. To overcome these challenges, an augmented reality guidance system is proposed. Using intra-operative ultrasound, a single navigation aid, and surgical instrument tracking, four augmentations of guidance information are provided during tumour excision. Qualitative and quantitative system benefits are measured in simulated robot-assisted partial nephrectomies. Robot-to-camera calibration achieved a total registration error of 1.0 ± 0.4 mm while the total system error is 2.5 ± 0.5 mm. The system significantly reduced healthy tissue excised from an average (±standard deviation) of 30.6 ± 5.5 to 17.5 ± 2.4 cm3 (p < 0.05) and reduced the depth from the tumor underside to cut from an average (±standard deviation) of 10.2 ± 4.1 to 3.3 ± 2.3 mm (p < 0.05). Further evaluation is required in vivo, but the system has promising potential to reduce the amount of healthy parenchymal tissue excised.
Collapse
Affiliation(s)
- Rohit Singla
- Department of Electrical and Computer Engineering, University of British Columbia, Vancouver, CanadaV6T1Z4
| | - Philip Edgcumbe
- MD/PhD Program, University of British Columbia, Vancouver, CanadaV6T1Z4
| | - Philip Pratt
- Department of Surgery and Cancer, Imperial College London, UK, SW72BX
| | - Christopher Nguan
- Department of Urological Sciences, University of British Columbia, Vancouver, CanadaV6T1Z4
| | - Robert Rohling
- Department of Electrical and Computer Engineering, University of British Columbia, Vancouver, CanadaV6T1Z4.,Department of Mechanical Engineering, University of British Columbia, Vancouver, CanadaV6T1Z4
| |
Collapse
|
8
|
Detmer FJ, Hettig J, Schindele D, Schostak M, Hansen C. Virtual and Augmented Reality Systems for Renal Interventions: A Systematic Review. IEEE Rev Biomed Eng 2017; 10:78-94. [PMID: 28885161 DOI: 10.1109/rbme.2017.2749527] [Citation(s) in RCA: 28] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/05/2022]
Abstract
PURPOSE Many virtual and augmented reality systems have been proposed to support renal interventions. This paper reviews such systems employed in the treatment of renal cell carcinoma and renal stones. METHODS A systematic literature search was performed. Inclusion criteria were virtual and augmented reality systems for radical or partial nephrectomy and renal stone treatment, excluding systems solely developed or evaluated for training purposes. RESULTS In total, 52 research papers were identified and analyzed. Most of the identified literature (87%) deals with systems for renal cell carcinoma treatment. About 44% of the systems have already been employed in clinical practice, but only 20% in studies with ten or more patients. Main challenges remaining for future research include the consideration of organ movement and deformation, human factor issues, and the conduction of large clinical studies. CONCLUSION Augmented and virtual reality systems have the potential to improve safety and outcomes of renal interventions. In the last ten years, many technical advances have led to more sophisticated systems, which are already applied in clinical practice. Further research is required to cope with current limitations of virtual and augmented reality assistance in clinical environments.
Collapse
|
9
|
The status of augmented reality in laparoscopic surgery as of 2016. Med Image Anal 2017; 37:66-90. [DOI: 10.1016/j.media.2017.01.007] [Citation(s) in RCA: 183] [Impact Index Per Article: 26.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/10/2016] [Revised: 01/16/2017] [Accepted: 01/23/2017] [Indexed: 12/27/2022]
|
10
|
Simpfendörfer T, Gasch C, Hatiboglu G, Müller M, Maier-Hein L, Hohenfellner M, Teber D. Intraoperative Computed Tomography Imaging for Navigated Laparoscopic Renal Surgery: First Clinical Experience. J Endourol 2016; 30:1105-1111. [PMID: 27530774 DOI: 10.1089/end.2016.0385] [Citation(s) in RCA: 25] [Impact Index Per Article: 3.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/20/2023] Open
Abstract
INTRODUCTION Laparoscopic partial nephrectomy (LPN) remains challenging in endophytic and complex kidney tumors as the clear understanding of tumor location and spreading depends on a precise analysis of available imaging. The purpose of this study was to investigate navigated kidney surgery using intraoperative cone-beam computed tomography (CBCT) images in conjunction with a previously proposed method for augmented reality (AR) guidance for safe LPN. MATERIALS AND METHODS The concept proposed is based on using an intraoperative CBCT scan for (1) marker-based AR guidance for fast and reliable tumor access and (2) enhancement of real-time fluoroscopy images for accurate tumor resection. Workflow and accuracy of the system were assessed using a porcine kidney model. Ten patients with complex or endophytic tumor localization and R.E.N.A.L. Nephrometry Score of at least nine scheduled for LPN were included in this study. Patients received an intraoperative CBCT after marker placement. Defining the resection line was assisted by AR. In addition, fluoroscopy imaging for depth perception was used for assistance during dissection. Feasibility and performance were assessed by histopathological results, peri- and postoperative data. RESULTS Surgery was performed successfully and negative margins were found in all cases. Segmental branches of the renal artery shifted as much as 10 mm in the vertical and 11 mm in the sagittal axis intraoperatively compared to preoperative imaging. Fluoroscopy to intraoperative computed tomography image fusion enabled enhanced depth perception during dissection in all cases. Radiation dose area product was 4.8 mGym2. CONCLUSIONS The application of the navigation system is feasible and allows for safe and direct access to complex or endophytic renal masses. Radiation limits the application to selected indications.
Collapse
Affiliation(s)
| | - Claudia Gasch
- 1 Department of Urology, University of Heidelberg , Heidelberg, Germany
| | - Gencay Hatiboglu
- 1 Department of Urology, University of Heidelberg , Heidelberg, Germany
| | | | - Lena Maier-Hein
- 3 Junior Group Computer-Assisted Interventions, German Cancer Research Center (DKFZ) , Heidelberg, Germany
| | | | - Dogu Teber
- 1 Department of Urology, University of Heidelberg , Heidelberg, Germany
| |
Collapse
|
11
|
Malthouse T, Kasivisvanathan V, Raison N, Lam W, Challacombe B. The future of partial nephrectomy. Int J Surg 2016; 36:560-567. [PMID: 26975430 DOI: 10.1016/j.ijsu.2016.03.024] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/08/2016] [Accepted: 03/10/2016] [Indexed: 12/29/2022]
Abstract
Innovation in recent times has accelerated due to factors such as the globalization of communication; but there are also more barriers/safeguards in place than ever before as we strive to streamline this process. From the first planned partial nephrectomy completed in 1887, it took over a century to become recommended practice for small renal tumours. At present, identified areas for improvement/innovation are 1) to preserve renal parenchyma, 2) to optimise pre-operative eGFR and 3) to reduce global warm ischaemia time. All 3 of these, are statistically significant predictors of post-operative renal function. Urologists, have a proud history of embracing innovation & have experimented with different clamping techniques of the renal vasculature, image guidance in robotics, renal hypothermia, lasers and new robots under development. The DaVinci model may soon no longer have a monopoly on this market, as it loses its stranglehold with novel technology emerging including added features, such as haptic feedback with reduced costs. As ever, our predictions of the future may well fall wide of the mark, but in order to progress, one must open the mind to the possibilities that already exist, as evolution of existing technology often appears to be a revolution in hindsight.
Collapse
Affiliation(s)
- Theo Malthouse
- Guy's and St Thomas' NHS Foundation Trust, Great Maze Pond, London SE1 9RT, United Kingdom.
| | - Veeru Kasivisvanathan
- University College London Hospital, 235 Euston Rd, Fitzrovia, London NW1 2BU, United Kingdom
| | - Nicholas Raison
- King's College Hospital NHS Foundation Trust, Denmark Hill, London SE5 9RS, United Kingdom
| | - Wayne Lam
- Guy's and St Thomas' NHS Foundation Trust, Great Maze Pond, London SE1 9RT, United Kingdom
| | - Ben Challacombe
- Guy's and St Thomas' NHS Foundation Trust, Great Maze Pond, London SE1 9RT, United Kingdom
| |
Collapse
|
12
|
Hughes-Hallett A, Mayer EK, Marcus HJ, Cundy TP, Pratt PJ, Darzi AW, Vale JA. Augmented reality partial nephrectomy: examining the current status and future perspectives. Urology 2013; 83:266-73. [PMID: 24149104 DOI: 10.1016/j.urology.2013.08.049] [Citation(s) in RCA: 55] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/10/2013] [Revised: 06/28/2013] [Accepted: 08/24/2013] [Indexed: 10/26/2022]
Abstract
A minimal access approach to partial nephrectomy has historically been under-utilized, but is now becoming more popular with the growth of robot-assisted laparoscopy. One of the criticisms of minimal access partial nephrectomy is the loss of haptic feedback. Augmented reality operating environments are forecast to play a major enabling role in the future of minimal access partial nephrectomy by integrating enhanced visual information to supplement this loss of haptic sensation. In this article, we systematically examine the current status of augmented reality in partial nephrectomy by identifying existing research challenges and exploring future agendas for this technology to achieve wider clinical translation.
Collapse
Affiliation(s)
| | - Erik K Mayer
- Department of Surgery and Cancer, Imperial College London, United Kingdom.
| | - Hani J Marcus
- The Hamlyn Centre, Institute of Global Health Innovation, Imperial College London, United Kingdom
| | - Thomas P Cundy
- The Hamlyn Centre, Institute of Global Health Innovation, Imperial College London, United Kingdom
| | - Philip J Pratt
- The Hamlyn Centre, Institute of Global Health Innovation, Imperial College London, United Kingdom
| | - Ara W Darzi
- Department of Surgery and Cancer, Imperial College London, United Kingdom; The Hamlyn Centre, Institute of Global Health Innovation, Imperial College London, United Kingdom
| | - Justin A Vale
- Department of Surgery and Cancer, Imperial College London, United Kingdom
| |
Collapse
|