1
|
Peek JJ, Zhang X, Hildebrandt K, Max SA, Sadeghi AH, Bogers AJJC, Mahtab EAF. A novel 3D image registration technique for augmented reality vision in minimally invasive thoracoscopic pulmonary segmentectomy. Int J Comput Assist Radiol Surg 2024:10.1007/s11548-024-03308-7. [PMID: 39707038 DOI: 10.1007/s11548-024-03308-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/19/2024] [Accepted: 12/04/2024] [Indexed: 12/23/2024]
Abstract
PURPOSE In this feasibility study, we aimed to create a dedicated pulmonary augmented reality (AR) workflow to enable a semi-automated intraoperative overlay of the pulmonary anatomy during video-assisted thoracoscopic surgery (VATS) or robot-assisted thoracoscopic surgery (RATS). METHODS Initially, the stereoscopic cameras were calibrated to obtain the intrinsic camera parameters. Intraoperatively, stereoscopic images were recorded and a 3D point cloud was generated from these images. By manually selecting the bifurcation key points, the 3D segmentation (from the diagnostic CT scan) was registered onto the intraoperative 3D point cloud. RESULTS Image reprojection errors were 0.34 and 0.22 pixels for the VATS and RATS cameras, respectively. We created disparity maps and point clouds for all eight patients. Time for creation of the 3D AR overlay was 5 min. Validation of the point clouds was performed, resulting in a median absolute error of 0.20 mm [IQR 0.10-0.54]. We were able to visualize the AR overlay and identify the arterial bifurcations adequately for five patients. In addition to creating AR overlays of the visible or invisible structures intraoperatively, we successfully visualized branch labels and altered the transparency of the overlays. CONCLUSION An algorithm was developed transforming the operative field into a 3D point cloud surface. This allowed for an accurate registration and visualization of preoperative 3D models. Using this system, surgeons can navigate through the patient's anatomy intraoperatively, especially during crucial moments, by visualizing otherwise invisible structures. This proposed registration method lays the groundwork for automated intraoperative AR navigation during minimally invasive pulmonary resections.
Collapse
Affiliation(s)
- J J Peek
- Department of Cardiothoracic Surgery, Erasmus University Medical Center, Rotterdam, The Netherlands
| | - X Zhang
- Computer Vision Lab, TU Delft, Delft, The Netherlands
| | - K Hildebrandt
- Computer Graphics and Visualization Lab, TU Delft, Delft, The Netherlands
| | - S A Max
- Department of Cardiothoracic Surgery, Leiden University Medical Center, Leiden, The Netherlands
| | - A H Sadeghi
- Department of Cardiothoracic Surgery, Erasmus University Medical Center, Rotterdam, The Netherlands
- Department of Cardiothoracic Surgery, University Medical Center Utrecht, Utrecht, The Netherlands
| | - A J J C Bogers
- Department of Cardiothoracic Surgery, Erasmus University Medical Center, Rotterdam, The Netherlands
| | - E A F Mahtab
- Department of Cardiothoracic Surgery, Erasmus University Medical Center, Rotterdam, The Netherlands.
- Department of Cardiothoracic Surgery, Leiden University Medical Center, Leiden, The Netherlands.
| |
Collapse
|
2
|
Asadi Z, Asadi M, Kazemipour N, Léger É, Kersten-Oertel M. A decade of progress: bringing mixed reality image-guided surgery systems in the operating room. Comput Assist Surg (Abingdon) 2024; 29:2355897. [PMID: 38794834 DOI: 10.1080/24699322.2024.2355897] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/26/2024] Open
Abstract
Advancements in mixed reality (MR) have led to innovative approaches in image-guided surgery (IGS). In this paper, we provide a comprehensive analysis of the current state of MR in image-guided procedures across various surgical domains. Using the Data Visualization View (DVV) Taxonomy, we analyze the progress made since a 2013 literature review paper on MR IGS systems. In addition to examining the current surgical domains using MR systems, we explore trends in types of MR hardware used, type of data visualized, visualizations of virtual elements, and interaction methods in use. Our analysis also covers the metrics used to evaluate these systems in the operating room (OR), both qualitative and quantitative assessments, and clinical studies that have demonstrated the potential of MR technologies to enhance surgical workflows and outcomes. We also address current challenges and future directions that would further establish the use of MR in IGS.
Collapse
Affiliation(s)
- Zahra Asadi
- Department of Computer Science and Software Engineering, Concordia University, Montréal, Canada
| | - Mehrdad Asadi
- Department of Computer Science and Software Engineering, Concordia University, Montréal, Canada
| | - Negar Kazemipour
- Department of Computer Science and Software Engineering, Concordia University, Montréal, Canada
| | - Étienne Léger
- Montréal Neurological Institute & Hospital (MNI/H), Montréal, Canada
- McGill University, Montréal, Canada
| | - Marta Kersten-Oertel
- Department of Computer Science and Software Engineering, Concordia University, Montréal, Canada
| |
Collapse
|
3
|
Doornbos MCJ, Peek JJ, Maat APWM, Ruurda JP, De Backer P, Cornelissen BMW, Mahtab EAF, Sadeghi AH, Kluin J. Augmented Reality Implementation in Minimally Invasive Surgery for Future Application in Pulmonary Surgery: A Systematic Review. Surg Innov 2024; 31:646-658. [PMID: 39370802 PMCID: PMC11475712 DOI: 10.1177/15533506241290412] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/08/2024]
Abstract
OBJECTIVE This systematic review investigates of Augmented Reality (AR) systems used in minimally invasive surgery of deformable organs, focusing on initial registration, dynamic tracking, and visualization. The objective is to acquire a comprehensive understanding of the current knowledge, applications, and challenges associated with current AR-techniques, aiming to leverage these insights for developing a dedicated AR pulmonary Video or Robotic Assisted Thoracic Surgery (VATS/RATS) workflow. METHODS A systematic search was conducted within Embase, Medline (Ovid) and Web of Science on April 16, 2024, following the Preferred Reporting items for Systematic Reviews and Meta-Analyses (PRISMA). The search focused on intraoperative AR applications and intraoperative navigational purposes for deformable organs. Quality assessment was performed and studies were categorized according to initial registration and dynamic tracking methods. RESULTS 33 articles were included, of which one involved pulmonary surgery. Studies used both manual and (semi-) automatic registration methods, established through anatomical landmark-based, fiducial-based, or surface-based techniques. Diverse outcome measures were considered, including surgical outcomes and registration accuracy. The majority of studies that reached an registration accuracy below 5 mm applied surface-based registration. CONCLUSIONS AR can potentially aid surgeons with real-time navigation and decision making during anatomically complex minimally invasive procedures. Future research for pulmonary applications should focus on exploring surface-based registration methods, considering their non-invasive, marker-less nature, and promising accuracy. Additionally, vascular-labeling-based methods are worth exploring, given the importance and relative stability of broncho-vascular anatomy in pulmonary VATS/RATS. Assessing clinical feasibility of these approaches is crucial, particularly concerning registration accuracy and potential impact on surgical outcomes.
Collapse
Affiliation(s)
- Marie-Claire J. Doornbos
- Department of Cardiothoracic Surgery, Thoraxcenter, Erasmus MC, Rotterdam, The Netherlands
- Educational Program Technical Medicine, Leiden University Medical Center, Delft University of Technology & Erasmus University Medical Center Rotterdam, Leiden, The Netherlands
| | - Jette J. Peek
- Department of Cardiothoracic Surgery, Thoraxcenter, Erasmus MC, Rotterdam, The Netherlands
| | | | - Jelle P. Ruurda
- Department of Surgery, University Medical Center Utrecht, Utrecht, The Netherlands
| | | | | | - Edris A. F. Mahtab
- Department of Cardiothoracic Surgery, Thoraxcenter, Erasmus MC, Rotterdam, The Netherlands
- Department of Cardiothoracic Surgery, Leiden University Medical Center, Leiden, The Netherlands
| | - Amir H. Sadeghi
- Department of Cardiothoracic Surgery, Thoraxcenter, Erasmus MC, Rotterdam, The Netherlands
- Department of Cardiothoracic Surgery, University Medical Center Utrecht, The Netherlands
| | - Jolanda Kluin
- Department of Cardiothoracic Surgery, Thoraxcenter, Erasmus MC, Rotterdam, The Netherlands
| |
Collapse
|
4
|
Li Y, Raison N, Ourselin S, Mahmoodi T, Dasgupta P, Granados A. AI solutions for overcoming delays in telesurgery and telementoring to enhance surgical practice and education. J Robot Surg 2024; 18:403. [PMID: 39527379 PMCID: PMC11554828 DOI: 10.1007/s11701-024-02153-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/30/2024] [Accepted: 10/26/2024] [Indexed: 11/16/2024]
Abstract
Artificial intelligence (AI) has emerged as a transformative tool in surgery, particularly in telesurgery and telementoring. However, its potential to enhance data transmission efficiency and reliability in these fields remains unclear. While previous reviews have explored the general applications of telesurgery and telementoring in specific surgical contexts, this review uniquely focuses on AI models designed to optimise data transmission and mitigate delays. We conducted a comprehensive literature search on PubMed and IEEE Xplore for studies published in English between 2010 and 2023, focusing on AI-driven, surgery-related, telemedicine, and delay-related research. This review includes methodologies from journals, conferences, and symposiums. Our analysis identified a total of twelve AI studies that focus on optimising network resources, enhancing edge computing, and developing delay-robust predictive applications. Specifically, three studies addressed wireless network resource optimisation, two proposed low-latency control and transfer learning algorithms for edge computing, and seven developed delay-robust applications, five of which focused on motion data, with the remaining two addressing visual and haptic data. These advancements lay the foundation for a truly holistic and context-aware telesurgical experience, significantly transforming remote surgical practice and education. By mapping the current role of AI in addressing delay-related challenges, this review highlights the pressing need for collaborative research to drive the evolution of telesurgery and telementoring in modern robotic surgery.
Collapse
Affiliation(s)
- Yang Li
- Surgical and Interventional Engineering, King's College London, London, UK
| | - Nicholas Raison
- Surgical and Interventional Engineering, King's College London, London, UK
- Department of Urology, Guy's Hospital, London, UK
| | - Sebastien Ourselin
- Surgical and Interventional Engineering, King's College London, London, UK
| | - Toktam Mahmoodi
- Department of Engineering, King's College London, London, UK
| | - Prokar Dasgupta
- Surgical and Interventional Engineering, King's College London, London, UK
- Department of Urology, Guy's Hospital, London, UK
| | - Alejandro Granados
- Surgical and Interventional Engineering, King's College London, London, UK.
| |
Collapse
|
5
|
Kotsifa E, Mavroeidis VK. Present and Future Applications of Artificial Intelligence in Kidney Transplantation. J Clin Med 2024; 13:5939. [PMID: 39407999 PMCID: PMC11478249 DOI: 10.3390/jcm13195939] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/03/2024] [Revised: 09/27/2024] [Accepted: 10/02/2024] [Indexed: 10/15/2024] Open
Abstract
Artificial intelligence (AI) has a wide and increasing range of applications across various sectors. In medicine, AI has already made an impact in numerous fields, rapidly transforming healthcare delivery through its growing applications in diagnosis, treatment and overall patient care. Equally, AI is swiftly and essentially transforming the landscape of kidney transplantation (KT), offering innovative solutions for longstanding problems that have eluded resolution through traditional approaches outside its spectrum. The purpose of this review is to explore the present and future applications of artificial intelligence in KT, with a focus on pre-transplant evaluation, surgical assistance, outcomes and post-transplant care. We discuss its great potential and the inevitable limitations that accompany these technologies. We conclude that by fostering collaboration between AI technologies and medical practitioners, we can pave the way for a future where advanced, personalised care becomes the standard in KT and beyond.
Collapse
Affiliation(s)
- Evgenia Kotsifa
- Second Propaedeutic Department of Surgery, National and Kapodistrian University of Athens, General Hospital of Athens “Laiko”, Agiou Thoma 17, 157 72 Athens, Greece
| | - Vasileios K. Mavroeidis
- Department of Transplant Surgery, North Bristol NHS Trust, Southmead Hospital, Bristol BS10 5NB, UK
| |
Collapse
|
6
|
Chen Z, Cruciani L, Fan K, Fontana M, Lievore E, De Cobelli O, Musi G, Ferrigno G, De Momi E. Towards safer robot-assisted surgery: A markerless augmented reality framework. Neural Netw 2024; 178:106469. [PMID: 38925030 DOI: 10.1016/j.neunet.2024.106469] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/28/2024] [Revised: 06/01/2024] [Accepted: 06/16/2024] [Indexed: 06/28/2024]
Abstract
Robot-assisted surgery is rapidly developing in the medical field, and the integration of augmented reality shows the potential to improve the operation performance of surgeons by providing more visual information. In this paper, we proposed a markerless augmented reality framework to enhance safety by avoiding intra-operative bleeding, which is a high risk caused by collision between surgical instruments and delicate blood vessels (arteries or veins). Advanced stereo reconstruction and segmentation networks are compared to find the best combination to reconstruct the intra-operative blood vessel in 3D space for registration with the pre-operative model, and the minimum distance detection between the instruments and the blood vessel is implemented. A robot-assisted lymphadenectomy is emulated on the da Vinci Research Kit in a dry lab, and ten human subjects perform this operation to explore the usability of the proposed framework. The result shows that the augmented reality framework can help the users to avoid the dangerous collision between the instruments and the delicate blood vessel while not introducing an extra load. It provides a flexible framework that integrates augmented reality into the medical robotic platform to enhance safety during surgery.
Collapse
Affiliation(s)
- Ziyang Chen
- Politecnico di Milano, Department of Electronics, Information and Bioengineering, Milano, 20133, Italy
| | - Laura Cruciani
- Politecnico di Milano, Department of Electronics, Information and Bioengineering, Milano, 20133, Italy
| | - Ke Fan
- Politecnico di Milano, Department of Electronics, Information and Bioengineering, Milano, 20133, Italy.
| | - Matteo Fontana
- European Institute of Oncology, Department of Urology, IRCCS, Milan, 20141, Italy
| | - Elena Lievore
- European Institute of Oncology, Department of Urology, IRCCS, Milan, 20141, Italy
| | - Ottavio De Cobelli
- European Institute of Oncology, Department of Urology, IRCCS, Milan, 20141, Italy; University of Milan, Department of Oncology and Onco-haematology, Faculty of Medicine and Surgery, Milan, 20122, Italy
| | - Gennaro Musi
- European Institute of Oncology, Department of Urology, IRCCS, Milan, 20141, Italy; University of Milan, Department of Oncology and Onco-haematology, Faculty of Medicine and Surgery, Milan, 20122, Italy
| | - Giancarlo Ferrigno
- Politecnico di Milano, Department of Electronics, Information and Bioengineering, Milano, 20133, Italy
| | - Elena De Momi
- Politecnico di Milano, Department of Electronics, Information and Bioengineering, Milano, 20133, Italy; European Institute of Oncology, Department of Urology, IRCCS, Milan, 20141, Italy
| |
Collapse
|
7
|
Augmented Reality unterstützt roboterassistierte Nierenchirurgie. Aktuelle Urol 2024; 55:286. [PMID: 39047742 DOI: 10.1055/a-2188-9609] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/27/2024]
|
8
|
Checcucci E, Piana A, Volpi G, Quarà A, De Cillis S, Piramide F, Burgio M, Meziere J, Cisero E, Colombo M, Bignante G, Sica M, Granato S, Verri P, Gatti C, Alessio P, Di Dio M, Alba S, Fiori C, Amparore D, Porpiglia F. Visual extended reality tools in image-guided surgery in urology: a systematic review. Eur J Nucl Med Mol Imaging 2024; 51:3109-3134. [PMID: 38589511 DOI: 10.1007/s00259-024-06699-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/29/2023] [Accepted: 03/19/2024] [Indexed: 04/10/2024]
Abstract
PURPOSE The aim of this systematic review is to assess the clinical implications of employing various Extended Reality (XR) tools for image guidance in urological surgery. METHODS In June 2023, a systematic electronic literature search was conducted using the Medline database (via PubMed), Embase (via Ovid), Scopus, and Web of Science. The search strategy was designed based on the PICO (Patients, Intervention, Comparison, Outcome) criteria. Study protocol was registered on PROSPERO (registry number CRD42023449025). We incorporated retrospective and prospective comparative studies, along with single-arm studies, which provided information on the use of XR, Mixed Reality (MR), Augmented Reality (AR), and Virtual Reality (VR) in urological surgical procedures. Studies that were not written in English, non-original investigations, and those involving experimental research on animals or cadavers were excluded from our analysis. The quality assessment of comparative and cohort studies was conducted utilizing the Newcastle-Ottawa scale, whilst for randomized controlled trials (RCTs), the Jadad scale was adopted. The level of evidence for each study was determined based on the guidelines provided by the Oxford Centre for Evidence-Based Medicine. RESULTS The initial electronic search yielded 1,803 papers after removing duplicates. Among these, 58 publications underwent a comprehensive review, leading to the inclusion of 40 studies that met the specified criteria for analysis. 11, 20 and 9 studies tested XR on prostate cancer, kidney cancer and miscellaneous, including bladder cancer and lithiasis surgeries, respectively. Focusing on the different technologies 20, 15 and 5 explored the potential of VR, AR and MR. The majority of the included studies (i.e., 22) were prospective non-randomized, whilst 7 and 11 were RCT and retrospective studies respectively. The included studies that revealed how these new tools can be useful both in preoperative and intraoperative setting for a tailored surgical approach. CONCLUSIONS AR, VR and MR techniques have emerged as highly effective new tools for image-guided surgery, especially for urologic oncology. Nevertheless, the complete clinical advantages of these innovations are still in the process of evaluation.
Collapse
Affiliation(s)
- Enrico Checcucci
- Department of Surgery, Candiolo Cancer Institute, FPO-IRCCS, Strada Provinciale 142, km 3,95, Candiolo, Turin, 10060, Italy.
| | - Alberto Piana
- Department of Urology, Romolo Hospital, Rocca di Neto, Italy
| | - Gabriele Volpi
- Department of Surgery, Candiolo Cancer Institute, FPO-IRCCS, Strada Provinciale 142, km 3,95, Candiolo, Turin, 10060, Italy
| | - Alberto Quarà
- Department of Oncology, Division of Urology, University of Turin, San Luigi Gonzaga Hospital, Turin, Italy
| | - Sabrina De Cillis
- Department of Oncology, Division of Urology, University of Turin, San Luigi Gonzaga Hospital, Turin, Italy
| | - Federico Piramide
- Department of Oncology, Division of Urology, University of Turin, San Luigi Gonzaga Hospital, Turin, Italy
| | - Mariano Burgio
- Department of Oncology, Division of Urology, University of Turin, San Luigi Gonzaga Hospital, Turin, Italy
| | - Juliette Meziere
- Department of Oncology, Division of Urology, University of Turin, San Luigi Gonzaga Hospital, Turin, Italy
| | - Edoardo Cisero
- Department of Oncology, Division of Urology, University of Turin, San Luigi Gonzaga Hospital, Turin, Italy
| | - Marco Colombo
- Department of Oncology, Division of Urology, University of Turin, San Luigi Gonzaga Hospital, Turin, Italy
| | - Gabriele Bignante
- Department of Oncology, Division of Urology, University of Turin, San Luigi Gonzaga Hospital, Turin, Italy
| | - Michele Sica
- Department of Oncology, Division of Urology, University of Turin, San Luigi Gonzaga Hospital, Turin, Italy
| | - Stefano Granato
- Department of Oncology, Division of Urology, University of Turin, San Luigi Gonzaga Hospital, Turin, Italy
| | - Paolo Verri
- Department of Oncology, Division of Urology, University of Turin, San Luigi Gonzaga Hospital, Turin, Italy
| | - Cecilia Gatti
- Department of Surgery, Candiolo Cancer Institute, FPO-IRCCS, Strada Provinciale 142, km 3,95, Candiolo, Turin, 10060, Italy
| | - Paolo Alessio
- Department of Surgery, Candiolo Cancer Institute, FPO-IRCCS, Strada Provinciale 142, km 3,95, Candiolo, Turin, 10060, Italy
| | - Michele Di Dio
- Dept. of Surgery, Division of Urology, SS Annunziata Hospital, Cosenza, Italy
| | - Stefano Alba
- Department of Urology, Romolo Hospital, Rocca di Neto, Italy
| | - Cristian Fiori
- Department of Oncology, Division of Urology, University of Turin, San Luigi Gonzaga Hospital, Turin, Italy
| | - Daniele Amparore
- Department of Oncology, Division of Urology, University of Turin, San Luigi Gonzaga Hospital, Turin, Italy
| | - Francesco Porpiglia
- Department of Oncology, Division of Urology, University of Turin, San Luigi Gonzaga Hospital, Turin, Italy
| |
Collapse
|
9
|
Lanfranchi G, Costanzo S, Selvaggio GGO, Gallotta C, Milani P, Rizzetto F, Musitelli A, Vertemati M, Santaniello T, Campari A, Paraboschi I, Camporesi A, Marinaro M, Calcaterra V, Pierucci UM, Pelizzo G. Virtual Reality Head-Mounted Display (HMD) and Preoperative Patient-Specific Simulation: Impact on Decision-Making in Pediatric Urology: Preliminary Data. Diagnostics (Basel) 2024; 14:1647. [PMID: 39125523 PMCID: PMC11311633 DOI: 10.3390/diagnostics14151647] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/21/2024] [Revised: 07/19/2024] [Accepted: 07/29/2024] [Indexed: 08/12/2024] Open
Abstract
AIM OF THE STUDY To assess how virtual reality (VR) patient-specific simulations can support decision-making processes and improve care in pediatric urology, ultimately improving patient outcomes. PATIENTS AND METHODS Children diagnosed with urological conditions necessitating complex procedures were retrospectively reviewed and enrolled in the study. Patient-specific VR simulations were developed with medical imaging specialists and VR technology experts. Routine CT images were utilized to create a VR environment using advanced software platforms. The accuracy and fidelity of the VR simulations was validated through a multi-step process. This involved comparing the virtual anatomical models to the original medical imaging data and conducting feedback sessions with pediatric urology experts to assess VR simulations' realism and clinical relevance. RESULTS A total of six pediatric patients were reviewed. The median age of the participants was 5.5 years (IQR: 3.5-8.5 years), with an equal distribution of males and females across both groups. A minimally invasive laparoscopic approach was performed for adrenal lesions (n = 3), Wilms' tumor (n = 1), bilateral nephroblastomatosis (n = 1), and abdominal trauma in complex vascular and renal malformation (ptotic and hypoplastic kidney) (n = 1). Key benefits included enhanced visualization of the segmental arteries and the deep vascularization of the kidney and adrenal glands in all cases. The high depth perception and precision in the orientation of the arteries and veins to the parenchyma changed the intraoperative decision-making process in five patients. Preoperative VR patient-specific simulation did not offer accuracy in studying the pelvic and calyceal anatomy. CONCLUSIONS VR patient-specific simulations represent an empowering tool in pediatric urology. By leveraging the immersive capabilities of VR technology, preoperative planning and intraoperative navigation can greatly impact surgical decision-making. As we continue to advance in medical simulation, VR holds promise in educational programs to include even surgical treatment of more complex urogenital malformations.
Collapse
Affiliation(s)
- Giulia Lanfranchi
- Department of Pediatric Surgery, Children’s Hospital “Vittore Buzzi”, 20154 Milan, Italy; (G.L.); (S.C.); (G.G.O.S.); (A.M.); (M.M.); (U.M.P.)
| | - Sara Costanzo
- Department of Pediatric Surgery, Children’s Hospital “Vittore Buzzi”, 20154 Milan, Italy; (G.L.); (S.C.); (G.G.O.S.); (A.M.); (M.M.); (U.M.P.)
| | - Giorgio Giuseppe Orlando Selvaggio
- Department of Pediatric Surgery, Children’s Hospital “Vittore Buzzi”, 20154 Milan, Italy; (G.L.); (S.C.); (G.G.O.S.); (A.M.); (M.M.); (U.M.P.)
| | - Cristina Gallotta
- Department of Biomedical and Clinical Sciences “L Sacco”, University of Milano, 20157 Milan, Italy; (C.G.); (M.V.); (I.P.)
| | - Paolo Milani
- CIMaINa (Interdisciplinary Centre for Nanostructured Materials and Interfaces), University of Milano, 20133 Milan, Italy; (P.M.); (T.S.)
- Department of Physics “Aldo Pontremoli”, University of Milano, 20133 Milan, Italy
| | - Francesco Rizzetto
- Department of Radiology, ASST Grande Ospedale Metropolitano Niguarda, 20162 Milan, Italy;
- Postgraduate School of Diagnostic and Interventional Radiology, University of Milano, 20122 Milan, Italy
| | - Alessia Musitelli
- Department of Pediatric Surgery, Children’s Hospital “Vittore Buzzi”, 20154 Milan, Italy; (G.L.); (S.C.); (G.G.O.S.); (A.M.); (M.M.); (U.M.P.)
| | - Maurizio Vertemati
- Department of Biomedical and Clinical Sciences “L Sacco”, University of Milano, 20157 Milan, Italy; (C.G.); (M.V.); (I.P.)
- CIMaINa (Interdisciplinary Centre for Nanostructured Materials and Interfaces), University of Milano, 20133 Milan, Italy; (P.M.); (T.S.)
| | - Tommaso Santaniello
- CIMaINa (Interdisciplinary Centre for Nanostructured Materials and Interfaces), University of Milano, 20133 Milan, Italy; (P.M.); (T.S.)
- Department of Physics “Aldo Pontremoli”, University of Milano, 20133 Milan, Italy
| | - Alessandro Campari
- Pediatric Radiology and Neuroradiology Unit, “Vittore Buzzi” Children’s Hospital, 20154 Milan, Italy;
| | - Irene Paraboschi
- Department of Biomedical and Clinical Sciences “L Sacco”, University of Milano, 20157 Milan, Italy; (C.G.); (M.V.); (I.P.)
| | - Anna Camporesi
- Pediatric Anesthesia and Intensive Care Unit, “Vittore Buzzi“ Children’s Hospital, 20154 Milan, Italy;
| | - Michela Marinaro
- Department of Pediatric Surgery, Children’s Hospital “Vittore Buzzi”, 20154 Milan, Italy; (G.L.); (S.C.); (G.G.O.S.); (A.M.); (M.M.); (U.M.P.)
| | - Valeria Calcaterra
- Pediatrics and Adolescentology Unit, Department of Internal Medicine, University of Pavia, 27100 Pavia, Italy;
- Pediatric Department, “Vittore Buzzi” Children’s Hospital, 20154 Milan, Italy
| | - Ugo Maria Pierucci
- Department of Pediatric Surgery, Children’s Hospital “Vittore Buzzi”, 20154 Milan, Italy; (G.L.); (S.C.); (G.G.O.S.); (A.M.); (M.M.); (U.M.P.)
| | - Gloria Pelizzo
- Department of Pediatric Surgery, Children’s Hospital “Vittore Buzzi”, 20154 Milan, Italy; (G.L.); (S.C.); (G.G.O.S.); (A.M.); (M.M.); (U.M.P.)
- Department of Biomedical and Clinical Sciences “L Sacco”, University of Milano, 20157 Milan, Italy; (C.G.); (M.V.); (I.P.)
| |
Collapse
|
10
|
De Backer P, Simoens J, Mestdagh K, Hofman J, Eckhoff JA, Jobczyk M, Van Eetvelde E, D’Hondt M, Moschovas MC, Patel V, Van Praet C, Fuchs HF, Debbaut C, Decaestecker K, Mottrie A. Privacy-proof Live Surgery Streaming: Development and Validation of a Low-cost, Real-time Robotic Surgery Anonymization Algorithm. Ann Surg 2024; 280:13-20. [PMID: 38390732 PMCID: PMC11161223 DOI: 10.1097/sla.0000000000006245] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/24/2024]
Abstract
OBJECTIVE Develop a pioneer surgical anonymization algorithm for reliable and accurate real-time removal of out-of-body images validated across various robotic platforms. BACKGROUND The use of surgical video data has become a common practice in enhancing research and training. Video sharing requires complete anonymization, which, in the case of endoscopic surgery, entails the removal of all nonsurgical video frames where the endoscope can record the patient or operating room staff. To date, no openly available algorithmic solution for surgical anonymization offers reliable real-time anonymization for video streaming, which is also robotic-platform and procedure-independent. METHODS A data set of 63 surgical videos of 6 procedures performed on four robotic systems was annotated for out-of-body sequences. The resulting 496.828 images were used to develop a deep learning algorithm that automatically detected out-of-body frames. Our solution was subsequently benchmarked against existing anonymization methods. In addition, we offer a postprocessing step to enhance the performance and test a low-cost setup for real-time anonymization during live surgery streaming. RESULTS Framewise anonymization yielded a receiver operating characteristic area under the curve score of 99.46% on unseen procedures, increasing to 99.89% after postprocessing. Our Robotic Anonymization Network outperforms previous state-of-the-art algorithms, even on unseen procedural types, despite the fact that alternative solutions are explicitly trained using these procedures. CONCLUSIONS Our deep learning model, Robotic Anonymization Network, offers reliable, accurate, and safe real-time anonymization during complex and lengthy surgical procedures regardless of the robotic platform. The model can be used in real time for surgical live streaming and is openly available.
Collapse
Affiliation(s)
- Pieter De Backer
- ORSI Academy, Belgium
- Department of Human Structure and Repair, Faculty of Medicine and Health Sciences, Ghent University, Belgium
- IBiTech-Biommeda, Faculty of Engineering and Architecture, CRIG, Ghent University, Belgium
- Urology Department, Ghent University Hospital, Belgium
| | | | - Kenzo Mestdagh
- Department of Human Structure and Repair, Faculty of Medicine and Health Sciences, Ghent University, Belgium
| | | | - Jennifer A. Eckhoff
- Robotic Innovation Laboratory, Department of General, University Hospital Cologne, Visceral, Tumor and Transplantsurgery, Germany
| | - Mateusz Jobczyk
- Urology Department, Salve Medica Hospital Lodz, Lodz, Poland
| | | | - Mathieu D’Hondt
- Department of Digestive and Hepatobiliary/Pancreatic Surgery, AZ Groeninge Hospital Kortrijk, Belgium
| | | | - Vipul Patel
- Urology Department, AdventHealth Global Robotics Institute, Celebration, FL
| | | | - Hans F. Fuchs
- Robotic Innovation Laboratory, Department of General, University Hospital Cologne, Visceral, Tumor and Transplantsurgery, Germany
| | - Charlotte Debbaut
- IBiTech-Biommeda, Faculty of Engineering and Architecture, CRIG, Ghent University, Belgium
| | - Karel Decaestecker
- Urology Department, Ghent University Hospital, Belgium
- Urology Department, AZ Maria Middelares Hospital Ghent, Ghent, Belgium
| | - Alexandre Mottrie
- ORSI Academy, Belgium
- Urology Department, OLV Hospital Aalst-Asse-Ninove, Belgium
| |
Collapse
|
11
|
Piana A, Pecoraro A, Dönmez Mİ, Prudhomme T, Bañuelos Marco B, López Abad A, Campi R, Boissier R, Checcucci E, Amparore D, Porpiglia F, Breda A, Territo A. New frontiers in kidney transplantation: Towards the extended reality. Actas Urol Esp 2024; 48:337-339. [PMID: 37981169 DOI: 10.1016/j.acuroe.2023.11.005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/02/2023] [Accepted: 10/04/2023] [Indexed: 11/21/2023]
Affiliation(s)
- Alberto Piana
- Departmento de Urología, Universidad de Turín, Turín, Italy; Servicio de Urología, Hospital Romolo, Rocca di Neto, Italy.
| | - Alessio Pecoraro
- Departmento de Medicina Experimental y Clínica, Universidad de Florencia, Florencia, Italy
| | - Muhammet İrfan Dönmez
- Departmento de Urología, Facultad de Medicina de la Universidad de Estambul, Estambul, Turkey
| | - Thomas Prudhomme
- Servicio de Urología, Trasplante Renal y Andrología, Hospital Universitario de Rangueil, Toulouse, France
| | - Beatriz Bañuelos Marco
- Sección de Trasplante Renal y Urología Reconstructiva, Hospital Universitario Clínico San Carlos, Madrid, Spain
| | - Alicia López Abad
- Departmento de Medicina Experimental y Clínica, Universidad de Florencia, Florencia, Italy; Servicio de Urología, Hospital Clínico Universitario Virgen de la Arrixaca, Murcia, Spain
| | - Riccardo Campi
- Departmento de Medicina Experimental y Clínica, Universidad de Florencia, Florencia, Italy
| | - Romain Boissier
- Servicio de Urología y Trasplante Renal, Hospital Universitario La Conception, Marsella, France
| | - Enrico Checcucci
- Servicio de Cirugía, Instituto de Candiolo FPO-IRCCS, Candiolo, Turín, Italy
| | | | | | - Alberto Breda
- Unidad de Uro-oncología y Trasplante Renal, Servicio de Urología, Fundación Puigvert, Universidad Autónoma de Barcelona (UAB), Barcelona, Spain
| | - Angelo Territo
- Unidad de Uro-oncología y Trasplante Renal, Servicio de Urología, Fundación Puigvert, Universidad Autónoma de Barcelona (UAB), Barcelona, Spain
| |
Collapse
|
12
|
Daunt R, Curtin D, O'Mahony D. Optimizing drug therapy for older adults: shifting away from problematic polypharmacy. Expert Opin Pharmacother 2024; 25:1199-1208. [PMID: 38940370 DOI: 10.1080/14656566.2024.2374048] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/22/2024] [Accepted: 06/25/2024] [Indexed: 06/29/2024]
Abstract
INTRODUCTION The accelerated discovery and production of pharmaceutical products has resulted in many positive outcomes. However, this progress has also contributed to problematic polypharmacy, one of the rapidly growing threats to public health in this century. Problematic polypharmacy results in adverse patient outcomes and imposes increased strain and financial burden on healthcare systems. AREAS COVERED A review was conducted on the current body of evidence concerning factors contributing to and consequences of problematic polypharmacy. Recent trials investigating interventions that target polypharmacy and emerging solutions, including incorporation of artificial intelligence, are also examined in this article. EXPERT OPINION To shift away from problematic polypharmacy, a multifaceted interdisciplinary approach is necessary. Any potentially successful strategy must be adapted to suit various healthcare settings and must utilize all available resources, including artificial intelligence.
Collapse
Affiliation(s)
- Ruth Daunt
- Department of Medicine (Geriatrics), School of Medicine, University College Cork, Cork, Ireland
- Department of Geriatric Medicine, Cork University Hospital, Cork, Ireland
| | - Denis Curtin
- Department of Medicine (Geriatrics), School of Medicine, University College Cork, Cork, Ireland
- Department of Geriatric Medicine, Cork University Hospital, Cork, Ireland
| | - Denis O'Mahony
- Department of Medicine (Geriatrics), School of Medicine, University College Cork, Cork, Ireland
- Department of Geriatric Medicine, Cork University Hospital, Cork, Ireland
| |
Collapse
|
13
|
Bellos T, Manolitsis I, Katsimperis S, Juliebø-Jones P, Feretzakis G, Mitsogiannis I, Varkarakis I, Somani BK, Tzelves L. Artificial Intelligence in Urologic Robotic Oncologic Surgery: A Narrative Review. Cancers (Basel) 2024; 16:1775. [PMID: 38730727 PMCID: PMC11083167 DOI: 10.3390/cancers16091775] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/26/2024] [Revised: 04/29/2024] [Accepted: 05/02/2024] [Indexed: 05/13/2024] Open
Abstract
With the rapid increase in computer processing capacity over the past two decades, machine learning techniques have been applied in many sectors of daily life. Machine learning in therapeutic settings is also gaining popularity. We analysed current studies on machine learning in robotic urologic surgery. We searched PubMed/Medline and Google Scholar up to December 2023. Search terms included "urologic surgery", "artificial intelligence", "machine learning", "neural network", "automation", and "robotic surgery". Automatic preoperative imaging, intraoperative anatomy matching, and bleeding prediction has been a major focus. Early artificial intelligence (AI) therapeutic outcomes are promising. Robot-assisted surgery provides precise telemetry data and a cutting-edge viewing console to analyse and improve AI integration in surgery. Machine learning enhances surgical skill feedback, procedure effectiveness, surgical guidance, and postoperative prediction. Tension-sensors on robotic arms and augmented reality can improve surgery. This provides real-time organ motion monitoring, improving precision and accuracy. As datasets develop and electronic health records are used more and more, these technologies will become more effective and useful. AI in robotic surgery is intended to improve surgical training and experience. Both seek precision to improve surgical care. AI in ''master-slave'' robotic surgery offers the detailed, step-by-step examination of autonomous robotic treatments.
Collapse
Affiliation(s)
- Themistoklis Bellos
- 2nd Department of Urology, Sismanoglio General Hospital of Athens, 15126 Athens, Greece; (T.B.); (I.M.); (S.K.); (I.M.); (I.V.)
| | - Ioannis Manolitsis
- 2nd Department of Urology, Sismanoglio General Hospital of Athens, 15126 Athens, Greece; (T.B.); (I.M.); (S.K.); (I.M.); (I.V.)
| | - Stamatios Katsimperis
- 2nd Department of Urology, Sismanoglio General Hospital of Athens, 15126 Athens, Greece; (T.B.); (I.M.); (S.K.); (I.M.); (I.V.)
| | | | - Georgios Feretzakis
- School of Science and Technology, Hellenic Open University, 26335 Patras, Greece;
| | - Iraklis Mitsogiannis
- 2nd Department of Urology, Sismanoglio General Hospital of Athens, 15126 Athens, Greece; (T.B.); (I.M.); (S.K.); (I.M.); (I.V.)
| | - Ioannis Varkarakis
- 2nd Department of Urology, Sismanoglio General Hospital of Athens, 15126 Athens, Greece; (T.B.); (I.M.); (S.K.); (I.M.); (I.V.)
| | - Bhaskar K. Somani
- Department of Urology, University of Southampton, Southampton SO16 6YD, UK;
| | - Lazaros Tzelves
- 2nd Department of Urology, Sismanoglio General Hospital of Athens, 15126 Athens, Greece; (T.B.); (I.M.); (S.K.); (I.M.); (I.V.)
| |
Collapse
|
14
|
Banerjee A, Babu R, Jayaraman D, Chilukuri S. Preoperative three-dimensional modelling and virtual reality planning aids nephron sparing surgery in a child with bilateral Wilms tumour. BMJ Case Rep 2024; 17:e260600. [PMID: 38642931 PMCID: PMC11033631 DOI: 10.1136/bcr-2024-260600] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/22/2024] Open
Abstract
Bilateral Wilms tumour (BWT) is a surgically challenging condition. Virtual reality (VR) reconstruction aids surgeons to foresee the anatomy ahead of Nephron Sparing Surgery (NSS). Three-dimensional (3D) visualisation improves the anatomical orientation of surgeons performing NSS. We herewith report a case of BWT where VR planning and 3D printing were used to aid NSS. Conventional imaging is often found to be inadequate while assessing the tumour-organ-vascular anatomy. Advances like VR and 3D printing help surgeons plan better for complex surgeries like bilateral NSS. Next-generation extended reality tools will likely aid robotic-assisted precision NSS and improve patient outcomes.
Collapse
Affiliation(s)
- Avijit Banerjee
- Urology, Sri Ramachandra Institute of Higher Education and Research, Chennai, India
| | - Ramesh Babu
- Pediatric Urology, Sri Ramachandra University Medical College, Chennai, India
| | - Dhaarani Jayaraman
- Paediatric Hematology and Oncology, Sri Ramachandra Institute of Higher Education and Research (Deemed to be University), Chennai, India
| | | |
Collapse
|
15
|
Wiklund P, Rebuffo S, Frego N, Mottrie A. What More Can We Ask of Robotics? Eur Urol 2024; 85:315-316. [PMID: 37919191 DOI: 10.1016/j.eururo.2023.10.013] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/10/2023] [Accepted: 10/17/2023] [Indexed: 11/04/2023]
Abstract
The future of robotics relies heavily on the ongoing synergy between robotic surgery and artificial intelligence. To unlock their full potential, we should address issues such as accessibility, education, data privacy, and ethics.
Collapse
Affiliation(s)
- Peter Wiklund
- Department of Urology, Icahn School of Medicine at Mount Sinai, New York, NY, USA
| | - Silvia Rebuffo
- Department of Urology, Onze-Lieve-Vrouwziekenhuis, Aalst, Belgium; ORSI Academy, Ghent, Belgium; Department of Urology, Policlinico San Martino Hospital, University of Genoa, Genoa, Italy
| | - Nicola Frego
- Department of Urology, Onze-Lieve-Vrouwziekenhuis, Aalst, Belgium; ORSI Academy, Ghent, Belgium; Department of Urology, IRCCS Humanitas Research Hospital, Rozzano, Italy
| | - Alexandre Mottrie
- Department of Urology, Onze-Lieve-Vrouwziekenhuis, Aalst, Belgium; ORSI Academy, Ghent, Belgium.
| |
Collapse
|
16
|
Hofman J, De Backer P, Manghi I, Simoens J, De Groote R, Van Den Bossche H, D'Hondt M, Oosterlinck T, Lippens J, Van Praet C, Ferraguti F, Debbaut C, Li Z, Kutter O, Mottrie A, Decaestecker K. First-in-human real-time AI-assisted instrument deocclusion during augmented reality robotic surgery. Healthc Technol Lett 2024; 11:33-39. [PMID: 38638494 PMCID: PMC11022222 DOI: 10.1049/htl2.12056] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/09/2023] [Accepted: 11/21/2023] [Indexed: 04/20/2024] Open
Abstract
The integration of Augmented Reality (AR) into daily surgical practice is withheld by the correct registration of pre-operative data. This includes intelligent 3D model superposition whilst simultaneously handling real and virtual occlusions caused by the AR overlay. Occlusions can negatively impact surgical safety and as such deteriorate rather than improve surgical care. Robotic surgery is particularly suited to tackle these integration challenges in a stepwise approach as the robotic console allows for different inputs to be displayed in parallel to the surgeon. Nevertheless, real-time de-occlusion requires extensive computational resources which further complicates clinical integration. This work tackles the problem of instrument occlusion and presents, to the authors' best knowledge, the first-in-human on edge deployment of a real-time binary segmentation pipeline during three robot-assisted surgeries: partial nephrectomy, migrated endovascular stent removal, and liver metastasectomy. To this end, a state-of-the-art real-time segmentation and 3D model pipeline was implemented and presented to the surgeon during live surgery. The pipeline allows real-time binary segmentation of 37 non-organic surgical items, which are never occluded during AR. The application features real-time manual 3D model manipulation for correct soft tissue alignment. The proposed pipeline can contribute towards surgical safety, ergonomics, and acceptance of AR in minimally invasive surgery.
Collapse
Affiliation(s)
| | - Pieter De Backer
- ORSI AcademyMelleBelgium
- Faculty of Medicine and Health Sciences, Department of Human Structure and RepairGhent UniversityGhentBelgium
- IBiTech‐Biommeda, Faculty of Engineering and Architecture, and CRIGGhent UniversityGhentBelgium
- Department of UrologyGhent University HospitalGhentBelgium
| | - Ilaria Manghi
- Department of Sciences and Methods for EngineeringUniversity of Modena and Reggio EmiliaModenaItaly
| | | | - Ruben De Groote
- ORSI AcademyMelleBelgium
- Department of UrologyOLV HospitalAalstBelgium
| | | | - Mathieu D'Hondt
- Department of Digestive and Hepatobiliary/Pancreatic SurgeryAZ Groeninge HospitalKortrijkBelgium
| | | | - Julie Lippens
- Faculty of Medicine and Health Sciences, Department of Human Structure and RepairGhent UniversityGhentBelgium
| | | | - Federica Ferraguti
- Department of Sciences and Methods for EngineeringUniversity of Modena and Reggio EmiliaModenaItaly
| | - Charlotte Debbaut
- IBiTech‐Biommeda, Faculty of Engineering and Architecture, and CRIGGhent UniversityGhentBelgium
| | | | | | - Alexandre Mottrie
- ORSI AcademyMelleBelgium
- Department of UrologyOLV HospitalAalstBelgium
| | - Karel Decaestecker
- Faculty of Medicine and Health Sciences, Department of Human Structure and RepairGhent UniversityGhentBelgium
- Department of UrologyAZ Maria Middelares HospitalGhentBelgium
| |
Collapse
|
17
|
Piana A, Amparore D, Sica M, Volpi G, Checcucci E, Piramide F, De Cillis S, Busacca G, Scarpelli G, Sidoti F, Alba S, Piazzolla P, Fiori C, Porpiglia F, Di Dio M. Automatic 3D Augmented-Reality Robot-Assisted Partial Nephrectomy Using Machine Learning: Our Pioneer Experience. Cancers (Basel) 2024; 16:1047. [PMID: 38473404 DOI: 10.3390/cancers16051047] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/23/2024] [Revised: 02/26/2024] [Accepted: 02/29/2024] [Indexed: 03/14/2024] Open
Abstract
The aim of "Precision Surgery" is to reduce the impact of surgeries on patients' global health. In this context, over the last years, the use of three-dimensional virtual models (3DVMs) of organs has allowed for intraoperative guidance, showing hidden anatomical targets, thus limiting healthy-tissue dissections and subsequent damage during an operation. In order to provide an automatic 3DVM overlapping in the surgical field, we developed and tested a new software, called "ikidney", based on convolutional neural networks (CNNs). From January 2022 to April 2023, patients affected by organ-confined renal masses amenable to RAPN were enrolled. A bioengineer, a software developer, and a surgeon collaborated to create hyper-accurate 3D models for automatic 3D AR-guided RAPN, using CNNs. For each patient, demographic and clinical data were collected. A total of 13 patients were included in the present study. The average anchoring time was 11 (6-13) s. Unintended 3D-model automatic co-registration temporary failures happened in a static setting in one patient, while this happened in one patient in a dynamic setting. There was one failure; in this single case, an ultrasound drop-in probe was used to detect the neoplasm, and the surgery was performed under ultrasound guidance instead of AR guidance. No major intraoperative nor postoperative complications (i.e., Clavien Dindo > 2) were recorded. The employment of AI has unveiled several new scenarios in clinical practice, thanks to its ability to perform specific tasks autonomously. We employed CNNs for an automatic 3DVM overlapping during RAPN, thus improving the accuracy of the superimposition process.
Collapse
Affiliation(s)
- Alberto Piana
- Division of Urology, Department of Oncology, School of Medicine, University of Turin, San Luigi Hospital, 10043 Turin, Italy
| | - Daniele Amparore
- Division of Urology, Department of Oncology, School of Medicine, University of Turin, San Luigi Hospital, 10043 Turin, Italy
| | - Michele Sica
- Department of Surgery, Candiolo Cancer Institute FPO-IRCCS, 10060 Turin, Italy
| | - Gabriele Volpi
- Department of Surgery, Candiolo Cancer Institute FPO-IRCCS, 10060 Turin, Italy
| | - Enrico Checcucci
- Department of Surgery, Candiolo Cancer Institute FPO-IRCCS, 10060 Turin, Italy
| | - Federico Piramide
- Division of Urology, Department of Oncology, School of Medicine, University of Turin, San Luigi Hospital, 10043 Turin, Italy
| | - Sabrina De Cillis
- Division of Urology, Department of Oncology, School of Medicine, University of Turin, San Luigi Hospital, 10043 Turin, Italy
| | - Giovanni Busacca
- Division of Urology, Department of Oncology, School of Medicine, University of Turin, San Luigi Hospital, 10043 Turin, Italy
| | | | | | | | - Pietro Piazzolla
- Division of Urology, Department of Oncology, School of Medicine, University of Turin, San Luigi Hospital, 10043 Turin, Italy
| | - Cristian Fiori
- Division of Urology, Department of Oncology, School of Medicine, University of Turin, San Luigi Hospital, 10043 Turin, Italy
| | - Francesco Porpiglia
- Division of Urology, Department of Oncology, School of Medicine, University of Turin, San Luigi Hospital, 10043 Turin, Italy
| | - Michele Di Dio
- Division of Urology, Department of Surgery, Annunziata Hospital, 87100 Cosenza, Italy
| |
Collapse
|
18
|
Knudsen JE, Ghaffar U, Ma R, Hung AJ. Clinical applications of artificial intelligence in robotic surgery. J Robot Surg 2024; 18:102. [PMID: 38427094 PMCID: PMC10907451 DOI: 10.1007/s11701-024-01867-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/12/2024] [Accepted: 02/10/2024] [Indexed: 03/02/2024]
Abstract
Artificial intelligence (AI) is revolutionizing nearly every aspect of modern life. In the medical field, robotic surgery is the sector with some of the most innovative and impactful advancements. In this narrative review, we outline recent contributions of AI to the field of robotic surgery with a particular focus on intraoperative enhancement. AI modeling is allowing surgeons to have advanced intraoperative metrics such as force and tactile measurements, enhanced detection of positive surgical margins, and even allowing for the complete automation of certain steps in surgical procedures. AI is also Query revolutionizing the field of surgical education. AI modeling applied to intraoperative surgical video feeds and instrument kinematics data is allowing for the generation of automated skills assessments. AI also shows promise for the generation and delivery of highly specialized intraoperative surgical feedback for training surgeons. Although the adoption and integration of AI show promise in robotic surgery, it raises important, complex ethical questions. Frameworks for thinking through ethical dilemmas raised by AI are outlined in this review. AI enhancements in robotic surgery is some of the most groundbreaking research happening today, and the studies outlined in this review represent some of the most exciting innovations in recent years.
Collapse
Affiliation(s)
- J Everett Knudsen
- Keck School of Medicine, University of Southern California, Los Angeles, USA
| | | | - Runzhuo Ma
- Cedars-Sinai Medical Center, Los Angeles, USA
| | | |
Collapse
|
19
|
Amparore D, Sica M, Verri P, Piramide F, Checcucci E, De Cillis S, Piana A, Campobasso D, Burgio M, Cisero E, Busacca G, Di Dio M, Piazzolla P, Fiori C, Porpiglia F. Computer Vision and Machine-Learning Techniques for Automatic 3D Virtual Images Overlapping During Augmented Reality Guided Robotic Partial Nephrectomy. Technol Cancer Res Treat 2024; 23:15330338241229368. [PMID: 38374643 PMCID: PMC10878218 DOI: 10.1177/15330338241229368] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/21/2024] Open
Abstract
OBJECTIVES The research's purpose is to develop a software that automatically integrates and overlay 3D virtual models of kidneys harboring renal masses into the Da Vinci robotic console, assisting surgeon during the intervention. INTRODUCTION Precision medicine, especially in the field of minimally-invasive partial nephrectomy, aims to use 3D virtual models as a guidance for augmented reality robotic procedures. However, the co-registration process of the virtual images over the real operative field is performed manually. METHODS In this prospective study, two strategies for the automatic overlapping of the model over the real kidney were explored: the computer vision technology, leveraging the super-enhancement of the kidney allowed by the intraoperative injection of Indocyanine green for superimposition and the convolutional neural network technology, based on the processing of live images from the endoscope, after a training of the software on frames from prerecorded videos of the same surgery. The work-team, comprising a bioengineer, a software-developer and a surgeon, collaborated to create hyper-accuracy 3D models for automatic 3D-AR-guided RAPN. For each patient, demographic and clinical data were collected. RESULTS Two groups (group A for the first technology with 12 patients and group B for the second technology with 8 patients) were defined. They showed comparable preoperative and post-operative characteristics. Concerning the first technology the average co-registration time was 7 (3-11) seconds while in the case of the second technology 11 (6-13) seconds. No major intraoperative or postoperative complications were recorded. There were no differences in terms of functional outcomes between the groups at every time-point considered. CONCLUSION The first technology allowed a successful anchoring of the 3D model to the kidney, despite minimal manual refinements. The second technology improved kidney automatic detection without relying on indocyanine injection, resulting in better organ boundaries identification during tests. Further studies are needed to confirm this preliminary evidence.
Collapse
Affiliation(s)
- Daniele Amparore
- Division of Urology, Dept. of Oncology, School of Medicine, University of Turin, San Luigi Hospital, Orbassano (Turin), Italy
| | - Michele Sica
- Division of Urology, Dept. of Oncology, School of Medicine, University of Turin, San Luigi Hospital, Orbassano (Turin), Italy
| | - Paolo Verri
- Division of Urology, Dept. of Oncology, School of Medicine, University of Turin, San Luigi Hospital, Orbassano (Turin), Italy
| | - Federico Piramide
- Division of Urology, Dept. of Oncology, School of Medicine, University of Turin, San Luigi Hospital, Orbassano (Turin), Italy
| | - Enrico Checcucci
- Department of Surgery, Candiolo Cancer Institute FPO-IRCCS, Candiolo, Italy
| | - Sabrina De Cillis
- Division of Urology, Dept. of Oncology, School of Medicine, University of Turin, San Luigi Hospital, Orbassano (Turin), Italy
| | - Alberto Piana
- Division of Urology, Dept. of Oncology, School of Medicine, University of Turin, San Luigi Hospital, Orbassano (Turin), Italy
- Department of Urology, Romolo Hospital, Rocca di Neto (KR), Italy
| | - Davide Campobasso
- Urology Unit, University Hospital of Parma, Parma, Italy
- 2 Level Master Degree Program in Advanced Robotic and Laparoscopic Surgery in Urology, Division of Urology, Dept. of Oncology, School of Medicine, University of Turin, San Luigi, Italy
| | - Mariano Burgio
- Division of Urology, Dept. of Oncology, School of Medicine, University of Turin, San Luigi Hospital, Orbassano (Turin), Italy
| | - Edoardo Cisero
- Division of Urology, Dept. of Oncology, School of Medicine, University of Turin, San Luigi Hospital, Orbassano (Turin), Italy
| | - Giovanni Busacca
- Division of Urology, Dept. of Oncology, School of Medicine, University of Turin, San Luigi Hospital, Orbassano (Turin), Italy
| | - Michele Di Dio
- Division of Urology, Department of Surgery, Annunziata Hospital, Cosenza, Italy
| | - Pietro Piazzolla
- Department of Management and Production Engineer, Polytechnic University of Turin, Turin, Italy
| | - Cristian Fiori
- Division of Urology, Dept. of Oncology, School of Medicine, University of Turin, San Luigi Hospital, Orbassano (Turin), Italy
| | - Francesco Porpiglia
- Division of Urology, Dept. of Oncology, School of Medicine, University of Turin, San Luigi Hospital, Orbassano (Turin), Italy
| |
Collapse
|
20
|
Eckhoff JA, Meireles O. Could Artificial Intelligence guide surgeons' hands? Rev Col Bras Cir 2023; 50:e20233696EDIT01. [PMID: 38088637 PMCID: PMC10668586 DOI: 10.1590/0100-6991e-20233696edit01-en] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/23/2023] [Accepted: 11/23/2023] [Indexed: 12/18/2023] Open
Affiliation(s)
- Jennifer A. Eckhoff
- - Harvard Medical School, Surgical Artificial Intelligence and Innovation Laboratory, Massachusetts General Hospital - Boston - MA - Estados Unidos
| | - Ozanan Meireles
- - Harvard Medical School, Surgical Artificial Intelligence and Innovation Laboratory, Massachusetts General Hospital - Boston - MA - Estados Unidos
| |
Collapse
|
21
|
De Backer P, Vangeneugden J, Berquin C, Vermijs S, Dekuyper P, Mottrie A, Debbaut C, Quackels T, Van Praet C, Decaestecker K. Robot-assisted Partial Nephrectomy Using Intra-arterial Renal Hypothermia for Highly Complex Endophytic or Hilar Tumors: Case Series and Description of Surgical Technique. EUR UROL SUPPL 2023; 58:19-27. [PMID: 38028235 PMCID: PMC10660005 DOI: 10.1016/j.euros.2023.10.004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 10/16/2023] [Indexed: 12/01/2023] Open
Abstract
Background In partial nephrectomy for highly complex tumors with expected long ischemia time, renal hypothermia can be used to minimize ischemic parenchymal damage. Objective To describe our case series, surgical technique, and early outcomes for robot-assisted partial nephrectomy (RAPN) using intra-arterial cold perfusion through arteriotomy. Design setting and participants A retrospective analysis was conducted of ten patients with renal tumors (PADUA score 9-13) undergoing RAPN between March 2020 and March 2023 with intra-arterial cooling because of expected arterial clamping times longer than 25 min. Surgical procedure Multiport transperitoneal RAPN with full renal mobilization and arterial, venous, and ureteral clamping was performed. After arteriotomy and venotomy, 4°C heparinized saline is administered intravascular through a Fogarty catheter to maintain renal hypothermia while performing RAPN. Measurements Demographic data, renal function, console and ischemia times, surgical margin status, hospital stay, estimated blood loss, and complications were analyzed. Results and limitations The median warm and cold ischemia times were 4 min (interquartile range [IQR] 3-7 min) and 60 min (IQR 33-75 min), respectively. The median rewarming ischemia time was 10.5 min (IQR 6.5-23.75 min). The median pre- and postoperative estimated glomerular filtration rate values at least 1 mo after surgery were 90 ml/min (IQR 78.35-90 ml/min) and 86.9 ml/min (IQR 62.08-90 ml/min), respectively. Limitations include small cohort size and short median follow-up (13 [IQR 9.1-32.4] mo). Conclusions We demonstrate the feasibility and first case series for RAPN using intra-arterial renal hypothermia through arteriotomy. This approach broadens the scope for minimal invasive nephron-sparing surgery in highly complex renal masses. Patient summary We demonstrate a minimally invasive surgical technique that reduces kidney infarction during complex kidney tumor removal where surrounding healthy kidney tissue is spared. The technique entails arterial cold fluid irrigation, which temporarily decreases renal metabolism and allows more kidneys to be salvaged.
Collapse
Affiliation(s)
- Pieter De Backer
- Department of Urology, ERN eUROGEN Accredited Centre, Ghent University Hospital, Ghent, Belgium
- IBiTech-Biommeda, Department of Electronics and Information Systems, Faculty of Engineering and Architecture, Ghent University, Ghent, Belgium
- ORSI Academy, Melle, Belgium
| | - Joris Vangeneugden
- Department of Urology, ERN eUROGEN Accredited Centre, Ghent University Hospital, Ghent, Belgium
| | - Camille Berquin
- Department of Urology, ERN eUROGEN Accredited Centre, Ghent University Hospital, Ghent, Belgium
| | - Saar Vermijs
- IBiTech-Biommeda, Department of Electronics and Information Systems, Faculty of Engineering and Architecture, Ghent University, Ghent, Belgium
| | - Peter Dekuyper
- Department of Urology, AZ Maria Middelares Hospital, Ghent, Belgium
| | - Alexandre Mottrie
- ORSI Academy, Melle, Belgium
- Department of Urology, Onze-Lieve-Vrouwziekenhuis Hospital, Aalst, Belgium
| | - Charlotte Debbaut
- IBiTech-Biommeda, Department of Electronics and Information Systems, Faculty of Engineering and Architecture, Ghent University, Ghent, Belgium
| | | | - Charles Van Praet
- Department of Urology, ERN eUROGEN Accredited Centre, Ghent University Hospital, Ghent, Belgium
| | - Karel Decaestecker
- Department of Urology, ERN eUROGEN Accredited Centre, Ghent University Hospital, Ghent, Belgium
- Department of Urology, AZ Maria Middelares Hospital, Ghent, Belgium
| |
Collapse
|
22
|
Rodriguez Peñaranda N, Eissa A, Ferretti S, Bianchi G, Di Bari S, Farinha R, Piazza P, Checcucci E, Belenchón IR, Veccia A, Gomez Rivas J, Taratkin M, Kowalewski KF, Rodler S, De Backer P, Cacciamani GE, De Groote R, Gallagher AG, Mottrie A, Micali S, Puliatti S. Artificial Intelligence in Surgical Training for Kidney Cancer: A Systematic Review of the Literature. Diagnostics (Basel) 2023; 13:3070. [PMID: 37835812 PMCID: PMC10572445 DOI: 10.3390/diagnostics13193070] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/22/2023] [Revised: 09/17/2023] [Accepted: 09/24/2023] [Indexed: 10/15/2023] Open
Abstract
The prevalence of renal cell carcinoma (RCC) is increasing due to advanced imaging techniques. Surgical resection is the standard treatment, involving complex radical and partial nephrectomy procedures that demand extensive training and planning. Furthermore, artificial intelligence (AI) can potentially aid the training process in the field of kidney cancer. This review explores how artificial intelligence (AI) can create a framework for kidney cancer surgery to address training difficulties. Following PRISMA 2020 criteria, an exhaustive search of PubMed and SCOPUS databases was conducted without any filters or restrictions. Inclusion criteria encompassed original English articles focusing on AI's role in kidney cancer surgical training. On the other hand, all non-original articles and articles published in any language other than English were excluded. Two independent reviewers assessed the articles, with a third party settling any disagreement. Study specifics, AI tools, methodologies, endpoints, and outcomes were extracted by the same authors. The Oxford Center for Evidence-Based Medicine's evidence levels were employed to assess the studies. Out of 468 identified records, 14 eligible studies were selected. Potential AI applications in kidney cancer surgical training include analyzing surgical workflow, annotating instruments, identifying tissues, and 3D reconstruction. AI is capable of appraising surgical skills, including the identification of procedural steps and instrument tracking. While AI and augmented reality (AR) enhance training, challenges persist in real-time tracking and registration. The utilization of AI-driven 3D reconstruction proves beneficial for intraoperative guidance and preoperative preparation. Artificial intelligence (AI) shows potential for advancing surgical training by providing unbiased evaluations, personalized feedback, and enhanced learning processes. Yet challenges such as consistent metric measurement, ethical concerns, and data privacy must be addressed. The integration of AI into kidney cancer surgical training offers solutions to training difficulties and a boost to surgical education. However, to fully harness its potential, additional studies are imperative.
Collapse
Affiliation(s)
- Natali Rodriguez Peñaranda
- Department of Urology, Azienda Ospedaliero-Universitaria di Modena, Via Pietro Giardini, 1355, 41126 Baggiovara, Italy; (N.R.P.); (A.E.); (S.F.); (G.B.); (S.D.B.); (S.M.)
| | - Ahmed Eissa
- Department of Urology, Azienda Ospedaliero-Universitaria di Modena, Via Pietro Giardini, 1355, 41126 Baggiovara, Italy; (N.R.P.); (A.E.); (S.F.); (G.B.); (S.D.B.); (S.M.)
- Department of Urology, Faculty of Medicine, Tanta University, Tanta 31527, Egypt
| | - Stefania Ferretti
- Department of Urology, Azienda Ospedaliero-Universitaria di Modena, Via Pietro Giardini, 1355, 41126 Baggiovara, Italy; (N.R.P.); (A.E.); (S.F.); (G.B.); (S.D.B.); (S.M.)
| | - Giampaolo Bianchi
- Department of Urology, Azienda Ospedaliero-Universitaria di Modena, Via Pietro Giardini, 1355, 41126 Baggiovara, Italy; (N.R.P.); (A.E.); (S.F.); (G.B.); (S.D.B.); (S.M.)
| | - Stefano Di Bari
- Department of Urology, Azienda Ospedaliero-Universitaria di Modena, Via Pietro Giardini, 1355, 41126 Baggiovara, Italy; (N.R.P.); (A.E.); (S.F.); (G.B.); (S.D.B.); (S.M.)
| | - Rui Farinha
- Orsi Academy, 9090 Melle, Belgium; (R.F.); (P.D.B.); (R.D.G.); (A.G.G.); (A.M.)
- Urology Department, Lusíadas Hospital, 1500-458 Lisbon, Portugal
| | - Pietro Piazza
- Division of Urology, IRCCS Azienda Ospedaliero-Universitaria di Bologna, 40138 Bologna, Italy;
| | - Enrico Checcucci
- Department of Surgery, FPO-IRCCS Candiolo Cancer Institute, 10060 Turin, Italy;
| | - Inés Rivero Belenchón
- Urology and Nephrology Department, Virgen del Rocío University Hospital, 41013 Seville, Spain;
| | - Alessandro Veccia
- Department of Urology, University of Verona, Azienda Ospedaliera Universitaria Integrata, 37126 Verona, Italy;
| | - Juan Gomez Rivas
- Department of Urology, Hospital Clinico San Carlos, 28040 Madrid, Spain;
| | - Mark Taratkin
- Institute for Urology and Reproductive Health, Sechenov University, 119435 Moscow, Russia;
| | - Karl-Friedrich Kowalewski
- Department of Urology and Urosurgery, University Medical Center Mannheim, Medical Faculty Mannheim, Heidelberg University, 68167 Mannheim, Germany;
| | - Severin Rodler
- Department of Urology, University Hospital LMU Munich, 80336 Munich, Germany;
| | - Pieter De Backer
- Orsi Academy, 9090 Melle, Belgium; (R.F.); (P.D.B.); (R.D.G.); (A.G.G.); (A.M.)
- Department of Human Structure and Repair, Faculty of Medicine and Health Sciences, Ghent University, 9000 Ghent, Belgium
| | - Giovanni Enrico Cacciamani
- USC Institute of Urology, Catherine and Joseph Aresty Department of Urology, Keck School of Medicine, University of Southern California, Los Angeles, CA 90089, USA;
- AI Center at USC Urology, USC Institute of Urology, University of Southern California, Los Angeles, CA 90089, USA
| | - Ruben De Groote
- Orsi Academy, 9090 Melle, Belgium; (R.F.); (P.D.B.); (R.D.G.); (A.G.G.); (A.M.)
| | - Anthony G. Gallagher
- Orsi Academy, 9090 Melle, Belgium; (R.F.); (P.D.B.); (R.D.G.); (A.G.G.); (A.M.)
- Faculty of Life and Health Sciences, Ulster University, Derry BT48 7JL, UK
| | - Alexandre Mottrie
- Orsi Academy, 9090 Melle, Belgium; (R.F.); (P.D.B.); (R.D.G.); (A.G.G.); (A.M.)
| | - Salvatore Micali
- Department of Urology, Azienda Ospedaliero-Universitaria di Modena, Via Pietro Giardini, 1355, 41126 Baggiovara, Italy; (N.R.P.); (A.E.); (S.F.); (G.B.); (S.D.B.); (S.M.)
| | - Stefano Puliatti
- Department of Urology, Azienda Ospedaliero-Universitaria di Modena, Via Pietro Giardini, 1355, 41126 Baggiovara, Italy; (N.R.P.); (A.E.); (S.F.); (G.B.); (S.D.B.); (S.M.)
| |
Collapse
|
23
|
Liu X, Shi J, Li Z, Huang Y, Zhang Z, Zhang C. The Present and Future of Artificial Intelligence in Urological Cancer. J Clin Med 2023; 12:4995. [PMID: 37568397 PMCID: PMC10419644 DOI: 10.3390/jcm12154995] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/05/2023] [Revised: 07/10/2023] [Accepted: 07/27/2023] [Indexed: 08/13/2023] Open
Abstract
Artificial intelligence has drawn more and more attention for both research and application in the field of medicine. It has considerable potential for urological cancer detection, therapy, and prognosis prediction due to its ability to choose features in data to complete a particular task autonomously. Although the clinical application of AI is still immature and faces drawbacks such as insufficient data and a lack of prospective clinical trials, AI will play an essential role in individualization and the whole management of cancers as research progresses. In this review, we summarize the applications and studies of AI in major urological cancers, including tumor diagnosis, treatment, and prognosis prediction. Moreover, we discuss the current challenges and future applications of AI.
Collapse
Affiliation(s)
| | | | | | | | - Zhihong Zhang
- Tianjin Institute of Urology, The Second Hospital of Tianjin Medical University, Tianjin 300211, China; (X.L.)
| | - Changwen Zhang
- Tianjin Institute of Urology, The Second Hospital of Tianjin Medical University, Tianjin 300211, China; (X.L.)
| |
Collapse
|