How molecular imaging will enable robotic precision surgery : The role of artificial intelligence, augmented reality, and navigation

The Rise of Molecular Image-Guided Robotic Surgery

Following early acceptance by urologists, the use of surgical robotic platforms is rapidly spreading to other surgical fields. This empowerment of surgical perception via robotic advances occurs in parallel to developments in intraoperative molecular imaging. Convergence of these efforts creates a logical incentive to advance the decades-old image-guided robotics paradigm. This yields new radioguided surgery strategies set to optimally exploit the symbiosis between the growing clinical translation of robotics and molecular imaging. These strategies intend to advance surgical precision by increasing dexterity and optimizing surgical decision-making. In this state-of-the-art review, topic-related developments in chemistry (tracer development) and engineering (medical device development) are discussed, and future scientific robotic growth markets for molecular imaging are presented.

Lessons learned in application driven imaging agent design for image-guided surgery

To meet the growing demand for intraoperative molecular imaging, the development of compatible imaging agents plays a crucial role. Given the unique requirements of surgical applications compared to diagnostics and therapy, maximizing translational potential necessitates distinctive imaging agent designs. For effective surgical guidance, exogenous signatures are essential and are achievable through a diverse range of imaging labels such as (radio)isotopes, fluorescent dyes, or combinations thereof. To achieve optimal in vivo utility a balanced molecular design of the tracer as a whole is required, which ensures a harmonious effect of the imaging label with the affinity and specificity (e.g., pharmacokinetics) of a pharmacophore/targeting moiety. This review outlines common design strategies and the effects of refinements in the molecular imaging agent design on the agent's pharmacological profile. This includes the optimization of affinity, pharmacokinetics (including serum binding and target mediated background), biological clearance route, the achievable signal intensity, and the effect of dosing hereon.

Towards safer robot-assisted surgery: A markerless augmented reality framework

Robot-assisted surgery is rapidly developing in the medical field, and the integration of augmented reality shows the potential to improve the operation performance of surgeons by providing more visual information. In this paper, we proposed a markerless augmented reality framework to enhance safety by avoiding intra-operative bleeding, which is a high risk caused by collision between surgical instruments and delicate blood vessels (arteries or veins). Advanced stereo reconstruction and segmentation networks are compared to find the best combination to reconstruct the intra-operative blood vessel in 3D space for registration with the pre-operative model, and the minimum distance detection between the instruments and the blood vessel is implemented. A robot-assisted lymphadenectomy is emulated on the da Vinci Research Kit in a dry lab, and ten human subjects perform this operation to explore the usability of the proposed framework. The result shows that the augmented reality framework can help the users to avoid the dangerous collision between the instruments and the delicate blood vessel while not introducing an extra load. It provides a flexible framework that integrates augmented reality into the medical robotic platform to enhance safety during surgery.

Photonics-powered augmented reality skin electronics for proactive healthcare: multifaceted opportunities

Rapid technological advancements have created opportunities for new solutions in various industries, including healthcare. One exciting new direction in this field of innovation is the combination of skin-based technologies and augmented reality (AR). These dermatological devices allow for the continuous and non-invasive measurement of vital signs and biomarkers, enabling the real-time diagnosis of anomalies, which have applications in telemedicine, oncology, dermatology, and early diagnostics. Despite its many potential benefits, there is a substantial information vacuum regarding using flexible photonics in conjunction with augmented reality for medical purposes. This review explores the current state of dermal augmented reality and flexible optics in skin-conforming sensing platforms by examining the obstacles faced thus far, including technical hurdles, demanding clinical validation standards, and problems with user acceptance. Our main areas of interest are skills, chiroptical properties, and health platform applications, such as optogenetic pixels, spectroscopic imagers, and optical biosensors. My skin-enhanced spherical dichroism and powerful spherically polarized light enable thorough physical inspection with these augmented reality devices: diabetic tracking, skin cancer diagnosis, and cardiovascular illness: preventative medicine, namely blood pressure screening. We demonstrate how to accomplish early prevention using case studies and emergency detection. Finally, it addresses real-world obstacles that hinder fully realizing these materials' extraordinary potential in advancing proactive and preventative personalized medicine, including technical constraints, clinical validation gaps, and barriers to widespread adoption.

Current Application of Navigation Systems in Robotic-Assisted and Laparoscopic Partial Nephrectomy: Focus on the Improvement of Surgical Performance and Outcomes

Kidney cancer represents the third most prevalent malignancy among all types of genitourinary cancer worldwide. Currently, there is a growing trend of employing partial nephrectomy for the management of large and complex tumors. Surgical outcomes are associated with some amendable surgical factors, including warm ischemic time, pedicle clamping, preserved volume of renal parenchyma, appropriate surgical strategy, and precise resection of the tumor. Improving surgical performance is pivotal for achieving favorable surgical outcomes. Due to advancements in imaging visualization technology and the shift of the medical paradigm toward precision medicine, an increasing number of navigation systems have been implemented in partial nephrectomy procedures. The navigation system can assist surgeons in formulating optimal surgical strategies and enhance the safety, precision, and feasibility of resecting complex renal tumors. In this review, we provide an overview of currently available navigation systems and their feasible applications, with a focus on how they contribute to the improvement of surgical performance and outcomes during robotic-assisted and laparoscopic partial nephrectomy.

Molecular image-guided surgery in gynaecological cancer: where do we stand?

PURPOSE

The aim of this review is to give an overview of the current status of molecular image-guided surgery in gynaecological malignancies, from both clinical and technological points of view.

METHODS

A narrative approach was taken to describe the relevant literature, focusing on clinical applications of molecular image-guided surgery in gynaecology, preoperative imaging as surgical roadmap, and intraoperative devices.

RESULTS

The most common clinical application in gynaecology is sentinel node biopsy (SNB). Other promising approaches are receptor-target modalities and occult lesion localisation. Preoperative SPECT/CT and PET/CT permit a roadmap for adequate surgical planning. Intraoperative detection modalities span from 1D probes to 2D portable cameras and 3D freehand imaging.

CONCLUSION

After successful application of radio-guided SNB and SPECT, innovation is leaning towards hybrid modalities, such as hybrid tracer and fusion of imaging approaches including SPECT/CT and PET/CT. Robotic surgery, as well as augmented reality and virtual reality techniques, is leading to application of these innovative technologies to the clinical setting, guiding surgeons towards a precise, personalised, and minimally invasive approach.

Revolutionising Breast Surgery: A Comprehensive Review of Robotic Innovations in Breast Surgery and Reconstruction

Robotic innovations in breast surgery have ushered in a new era of precision, safety, and patient-centred care. This comprehensive review explores the multifaceted realm of robotic breast surgery, from preoperative planning to postoperative outcomes, learning curves for surgeons, and the implications for healthcare policies. We examine the ethical considerations, cost-effectiveness, and future directions, including integrating artificial intelligence and telesurgery. Key findings reveal that robotic systems provide improved surgical precision, reduced complications, and enhanced patient satisfaction. Ethical concerns encompass informed consent, resource allocation, and equitable access. The future of breast surgery lies in continued research and development, ensuring that robotics becomes a standard of care accessible to all patients. This technology is reshaping breast surgery and offering new possibilities for minimally invasive, patient-centred care, ultimately redefining the standards of care in this critical field of medicine.

Continuous innovation in precision radio-guided surgery

Since its origins, nuclear medicine has faced technological changes that led to modifying operating modes and adapting protocols. In the field of radioguided surgery, the incorporation of preoperative scintigraphic imaging and intraoperative detection with the gamma probe provided a definitive boost to sentinel lymph node biopsy to become a standard procedure for melanoma and breast cancer. The various technological innovations and consequent adaptation of protocols come together in the coexistence of the disruptive and the gradual. As obvious examples we have the introduction of SPECT/CT in the preoperative field and Drop-in probes in the intraoperative field. Other innovative aspects with possible application in radio-guided surgery are based on the application of artificial intelligence, navigation and telecare.

Setting-up a training programme for intraoperative molecular imaging and sentinel node mapping: how to teach? How to learn?

BACKGROUND

The current expansion of image-guided surgery is closely related to the role played by radio-guided surgery in supporting the sentinel node (SN) procedure during more than three decades. The so-called triple approach (lymphoscintigraphy, gamma probe detection and blue dye) was not only essential in the seminal validation of the SN procedure but also a first collective learning effort based on skill transfer and outcome-related evaluation which laid the fundaments to delineate the field of intraoperative molecular imaging (IMI) based on a similar multimodality approach and multidisciplinary practice.

METHODS

These elements are also becoming valid in the current incorporation of SPECT/CT and PET/CT to existing and new protocols of IMI procedures and SN mapping concerning other clinical applications. On the other hand, there is a growing tendency to combine novel modern technologies in an allied role with gamma guidance in the operating room following the development of hybrid tracers and multimodal detection approaches. Against this background, learning initiatives are required for professionals working in this area.

RESULTS

This objective has led to a group of European practitioners with large experience in SN mapping and IMI applications to give shape to a programme made up out of specific learning modules aimed to be used as a conductive thread in peripherical or centralised training instances concerning the topic.

CONCLUSION

The presented work, written as a tutorial review, is placed in an available prior-art context and is primarily aimed at medical and paramedical practitioners as well as at hardware and software developers.

Delphi consensus project on prostate-specific membrane antigen (PSMA)-targeted surgery-outcomes from an international multidisciplinary panel

PURPOSE

Prostate-specific membrane antigen (PSMA) is increasingly considered as a molecular target to achieve precision surgery for prostate cancer. A Delphi consensus was conducted to explore expert views in this emerging field and to identify knowledge and evidence gaps as well as unmet research needs that may help change practice and improve oncological outcomes for patients.

METHODS

One hundred and five statements (scored by a 9-point Likert scale) were distributed through SurveyMonkey®. Following evaluation, a consecutive second round was performed to evaluate consensus (16 statements; 89% response rate). Consensus was defined using the disagreement index, assessed by the research and development project/University of California, Los Angeles appropriateness method.

RESULTS

Eighty-six panel participants (72.1% clinician, 8.1% industry, 15.1% scientists, and 4.7% other) participated, most with a urological background (57.0%), followed by nuclear medicine (22.1%). Consensus was obtained on the following: (1) The diagnostic PSMA-ligand PET/CT should ideally be taken < 1 month before surgery, 1-3 months is acceptable; (2) a 16-20-h interval between injection of the tracer and surgery seems to be preferred; (3) PSMA targeting is most valuable for identification of nodal metastases; (4) gamma, fluorescence, and hybrid imaging are the preferred guidance technologies; and (5) randomized controlled clinical trials are required to define oncological value. Regarding surgical margin assessment, the view on the value of PSMA-targeted surgery was neutral or inconclusive. A high rate of "cannot answer" responses indicates further study is necessary to address knowledge gaps (e.g., Cerenkov or beta-emissions).

CONCLUSIONS

This Delphi consensus provides guidance for clinicians and researchers that implement or develop PSMA-targeted surgery technologies. Ultimately, however, the consensus should be backed by randomized clinical trial data before it may be implemented within the guidelines.

Artificial Intelligence in Gynaecological Malignancies: Perspectives of a Clinical Oncologist

Gynecological malignancies are treated with a multi-disciplinary approach. There are two important factors, the stage and Karnofsky performance status scale (KPS) of patients, which guide the treatment strategy by single or multiple modalities in terms of surgery, radiotherapy, or chemotherapy. Various aspects are included in the workflow of gynecological malignancies, like screening, diagnosis in each individual, treatment modalities, and finally, follow-up to see for outcomes leading to the development of new research protocols. The quality data plays an important role in every step. Artificial Intelligence (AI) will play an important role if it is developed in the above-mentioned steps. AI is already established partially in every aspect of the management of gynecological cancer. It needs to be strengthened and incorporated further in a more robust form. This needs an association between clinicians, software engineers, and stakeholders. This article reviews the role of AI in various steps of the workflow of gynecological malignancies and discusses a few clinical aspects that may be researched to find solutions by AI.

Randomized controlled trials and alternative study designs in surgical oncology

Surgery is central to the cure of most solid cancers and an integral part of modern multimodal cancer management for early and advanced stage cancers. Decisions made by surgeons and multidisciplinary team members are based on best available knowledge for the defined clinical situation at hand. While surgery is both an art and a science, good decision-making requires data that are robust, valid, representative and, applicable to most if not all patients with a specific cancer. Such data largely comes from clinical observations and registries, and more preferably from trials conducted with the specific purpose of arriving at new answers. As part of the ESSO core curriculum development an increased focus has been put on the need to enhance research literacy among surgical candidates. As an expansion of the curriculum catalogue list and to enhance the educational value, we here present a set of principles and emerging concepts which applies to surgical oncologist for reading, understanding, planning and contributing to future surgeon-led cancer trials.

The Role of Augmented Reality in the Advancement of Minimally Invasive Surgery Procedures: A Scoping Review

The purpose of this review was to analyze the evidence on the role of augmented reality (AR) in the improvement of minimally invasive surgical (MIS) procedures. A scoping literature search of the PubMed and ScienceDirect databases was performed to identify articles published in the last five years that addressed the direct impact of AR technology on MIS procedures or that addressed an area of education or clinical care that could potentially be used for MIS development. A total of 359 studies were screened and 31 articles were reviewed in depth and categorized into three main groups: Navigation, education and training, and user-environment interfaces. A comparison of studies within the different application groups showed that AR technology can be useful in various disciplines to advance the development of MIS. Although AR-guided navigation systems do not yet offer a precision advantage, benefits include improved ergonomics and visualization, as well as reduced surgical time and blood loss. Benefits can also be seen in improved education and training conditions and improved user-environment interfaces that can indirectly influence MIS procedures. However, there are still technical challenges that need to be addressed to demonstrate added value to patient care and should be evaluated in clinical trials with sufficient patient numbers or even in systematic reviews or meta-analyses.

Click-on fluorescence detectors: using robotic surgical instruments to characterize molecular tissue aspects

Fluorescence imaging is increasingly being implemented in surgery. One of the drawbacks of its application is the need to switch back-and-forth between fluorescence- and white-light-imaging settings and not being able to dissect safely under fluorescence guidance. The aim of this study was to engineer 'click-on' fluorescence detectors that transform standard robotic instruments into molecular sensing devices that enable the surgeon to detect near-infrared (NIR) fluorescence in a white-light setting. This NIR-fluorescence detector setup was engineered to be press-fitted onto standard forceps instruments of the da Vinci robot. Following system characterization in a phantom setting (i.e., spectral properties, sensitivity and tissue signal attenuation), the performance with regard to different clinical indocyanine green (ICG) indications (e.g., angiography and lymphatic mapping) was determined via robotic surgery in pigs. To evaluate in-human applicability, the setup was also used for ICG-containing lymph node specimens from robotic prostate cancer surgery. The resulting Click-On device allowed for NIR ICG signal identification down to a concentration of 4.77 × 10^-6 mg/ml. The fully assembled system could be introduced through the trocar and grasping, and movement abilities of the instrument were preserved. During surgery, the system allowed for the identification of blood vessels and assessment of vascularization (i.e., bowel, bladder and kidney), as well as localization of pelvic lymph nodes. During human specimen evaluation, it was able to distinguish sentinel from non-sentinel lymph nodes. With this introduction of a NIR-fluorescence Click-On sensing detector, a next step is made towards using surgical instruments in the characterization of molecular tissue aspects.

Where Robotic Surgery Meets the Metaverse

With a focus on hepatobiliary surgery, the review by Giannone et al [...].

The current status and future prospects for molecular imaging-guided precision surgery

Molecular imaging technologies are increasingly used to diagnose, monitor, and guide treatment of i.e., cancer. In this review, the current status and future prospects of the use of molecular imaging as an instrument to help realize precision surgery is addressed with focus on the main components that form the conceptual basis of intraoperative molecular imaging. Paramount for successful interventions is the relevance and accessibility of surgical targets. In addition, selection of the correct combination of imaging agents and modalities is critical to visualize both microscopic and bulk disease sites with high affinity and specificity. In this context developments within engineering/imaging physics continue to drive the growth of image-guided surgery. Particularly important herein is enhancement of sensitivity through improved contrast and spatial resolution, features that are critical if sites of cancer involvement are not to be overlooked during surgery. By facilitating the connection between surgical planning and surgical execution, digital surgery technologies such as computer-aided visualization nicely complement these technologies. The complexity of image guidance, combined with the plurality of technologies that are becoming available, also drives the need for evaluation mechanisms that can objectively score the impact that technologies exert on the performance of healthcare professionals and outcome improvement for patients.

Artificial Intelligence: Present and Future Potential for Solid Organ Transplantation

Artificial intelligence (AI) refers to computer algorithms used to complete tasks that usually require human intelligence. Typical examples include complex decision-making and- image or speech analysis. AI application in healthcare is rapidly evolving and it undoubtedly holds an enormous potential for the field of solid organ transplantation. In this review, we provide an overview of AI-based approaches in solid organ transplantation. Particularly, we identified four key areas of transplantation which could be facilitated by AI: organ allocation and donor-recipient pairing, transplant oncology, real-time immunosuppression regimes, and precision transplant pathology. The potential implementations are vast—from improved allocation algorithms, smart donor-recipient matching and dynamic adaptation of immunosuppression to automated analysis of transplant pathology. We are convinced that we are at the beginning of a new digital era in transplantation, and that AI has the potential to improve graft and patient survival. This manuscript provides a glimpse into how AI innovations could shape an exciting future for the transplantation community.

EGFR-targeted fluorescent imaging using the da Vinci® Firefly™ camera for gallbladder cancer

Background

Fluorescent imaging may aid with the precise diagnosis and treatment of patients with gallbladder cancer. In this study, we sought to demonstrate whether the da Vinci® surgical system and Firefly™ camera could detect EGFR-targeted fluorescent images in orthotopic mouse models of gallbladder cancer.

Methods

An orthotopic mouse model of gallbladder cancer was created by injecting NOZ gallbladder cancer cells mixed with Matrigel into the gallbladder. In vivo imaging of subcutaneous and orthotopic gallbladder tumors was performed after the injection of DyLight 650- or 800-conjugated EGFR antibody.

Results

Western blotting, flow cytometry, and confocal microscopy showed the presence of EGFR in NOZ cells, but not in HEK293 cells. Subcutaneous NOZ cell tumors fluoresced after injection with fluorescent EGFR antibody, but subcutaneous HEK293 tumors did not. Fluorescent EGFR antibody made orthotopic NOZ tumors fluoresce, with an intensity stronger than that in the surrounding normal tissues. Histochemical examination confirmed the location of the tumors inside the gallbladder and adjacent liver parenchyma. Fluorescent signal was also detected in orthotopic gallbladder tumors with Firefly™ camera.

Conclusion

Our study showed that fluorescent EGFR antibodies and the Firefly camera in the da Vinci system can detect fluorescing gallbladder tumors, which demonstrates their potential use for molecular imaging-based prevision surgery in the near future.

Interventional nuclear medicine's contribution to molecularly targeted precision surgery

The surgical approach to different pathologies, not only oncological, has evolved. As Veronesi's group has coined very graphically, we are moving from "maximum tolerable treatments to minimum effective treatments" and this journey cannot be carried out in any other way than through a multidisciplinary and multimodality approach. Multidisciplinary, because collaboration between surgeons, oncologists, radiologists, nuclear physicians, pathologists, and all those involved in patient follow-up is necessary, and multimodality, because we must move towards precision surgery tailored to each patient in which, on the part of Nuclear Medicine, hybrid imaging (SPECT/CT and PET/CT), bimodal tracers, the use of new allies such as ultrasound or our own adaptation to robotic surgery have a great deal to say. A wide range of possibilities is built on the solid foundation of preoperative scintigraphy, which makes it possible to identify the target tissues and whose knowledge prior to surgery allows the necessary surgical approach to be considered for each patient.

Looking for a perfect match: multimodal combinations of Raman spectroscopy for biomedical applications

Raman spectroscopy has shown very promising results in medical diagnostics by providing label-free and highly specific molecular information of pathological tissue ex vivo and in vivo. Nevertheless, the high specificity of Raman spectroscopy comes at a price, i.e., low acquisition rate, no direct access to depth information, and limited sampling areas. However, a similar case regarding advantages and disadvantages can also be made for other highly regarded optical modalities, such as optical coherence tomography, autofluorescence imaging and fluorescence spectroscopy, fluorescence lifetime microscopy, second-harmonic generation, and others. While in these modalities the acquisition speed is significantly higher, they have no or only limited molecular specificity and are only sensitive to a small group of molecules. It can be safely stated that a single modality provides only a limited view on a specific aspect of a biological specimen and cannot assess the entire complexity of a sample. To solve this issue, multimodal optical systems, which combine different optical modalities tailored to a particular need, become more and more common in translational research and will be indispensable diagnostic tools in clinical pathology in the near future. These systems can assess different and partially complementary aspects of a sample and provide a distinct set of independent biomarkers. Here, we want to give an overview on the development of multimodal systems that use RS in combination with other optical modalities to improve the diagnostic performance.

The Click-On gamma probe, a second-generation tethered robotic gamma probe that improves dexterity and surgical decision-making

Purpose

Decision-making and dexterity, features that become increasingly relevant in (robot-assisted) minimally invasive surgery, are considered key components in improving the surgical accuracy. Recently, DROP-IN gamma probes were introduced to facilitate radioguided robotic surgery. We now studied if robotic DROP-IN radioguidance can be further improved using tethered Click-On designs that integrate gamma detection onto the robotic instruments themselves.

Methods

Using computer-assisted drawing software, 3D printing and precision machining, we created a Click-On probe containing two press-fit connections and an additional grasping moiety for a ProGrasp instrument combined with fiducials that could be video tracked using the Firefly laparoscope. Using a dexterity phantom, the duration of the specific tasks and the path traveled could be compared between use of the Click-On or DROP-IN probe. To study the impact on surgical decision-making, we performed a blinded study, in porcine models, wherein surgeons had to identify a hidden ⁵⁷Co-source using either palpation or Click-On radioguidance.

Results

When assembled onto a ProGrasp instrument, while preserving grasping function and rotational freedom, the fully functional prototype could be inserted through a 12-mm trocar. In dexterity assessments, the Click-On provided a 40% reduction in movements compared to the DROP-IN, which converted into a reduction in time, path length, and increase in straightness index. Radioguidance also improved decision-making; task-completion rate increased by 60%, procedural time was reduced, and movements became more focused.

Conclusion

The Click-On gamma probe provides a step toward full integration of radioguidance in minimal invasive surgery. The value of this concept was underlined by its impact on surgical dexterity and decision-making.