Intraoperative Imaging Techniques in Oncology

Imaging-based procedures have become well integrated into the diagnosis and management of oncological patients and play a significant role in reducing morbidity and mortality rates. Here we describe the established and upcoming surgical oncological imaging techniques and their impact on cancer management.

Letter to the editor regarding, "Beneath the surface: A systematic review on intraoperative imaging techniques for deep margin assessment in oral squamous cell carcinoma"

Surgical methods for oral squamous cell carcinoma have the potential to improve patient outcomes with the integration of modern imaging tools for deep margin evaluation. This articlesummarises the potential benefits of MRI, FMI, and ultrasound modalities for improving surgical accuracy, based on a wide range of research. Theuses of intraoperative imaging in oral pathology are also covered, along with difficulties including ethical and technological constraints. Important insights to direct future research and implementation efforts in the field of oral cancer surgery are provided, which also examines implications for clinical education and innovation.

Surgical optomics: hyperspectral imaging and deep learning towards precision intraoperative automatic tissue recognition-results from the EX-MACHYNA trial

BACKGROUND

Hyperspectral imaging (HSI), combined with machine learning, can help to identify characteristic tissue signatures enabling automatic tissue recognition during surgery. This study aims to develop the first HSI-based automatic abdominal tissue recognition with human data in a prospective bi-center setting.

METHODS

Data were collected from patients undergoing elective open abdominal surgery at two international tertiary referral hospitals from September 2020 to June 2021. HS images were captured at various time points throughout the surgical procedure. Resulting RGB images were annotated with 13 distinct organ labels. Convolutional Neural Networks (CNNs) were employed for the analysis, with both external and internal validation settings utilized.

RESULTS

A total of 169 patients were included, 73 (43.2%) from Strasbourg and 96 (56.8%) from Verona. The internal validation within centers combined patients from both centers into a single cohort, randomly allocated to the training (127 patients, 75.1%, 585 images) and test sets (42 patients, 24.9%, 181 images). This validation setting showed the best performance. The highest true positive rate was achieved for the skin (100%) and the liver (97%). Misclassifications included tissues with a similar embryological origin (omentum and mesentery: 32%) or with overlaying boundaries (liver and hepatic ligament: 22%). The median DICE score for ten tissue classes exceeded 80%.

CONCLUSION

To improve automatic surgical scene segmentation and to drive clinical translation, multicenter accurate HSI datasets are essential, but further work is needed to quantify the clinical value of HSI. HSI might be included in a new omics science, namely surgical optomics, which uses light to extract quantifiable tissue features during surgery.

Knowledge, attitudes and practices of using Indocyanine Green (ICG) fluorescence in emergency surgery: an international web-based survey in the ARtificial Intelligence in Emergency and trauma Surgery (ARIES)-WSES project

Fluorescence imaging is a real-time intraoperative navigation modality to enhance surgical vision and it can guide emergency surgeons while performing difficult, high-risk surgical procedures. The aim of this study is to assess current knowledge, attitudes, and practices of emergency surgeons in the use of indocyanine green (ICG) in emergency settings. Between March 08, 2023 and April 10, 2023, a questionnaire composed of 27 multiple choice and open-ended questions was sent to 200 emergency surgeons who had previously joined the ARtificial Intelligence in Emergency and trauma Surgery (ARIES) project promoted by the WSES. The questionnaire was developed by an emergency surgeon with an interest in advanced technologies and artificial intelligence. The response rate was 96% (192/200). Responders affirmed that ICG fluorescence can support the performance of difficult surgical procedures in the emergency setting, particularly in the presence of severe inflammation and in evaluating bowel viability. Nevertheless, there were concerns regarding accessibility and availability of fluorescence imaging in emergency settings. Eighty-seven out of 192 (45.3%) respondents have a fluorescence imaging system of vision for both elective and emergency surgical procedures; 32.3% of respondents have this system solely for elective procedures; 21.4% of respondents do not have this system, 15% do not have experience with it, and 38% do not use this imaging in emergency surgery. Less than 1% (2/192) affirmed that ICG fluorescence changed always their intraoperative decision-making. Precision surgery effectively tailors surgical interventions to individual patient characteristics using advanced technology, data analysis and artificial intelligence. ICG fluorescence can serve as a valid and safe tool to guide emergency surgery in different scenarios, such as intestinal ischemia and severe acute cholecystitis. Due to the lack of high-level evidence within this field, a consensus of expert emergency surgeons is needed to encourage stakeholders to increase the availability of fluorescence imaging systems and to support emergency surgeons in implementing ICG fluorescence in their daily practice.

HeiPorSPECTRAL - the Heidelberg Porcine HyperSPECTRAL Imaging Dataset of 20 Physiological Organs

Hyperspectral Imaging (HSI) is a relatively new medical imaging modality that exploits an area of diagnostic potential formerly untouched. Although exploratory translational and clinical studies exist, no surgical HSI datasets are openly accessible to the general scientific community. To address this bottleneck, this publication releases HeiPorSPECTRAL ( https://www.heiporspectral.org ; https://doi.org/10.5281/zenodo.7737674 ), the first annotated high-quality standardized surgical HSI dataset. It comprises 5,758 spectral images acquired with the TIVITA® Tissue and annotated with 20 physiological porcine organs from 8 pigs per organ distributed over a total number of 11 pigs. Each HSI image features a resolution of 480 × 640 pixels acquired over the 500-1000 nm wavelength range. The acquisition protocol has been designed such that the variability of organ spectra as a function of several parameters including the camera angle and the individual can be assessed. A comprehensive technical validation confirmed both the quality of the raw data and the annotations. We envision potential reuse within this dataset, but also its reuse as baseline data for future research questions outside this dataset. Measurement(s) Spectral Reflectance Technology Type(s) Hyperspectral Imaging Sample Characteristic - Organism Sus scrofa.

Application of indocyanine green in surgery: A review of current evidence and implementation in trauma patients

Background: Modern surgical medicine strives to manage trauma while improving outcomes using functional imaging. Identification of viable tissues is crucial for the surgical management of polytrauma and burn patients presenting with soft tissue and hollow viscus injuries. Bowel anastomosis after trauma-related resection is associated with a high rate of leakage. The ability of the surgeon’s bare eye to determine bowel viability remains limited, and the need for a more standardized objective assessment has not yet been fulfilled. Hence, there is a need for more precise diagnostic tools to enhance surgical evaluation and visualization to aid early diagnosis and timely management to minimize trauma-associated complications. Indocyanine green (ICG) coupled with fluorescence angiography is a potential solution for this problem. ICG is a fluorescent dye that responds to near-infrared irradiation. Methods: We conducted a narrative review to address the utility of ICG in the surgical management of patients with trauma as well as elective surgery. Discussion: ICG has many applications in different medical fields and has recently become an important clinical indicator for surgical guidance. However, there is a paucity of information regarding the use of this technology to treat traumas. Recently, angiography with ICG has been introduced in clinical practice to visualize and quantify organ perfusion under several conditions, leading to fewer cases of anastomotic insufficiency. This has great potential to bridge this gap and enhance the clinical outcomes of surgery and patient safety. However, there is no consensus on the ideal dose, time, and manner of administration nor the indications that ICG provides a genuine advantage through greater safety in trauma surgical settings. Conclusions: There is a scarcity of publications describing the use of ICG in trauma patients as a potentially useful strategy to facilitate intraoperative decisions and to limit the extent of surgical resection. This review will improve our understanding of the utility of intraoperative ICG fluorescence in guiding and assisting trauma surgeons to deal with the intraoperative challenges and thus improve the patients’ operative care and safety in the field of trauma surgery.

Quantification of bowel ischaemia using real-time multispectral Single Snapshot Imaging of Optical Properties (SSOP)

BACKGROUND

Single snapshot imaging of optical properties (SSOP) is a relatively new non-invasive, real-time, contrast-free optical imaging technology, which allows for the real-time quantitative assessment of physiological properties, including tissue oxygenation (StO2). This study evaluates the accuracy of multispectral SSOP in quantifying bowel ischaemia in a preclinical experimental model.

METHODS

In six pigs, an ischaemic bowel segment was created by dividing the arcade branches. Five regions of interest (ROIs) were identified on the bowel loop, as follows: ROI 1: central ischaemic; ROI 2: left marginal; ROI 3: left vascularised; ROI 4: right marginal; and ROI 5: right vascularised. The Trident imaging system, specifically developed for real-time tissue oxygenation imaging using SSOP, was used to image before (T0) and after ischaemia induction. Capillary and systemic lactates were measured at each time point (T0, T15, T30, T45, T60), as well as StO2 values acquired by means of SSOP (SSOP-StO2).

RESULTS

The mean value of SSOP-StO2 in ROI 1 was 30.08 ± 6.963 and was significantly lower when compared to marginal ROIs (ROI 2 + ROI 4: 45.67 ± 10.02 p =  < 0.0001), and to vascularised ROIs (ROI 3 + ROI 5: 48.08 ± 7.083 p =  < 0.0001). SSOP-StO2 was significantly correlated with normalised lactates r = - 0.5892 p < 0.0001 and with histology r =- 0.6251 p = 0.0002.

CONCLUSION

Multispectral SSOP allows for a contrast-free accurate assessment of small bowel perfusion identifying physiological tissue oxygenation as confirmed with perfusion biomarkers.

Intraoperative bowel perfusion quantification with hyperspectral imaging: a guidance tool for precision colorectal surgery

BACKGROUND

Poor anastomotic perfusion can cause anastomotic leaks (AL). Hyperspectral imaging (HSI), previously validated experimentally, provides accurate, real-time, contrast-free intestinal perfusion quantification. Clinical experience with HSI is limited. In this study, HSI was used to evaluate bowel perfusion intraoperatively.

METHODS

Fifty-two patients undergoing elective colorectal surgeries for neoplasia (n = 40) or diverticular disease (n = 12), were enrolled. Intestinal perfusion was assessed with HSI (TIVITA®, Diaspective Vision, Am Salzhaff, Germany). This device generates a perfusion heat map reflecting the tissue oxygen saturation (StO2) amount. Prior to anastomose creation, the clinical transection line (CTL) was highlighted on the proximal bowel and imaged with HSI. Upon StO2 heat map evaluation, the hyperspectral transection line (HTL) was identified. In case of CTL/HTL discrepancy > 5 mm, the bowel was always resected at the HTL. HSI outcomes were compared to the clinical ones.

RESULTS

AL occurred in one patient who underwent neoadjuvant radiochemotherapy and ultralow anterior resection for rectal cancer. HSI assessment was feasible in all patients, and StO2-values were significantly higher at proximal segments than distal ones. Twenty-six patients showed CTL/HTL discrepancy, and these patients had a lower mean StO2 (54.55 ± 21.30%) than patients without discrepancy (65.10 ± 21.30%, p = 0.000). Patients undergoing neoadjuvant radiochemotherapy showed a lower StO2 (51.41 ± 23.41%) than non-neoadjuvated patients (60.51 ± 24.98%, p = 0.010).

CONCLUSION

HSI is useful in detecting intraoperatively marginally perfused segments, for which the clinical appreciation is unreliable. Intestinal vascular supply is lower in patients undergoing neoadjuvant radiochemotherapy, and this novel finding together with the clinical impact of HSI perfusion quantification deserves further investigation in larger trials.

Artificial Intelligence in Colorectal Cancer Surgery: Present and Future Perspectives

Artificial intelligence (AI) and computer vision (CV) are beginning to impact medicine. While evidence on the clinical value of AI-based solutions for the screening and staging of colorectal cancer (CRC) is mounting, CV and AI applications to enhance the surgical treatment of CRC are still in their early stage. This manuscript introduces key AI concepts to a surgical audience, illustrates fundamental steps to develop CV for surgical applications, and provides a comprehensive overview on the state-of-the-art of AI applications for the treatment of CRC. Notably, studies show that AI can be trained to automatically recognize surgical phases and actions with high accuracy even in complex colorectal procedures such as transanal total mesorectal excision (TaTME). In addition, AI models were trained to interpret fluorescent signals and recognize correct dissection planes during total mesorectal excision (TME), suggesting CV as a potentially valuable tool for intraoperative decision-making and guidance. Finally, AI could have a role in surgical training, providing automatic surgical skills assessment in the operating room. While promising, these proofs of concept require further development, validation in multi-institutional data, and clinical studies to confirm AI as a valuable tool to enhance CRC treatment.

Spectral organ fingerprints for machine learning-based intraoperative tissue classification with hyperspectral imaging in a porcine model

Visual discrimination of tissue during surgery can be challenging since different tissues appear similar to the human eye. Hyperspectral imaging (HSI) removes this limitation by associating each pixel with high-dimensional spectral information. While previous work has shown its general potential to discriminate tissue, clinical translation has been limited due to the method’s current lack of robustness and generalizability. Specifically, the scientific community is lacking a comprehensive spectral tissue atlas, and it is unknown whether variability in spectral reflectance is primarily explained by tissue type rather than the recorded individual or specific acquisition conditions. The contribution of this work is threefold: (1) Based on an annotated medical HSI data set (9059 images from 46 pigs), we present a tissue atlas featuring spectral fingerprints of 20 different porcine organs and tissue types. (2) Using the principle of mixed model analysis, we show that the greatest source of variability related to HSI images is the organ under observation. (3) We show that HSI-based fully-automatic tissue differentiation of 20 organ classes with deep neural networks is possible with high accuracy (> 95%). We conclude from our study that automatic tissue discrimination based on HSI data is feasible and could thus aid in intraoperative decisionmaking and pave the way for context-aware computer-assisted surgery systems and autonomous robotics.

Border Line Definition Using Hyperspectral Imaging in Colorectal Resections

Simple Summary

Good oxygenation of both bowel ends is an important prerequisite to promote anastomotic healing after colorectal resections. Bowel oxygenation is usually assessed clinically. Hyperspectral imaging is a contactless and contrast-free tool that allows quantifying tissue oxygen intraoperatively. In this study, the results of 105 colorectal resections with hyperspectral imaging are reported.

Abstract

Background: A perfusion deficit is a well-defined and intraoperatively influenceable cause of anastomotic leak (AL). Current intraoperative perfusion assessment methods do not provide objective and quantitative results. In this study, the ability of hyperspectral imaging (HSI) to quantify tissue oxygenation intraoperatively was assessed. Methods: 115 patients undergoing colorectal resections were included in the final analysis. Before anastomotic formation, the bowel was extracted and the resection line was outlined and imaged using a compact HSI camera, in order to provide instantaneously quantitative perfusion assessment. Results: In 105 patients, a clear demarcation line was visible with HSI one minute after marginal artery transection, reaching a plateau after 3 min. In 58 (55.2%) patients, the clinically determined transection line matched with HSI. In 23 (21.9%) patients, the clinically established resection margin was entirely within the less perfused area. In 24 patients (22.8%), the HSI transection line had an irregular course and crossed the clinically established resection line. In four cases, HSI disclosed a clinically undetected lesion of the marginal artery. Conclusions: Intraoperative HSI is safe, well reproducible, and does not disrupt the surgical workflow. It also quantifies bowel surface perfusion. HSI might become an intraoperative guidance tool, potentially preventing postoperative complications.

Knowledge, attitude, and practice of artificial intelligence in emergency and trauma surgery, the ARIES project: an international web-based survey

Aim

We aimed to evaluate the knowledge, attitude, and practices in the application of AI in the emergency setting among international acute care and emergency surgeons.

Methods

An online questionnaire composed of 30 multiple choice and open-ended questions was sent to the members of the World Society of Emergency Surgery between 29th May and 28th August 2021. The questionnaire was developed by a panel of 11 international experts and approved by the WSES steering committee.

Results

200 participants answered the survey, 32 were females (16%). 172 (86%) surgeons thought that AI will improve acute care surgery. Fifty surgeons (25%) were trained, robotic surgeons and can perform it. Only 19 (9.5%) were currently performing it. 126 (63%) surgeons do not have a robotic system in their institution, and for those who have it, it was mainly used for elective surgery. Only 100 surgeons (50%) were able to define different AI terminology. Participants thought that AI is useful to support training and education (61.5%), perioperative decision making (59.5%), and surgical vision (53%) in emergency surgery. There was no statistically significant difference between males and females in ability, interest in training or expectations of AI (p values 0.91, 0.82, and 0.28, respectively, Mann–Whitney U test). Ability was significantly correlated with interest and expectations (p < 0.0001 Pearson rank correlation, rho 0.42 and 0.47, respectively) but not with experience (p = 0.9, rho − 0.01).

Conclusions

The implementation of artificial intelligence in the emergency and trauma setting is still in an early phase. The support of emergency and trauma surgeons is essential for the progress of AI in their setting which can be augmented by proper research and training programs in this area.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13017-022-00413-3.

Surgical data science - from concepts toward clinical translation

Recent developments in data science in general and machine learning in particular have transformed the way experts envision the future of surgery. Surgical Data Science (SDS) is a new research field that aims to improve the quality of interventional healthcare through the capture, organization, analysis and modeling of data. While an increasing number of data-driven approaches and clinical applications have been studied in the fields of radiological and clinical data science, translational success stories are still lacking in surgery. In this publication, we shed light on the underlying reasons and provide a roadmap for future advances in the field. Based on an international workshop involving leading researchers in the field of SDS, we review current practice, key achievements and initiatives as well as available standards and tools for a number of topics relevant to the field, namely (1) infrastructure for data acquisition, storage and access in the presence of regulatory constraints, (2) data annotation and sharing and (3) data analytics. We further complement this technical perspective with (4) a review of currently available SDS products and the translational progress from academia and (5) a roadmap for faster clinical translation and exploitation of the full potential of SDS, based on an international multi-round Delphi process.

Clinical applications for intraoperative optical coherence tomography: a systematic review

In this systematic review, we provide an overview of the current state of intraoperative optical coherence tomography (iOCT). As iOCT technology is increasingly utilized, its current clinical applications and potential uses warrant attention. Here, we categorize the findings of various studies by their respective fields, including the use of iOCT in vitreoretinal surgery, corneal surgery, glaucoma surgery, cataract surgery, and pediatric ophthalmology. The trend observed in recent decades towards performing minimally invasive ophthalmic surgery has caused practitioners to recognize the limitations of using a conventional surgical microscope for intraoperative visualization. Thus, the superior visualization provided by iOCT can improve the safety of these surgical techniques and promote the development of new minimally invasive ophthalmic surgeries. Landmark prospective studies found that iOCT can significantly affect surgical decision making and can cause a subsequent change in surgical strategy, and the use of iOCT has potential to improve surgical outcome. Despite these advantages, however, iOCT is still a relatively new technique, and beginning users of iOCT can encounter limitations that can preclude their reaching the full potential of iOCT and in this respect several improvements are needed.

Total Variation Constrained Graph Manifold Learning Strategy for Cerenkov Luminescence Tomography

Harnessing the power and flexibility of radiolabeled molecules, Cerenkov luminescence tomography (CLT) provides a novel technique for non-invasive visualisation and quantification of viable tumour cells in a living organism. However, owing to the photon scattering effect and the ill-posed inverse problem, CLT still suffers from insufficient spatial resolution and shape recovery in various preclinical applications. In this study, we proposed a total variation constrained graph manifold learning (TV-GML) strategy for achieving accurate spatial location, dual-source resolution, and tumour morphology. TV-GML integrates the isotropic total variation term and dynamic graph Laplacian constraint to make a trade-off between edge preservation and piecewise smooth region reconstruction. Meanwhile, the tetrahedral mesh-Cartesian grid pair method based on the k-nearest neighbour, and the adaptive and composite Barzilai-Borwein method, were proposed to ensure global super linear convergence of the solution of TV-GML. The comparison results of both simulation experiments and in vivo experiments further indicated that TV-GML achieved superior reconstruction performance in terms of location accuracy, dual-source resolution, shape recovery capability, robustness, and in vivo practicability. Significance: We believe that this novel method will be beneficial to the application of CLT for quantitative analysis and morphological observation of various preclinical applications and facilitate the development of the theory of solving inverse problem.

Fundamentals and developments in fluorescence-guided cancer surgery

Fluorescence-guided surgery using tumour-targeted imaging agents has emerged over the past decade as a promising and effective method of intraoperative cancer detection. An impressive number of fluorescently labelled antibodies, peptides, particles and other molecules related to cancer hallmarks have been developed for the illumination of target lesions. New approaches are being implemented to translate these imaging agents into the clinic, although only a few have made it past early-phase clinical trials. For this translational process to succeed, target selection, imaging agents and their related detection systems and clinical implementation have to operate in perfect harmony to enable real-time intraoperative visualization that can benefit patients. Herein, we review key aspects of this imaging cascade and focus on imaging approaches and methods that have helped to shed new light onto the field of intraoperative fluorescence-guided cancer surgery with the singular goal of improving patient outcomes.

Novel Intraoperative Imaging of Gastric Tube Perfusion during Oncologic Esophagectomy—A Pilot Study Comparing Hyperspectral Imaging (HSI) and Fluorescence Imaging (FI) with Indocyanine Green (ICG)

Background: Novel intraoperative imaging techniques, namely, hyperspectral (HSI) and fluorescence imaging (FI), are promising with respect to reducing severe postoperative complications, thus increasing patient safety. Both tools have already been used to evaluate perfusion of the gastric conduit after esophagectomy and before anastomosis. To our knowledge, this is the first study evaluating both modalities simultaneously during esophagectomy. Methods: In our pilot study, 13 patients, who underwent Ivor Lewis esophagectomy and gastric conduit reconstruction, were analyzed prospectively. HSI and FI were recorded before establishing the anastomosis in order to determine its optimum position. Results: No anastomotic leak occurred during this pilot study. In five patients, the imaging methods resulted in a more peripheral adaptation of the anastomosis. There were no significant differences between the two imaging tools, and no adverse events due to the imaging methods or indocyanine green (ICG) injection occurred. Conclusions: Simultaneous intraoperative application of both modalities was feasible and not time consuming. They are complementary with regard to the ideal anastomotic position and may contribute to better surgical outcomes. The impact of their simultaneous application will be proven in consecutive prospective trials with a large patient cohort.

Proof of concept for the sapphire scalpel combining tissue dissection and optical diagnosis

OBJECTIVES

The development of compact diagnostic probes and instruments with an ability to direct access to organs and tissues and integration of these instruments into surgical workflows is an important task of modern physics and medicine. The need for such tools is essential for surgical oncology, where intraoperative visualization and demarcation of tumor margins define further prognosis and survival of patients. In this paper, the possible solution for this intraoperative imaging problem is proposed and its feasibility to detect tumorous tissue is studied experimentally.

METHODS

For this aim, the sapphire scalpel was developed and fabricated using the edge-defined film-fed growth technique aided by mechanical grinding, polishing, and chemical sharpening of the cutting edge. It possesses optical transparency, mechanical strength, chemical inertness, and thermal resistance alongside the presence of the as-grown hollow capillary channels in its volume for accommodating optical fibers. The rounding of the cutting edge exceeds the same for metal scalpels and can be as small as 110 nm. Thanks to these features, sapphire scalpel combines tissue dissection with light delivering and optical diagnosis. The feasibility for the tumor margin detection was studied, including both gelatin-based tissue phantoms and ex vivo freshly excised specimens of the basal cell carcinoma from humans and the glioma model 101.8 from rats. These tumors are commonly diagnosed either non-invasively or intraoperatively using different modalities of fluorescence spectroscopy and imaging, which makes them ideal candidates for our feasibility test. For this purpose, fiber-based spectroscopic measurements of the backscattered laser radiation and the fluorescence signals were carried out in the visible range.

RESULTS

Experimental studies show the feasibility of the proposed sapphire scalpel to provide a 2-mm-resolution of the tumor margins' detection, along with an ability to distinguish the tumor invasion region, which results from analysis of the backscattered optical fields and the endogenous or exogenous fluorescence data.

CONCLUSIONS

Our findings justified a strong potential of the sapphire scalpel for surgical oncology. However, further research and engineering efforts are required to optimize the sapphire scalpel geometry and the optical diagnosis protocols to meet the requirements of oncosurgery, including diagnosis and resection of neoplasms with different localizations and nosologies.

Intraoperative Guidance Using Hyperspectral Imaging: A Review for Surgeons

Hyperspectral imaging (HSI) is a novel optical imaging modality, which has recently found diverse applications in the medical field. HSI is a hybrid imaging modality, combining a digital photographic camera with a spectrographic unit, and it allows for a contactless and non-destructive biochemical analysis of living tissue. HSI provides quantitative and qualitative information of the tissue composition at molecular level in a contrast-free manner, hence making it possible to objectively discriminate between different tissue types and between healthy and pathological tissue. Over the last two decades, HSI has been increasingly used in the medical field, and only recently it has found an application in the operating room. In the last few years, several research groups have used this imaging modality as an intraoperative guidance tool within different surgical disciplines. Despite its great potential, HSI still remains far from being routinely used in the daily surgical practice, since it is still largely unknown to most of the surgical community. The aim of this study is to provide clinical surgeons with an overview of the capabilities, current limitations, and future directions of HSI for intraoperative guidance.

Deep Learning Analysis of In Vivo Hyperspectral Images for Automated Intraoperative Nerve Detection

Nerves are critical structures that may be difficult to recognize during surgery. Inadvertent nerve injuries can have catastrophic consequences for the patient and lead to life-long pain and a reduced quality of life. Hyperspectral imaging (HSI) is a non-invasive technique combining photography with spectroscopy, allowing non-invasive intraoperative biological tissue property quantification. We show, for the first time, that HSI combined with deep learning allows nerves and other tissue types to be automatically recognized in in vivo hyperspectral images. An animal model was used, and eight anesthetized pigs underwent neck midline incisions, exposing several structures (nerve, artery, vein, muscle, fat, skin). State-of-the-art machine learning models were trained to recognize these tissue types in HSI data. The best model was a convolutional neural network (CNN), achieving an overall average sensitivity of 0.91 and a specificity of 1.0, validated with leave-one-patient-out cross-validation. For the nerve, the CNN achieved an average sensitivity of 0.76 and a specificity of 0.99. In conclusion, HSI combined with a CNN model is suitable for in vivo nerve recognition.

Fluorescence Image-Guided Surgery for Thyroid Cancer: Utility for Preventing Hypoparathyroidism

Background: Hypoparathyroidism is one of the most frequent complications of thyroid surgery, especially when associated with lymph node dissection in cases of thyroid cancer. Fluorescence-guided surgery is an emerging tool that appears to help reduce the rate of this complication. The present review aims to highlight the utility of fluorescence imaging in preserving parathyroid glands during thyroid cancer surgery. Methods: We performed a systematic review of the literature according to PRISMA guidelines to identify published studies on fluorescence-guided thyroid surgery with a particular focus on thyroid cancer. Articles were selected and analyzed per indication and type of surgery, autofluorescence or exogenous dye usage, and outcomes. The Methodological Index for Non-Randomized Studies (MINORS) was used to assess the methodological quality of the included articles. Results: Twenty-five studies met the inclusion criteria, with three studies exclusively assessing patients with thyroid cancer. The remaining studies assessed mixed cohorts with thyroid cancer and other thyroid or parathyroid diseases. The majority of the papers support the potential benefit of fluorescence imaging in preserving parathyroid glands in thyroid surgery. Conclusions: Fluorescence-guided surgery is useful in the prevention of post-thyroidectomy hypoparathyroidism via enhanced early identification, visualization, and preservation of the parathyroid glands. These aspects are notably beneficial in cases of associated lymphadenectomy for thyroid cancer.

Computer-assisted technology for enhanced abdominal surgery

The application of computer-based technology to surgery has the potential to enhance the accuracy and outcomes of surgical procedures and perioperative care. Such innovative technologies include the integration of artificial intelligence into surgical decision-making, and the use of three-dimensional (3D) visual imaging, other real-time imaging techniques, and 3D printing technology.

Artificial intelligence indocyanine green (ICG) perfusion for colorectal cancer intra-operative tissue classification

A new concept in intraoperative decision support for tumour delineation is proposed showing that artificial intelligence provides categorising information and interpretation from the images captured during fluorescence-guided colorectal cancer operations. This is potentially applicable to all cancer subtypes and is pertinent to new fluorophore development.A new concept in intraoperative decision support for tumour delineation is proposed showing that artificial intelligence provides categorising information and interpretation from the images captured during fluorescence-guided colorectal cancer operations. This is potentially applicable to all cancer subtypes and is pertinent to new fluorophore development. A new concept in intraoperative decision support for tumour delineation is proposed showing that artificial intelligence provides categorising information and interpretation from the images captured during fluorescence-guided colorectal cancer operations. This is potentially applicable to all cancer subtypes and is pertinent to new fluorophore development.

Shows promise

Perspective on Multimodal Imaging Techniques Coupling Mass Spectrometry and Vibrational Spectroscopy: Picturing the Best of Both Worlds

Studies on complex biological phenomena often combine two or more imaging techniques to collect high-quality comprehensive data directly in situ, preserving the biological context. Mass spectrometry imaging (MSI) and vibrational spectroscopy imaging (VSI) complement each other in terms of spatial resolution and molecular information. In the past decade, several combinations of such multimodal strategies arose in research fields as diverse as microbiology, cancer, and forensics, overcoming many challenges toward the unification of these techniques. Here we focus on presenting the advantages and challenges of multimodal imaging from the point of view of studying biological samples as well as giving a perspective on the upcoming trends regarding this topic. The latest efforts in the field are discussed, highlighting the purpose of the technique for clinical applications.

Super-Selective Intra-Arterial Indocyanine Green Administration for Near-Infrared Fluorescence-Based Positive Staining of Hepatic Segmentation: A Feasibility Study

BACKGROUND

Despite the increasing number of laparoscopic hepatic procedures for the resection of hepatocellular carcinoma (HCC), intraoperative tumor localization and demarcation remains challenging in comparison to open surgery. In this study, we evaluated the feasibility of positive liver segment staining through the super-selective intra-arterial indocyanine green (ICG) administration.

METHODS

Eight patients presenting with a single HCC underwent an interventional vascular procedure followed by laparoscopic surgery. A microcatheter was advanced into the hepatic artery branches perfusing the HCC followed by digital subtraction angiography and angiography computed tomography (angio-CT). Patients were then transferred to the operating room, and a laparoscopic hepatectomy was performed under ultrasound guidance. A 5 mL bolus of ICG with a concentration of .125 mg/mL was injected through the microcatheter, and a near-infrared laparoscope was used to detect the fluorescence signal to assess the correspondence between the fluorescence-based demarcation and the intraoperative ultrasound-based demarcation.

RESULTS

The duration for the angiography procedure was 32.7 +/- 5.3 min, and it took 242 +/- 118 min from the end of angiography procedure until the start of the surgical procedure. In all cases, the fluorescent liver segment was corresponding to the angio-CT findings. In 6/8 cases, fluorescence imaging was considered helpful in the identification of the resection line. In 3 patients, the resection line was changed according to the positively stained liver segment.

CONCLUSION

We successfully demonstrated the feasibility of the super-selective intra-arterial ICG administration for fluorescence-based positive staining of hepatic segmentation during laparoscopic surgery for HCC (NCT04266548).

OR black box and surgical control tower: Recording and streaming data and analytics to improve surgical care

Effective and safe surgery results from a complex sociotechnical process prone to human error. Acquiring large amount of data on surgical care and modelling the process of surgery with artificial intelligence's computational methods could shed lights on system strengths and limitations and enable computer-based smart assistance. With this vision in mind, surgeons and computer scientists have joined forces in a novel discipline called Surgical Data Science. In this regard, operating room (OR) black boxes and surgical control towers are being developed to systematically capture comprehensive data on surgical procedures and to oversee and assist during operating rooms activities, respectively. Most of the early Surgical Data Science works have focused on understanding risks and resilience factors affecting surgical safety, the context and workflow of procedures, and team behaviors. These pioneering efforts in sensing and analyzing surgical activities, together with the advent of precise robotic actuators, bring surgery on the verge of a fourth revolution characterized by smart assistance in perceptual, cognitive and physical tasks. Barriers to implement this vision exist, but the surgical-technical partnerships set by ambitious efforts such as the OR black box and the surgical control tower are working to overcome these roadblocks and translate the vision and early works described in the manuscript into value for patients, surgeons and health systems.

Application of Indocyanine Green Fluorescence as an Adjuvant to Laparoscopic Ultrasound in Minimally Invasive Liver Resection

Background: Indocyanine green (ICG) fluorescence imaging has been extensively used in a variety of applications in visceral surgery. In minimally invasive liver resections, the detection of small superficial hepatic lesions using an intravenous injection of ICG before surgery represents a promising application. Methods: We analyzed 18 consecutive patients who underwent laparoscopic liver resection for superficial malignant tumors, namely 11 patients with hepatocellular carcinoma (HCC), 5 patients with colorectal liver metastases (CRLM), 1 patient with intrahepatic cholangiocarcinoma (ICC), and 1 patient with thyroid cancer metastasis, using ICG fluorescence as an adjuvant tool to intraoperative laparoscopic ultrasound (LUS). Results: An optimal ICG 15-minute clearance retention rate (R15 < 10%) and ICG plasma disappearance rate (<18%/minute) were present in 11 patients (61.1%) and in 14 patients (77.7%), respectively. Liver tumors were 29 in total, including 14 HCCs (48.3%), 13 CRLMs (44.8%), 1 ICC (3.4%), and 1 thyroid cancer metastasis (3.4%). Twenty-nine tumors (100%) were correctly visualized with ICG/fluorescence, as compared with 21 tumors identified with LUS (72.4%). After complete liver mobilization, ICG staining allowed to identify more superficial lesions (early HCC and small CRLM) in posterolateral segments (Segments 6 and 7) as compared with LUS (14 versus 10 lesions). In addition, in segments usually treated laparoscopically (e.g., left lateral segments), ICG was superior to LUS (10 versus 6 lesions) to identify superficial early HCC in patients with macronodular cirrhosis. Conclusions: ICG visual feedback might substitute the tactile feedback of the hand and might in some cases act as a "booster" of LUS for superficial hepatic lesions.

Computer Vision in the Operating Room: Opportunities and Caveats

Effectiveness of computer vision techniques has been demonstrated through a number of applications, both within and outside healthcare. The operating room environment specifically is a setting with rich data sources compatible with computational approaches and high potential for direct patient benefit. The aim of this review is to summarize major topics in computer vision for surgical domains. The major capabilities of computer vision are described as an aid to surgical teams to improve performance and contribute to enhanced patient safety. Literature was identified through leading experts in the fields of surgery, computational analysis and modeling in medicine, and computer vision in healthcare. The literature supports the application of computer vision principles to surgery. Potential applications within surgery include operating room vigilance, endoscopic vigilance, and individual and team-wide behavioral analysis. To advance the field, we recommend collecting and publishing carefully annotated datasets. Doing so will enable the surgery community to collectively define well-specified common objectives for automated systems, spur academic research, mobilize industry, and provide benchmarks with which we can track progress. Leveraging computer vision approaches through interdisciplinary collaboration and advanced approaches to data acquisition, modeling, interpretation, and integration promises a powerful impact on patient safety, public health, and financial costs.

Evaluation of a novel navigation platform for laparoscopic liver surgery with organ deformation compensation using injected fiducials

In laparoscopic liver resection, surgeons conventionally rely on anatomical landmarks detected through a laparoscope, preoperative volumetric images and laparoscopic ultrasound to compensate for the challenges of minimally invasive access. Image guidance using optical tracking and registration procedures is a promising tool, although often undermined by its inaccuracy. This study evaluates a novel surgical navigation solution that can compensate for liver deformations using an accurate and effective registration method. The proposed solution relies on a robotic C-arm to perform registration to preoperative CT/MRI image data and allows for intraoperative updates during resection using fluoroscopic images. Navigation is offered both as a 3D liver model with real-time instrument visualization, as well as an augmented reality overlay on the laparoscope camera view. Testing was conducted through a pre-clinical trial which included four porcine models. Accuracy of the navigation system was measured through two evaluation methods: liver surface fiducials reprojection and a comparison between planned and navigated resection margins. Target Registration Error with the fiducials evaluation shows that the accuracy in the vicinity of the lesion was 3.78±1.89 mm. Resection margin evaluations resulted in an overall median accuracy of 4.44 mm with a maximum error of 9.75 mm over the four subjects. The presented solution is accurate enough to be potentially clinically beneficial for surgical guidance in laparoscopic liver surgery.

Simultaneous computer-assisted assessment of mucosal and serosal perfusion in a model of segmental colonic ischemia

BACKGROUND

Fluorescence-based enhanced reality (FLER) enables the quantification of fluorescence signal dynamics, which can be superimposed onto real-time laparoscopic images by using a virtual perfusion cartogram. The current practice of perfusion assessment relies on visualizing the bowel serosa. The aim of this experimental study was to quantify potential differences in mucosal and serosal perfusion levels in an ischemic colon segment.

METHODS

An ischemic colon segment was created in 12 pigs. Simultaneous quantitative mucosal and serosal fluorescence imaging was obtained via intravenous indocyanine green injection (0.2 mg/kg), using two near-infrared camera systems, and computer-assisted FLER analysis. Lactate levels were measured in capillary blood of the colonic wall at seven regions of interest (ROIs) as determined with FLER perfusion cartography: the ischemic zone (I), the proximal and distal vascularized areas (PV, DV), and the 50% perfusion threshold proximally and distally at the mucosal and serosal side (P50M, P50S, D50M, D50S).

RESULTS

The mean ischemic zone as measured (mm) for the mucosal side was significantly larger than the serosal one (56.3 ± 21.3 vs. 40.8 ± 14.9, p = 0.001) with significantly lower lactate values at the mucosal ROIs. There was a significant weak inverse correlation between lactate and slope values for the defined ROIs (r = - 0.2452, p = 0.0246).

CONCLUSIONS

Mucosal ischemic zones were larger than serosal zones. These results suggest that an assessment of bowel perfusion from the serosal side only can underestimate the extent of ischemia. Further studies are required to predict the optimal resection margin and anastomotic site.

Demarcation Line Assessment in Anatomical Liver Resection: An Overview

Anatomical liver resection (ALR) is the preferred oncological approach for the treatment of primary liver malignancies, such as hepatocellular carcinoma and intrahepatic cholangiocarcinoma. The demarcation line (DL) is formed by means of selective vascular occlusion and is used by surgeons to guide ALR. Emerging intraoperative technologies are playing a major role to enhance the surgeon’s vision and ensure a precise oncologic surgery. In this article, a brief overview of modalities to assess the DL during ALRs is presented, from the established conventional techniques to future perspectives.

Intraoperative imaging for remnant viability assessment in bilateral posterior retroperitoneoscopic partial adrenalectomy in an experimental model

BACKGROUND

A surgical approach preserving functional adrenal tissue allows biochemical cure while avoiding the need for lifelong steroid replacement. The aim of this experimental study was to evaluate the impact of intraoperative imaging during bilateral partial adrenalectomy on remnant perfusion and function.

METHODS

Five pigs underwent bilateral posterior retroperitoneoscopic central adrenal gland division (9 divided glands, 1 undivided). Intraoperative perfusion assessment included computer-assisted quantitative fluorescence imaging, contrast-enhanced CT, confocal laser endomicroscopy (CLE) and local lactate sampling. Specimen analysis after completion adrenalectomy (10 adrenal glands) comprised mitochondrial activity and electron microscopy.

RESULTS

Fluorescence signal intensity evolution over time was significantly lower in the cranial segment of each adrenal gland (mean(s.d.) 0·052(0·057) versus 0·133(0·057) change in intensity per s for cranial versus caudal parts respectively; P = 0·020). Concordantly, intraoperative CT in the portal phase demonstrated significantly lower contrast uptake in cranial segments (P = 0·031). In CLE, fluorescein contrast was observed in all caudal segments, but in only four of nine cranial segments (P = 0·035). Imaging findings favouring caudal perfusion were congruent, with significantly lower local capillary lactate levels caudally (mean(s.d.) 5·66(5·79) versus 11·58(6·53) mmol/l for caudal versus cranial parts respectively; P = 0·008). Electron microscopy showed more necrotic cells cranially (P = 0·031). There was no disparity in mitochondrial activity (respiratory rates, reactive oxygen species and hydrogen peroxide production) between the different segments.

CONCLUSION

In a model of bilateral partial adrenalectomy, three intraoperative imaging modalities consistently discriminated between regular and reduced adrenal remnant perfusion. By avoiding circumferential dissection, mitochondrial function was preserved in each segment of the adrenal glands. Surgical relevance Preservation of adrenal tissue to maintain postoperative function is essential in bilateral and hereditary adrenal pathologies. There is interindividual variation in residual adrenocortical stress capacity, and the minimal functional remnant size is unknown. New intraoperative imaging technologies allow improved remnant size and perfusion assessment. Fluorescence imaging and contrast-enhanced intraoperative CT showed congruent results in evaluation of perfusion. Intraoperative imaging can help to visualize the remnant vascular supply in partial adrenalectomy. Intraoperative assessment of perfusion may foster maximal functional tissue preservation in bilateral adrenal pathologies and procedures.

Fluorescence-based cholangiography: preliminary results from the IHU-IRCAD-EAES EURO-FIGS registry

INTRODUCTION

Near-infrared fluorescence cholangiography (NIRF-C) is a popular application of fluorescence image-guided surgery (FIGS). NIRF-C requires near-infrared optimized laparoscopes and the injection of a fluorophore, most frequently Indocyanine Green (ICG), to highlight the biliary anatomy. It is investigated as a tool to increase safety during cholecystectomy. The European registry on FIGS (EURO-FIGS: www.euro-figs.eu ) aims to obtain a snapshot of the current practices of FIGS across Europe. Data on NIRF-C are presented.

METHODS

EURO-FIGS is a secured online database which collects anonymized data on surgical procedures performed using FIGS. Data collected for NIRF-C include gender, age, Body Mass Index (BMI), pathology, NIR device, ICG dose, ICG timing of administration before intraoperative visualization, visualization (Y/N) of biliary structures such as the cystic duct (CD), the common bile duct (CBD), the CD-CBD junction, the common hepatic duct (CHD), Visualization scores, adverse reactions to ICG, operative time, and surgical complications.

RESULTS

Fifteen surgeons (12 European surgical centers) uploaded 314 cases of NIRF-C during cholecystectomy (cholelithiasis n = 249, cholecystitis n = 58, polyps n = 7), using 4 different NIR devices. ICG doses (mg/kg) varied largely (mean 0.28 ± 0.17, median 0.3, range: 0.02-0.62). Similarly, injection-to-visualization timing (minutes) varied largely (mean 217 ± 357; median 57), ranging from 1 min (direct intragallbladder injection in 2 cases) to 3120 min (n = 2 cases). Visualization scores before dissection were significantly correlated, at univariate analysis, with ICG timing (all structures), ICG dose (CD-CBD), device (CD and CD-CBD), surgeon (CD and CD-CBD), and pathology (CD and CD-CBD). BMI was not correlated. At multivariate analysis, pathology and timing remained significant factors affecting the visualization scores of all three structures, whereas ICG dose remained correlated with HD visualization only.

CONCLUSIONS

The EURO-FIGS registry has confirmed a wide disparity in ICG dose and timing in NIRF-C. EURO-FIGS can represent a valuable tool to promote and monitor FIGS-related educational and consensus activities in Europe.

Computer-assisted quantification and visualization of bowel perfusion using fluorescence-based enhanced reality in left-sided colonic resections

BACKGROUND

Fluorescence-based enhanced reality (FLER) is a computer-based quantification method of fluorescence angiographies to evaluate bowel perfusion. The aim of this prospective trial was to assess the clinical feasibility and to correlate FLER with metabolic markers of perfusion, during colorectal resections.

METHODS

FLER analysis and visualization was performed in 22 patients (diverticulitis n = 17; colorectal cancer n = 5) intra- and extra-abdominally during distal and proximal resection, respectively. The fluorescence signal of indocyanine green (0.2 mg/kg) was captured using a near-infrared camera and computed to create a virtual color-coded cartography. This was overlaid onto the bowel (enhanced reality). It helped to identify regions of interest (ROIs) where samples were subsequently obtained. Resections were performed strictly guided according to clinical decision. On the surgical specimen, samplings were made at different ROIs to measure intestinal lactates (mmol/L) and mitochondria efficiency as acceptor control ratio (ACR).

RESULTS

The native (unquantified) fluorescent signal diffused to obvious ischemic areas during the distal appreciation. Proximally, a lower diffusion of ICG was observed. Five anastomotic complications occurred. The expected values of local capillary lactates were correlated with the measured values both proximally (3.62 ± 2.48 expected vs. 3.17 ± 2.8 actual; rho 0.89; p = 0.0006) and distally (4.5 ± 3 expected vs. 4 ± 2.5 actual; rho 0.73; p = 0.0021). FLER values correlated with ACR at the proximal site (rho 0.76; p = 0.04) and at the ischemic zone (rho 0.71; p = 0.01). In complicated cases, lactates at the proximal resection site were higher (5.8 ± 4.5) as opposed to uncomplicated cases (2.45 ± 1.5; p = 0.008). ACR was reduced proximally in complicated (1.3 ± 0.18) vs. uncomplicated cases (1.68 ± 0.3; p = 0.023).

CONCLUSIONS

FLER allows to image the quantified fluorescence signal in augmented reality and provides a reproducible estimation of bowel perfusion (NCT02626091).

Effectiveness of near-infrared fluorescent cholangiography in the identification of cystic duct-common hepatic duct anatomy in comparison to magnetic resonance cholangio-pancreatography: a preliminary study

INTRODUCTION

Bile duct injury represents the most serious complication of LC, with an incidence of 0.3-0.7% resulting in a significant impact on quality-of-life, overall survival, and frequent medico-legal litigations. Near-infrared fluorescent cholangiography (NIRF-C) represents a novel intra-operative imaging technique that allows a real-time enhanced visualization of the extrahepatic biliary tree by fluorescence. The role of routine use of pre-operative magnetic resonance cholangio-pancreatography (MRCP) to better clarify the biliary anatomy before laparoscopic cholecystectomy is still a matter of debate. The primary aim of this study was to evaluate the effectiveness of NIRF-C in the detection of cystic duct-common hepatic duct anatomy intra-operatively in comparison with pre-operative MRCP.

METHODS

Data from 26 consecutive patients with symptomatic cholelithiasis or chronic cholecystitis, who underwent elective laparoscopic cholecystectomy with intra-operative fluorescent cholangiography and pre-operative MRCP examination between January 2018 and May 2018, were analyzed. Three selected features of the cystic duct-common hepatic duct anatomy were identified and analyzed by the two different imaging methods: insertion of cystic duct, cystic duct-common hepatic duct junction, and cystic duct course.

RESULTS

Fluorescent cholangiography was performed successfully in all twenty-six patients undergoing elective laparoscopic cholecystectomy. The visualization of cystic duct was reported in 23 out of 26 cases, showing an overall diagnostic accuracy of 86.9%. The level of insertion, course, and wall implantation of cystic duct were achieved by NIRF-C with diagnostic accuracy values of 65.2%, 78.3%, and 91.3%, respectively in comparison with MRCP data. No bile duct injuries were reported.

CONCLUSION

Fluorescent cholangiography can be considered a useful imaging diagnostic tool comparable to MRCP for detailed intra-operative visualization of the cystic duct-common hepatic duct anatomy during elective laparoscopic cholecystectomies.

Quantitative fluorescence angiography versus hyperspectral imaging to assess bowel ischemia: A comparative study in enhanced reality

BACKGROUND

Fluorescence-based enhanced reality is a software that provides quantitative fluorescence angiography by computing the fluorescence intensity time-to-peak after intravenous indocyanine green. Hyperspectral imaging is a contrast-free, optical imaging modality which measures tissue oxygenation.

METHODS

In 8 pigs, an ischemic bowel segment created by dividing the arcade branches was imaged using hyperspectral imaging and fluorescence-based enhanced reality. Tissue oxygenation values were acquired through a hyperspectral imaging system. Subsequently, fluorescence angiography was performed using a near-infrared laparoscopic camera after intravenous injection of 0.2 mg/kg of indocyanine green. The time-to-peak fluorescence signal was analyzed through a proprietary software to realize a perfusion map. This was overlaid onto real-time images to obtain fluorescence-based enhanced reality. Simultaneously, 9 adjacent regions of interest were selected and superimposed onto the real-time video, thereby obtaining hyperspectral-based enhanced reality. Fluorescence-based enhanced reality and hyperspectral-based enhanced reality were superimposed allowing a comparison of both imaging modalities. Local capillary lactate levels were sampled at the regions of interest. Two prediction models using the local capillary lactate levels were extrapolated based on both imaging systems.

RESULTS

For all regions of interest, the mean local capillary lactate levels were 4.67 ± 4.34 mmol/L, the mean tissue oxygenation was 45.9 ± 18.9%, and the mean time-to-peak was 10 ± 9.4 seconds. Pearson's test between fluorescence-based enhanced reality-time-to-peak and hyperspectral imaging-tissue oxygenation at the corresponding regions of interest gave an R = -0.66 (P < .0001). The hyperspectral imaging lactate prediction model proved more accurate than the fluorescence-based enhanced reality-based model (P < .0001).

CONCLUSION

Bowel perfusion was quantified using hyperspectral imaging and fluorescence angiography. Hyperspectral imaging yielded more accurate results than fluorescence angiography. Hyperspectral-based enhanced reality may prove to be a useful, contrast-free intraoperative tool to quantify bowel ischemia.