Autofluorescence lifetime augmented reality as a means for real-time robotic surgery guidance in human patients

A decade of progress: bringing mixed reality image-guided surgery systems in the operating room

Advancements in mixed reality (MR) have led to innovative approaches in image-guided surgery (IGS). In this paper, we provide a comprehensive analysis of the current state of MR in image-guided procedures across various surgical domains. Using the Data Visualization View (DVV) Taxonomy, we analyze the progress made since a 2013 literature review paper on MR IGS systems. In addition to examining the current surgical domains using MR systems, we explore trends in types of MR hardware used, type of data visualized, visualizations of virtual elements, and interaction methods in use. Our analysis also covers the metrics used to evaluate these systems in the operating room (OR), both qualitative and quantitative assessments, and clinical studies that have demonstrated the potential of MR technologies to enhance surgical workflows and outcomes. We also address current challenges and future directions that would further establish the use of MR in IGS.

Dual excitation spectral autofluorescence lifetime and reflectance imaging for fast macroscopic characterization of tissues

Advancements in optical imaging techniques have revolutionized the field of biomedical research, allowing for the comprehensive characterization of tissues and their underlying biological processes. Yet, there is still a lack of tools to provide quantitative and objective characterization of tissues that can aid clinical assessment in vivo to enhance diagnostic and therapeutic interventions. Here, we present a clinically viable fiber-based imaging system combining time-resolved spectrofluorimetry and reflectance spectroscopy to achieve fast multiparametric macroscopic characterization of tissues. An essential feature of the setup is its ability to perform dual wavelength excitation in combination with recording time-resolved fluorescence data in several spectral intervals. Initial validation of this bimodal system was carried out in freshly resected human colorectal cancer specimens, where we demonstrated the ability of the system to differentiate normal from malignant tissues based on their autofluorescence and reflectance properties. To further highlight the complementarity of autofluorescence and reflectance measurements and demonstrate viability in a clinically relevant scenario, we also collected in vivo data from the skin of a volunteer. Altogether, integration of these modalities in a single platform can offer multidimensional characterization of tissues, thus facilitating a deeper understanding of biological processes and potentially advancing diagnostic and therapeutic approaches in various medical applications.

Fluorescence in depth: integration of spectroscopy and imaging with Raman, IR, and CD for advanced research

Fluorescence spectroscopy serves as a vital technique for studying the interaction between light and fluorescent molecules. It encompasses a range of methods, each presenting unique advantages and applications. This technique finds utility in various chemical studies. This review discusses Fluorescence spectroscopy, its branches such as Time-Resolved Fluorescence Spectroscopy (TRFS) and Fluorescence Lifetime Imaging Microscopy (FLIM), and their integration with other spectroscopic methods, including Raman, Infrared (IR), and Circular Dichroism (CD) spectroscopies. By delving into these methods, we aim to provide a comprehensive understanding of the capabilities and significance of fluorescence spectroscopy in scientific research, highlighting its diverse applications and the enhanced understanding it brings when combined with other spectroscopic methods. This review looks at each technique's unique features and applications. It discusses the prospects of their combined use in advancing scientific understanding and applications across various domains.

Single-sample image-fusion upsampling of fluorescence lifetime images

Fluorescence lifetime imaging microscopy (FLIM) provides detailed information about molecular interactions and biological processes. A major bottleneck for FLIM is image resolution at high acquisition speeds due to the engineering and signal-processing limitations of time-resolved imaging technology. Here, we present single-sample image-fusion upsampling, a data-fusion approach to computational FLIM super-resolution that combines measurements from a low-resolution time-resolved detector (that measures photon arrival time) and a high-resolution camera (that measures intensity only). To solve this otherwise ill-posed inverse retrieval problem, we introduce statistically informed priors that encode local and global correlations between the two "single-sample" measurements. This bypasses the risk of out-of-distribution hallucination as in traditional data-driven approaches and delivers enhanced images compared, for example, to standard bilinear interpolation. The general approach laid out by single-sample image-fusion upsampling can be applied to other image super-resolution problems where two different datasets are available.

Label-Free Assessment of Key Biological Autofluorophores: Material Characteristics and Opportunities for Clinical Applications

Autofluorophores are endogenous fluorescent compounds that naturally occur in the intra and extracellular spaces of all tissues and organs. Most have vital biological functions - like the metabolic cofactors NAD(P)H and FAD+, as well as the structural protein collagen. Others are considered to be waste products - like lipofuscin and advanced glycation end products - which accumulate with age and are associated with cellular dysfunction. Due to their natural fluorescence, these materials have great utility for enabling non-invasive, label-free assays with direct ties to biological function. Numerous technologies, with different advantages and drawbacks, are applied to their assessment, including fluorescence lifetime imaging microscopy, hyperspectral microscopy, and flow cytometry. Here, the applications of label-free autofluorophore assessment are reviewed for clinical and health-research applications, with specific attention to biomaterials, disease detection, surgical guidance, treatment monitoring, and tissue assessment - fields that greatly benefit from non-invasive methodologies capable of continuous, in vivo characterization.

Identification of new head and neck squamous cell carcinoma molecular imaging targets

OBJECTIVES

Intraoperative fluorescence imaging (FI) of head and neck squamous cell carcinoma (HNSCC) is performed to identify tumour-positive surgical margins, currently using epidermal growth factor receptor (EGFR) as imaging target. EGFR, not exclusively present in HNSCC, may result in non-specific tracer accumulation in normal tissues. We aimed to identify new potential HNSCC FI targets.

MATERIALS AND METHODS

Publicly available transcriptomic data were collected, and a biostatistical method (Transcriptional Adaptation to Copy Number Alterations (TACNA)-profiling) was applied. TACNA-profiling captures downstream effects of CNAs on mRNA levels, which may translate to protein-level overexpression. Overexpressed genes were identified by comparing HNSCC versus healthy oral mucosa. Potential targets, selected based on overexpression and plasma membrane expression, were immunohistochemically stained. Expression was compared to EGFR on paired biopsies of HNSCC, adjacent macroscopically suspicious mucosa, and healthy mucosa.

RESULTS

TACNA-profiling was applied on 111 healthy oral mucosa and 410 HNSCC samples, comparing expression levels of 19,635 genes. The newly identified targets were glucose transporter-1 (GLUT-1), placental cadherin (P-cadherin), monocarboxylate transporter-1 (MCT-1), and neural/glial antigen-2 (NG2), and were evaluated by IHC on samples of 31 patients. GLUT-1 was expressed in 100 % (median; range: 60-100 %) of tumour cells, P-cadherin in 100 % (50-100 %), EGFR in 70 % (0-100 %), MCT-1 in 30 % (0-100 %), and NG2 in 10 % (0-70 %). GLUT-1 and P-cadherin showed higher expression than EGFR (p < 0.001 and p = 0.015).

CONCLUSIONS

The immunohistochemical confirmation of TACNA-profiling results showed significantly higher GLUT-1 and P-cadherin expression than EGFR, warranting further investigation as HNSCC FI targets.

Fibre-based fluorescence-lifetime imaging microscopy: a real-time biopsy guidance tool for suspected lung cancer

Lung cancer is the most common cause of cancer-related deaths worldwide. Early detection improves outcomes, however, existing sampling techniques are associated with suboptimal diagnostic yield and procedure-related complications. Autofluorescence-based fluorescence-lifetime imaging microscopy (FLIM), a technique which measures endogenous fluorophore decay rates, may aid identification of optimal biopsy sites in suspected lung cancer. Our fibre-based fluorescence-lifetime imaging system, utilising 488 nm excitation, which is deliverable via existing diagnostic platforms, enables real-time visualisation and lifetime analysis of distal alveolar lung structure. We evaluated the diagnostic accuracy of the fibre-based fluorescence-lifetime imaging system to detect changes in fluorescence lifetime in freshly resected ex vivo lung cancer and adjacent healthy tissue as a first step towards future translation. The study compares paired non-small cell lung cancer (NSCLC) and non-cancerous tissues with gold standard diagnostic pathology to assess the performance of the technique. Paired NSCLC and non-cancerous lung tissues were obtained from thoracic resection patients (N=21). A clinically compatible 488 nm fluorescence-lifetime endomicroscopy platform was used to acquire simultaneous fluorescence intensity and lifetime images. Fluorescence lifetimes were calculated using a computationally-lightweight, rapid lifetime determination method. Fluorescence lifetime was significantly reduced in ex vivo lung cancer, compared with non-cancerous lung tissue [mean ± standard deviation (SD), 1.79±0.40 vs. 2.15±0.26 ns, P<0.0001], and fluorescence intensity images demonstrated distortion of alveolar elastin autofluorescence structure. Fibre-based fluorescence-lifetime imaging demonstrated good performance characteristics for distinguishing lung cancer, from adjacent non-cancerous tissue, with 81.0% sensitivity and 71.4% specificity. Our novel fibre-based fluorescence-lifetime imaging system, which enables label-free imaging and quantitative lifetime analysis, discriminates ex vivo lung cancer from adjacent healthy tissue. This minimally invasive technique has potential to be translated as a real-time biopsy guidance tool, capable of optimising diagnostic accuracy in lung cancer.

Metaverse Wearables for Immersive Digital Healthcare: A Review

The recent exponential growth of metaverse technology has been instrumental in reshaping a myriad of sectors, not least digital healthcare. This comprehensive review critically examines the landscape and future applications of metaverse wearables toward immersive digital healthcare. The key technologies and advancements that have spearheaded the metamorphosis of metaverse wearables are categorized, encapsulating all-encompassed extended reality, such as virtual reality, augmented reality, mixed reality, and other haptic feedback systems. Moreover, the fundamentals of their deployment in assistive healthcare (especially for rehabilitation), medical and nursing education, and remote patient management and treatment are investigated. The potential benefits of integrating metaverse wearables into healthcare paradigms are multifold, encompassing improved patient prognosis, enhanced accessibility to high-quality care, and high standards of practitioner instruction. Nevertheless, these technologies are not without their inherent challenges and untapped opportunities, which span privacy protection, data safeguarding, and innovation in artificial intelligence. In summary, future research trajectories and potential advancements to circumvent these hurdles are also discussed, further augmenting the incorporation of metaverse wearables within healthcare infrastructures in the post-pandemic era.

Anatomy-Specific Classification Model Using Label-Free FLIm to Aid Intraoperative Surgical Guidance of Head and Neck Cancer

Intraoperative identification of head and neck cancer tissue is essential to achieve complete tumor resection and mitigate tumor recurrence. Mesoscopic fluorescence lifetime imaging (FLIm) of intrinsic tissue fluorophores emission has demonstrated the potential to demarcate the extent of the tumor in patients undergoing surgical procedures of the oral cavity and the oropharynx. Here, we report FLIm-based classification methods using standard machine learning models that account for the diverse anatomical and biochemical composition across the head and neck anatomy to improve tumor region identification. Three anatomy-specific binary classification models were developed (i.e., "base of tongue," "palatine tonsil," and "oral tongue"). FLIm data from patients (N = 85) undergoing upper aerodigestive oncologic surgery were used to train and validate the classification models using a leave-one-patient-out cross-validation method. These models were evaluated for two classification tasks: (1) to discriminate between healthy and cancer tissue, and (2) to apply the binary classification model trained on healthy and cancer to discriminate dysplasia through transfer learning. This approach achieved superior classification performance compared to models that are anatomy-agnostic; specifically, a ROC-AUC of 0.94 was for the first task and 0.92 for the second. Furthermore, the model demonstrated detection of dysplasia, highlighting the generalization of the FLIm-based classifier. Current findings demonstrate that a classifier that accounts for tumor location can improve the ability to accurately identify surgical margins and underscore FLIm's potential as a tool for surgical guidance in head and neck cancer patients, including those subjects of robotic surgery.

3D Ultrasound and MRI in Assessing Resection Margins during Tongue Cancer Surgery: A Research Protocol for a Clinical Diagnostic Accuracy Study

Surgery is the primary treatment for tongue cancer. The goal is a complete resection of the tumor with an adequate margin of healthy tissue around the tumor.Inadequate margins lead to a high risk of local cancer recurrence and the need for adjuvant therapies. Ex vivo imaging of the resected surgical specimen has been suggested for margin assessment and improved surgical results. Therefore, we have developed a novel three-dimensional (3D) ultrasound imaging technique to improve the assessment of resection margins during surgery. In this research protocol, we describe a study comparing the accuracy of 3D ultrasound, magnetic resonance imaging (MRI), and clinical examination of the surgical specimen to assess the resection margins during cancer surgery. Tumor segmentation and margin measurement will be performed using 3D ultrasound and MRI of the ex vivo specimen. We will determine the accuracy of each method by comparing the margin measurements and the proportion of correctly classified margins (positive, close, and free) obtained by each technique with respect to the gold standard histopathology.

Intraoperative Imaging Techniques to Improve Surgical Resection Margins of Oropharyngeal Squamous Cell Cancer: A Comprehensive Review of Current Literature

Inadequate resection margins in head and neck squamous cell carcinoma surgery necessitate adjuvant therapies such as re-resection and radiotherapy with or without chemotherapy and imply increasing morbidity and worse prognosis. On the other hand, taking larger margins by extending the resection also leads to avoidable increased morbidity. Oropharyngeal squamous cell carcinomas (OPSCCs) are often difficult to access; resections are limited by anatomy and functionality and thus carry an increased risk for close or positive margins. Therefore, there is a need to improve intraoperative assessment of resection margins. Several intraoperative techniques are available, but these often lead to prolonged operative time and are only suitable for a subgroup of patients. In recent years, new diagnostic tools have been the subject of investigation. This study reviews the available literature on intraoperative techniques to improve resection margins for OPSCCs. A literature search was performed in Embase, PubMed, and Cochrane. Narrow band imaging (NBI), high-resolution microendoscopic imaging, confocal laser endomicroscopy, frozen section analysis (FSA), ultrasound (US), computed tomography scan (CT), (auto) fluorescence imaging (FI), and augmented reality (AR) have all been used for OPSCC. NBI, FSA, and US are most commonly used and increase the rate of negative margins. Other techniques will become available in the future, of which fluorescence imaging has high potential for use with OPSCC.

Click-on fluorescence detectors: using robotic surgical instruments to characterize molecular tissue aspects

Fluorescence imaging is increasingly being implemented in surgery. One of the drawbacks of its application is the need to switch back-and-forth between fluorescence- and white-light-imaging settings and not being able to dissect safely under fluorescence guidance. The aim of this study was to engineer 'click-on' fluorescence detectors that transform standard robotic instruments into molecular sensing devices that enable the surgeon to detect near-infrared (NIR) fluorescence in a white-light setting. This NIR-fluorescence detector setup was engineered to be press-fitted onto standard forceps instruments of the da Vinci robot. Following system characterization in a phantom setting (i.e., spectral properties, sensitivity and tissue signal attenuation), the performance with regard to different clinical indocyanine green (ICG) indications (e.g., angiography and lymphatic mapping) was determined via robotic surgery in pigs. To evaluate in-human applicability, the setup was also used for ICG-containing lymph node specimens from robotic prostate cancer surgery. The resulting Click-On device allowed for NIR ICG signal identification down to a concentration of 4.77 × 10^-6 mg/ml. The fully assembled system could be introduced through the trocar and grasping, and movement abilities of the instrument were preserved. During surgery, the system allowed for the identification of blood vessels and assessment of vascularization (i.e., bowel, bladder and kidney), as well as localization of pelvic lymph nodes. During human specimen evaluation, it was able to distinguish sentinel from non-sentinel lymph nodes. With this introduction of a NIR-fluorescence Click-On sensing detector, a next step is made towards using surgical instruments in the characterization of molecular tissue aspects.

Autofluorescence Image-Guided Endoscopy in the Management of Upper Aerodigestive Tract Tumors

At this juncture, autofluorescence and narrow-band imaging have resurfaced in the medicine arena in parallel with current technology advancement. The emergence of newly developed optical instrumentation in addition to the discovery of new fluorescence biomolecules have contributed to a refined management of diseases and tumors, especially in the management of upper aerodigestive tract tumors. The advancement in multispectral imaging and micro-endoscopy has also escalated the trends further in the setting of the management of this tumor, in order to gain not only the best treatment outcomes but also facilitate early tumor diagnosis. This includes the usage of autofluorescence endoscopy for screening, diagnosis and treatment of this tumor. This is crucial, as microtumoral deposit at the periphery of the gross tumor can be only assessed via an enhanced endoscopy and even more precisely with autofluorescence endoscopic techniques. Overall, with this new technique, optimum management can be achieved for these patients. Hence, the treatment outcomes can be improved and patients are able to attain better prognosis and survival.

Intraoperative In Vivo Imaging Modalities in Head and Neck Cancer Surgical Margin Delineation: A Systematic Review

Surgical margin status is one of the strongest prognosticators in predicting patient outcomes in head and neck cancer, yet head and neck surgeons continue to face challenges in the accurate detection of these margins with the current standard of care. Novel intraoperative imaging modalities have demonstrated great promise for potentially increasing the accuracy and efficiency in surgical margin delineation. In this current study, we collated and analyzed various intraoperative imaging modalities utilized in head and neck cancer to evaluate their use in discriminating malignant from healthy tissues. The authors conducted a systematic database search through PubMed/Medline, Web of Science, and EBSCOhost (CINAHL). Study screening and data extraction were performed and verified by the authors, and more studies were added through handsearching. Here, intraoperative imaging modalities are described, including optical coherence tomography, narrow band imaging, autofluorescence, and fluorescent-tagged probe techniques. Available sensitivities and specificities in delineating cancerous from healthy tissues ranged from 83.0% to 100.0% and 79.2% to 100.0%, respectively, across the different imaging modalities. Many of these initial studies are in small sample sizes, with methodological differences that preclude more extensive quantitative comparison. Thus, there is impetus for future larger studies examining and comparing the efficacy of these intraoperative imaging technologies.

Imaging approaches for monitoring three-dimensional cell and tissue culture systems

The past decade has seen an increasing demand for more complex, reproducible and physiologically relevant tissue cultures that can mimic the structural and biological features of living tissues. Monitoring the viability, development and responses of such tissues in real-time are challenging due to the complexities of cell culture physical characteristics and the environments in which these cultures need to be maintained in. Significant developments in optics, such as optical manipulation, improved detection and data analysis, have made optical imaging a preferred choice for many three-dimensional (3D) cell culture monitoring applications. The aim of this review is to discuss the challenges associated with imaging and monitoring 3D tissues and cell culture, and highlight topical label-free imaging tools that enable bioengineers and biophysicists to non-invasively characterise engineered living tissues.

Deep learning in macroscopic diffuse optical imaging

SIGNIFICANCE

Biomedical optics system design, image formation, and image analysis have primarily been guided by classical physical modeling and signal processing methodologies. Recently, however, deep learning (DL) has become a major paradigm in computational modeling and has demonstrated utility in numerous scientific domains and various forms of data analysis.

AIM

We aim to comprehensively review the use of DL applied to macroscopic diffuse optical imaging (DOI).

APPROACH

First, we provide a layman introduction to DL. Then, the review summarizes current DL work in some of the most active areas of this field, including optical properties retrieval, fluorescence lifetime imaging, and diffuse optical tomography.

RESULTS

The advantages of using DL for DOI versus conventional inverse solvers cited in the literature reviewed herein are numerous. These include, among others, a decrease in analysis time (often by many orders of magnitude), increased quantitative reconstruction quality, robustness to noise, and the unique capability to learn complex end-to-end relationships.

CONCLUSIONS

The heavily validated capability of DL's use across a wide range of complex inverse solving methodologies has enormous potential to bring novel DOI modalities, otherwise deemed impractical for clinical translation, to the patient's bedside.

Current Trends in Precision Medicine and Next-Generation Sequencing in Head and Neck Cancer

OPINION STATEMENT

As the field of oncology enters the era of precision medicine and targeted therapies, we have come to realize that there may be no single "magic bullet" for patients with head and neck cancer. While immune check point inhibitors and some targeted therapeutics have shown great promise in improving oncologic outcomes, the current standard of care in most patients with head and neck squamous cell carcinoma (HNSCC) remains a combination of surgery, radiation, and/or cytotoxic chemotherapy. Nevertheless, advances in precision medicine, next-generation sequencing (NGS), and targeted therapies have a potential future in the treatment of HNSCC. These roles include increased patient treatment stratification based on predictive biomarkers or targetable mutations and novel combinatorial regimens with existing HNSCC treatments. There remain challenges to precision medicine and NGS in HNSCC, including intertumor and intratumor heterogeneity, challenging targets, and need for further trials validating the utility of NGS and precision medicine. Additionally, there is a need for evidence-based practice guidelines to assist clinicians on how to appropriately incorporate NGS in care for HNSCC. In this review, we describe the current state of precision medicine and NGS in HNSCC and opportunities for future advances in this challenging but important field.

First in patient assessment of brain tumor infiltrative margins using simultaneous time-resolved measurements of 5-ALA-induced PpIX fluorescence and tissue autofluorescence

SIGNIFICANCE

5-aminolevulinic acid (5-ALA)-induced protoporphyrin IX (PpIX) fluorescence is currently used for image-guided glioma resection. Typically, this widefield imaging method highlights the bulk of high-grade gliomas, but it underperforms at the infiltrating edge where PpIX fluorescence is not visible to the eyes. Fluorescence lifetime imaging (FLIm) has the potential to detect PpIX fluorescence below the visible detection threshold. Moreover, simultaneous acquisition of time-resolved nicotinamide adenine (phosphate) dinucleotide [NAD(P)H] fluorescence may provide metabolic information from the tumor environment to further improve overall tumor detection.

AIM

We investigate the ability of pulse sampling, fiber-based FLIm to simultaneously image PpIX and NAD(P)H fluorescence of glioma infiltrative margins in patients.

APPROACH

A mesoscopic fiber-based point-scanning FLIm device (355 nm pulses) was used to simultaneously resolve the fluorescence decay of PpIX (629/53 nm) and NAD(P)H (470/28 nm). The FLIm device enabled data acquisition at room light and rapid (<33  ms) augmentation of FLIm parameters on the surgical field-of-view. FLIm measurements from superficial tumors and tissue areas around the resection margins were performed on three glioblastoma patients in vivo following inspection of PpIX visible fluorescence with a conventional neurosurgical microscope. Microbiopsies were collected from FLIm imaged areas for histopathological evaluation.

RESULTS

The average lifetime from PpIX and NAD(P)H fluorescence distinguished between tumor and surrounding tissue. FLIm measurements of resection margins presented a range of PpIX and NAD(P)H lifetime values (τPpIX       ∼  3 to 14 ns, τNAD(P)H  =  3 to 6 ns) associated with unaffected tissue and areas of low-density tumor infiltration.

CONCLUSIONS

Intraoperative FLIm could simultaneously detect the emission of PpIX and NAD(P)H from patients in vivo during craniotomy procedures. This approach doubles as a clinical tool to identify tumor areas while performing tissue resection and as a research tool to study tumor microenvironmental changes in vivo. Intraoperative FLIm of 5-ALA-induced PpIX and tissue autofluorescence makes a promising surgical adjunct to guide tumor resection surgery.

Toward Mechanochromic Soft Material-Based Visual Feedback for Electronics-Free Surgical Effectors

A chromogenically reversible, mechanochromic pressure sensor is integrated into a mininvasive surgical grasper compatible with the da Vinci robotic surgical system. The sensorized effector, also featuring two soft-material jaws, encompasses a mechanochromic polymeric inset doped with functionalized spiropyran (SP) molecule, designed to activate mechanochromism at a chosen pressure and providing a reversible color change. Considering such tools are systematically in the visual field of the operator during surgery, color change of the mechanochromic effector can help avoid tissue damage. No electronics is required to control the devised visual feedback. SP-doping of polydimethylsiloxane (2.5:1 prepolymer/curing agent weight ratio) permits to modulate the mechanochromic activation pressure, with lower values around 1.17 MPa for a 2% wt. SP concentration, leading to a shorter chromogenic recovery time of 150 s at room temperature (25 °C) under green light illumination. Nearly three-times shorter recovery time is observed at body temperature (37 °C). To the best of knowledge, this study provides the first demonstration of mechanochromic materials in surgery, in particular to sensorize unpowered surgical effectors, by avoiding dramatic increases in tool complexity due to additional electronics, thus fostering their application. The proposed sensing strategy can be extended to further tools and scopes.

Robotic and Endoscopic Approaches to Head and Neck Surgery

Technological developments have disrupted the practice of medicine throughout history. Endoscopic and robotic techniques in head and neck surgery have emerged over the past half-century and have been incrementally adapted to expanding indications within otolaryngology. Robotic and endoscopic surgery have an established role in treatment of oropharyngeal and laryngeal cancers, reducing surgical morbidity and improving survival relative to traditional open approaches. Surgical treatment of human papillomavirus-mediated oropharyngeal cancer via transoral robotic surgery offers equivalent oncologic and functional outcomes relative to radiotherapy. Newer iterations of single-port robotic systems continue to expand the scope of robotics in head and neck surgery.

Multispectral fluorescence lifetime imaging device with a silicon avalanche photodetector

We report the design, development, and characterization of a novel multi-spectral fluorescence lifetime measurement device incorporating solid-state detectors and automated gain control. For every excitation pulse (∼1 µJ, 600 ps), this device records complete fluorescence decay from multiple spectral channels simultaneously within microseconds, using a dedicated UV enhanced avalanche photodetector and analog to digital convert (2.5 GS/s) in each channel. Fast (<2 ms) channel-wise dynamic range adjustment maximizes the signal-to-noise ratio. Fluorophores with known lifetime ranging from 0.5-6.0 ns were used to demonstrate the device accuracy. Current results show the clear benefits of this device compared to existing devices employing microchannel-plate photomultiplier tubes. This is demonstrated by 5-fold reduction of lifetime measurement variability in identical conditions, independent gain adjustment in each spectral band, and 4-times faster imaging speed. The use of solid-state detectors will also facilitate future improved performance and miniaturization of the instrument.

Mesoscopic fluorescence lifetime imaging: Fundamental principles, clinical applications and future directions

Fluorescence lifetime imaging (FLIm) is an optical spectroscopic imaging technique capable of real-time assessments of tissue properties in clinical settings. Label-free FLIm is sensitive to changes in tissue structure and biochemistry resulting from pathological conditions, thus providing optical contrast to identify and monitor the progression of disease. Technical and methodological advances over the last two decades have enabled the development of FLIm instrumentation for real-time, in situ, mesoscopic imaging compatible with standard clinical workflows. Herein, we review the fundamental working principles of mesoscopic FLIm, discuss the technical characteristics of current clinical FLIm instrumentation, highlight the most commonly used analytical methods to interpret fluorescence lifetime data and discuss the recent applications of FLIm in surgical oncology and cardiovascular diagnostics. Finally, we conclude with an outlook on the future directions of clinical FLIm.

Precision head and neck surgery: robotics and surgical vision technology

PURPOSE OF REVIEW

As the molecular basis of head and neck cancer becomes more clearly defined, precision medicine has gradually refined the multidisciplinary treatment paradigm for patients with oropharyngeal cancer. Although precision medicine is often thought to refer to new molecular diagnostics or unique medical therapy, the recent adoption of robotic surgery has ushered in the era of 'precision' head and neck surgery.

RECENT FINDINGS

Surgeons now routinely utilize a virtual reality environment to reduce the morbidity of head and neck surgical care and improve oncologic and functional outcomes. The development and subsequent FDA approval of the da Vinci SP system, a single-arm system with a flexible camera, has allowed for improved visualization and access to the deep oropharynx and larynx. Groups investigating anti-EGFR antibodies tagged with fluorescent dye as well as inherent autofluorescence differences between normal and cancerous mucosal tissues show promise for improving positive surgical margin rates.

SUMMARY

The evolution of robotics and visualization for oropharyngeal cancer represents a novel and innovative utilization of new technology to improve care. Further advancements in techniques, from refinement of the robotic platform itself, to novel real-time tumour imaging options will be critical to further advance precision care for these patients.

Intraoperative Margin Assessment in Oral and Oropharyngeal Cancer Using Label-Free Fluorescence Lifetime Imaging and Machine Learning

OBJECTIVE

To demonstrate the diagnostic ability of label-free, point-scanning, fiber-based Fluorescence Lifetime Imaging (FLIm) as a means of intraoperative guidance during oral and oropharyngeal cancer removal surgery.

METHODS

FLIm point-measurements acquired from 53 patients (n = 67893 pre-resection in vivo, n = 89695 post-resection ex vivo) undergoing oral or oropharyngeal cancer removal surgery were used for analysis. Discrimination of healthy tissue and cancer was investigated using various FLIm-derived parameter sets and classifiers (Support Vector Machine, Random Forests, CNN). Classifier output for the acquired set of point-measurements was visualized through an interpolation-based approach to generate a probabilistic heatmap of cancer within the surgical field. Classifier output for dysplasia at the resection margins was also investigated.

RESULTS

Statistically significant change (P 0.01) between healthy and cancer was observed in vivo for the acquired FLIm signal parameters (e.g., average lifetime) linked with metabolic activity. Superior classification was achieved at the tissue region level using the Random Forests method (ROC-AUC: 0.88). Classifier output for dysplasia (% probability of cancer) was observed to lie between that of cancer and healthy tissue, highlighting FLIm's ability to distinguish various conditions.

CONCLUSION

The developed approach demonstrates the potential of FLIm for fast, reliable intraoperative margin assessment without the need for contrast agents.

SIGNIFICANCE

Fiber-based FLIm has the potential to be used as a diagnostic tool during cancer resection surgery, including Transoral Robotic Surgery (TORS), helping ensure complete resections and improve the survival rate of oral and oropharyngeal cancer patients.

Intraoperative Fluorescence-Guided Surgery in Head and Neck Squamous Cell Carcinoma

The rate of positive margins in head and neck cancers has remained stagnant over the past three decades and is consistently associated with poor overall survival. This suggests that significant improvements must be made intraoperatively to ensure negative margins. We discuss the important role of fluorescence imaging to guide surgical oncology in head and neck cancer. This review includes a general overview of the principles of fluorescence, available fluorophores used for fluorescence imaging, and specific clinical applications of fluorescence-guided surgery, as well as challenges and future directions in head and neck surgical oncology. Laryngoscope, 131:529-534, 2021.

Augmented Realities, Artificial Intelligence, and Machine Learning: Clinical Implications and How Technology Is Shaping the Future of Medicine

Technology has been integrated into every facet of human life, and whether it is completely advantageous remains unknown, but one thing is for sure; we are dependent on technology. Medical advances from the integration of artificial intelligence, machine learning, and augmented realities are widespread and have helped countless patients. Much of the advanced technology utilized by medical providers today has been borrowed and extrapolated from other industries. There remains no great collaboration between providers and engineers, which may be why medicine is only in its infancy of innovation with regards to advanced technologic integration. The purpose of this narrative review is to highlight the different technologies currently being utilized in a variety of medical specialties. Furthermore, we hope that by bringing attention to one shortcoming of the medical community, we may inspire future innovators to seek collaboration outside of the purely medical community for the betterment of all patients seeking care.

FLImBrush: dynamic visualization of intraoperative free-hand fiber-based fluorescence lifetime imaging

A free-hand scanning approach to medical imaging allows for flexible, lightweight probes to image intricate anatomies for modalities such as fluorescence lifetime imaging (FLIm), optical coherence tomography (OCT) and ultrasound. While very promising, this approach faces several key challenges including tissue motion during imaging, varying lighting conditions in the surgical field, and sparse sampling of the tissue surface. These challenges limit the coregistration accuracy and interpretability of the acquired imaging data. Here we report FLImBrush as a robust method for the localization and visualization of intraoperative free-hand fiber optic fluorescence lifetime imaging (FLIm). FLImBrush builds upon an existing method while employing deep learning-based image segmentation, block-matching based motion correction, and interpolation-based visualization to address the aforementioned challenges. Current results demonstrate that FLImBrush can provide accurate localization of FLIm point-measurements while producing interpretable and complete visualizations of FLIm data acquired from a tissue surface. Each of the main processing steps was shown to be capable of real-time processing (> 30 frames per second), highlighting the feasibility of FLImBrush for intraoperative imaging and surgical guidance. Current findings show the feasibility of integrating FLImBrush into a range of surgical applications including cancer margins assessment during head and neck surgery.

A Review on Advances in Intra-operative Imaging for Surgery and Therapy: Imagining the Operating Room of the Future

With the advent of Minimally Invasive Surgery (MIS), intra-operative imaging has become crucial for surgery and therapy guidance, allowing to partially compensate for the lack of information typical of MIS. This paper reviews the advancements in both classical (i.e. ultrasounds, X-ray, optical coherence tomography and magnetic resonance imaging) and more recent (i.e. multispectral, photoacoustic and Raman imaging) intra-operative imaging modalities. Each imaging modality was analyzed, focusing on benefits and disadvantages in terms of compatibility with the operating room, costs, acquisition time and image characteristics. Tables are included to summarize this information. New generation of hybrid surgical room and algorithms for real time/in room image processing were also investigated. Each imaging modality has its own (site- and procedure-specific) peculiarities in terms of spatial and temporal resolution, field of view and contrasted tissues. Besides the benefits that each technique offers for guidance, considerations about operators and patient risk, costs, and extra time required for surgical procedures have to be considered. The current trend is to equip surgical rooms with multimodal imaging systems, so as to integrate multiple information for real-time data extraction and computer-assisted processing. The future of surgery is to enhance surgeons eye to minimize intra- and after-surgery adverse events and provide surgeons with all possible support to objectify and optimize the care-delivery process.

Real-time multispectral fluorescence lifetime imaging using Single Photon Avalanche Diode arrays

Autofluorescence spectroscopy has emerged in recent years as a powerful tool to report label-free contrast between normal and diseased tissues, both in vivo and ex vivo. We report the development of an instrument employing Single Photon Avalanche Diode (SPAD) arrays to realize real-time multispectral autofluorescence lifetime imaging at a macroscopic scale using handheld single-point fibre optic probes, under bright background conditions. At the detection end, the fluorescence signal is passed through a transmission grating and both spectral and temporal information are encoded in the SPAD array. This configuration allows interrogation in the spectral range of interest in real time. Spatial information is provided by an external camera together with a guiding beam that provides a visual reference that is tracked in real-time. Through fast image processing and data analysis, fluorescence lifetime maps are augmented on white light images to provide feedback of the measurements in real-time. We validate and demonstrate the practicality of this technique in the reference fluorophores and in articular cartilage samples mimicking the degradation that occurs in osteoarthritis. Our results demonstrate that SPADs together with fibre probes can offer means to report autofluorescence spectral and lifetime contrast in real-time and thus are suitable candidates for in situ tissue diagnostics.

Fluorescence lifetime imaging microscopy: fundamentals and advances in instrumentation, analysis, and applications

SIGNIFICANCE

Fluorescence lifetime imaging microscopy (FLIM) is a powerful technique to distinguish the unique molecular environment of fluorophores. FLIM measures the time a fluorophore remains in an excited state before emitting a photon, and detects molecular variations of fluorophores that are not apparent with spectral techniques alone. FLIM is sensitive to multiple biomedical processes including disease progression and drug efficacy.

AIM

We provide an overview of FLIM principles, instrumentation, and analysis while highlighting the latest developments and biological applications.

APPROACH

This review covers FLIM principles and theory, including advantages over intensity-based fluorescence measurements. Fundamentals of FLIM instrumentation in time- and frequency-domains are summarized, along with recent developments. Image segmentation and analysis strategies that quantify spatial and molecular features of cellular heterogeneity are reviewed. Finally, representative applications are provided including high-resolution FLIM of cell- and organelle-level molecular changes, use of exogenous and endogenous fluorophores, and imaging protein-protein interactions with Förster resonance energy transfer (FRET). Advantages and limitations of FLIM are also discussed.

CONCLUSIONS

FLIM is advantageous for probing molecular environments of fluorophores to inform on fluorophore behavior that cannot be elucidated with intensity measurements alone. Development of FLIM technologies, analysis, and applications will further advance biological research and clinical assessments.

Real-time diagnosis and visualization of tumor margins in excised breast specimens using fluorescence lifetime imaging and machine learning

Tumor-free surgical margins are critical in breast-conserving surgery. In up to 38% of the cases, however, patients undergo a second surgery since malignant cells are found at the margins of the excised resection specimen. Thus, advanced imaging tools are needed to ensure clear margins at the time of surgery. The objective of this study was to evaluate a random forest classifier that makes use of parameters derived from point-scanning label-free fluorescence lifetime imaging (FLIm) measurements of breast specimens as a means to diagnose tumor at the resection margins and to enable an intuitive visualization of a probabilistic classifier on tissue specimen. FLIm data from fresh lumpectomy and mastectomy specimens from 18 patients were used in this study. The supervised training was based on a previously developed registration technique between autofluorescence imaging data and cross-sectional histology slides. A pathologist's histology annotations provide the ground truth to distinguish between adipose, fibrous, and tumor tissue. Current results demonstrate the ability of this approach to classify the tumor with 89% sensitivity and 93% specificity and to rapidly (∼ 20 frames per second) overlay the probabilistic classifier overlaid on excised breast specimens using an intuitive color scheme. Furthermore, we show an iterative imaging refinement that allows surgeons to switch between rapid scans with a customized, low spatial resolution to quickly cover the specimen and slower scans with enhanced resolution (400 μm per point measurement) in suspicious regions where more details are required. In summary, this technique provides high diagnostic prediction accuracy, rapid acquisition, adaptive resolution, nondestructive probing, and facile interpretation of images, thus holding potential for clinical breast imaging based on label-free FLIm.

Fluorescence lifetime imaging for intraoperative cancer delineation in transoral robotic surgery

This study evaluates the potential for fluorescence lifetime imaging (FLIm) to enhance intraoperative decisionmaking during robotic-assisted surgery of oropharyngeal cancer. Using a custom built FLIm instrument integrated with the da Vinci robotic surgical platform, we first demonstrate that cancer in epithelial tissue diagnosed by histopathology can be differentiated from surrounding healthy epithelial tissue imaged in vivo prior to cancer resection and ex vivo on the excised specimen. Second, we study the fluorescence properties of tissue imaged in vivo at surgical resection margins (tumor bed). Fluorescence lifetimes and spectral intensity ratios were calculated for three spectral channels, producing a set of six FLIm parameters. Current results from 10 patients undergoing TORS procedures demonstrate that healthy epithelium can be resolved from cancer (P < .001) for at least one FLIm parameter. We also showed that a multiparameter linear discriminant analysis approach provides superior discrimination to individual FLIm parameters for tissue imaged both in vivo and ex vivo. Overall, this study highlights the potential for FLIm to be developed into a diagnostic tool for clinical cancer applications of the oropharynx. This technique could help to circumvent the issues posed by the lack of tactile feedback associated with robotic surgical platforms to better enable cancer delineation.

Multiscale, multispectral fluorescence lifetime imaging using a double-clad fiber

Fiber-based imaging of tissue autofluorescence using ultraviolet (UV) excitation is a highly flexible tool used to probe structure and composition. In this Letter, we report, to the best of our knowledge, the first results from a single-fiber imaging system employing a custom double-clad fiber to acquire multispectral fluorescence lifetime images at two distinct spatial resolutions. We characterize the lateral point spread function and fluorescent background of the system and show how enhanced resolution can identify features such as trabeculae in ex vivo murine bone samples.