Automated detection of breast cancer in resected specimens with fluorescence lifetime imaging

Fluorescence Lifetime Imaging Techniques-A Review on Principles, Applications and Clinical Relevance

This article gives an overview of the most frequently used fluorescence-lifetime imaging (FLIM) techniques, their capabilities, and typical applications. Starting from a general introduction to fluorescence and phosphorescence lifetime, we will show that the fluorescence lifetime or, more accurately, the fluorescence decay function of a fluorophore is a direct indicator of the interaction with its molecular environment. FLIM is therefore more than a simple contrast technique in microscopy-it is a technique of molecular imaging. FLIM techniques can be classified into time-domain and frequency-domain techniques, analogue and photon counting techniques, and scanning and wide-field techniques. Starting from an overview of these general technical principles we will describe the features and peculiarities of the different FLIM techniques in use. An extended section is dedicated to TCSPC FLIM, addressing unique capabilities that make the technique especially interesting to FLIM of biological systems.

Fluorescence lifetime imaging with distance and ranging using a miniaturised SPAD system

In this work we demonstrate a miniaturised imaging system based around a time-gated SPAD array operating in a "chip-on-tip" manner. Two versions of the system are demonstrated, each measuring 23 mm × 23 mm × 28 mm with differing fields of view and working distances. Initial tests demonstrate contrast between materials in widefield fluorescence imaging (WFLIm) mode, with frame rates of > 2 Hz achievable. Following this, WFLIm images of autofluorescence in ovine lung tissue are obtained at frame rates of ~ 1 Hz. Finally, the ability of the second system to perform simultaneous WFLIm and time of flight (aka Flourescence Lifetime Imaging Distance and Ranging, FLImDAR) is also tested. This shows that the system is capable of 4 mm resolution of object separation when tested on 3D printed samples. It is further demonstrated as being able to perform scene reconstruction on autofluorescent lung tissue. This system is, to date, the smallest chip on tip WFLIm system published, and is the first demonstration of the FLImDAR technique in a compact, portable system.

Assessing the impact of extracellular matrix fiber orientation on breast cancer cellular metabolism

The extracellular matrix (ECM) is a dynamic and complex microenvironment that modulates cell behavior and cell fate. Changes in ECM composition and architecture have been correlated with development, differentiation, and disease progression in various pathologies, including breast cancer [1]. Studies have shown that aligned fibers drive a pro-metastatic microenvironment, promoting the transformation of mammary epithelial cells into invasive ductal carcinoma via the epithelial-to-mesenchymal transition (EMT) [2]. The impact of ECM orientation on breast cancer metabolism, however, is largely unknown. Here, we employ two non-invasive imaging techniques, fluorescence-lifetime imaging microscopy (FLIM) and intensity-based multiphoton microscopy, to assess the metabolic states of cancer cells cultured on ECM-mimicking nanofibers in a random and aligned orientation. By tracking the changes in the intrinsic fluorescence of nicotinamide adenine dinucleotide and flavin adenine dinucleotide, as well as expression levels of metastatic markers, we reveal how ECM fiber orientation alters cancer metabolism and EMT progression. Our study indicates that aligned cellular microenvironments play a key role in promoting metastatic phenotypes of breast cancer as evidenced by a more glycolytic metabolic signature on nanofiber scaffolds of aligned orientation compared to scaffolds of random orientation. This finding is particularly relevant for subsets of breast cancer marked by high levels of collagen remodeling (e.g. pregnancy associated breast cancer), and may serve as a platform for predicting clinical outcomes within these subsets [3-6].

Deformable multi-modal image registration for the correlation between optical measurements and histology images

Significance

The accurate correlation between optical measurements and pathology relies on precise image registration, often hindered by deformations in histology images. We investigate an automated multi-modal image registration method using deep learning to align breast specimen images with corresponding histology images.

Aim

We aim to explore the effectiveness of an automated image registration technique based on deep learning principles for aligning breast specimen images with histology images acquired through different modalities, addressing challenges posed by intensity variations and structural differences.

Approach

Unsupervised and supervised learning approaches, employing the VoxelMorph model, were examined using a dataset featuring manually registered images as ground truth.

Results

Evaluation metrics, including Dice scores and mutual information, demonstrate that the unsupervised model exceeds the supervised (and manual) approaches significantly, achieving superior image alignment. The findings highlight the efficacy of automated registration in enhancing the validation of optical technologies by reducing human errors associated with manual registration processes.

Conclusions

This automated registration technique offers promising potential to enhance the validation of optical technologies by minimizing human-induced errors and inconsistencies associated with manual image registration processes, thereby improving the accuracy of correlating optical measurements with pathology labels.

Macroscopic inelastic scattering imaging using a hyperspectral line-scanning system identifies invasive breast cancer in lumpectomy and mastectomy specimens

Significance

Of patients with early-stage breast cancer, 60% to 75% undergo breast-conserving surgery. Of those, 20% or more need a second surgery because of an incomplete tumor resection only discovered days after surgery. An intraoperative imaging technology allowing cancer detection on the margins of breast specimens could reduce re-excision procedure rates and improve patient survival.

Aim

We aimed to develop an experimental protocol using hyperspectral line-scanning Raman spectroscopy to image fresh breast specimens from cancer patients. Our objective was to determine whether macroscopic specimen images could be produced to distinguish invasive breast cancer from normal tissue structures.

Approach

A hyperspectral inelastic scattering imaging instrument was used to interrogate eight specimens from six patients undergoing breast cancer surgery. Machine learning models trained with a different system to distinguish cancer from normal breast structures were used to produce tissue maps with a field-of-view of 1    cm 2 classifying each pixel as either cancer, adipose, or other normal tissues. The predictive model results were compared with spatially correlated histology maps of the specimens.

Results

A total of eight specimens from six patients were imaged. Four of the hyperspectral images were associated with specimens containing cancer cells that were correctly identified by the new ex vivo pathology technique. The images associated with the remaining four specimens had no histologically detectable cancer cells, and this was also correctly predicted by the instrument.

Conclusions

We showed the potential of hyperspectral Raman imaging as an intraoperative breast cancer margin assessment technique that could help surgeons improve cosmesis and reduce the number of repeat procedures in breast cancer surgery.

Dual excitation spectral autofluorescence lifetime and reflectance imaging for fast macroscopic characterization of tissues

Advancements in optical imaging techniques have revolutionized the field of biomedical research, allowing for the comprehensive characterization of tissues and their underlying biological processes. Yet, there is still a lack of tools to provide quantitative and objective characterization of tissues that can aid clinical assessment in vivo to enhance diagnostic and therapeutic interventions. Here, we present a clinically viable fiber-based imaging system combining time-resolved spectrofluorimetry and reflectance spectroscopy to achieve fast multiparametric macroscopic characterization of tissues. An essential feature of the setup is its ability to perform dual wavelength excitation in combination with recording time-resolved fluorescence data in several spectral intervals. Initial validation of this bimodal system was carried out in freshly resected human colorectal cancer specimens, where we demonstrated the ability of the system to differentiate normal from malignant tissues based on their autofluorescence and reflectance properties. To further highlight the complementarity of autofluorescence and reflectance measurements and demonstrate viability in a clinically relevant scenario, we also collected in vivo data from the skin of a volunteer. Altogether, integration of these modalities in a single platform can offer multidimensional characterization of tissues, thus facilitating a deeper understanding of biological processes and potentially advancing diagnostic and therapeutic approaches in various medical applications.

Applications of machine learning in time-domain fluorescence lifetime imaging: a review

Many medical imaging modalities have benefited from recent advances in Machine Learning (ML), specifically in deep learning, such as neural networks. Computers can be trained to investigate and enhance medical imaging methods without using valuable human resources. In recent years, Fluorescence Lifetime Imaging (FLIm) has received increasing attention from the ML community. FLIm goes beyond conventional spectral imaging, providing additional lifetime information, and could lead to optical histopathology supporting real-time diagnostics. However, most current studies do not use the full potential of machine/deep learning models. As a developing image modality, FLIm data are not easily obtainable, which, coupled with an absence of standardisation, is pushing back the research to develop models which could advance automated diagnosis and help promote FLIm. In this paper, we describe recent developments that improve FLIm image quality, specifically time-domain systems, and we summarise sensing, signal-to-noise analysis and the advances in registration and low-level tracking. We review the two main applications of ML for FLIm: lifetime estimation and image analysis through classification and segmentation. We suggest a course of action to improve the quality of ML studies applied to FLIm. Our final goal is to promote FLIm and attract more ML practitioners to explore the potential of lifetime imaging.

Combined fluorescence lifetime and surface topographical imaging of biological tissue

In this work a combined fluorescence lifetime and surface topographical imaging system is demonstrated. Based around a 126 × 192 time resolved single photon avalanche diode (SPAD) array operating in time correlated single-photon counting (TCSPC) mode, both the fluorescence lifetime and time of flight (ToF) can be calculated on a pixel by pixel basis. Initial tests on fluorescent samples show it is able to provide 4 mm resolution in distance and 0.4 ns resolution in lifetime. This combined modality has potential biomedical applications such as surgical guidance, endoscopy, and diagnostic imaging. The system is demonstrated on both ovine and human pulmonary tissue samples, where it offers excellent fluorescence lifetime contrast whilst also giving a measure of the distance to the sample surface.

Fluorescence lifetime imaging for explosive detection

In this Letter, a time-resolved 120 × 128 pixel single-photon avalanche diode (SPAD) sensor is used in conjunction with an array of organic semiconductor films as a means of detecting the presence of explosive vapors. Using the spatial and temporal resolution of the sensor, both fluorescence intensity and fluorescence lifetime can be monitored on a pixel-by-pixel basis for each of the polymer films arranged in a 2 × 2 grid. This represents a significant improvement on similar systems demonstrated in the past, which either offer spatial resolution without the temporal resolution required to monitor lifetime or offer only a single bulk measurement of lifetime and intensity without the spatial resolution. The potential of the sensing system is demonstrated using vapors of DNT, and differing responses for each of the four polymer films is observed. This system has clear applications as the basis of a portable chemical fingerprinting tool with applications in humanitarian demining and security.

Probing organoid metabolism using fluorescence lifetime imaging microscopy (FLIM): The next frontier of drug discovery and disease understanding

Organoid models have been used to address important questions in developmental and cancer biology, tissue repair, advanced modelling of disease and therapies, among other bioengineering applications. Such 3D microenvironmental models can investigate the regulation of cell metabolism, and provide key insights into the mechanisms at the basis of cell growth, differentiation, communication, interactions with the environment and cell death. Their accessibility and complexity, based on 3D spatial and temporal heterogeneity, make organoids suitable for the application of novel, dynamic imaging microscopy methods, such as fluorescence lifetime imaging microscopy (FLIM) and related decay time-assessing readouts. Several biomarkers and assays have been proposed to study cell metabolism by FLIM in various organoid models. Herein, we present an expert-opinion discussion on the principles of FLIM and PLIM, instrumentation and data collection and analysis protocols, and general and emerging biosensor-based approaches, to highlight the pioneering work being performed in this field.

Handheld wide-field fluorescence lifetime imaging system based on a distally mounted SPAD array

In this work a handheld Fluorescent Lifetime IMaging (FLIM) system based on a distally mounted < 2 mm² 128 × 120 single photon avalanche diode (SPAD) array operating over a > 1 m long wired interface is demonstrated. The head of the system is ∼4.5 cm x 4.5 cm x 4.5 cm making it suitable for hand-held ex vivo applications. This is, to the best of the authors' knowledge, the first example of a SPAD array mounted on the distal end of a handheld FLIM system in this manner. All existing systems to date use a fibre to collect and relay fluorescent light to detectors at the proximal end of the system. This has clear potential biological and biomedical applications. To demonstrate this, the system is used to provide contrast between regions of differing tissue composition in ovine kidney samples, and between healthy and stressed or damaged plant leaves. Additionally, FLIM videos are provided showing that frame rates of > 1 Hz are achievable. It is thus an important step in realising an in vivo miniaturized chip-on-tip FLIM endoscopy system.

Automatic diagnosis and classification of breast surgical samples with dynamic full-field OCT and machine learning

Purpose

The adoption of emerging imaging technologies in the medical community is often hampered when they provide a new unfamiliar contrast that requires experience to be interpreted. Dynamic full-field optical coherence tomography (D-FF-OCT) microscopy is such an emerging technique. It provides fast, high-resolution images of excised tissues with a contrast comparable to H&E histology but without any tissue preparation and alteration.

Approach

We designed and compared two machine learning approaches to support interpretation of D-FF-OCT images of breast surgical specimens and thus provide tools to facilitate medical adoption. We conducted a pilot study on 51 breast lumpectomy and mastectomy surgical specimens and more than 1000 individual 1.3 × 1.3    mm 2 images and compared with standard H&E histology diagnosis.

Results

Using our automatic diagnosis algorithms, we obtained an accuracy above 88% at the image level (1.3 × 1.3    mm 2 ) and above 96% at the specimen level (above cm 2 ).

Conclusions

Altogether, these results demonstrate the high potential of D-FF-OCT coupled to machine learning to provide a rapid, automatic, and accurate histopathology diagnosis with minimal sample alteration.

Point-of-care devices based on fluorescence imaging and spectroscopy for tumor margin detection during breast cancer surgery: Towards breast conservation treatment

OBJECTIVE

Fluorescence-based methods are highly specific and sensitive and have potential in breast cancer detection. Simultaneous fluorescence imaging and spectroscopy during intraoperative procedures of breast cancer have great advantages in detection of tumor margin as well as in classification of tumor to healthy tissues. Intra-operative real-time confirmation of breast cancer tumor margin is the aim of surgeons, and therefore, there is an urgent need for such techniques and devices which fulfill the surgeon's priorities.

METHODS

In this article, we propose the development of fluorescence-based smartphone imaging and spectroscopic point-of-care multi-modal devices for detection of invasive ductal carcinoma in tumor margin during removal of tumor. These multimodal devices are portable, cost-effective, noninvasive, and user-friendly. Molecular level sensitivity of fluorescence process shows different behavior in normal, cancerous and marginal tissues. We observed significant spectral changes, such as, red-shift, full-width half maximum (FWHM), and increased intensity as we go towards tumor center from normal tissue. High contrast in fluorescence images and spectra are also recorded for cancer tissues compared to healthy tissues. Preliminary results for the initial trial of the devices are reported in this article.

RESULTS

A total 44 spectra from 11 patients of invasive ductal carcinoma (11 spectra for invasive ductal carcinoma and rest are normal and negative margins) are used. Principle component analysis is used for the classification of invasive ductal carcinoma with an accuracy of 93%, specificity of 75% and sensitivity of 92.8%. We obtained an average 6.17 ± 1.66 nm red shift for IDC with respect to normal tissue. The red shift and maximum fluorescence intensity indicates p < 0.01. These results described here are supported by histopathological examination of the same sample.

CONCLUSION

In the present manuscript, simultaneous fluorescence-based imaging and spectroscopy is accomplished for the classification of IDC tissues and breast cancer margin detection.

In situ Raman spectroscopy and machine learning unveil biomolecular alterations in invasive breast cancer

SIGNIFICANCE

As many as 60% of patients with early stage breast cancer undergo breast-conserving surgery. Of those, 20% to 35% need a second surgery because of incomplete resection of the lesions. A technology allowing in situ detection of cancer could reduce re-excision procedure rates and improve patient survival.

AIM

Raman spectroscopy was used to measure the spectral fingerprint of normal breast and cancer tissue ex-vivo. The aim was to build a machine learning model and to identify the biomolecular bands that allow one to detect invasive breast cancer.

APPROACH

The system was used to interrogate specimens from 20 patients undergoing lumpectomy, mastectomy, or breast reduction surgery. This resulted in 238 ex-vivo measurements spatially registered with standard histology classifying tissue as cancer, normal, or fat. A technique based on support vector machines led to the development of predictive models, and their performance was quantified using a receiver-operating-characteristic analysis.

RESULTS

Raman spectroscopy combined with machine learning detected normal breast from ductal or lobular invasive cancer with a sensitivity of 93% and a specificity of 95%. This was achieved using a model based on only two spectral bands, including the peaks associated with C-C stretching of proteins around 940    cm - 1 and the symmetric ring breathing at 1004    cm - 1 associated with phenylalanine.

CONCLUSIONS

Detection of cancer on the margins of surgically resected breast specimen is feasible with Raman spectroscopy.

Translational Potential of Fluorescence Polarization for Breast Cancer Cytopathology

Breast cancer is the most common malignancy in women. The standard of care for diagnosis involves invasive core needle biopsy followed by time-consuming histopathological evaluation. A rapid, accurate, and minimally invasive method to diagnose breast cancer would be invaluable. Therefore, this clinical study investigated the fluorescence polarization (Fpol) of the cytological stain methylene blue (MB) for the quantitative detection of breast cancer in fine needle aspiration (FNA) specimens. Cancerous, benign, and normal cells were aspirated from excess breast tissues immediately following surgery. The cells were stained in aqueous MB solution (0.05 mg/mL) and imaged using multimodal confocal microscopy. The system provided MB Fpol and fluorescence emission images of the cells. Results from optical imaging were compared to clinical histopathology. In total, we imaged and analyzed 3808 cells from 44 breast FNAs. Fpol images displayed quantitative contrast between cancerous and noncancerous cells, whereas fluorescence emission images showed the morphological features comparable to cytology. Statistical analysis demonstrated that MB Fpol is significantly higher (p < 0.0001) in malignant vs. benign/normal cells. It also revealed a correlation between MB Fpol values and tumor grade. The results indicate that MB Fpol could provide a reliable, quantitative diagnostic marker for breast cancer at the cellular level.

Deep learning in macroscopic diffuse optical imaging

SIGNIFICANCE

Biomedical optics system design, image formation, and image analysis have primarily been guided by classical physical modeling and signal processing methodologies. Recently, however, deep learning (DL) has become a major paradigm in computational modeling and has demonstrated utility in numerous scientific domains and various forms of data analysis.

AIM

We aim to comprehensively review the use of DL applied to macroscopic diffuse optical imaging (DOI).

APPROACH

First, we provide a layman introduction to DL. Then, the review summarizes current DL work in some of the most active areas of this field, including optical properties retrieval, fluorescence lifetime imaging, and diffuse optical tomography.

RESULTS

The advantages of using DL for DOI versus conventional inverse solvers cited in the literature reviewed herein are numerous. These include, among others, a decrease in analysis time (often by many orders of magnitude), increased quantitative reconstruction quality, robustness to noise, and the unique capability to learn complex end-to-end relationships.

CONCLUSIONS

The heavily validated capability of DL's use across a wide range of complex inverse solving methodologies has enormous potential to bring novel DOI modalities, otherwise deemed impractical for clinical translation, to the patient's bedside.

Proof of concept for the sapphire scalpel combining tissue dissection and optical diagnosis

OBJECTIVES

The development of compact diagnostic probes and instruments with an ability to direct access to organs and tissues and integration of these instruments into surgical workflows is an important task of modern physics and medicine. The need for such tools is essential for surgical oncology, where intraoperative visualization and demarcation of tumor margins define further prognosis and survival of patients. In this paper, the possible solution for this intraoperative imaging problem is proposed and its feasibility to detect tumorous tissue is studied experimentally.

METHODS

For this aim, the sapphire scalpel was developed and fabricated using the edge-defined film-fed growth technique aided by mechanical grinding, polishing, and chemical sharpening of the cutting edge. It possesses optical transparency, mechanical strength, chemical inertness, and thermal resistance alongside the presence of the as-grown hollow capillary channels in its volume for accommodating optical fibers. The rounding of the cutting edge exceeds the same for metal scalpels and can be as small as 110 nm. Thanks to these features, sapphire scalpel combines tissue dissection with light delivering and optical diagnosis. The feasibility for the tumor margin detection was studied, including both gelatin-based tissue phantoms and ex vivo freshly excised specimens of the basal cell carcinoma from humans and the glioma model 101.8 from rats. These tumors are commonly diagnosed either non-invasively or intraoperatively using different modalities of fluorescence spectroscopy and imaging, which makes them ideal candidates for our feasibility test. For this purpose, fiber-based spectroscopic measurements of the backscattered laser radiation and the fluorescence signals were carried out in the visible range.

RESULTS

Experimental studies show the feasibility of the proposed sapphire scalpel to provide a 2-mm-resolution of the tumor margins' detection, along with an ability to distinguish the tumor invasion region, which results from analysis of the backscattered optical fields and the endogenous or exogenous fluorescence data.

CONCLUSIONS

Our findings justified a strong potential of the sapphire scalpel for surgical oncology. However, further research and engineering efforts are required to optimize the sapphire scalpel geometry and the optical diagnosis protocols to meet the requirements of oncosurgery, including diagnosis and resection of neoplasms with different localizations and nosologies.

Metabolic Changes in Maternal and Cord Blood in One Case of Pregnancy-Associated Breast Cancer Seen by Fluorescence Lifetime Imaging Microscopy

Pregnancy-associated breast cancer (PABC) is a rare disease, which is frequently diagnosed at an advanced stage due to limitations in current diagnostic methods. In this study, fluorescence lifetime imaging microscopy (FLIM) was used to study the metabolic changes by measuring maternal blood and umbilical cord blood via the autofluorescence of coenzymes, reduced nicotinamide adenine dinucleotide (phosphate) (NAD(P)H), and flavin adenine dinucleotide (FAD). The NAD(P)H data showed that a PABC case had significant differences compared with normal cases, which may indicate increased glycolysis. The FAD data showed that both maternal and cord blood of PABC had shorter mean lifetimes and higher bound-FAD ratios. The significant differences suggested that FLIM testing of blood samples may be a potential method to assist in PABC non-radiative screening.

Exploring photoacoustic spectroscopy-based machine learning together with metabolomics to assess breast tumor progression in a xenograft model ex vivo

In the current study, a breast tumor xenograft was established in athymic nude mice by subcutaneous injection of the MCF-7 cell line and assessed the tumor progression by photoacoustic spectroscopy combined with machine learning tools. The advancement of breast tumors in nude mice was validated by tumor volume kinetics and histopathology and corresponding image analysis by TissueQuant software compared to controls. The ex vivo tumors in progressive conditions belonging to time points, day 5th, 10th, 15th & 20th, were excited with 281 nm pulsed laser light and recorded the corresponding photoacoustic spectra in time domain. The spectra were then pre-processed, augmented for a 10-fold increase in the data strength, and subjected to wavelet packet transformation for feature extraction and selection using MATLAB software. In the present study, the top 10 features from all the time point groups under study were selected based on their prediction ranking values using the mRMR algorithm. The chosen features of all the time-point groups were then subjected to multi-class Support Vector Machine (SVM) algorithms for learning and classifying into respective time point groups under study. The analysis demonstrated accuracy values of 95.2%, 99.5%, and 80.3% with SVM- Radial Basis Function (SVM-RBF), SVM-Polynomial & SVM-Linear, respectively. The serum metabolomic levels during tumor progression complemented photoacoustic patterns of tumor progression, depicting breast cancer pathophysiology.

Multispectral fluorescence lifetime imaging device with a silicon avalanche photodetector

We report the design, development, and characterization of a novel multi-spectral fluorescence lifetime measurement device incorporating solid-state detectors and automated gain control. For every excitation pulse (∼1 µJ, 600 ps), this device records complete fluorescence decay from multiple spectral channels simultaneously within microseconds, using a dedicated UV enhanced avalanche photodetector and analog to digital convert (2.5 GS/s) in each channel. Fast (<2 ms) channel-wise dynamic range adjustment maximizes the signal-to-noise ratio. Fluorophores with known lifetime ranging from 0.5-6.0 ns were used to demonstrate the device accuracy. Current results show the clear benefits of this device compared to existing devices employing microchannel-plate photomultiplier tubes. This is demonstrated by 5-fold reduction of lifetime measurement variability in identical conditions, independent gain adjustment in each spectral band, and 4-times faster imaging speed. The use of solid-state detectors will also facilitate future improved performance and miniaturization of the instrument.

Selective-sampling Raman imaging techniques for ex vivo assessment of surgical margins in cancer surgery

One of the main challenges in cancer surgery is to ensure the complete excision of the tumour while sparing as much healthy tissue as possible. Histopathology, the gold-standard technique used to assess the surgical margins on the excised tissue, is often impractical for intra-operative use because of the time-consuming tissue cryo-sectioning and staining, and availability of histopathologists to assess stained tissue sections. Raman micro-spectroscopy is a powerful technique that can detect microscopic residual tumours on ex vivo tissue samples with accuracy, based entirely on intrinsic chemical differences. However, raster-scanning Raman micro-spectroscopy is a slow imaging technique that typically requires long data acquisition times wich are impractical for intra-operative use. Selective-sampling Raman imaging overcomes these limitations by using information regarding the spatial properties of the tissue to reduce the number of Raman spectra. This paper reviews the latest advances in selective-sampling Raman techniques and applications, mainly based on multimodal optical imaging. We also highlight the latest results of clinical integration of a prototype device for non-melanoma skin cancer. These promising results indicate the potential impact of Raman spectroscopy for providing fast and objective assessment of surgical margins, helping surgeons ensure the complete removal of tumour cells while sparing as much healthy tissue as possible.

FLIM as a Promising Tool for Cancer Diagnosis and Treatment Monitoring

Fluorescence lifetime imaging microscopy (FLIM) has been rapidly developed over the past 30 years and widely applied in biomedical engineering. Recent progress in fluorophore-dyed probe design has widened the application prospects of fluorescence. Because fluorescence lifetime is sensitive to microenvironments and molecule alterations, FLIM is promising for the detection of pathological conditions. Current cancer-related FLIM applications can be divided into three main categories: (i) FLIM with autofluorescence molecules in or out of a cell, especially with reduced form of nicotinamide adenine dinucleotide, and flavin adenine dinucleotide for cellular metabolism research; (ii) FLIM with Förster resonance energy transfer for monitoring protein interactions; and (iii) FLIM with fluorophore-dyed probes for specific aberration detection. Advancements in nanomaterial production and efficient calculation systems, as well as novel cancer biomarker discoveries, have promoted FLIM optimization, offering more opportunities for medical research and applications to cancer diagnosis and treatment monitoring. This review summarizes cutting-edge researches from 2015 to 2020 on cancer-related FLIM applications and the potential of FLIM for future cancer diagnosis methods and anti-cancer therapy development. We also highlight current challenges and provide perspectives for further investigation.

Mesoscopic fluorescence lifetime imaging: Fundamental principles, clinical applications and future directions

Fluorescence lifetime imaging (FLIm) is an optical spectroscopic imaging technique capable of real-time assessments of tissue properties in clinical settings. Label-free FLIm is sensitive to changes in tissue structure and biochemistry resulting from pathological conditions, thus providing optical contrast to identify and monitor the progression of disease. Technical and methodological advances over the last two decades have enabled the development of FLIm instrumentation for real-time, in situ, mesoscopic imaging compatible with standard clinical workflows. Herein, we review the fundamental working principles of mesoscopic FLIm, discuss the technical characteristics of current clinical FLIm instrumentation, highlight the most commonly used analytical methods to interpret fluorescence lifetime data and discuss the recent applications of FLIm in surgical oncology and cardiovascular diagnostics. Finally, we conclude with an outlook on the future directions of clinical FLIm.

Intraoperative Mapping of Parathyroid Glands Using Fluorescence Lifetime Imaging

BACKGROUND

Hypoparathyroidism is a common complication following thyroidectomy. There is a need for technology to aid surgeons in identifying the parathyroid glands. In contrast to near infrared technologies, fluorescence lifetime imaging (FLIm) is not affected by ambient light and may be valuable in identifying parathyroid tissue, but has never been evaluated in this capacity.

METHODS

We used FLIm to measure the UV induced (355 nm) time-resolved autofluorescence signatures (average lifetimes in 3 spectral emission channels) of thyroid, parathyroid, lymphoid and adipose tissue in 21 patients undergoing thyroid and parathyroid surgery. The Mann-Whitney U test was used to assess the ability of FLIm to discriminate normocellular parathyroid from each of the other tissues. Various machine learning classifiers (random forests, neural network, support vector machine) were then evaluated to recognize parathyroid through a leave-one-out cross-validation.

RESULTS

Statistically significant differences in average lifetime were observed between parathyroid and each of the other tissue types in spectral channels 2 and 3 respectively. The largest change was observed between adipose tissue and parathyroid (P < 0.001), while less pronounced but still significant changes were observed when comparing parathyroid with lymphoid tissue (P < 0.05) and thyroid (P < 0.01). A random forest classifier trained on average lifetimes was found to detect parathyroid tissue with 100% sensitivity and 93% specificity at the acquisition run level.

CONCLUSION

We found that FLIm derived parameters can distinguish the parathyroid glands and other adjacent tissue types and has promise in scanning the surgical field to identify parathyroid tissue in real-time.

Intraoperative Margin Assessment in Oral and Oropharyngeal Cancer Using Label-Free Fluorescence Lifetime Imaging and Machine Learning

OBJECTIVE

To demonstrate the diagnostic ability of label-free, point-scanning, fiber-based Fluorescence Lifetime Imaging (FLIm) as a means of intraoperative guidance during oral and oropharyngeal cancer removal surgery.

METHODS

FLIm point-measurements acquired from 53 patients (n = 67893 pre-resection in vivo, n = 89695 post-resection ex vivo) undergoing oral or oropharyngeal cancer removal surgery were used for analysis. Discrimination of healthy tissue and cancer was investigated using various FLIm-derived parameter sets and classifiers (Support Vector Machine, Random Forests, CNN). Classifier output for the acquired set of point-measurements was visualized through an interpolation-based approach to generate a probabilistic heatmap of cancer within the surgical field. Classifier output for dysplasia at the resection margins was also investigated.

RESULTS

Statistically significant change (P 0.01) between healthy and cancer was observed in vivo for the acquired FLIm signal parameters (e.g., average lifetime) linked with metabolic activity. Superior classification was achieved at the tissue region level using the Random Forests method (ROC-AUC: 0.88). Classifier output for dysplasia (% probability of cancer) was observed to lie between that of cancer and healthy tissue, highlighting FLIm's ability to distinguish various conditions.

CONCLUSION

The developed approach demonstrates the potential of FLIm for fast, reliable intraoperative margin assessment without the need for contrast agents.

SIGNIFICANCE

Fiber-based FLIm has the potential to be used as a diagnostic tool during cancer resection surgery, including Transoral Robotic Surgery (TORS), helping ensure complete resections and improve the survival rate of oral and oropharyngeal cancer patients.

Combined total internal reflection AF spectral-imaging and Raman spectroscopy for fast assessment of surgical margins during breast cancer surgery

The standard treatment for breast cancer is surgical removal mainly through breast-conserving surgery (BCS). We developed a new technique based on auto-fluorescence (AF) spectral imaging and Raman spectroscopy for fast intraoperative assessment of excision margins in BCS. A new wide-field AF imaging unit based on total internal reflection (TIR) was combined with a Raman spectroscopy microscope equipped with a 785 nm laser. The wavelength of the AF excitation was optimized to 365 nm in order to maximize the discrimination of adipose tissue. This approach allows for the non-adipose regions of tissue, which are at a higher risk of containing a tumor, to be targeted more efficiently by the Raman spectroscopy measurements. The integrated TIR-AF-Raman was tested on small tissue samples as well as fresh wide local excisions, delivering the analysis of the entire cruciate surface of BCS specimens (5.1 × 7.6 cm²) in less than 45 minutes and also providing information regarding the location of the tumor in the specimen. Full automation of the instrument and selection of a faster translation stage would allow for the measurement of BCS specimens within an intraoperative time scale (20 minutes). This study demonstrates that the TIR-AF Raman microscope represents a feasible step towards the development of a technique for intraoperative assessment of large WLE within intraoperative timescales.

FLIm-Guided Raman Imaging to Study Cross-Linking and Calcification of Bovine Pericardium

Bovine pericardium (BP) is a vascular biomaterial used in cardiovascular surgery that is typically cross-linked for masking antigenicity and enhance stability. There is a need for biochemical evaluation of the tissue properties prior to implantation to ensure that quality and reliability standards are met. Here, engineered antigen removed BP (ARBP) that was cross-linked with 0.2% and 0.6% glutaraldehyde (GA), and further calcified in vitro to simulate graft calcifications upon implantation was characterized nondestructively using fluorescence lifetime imaging (FLIm) to identify regions of interest which were then assessed by Raman spectroscopy. We observed that the tissue fluorescence lifetime shortened, and that Raman bands at 856, 935, 1282, and 1682 cm-1 decreased, and at 1032 and 1627 cm-1 increased with increasing GA cross-linking. Independent classification analysis based on fluorescence lifetime and on Raman spectra discriminated between GA-ARBP and untreated ARBP with an accuracy of 91% and 66%, respectively. Pearson's correlation analysis showed a strong correlation between pyridinium cross-links measured with high-performance liquid chromatography and fluorescence lifetime measured at 380-400 nm (R = -0.76, p = 0.00094), as well as Raman bands at 856 cm-1 for hydroxy-proline (R = -0.68, p = 0.0056) and at 1032 cm-1 for hydroxy-pyridinium (R = 0.74, p = 0.0016). Calcified areas of GA cross-linked tissue showed characteristic hydroxyapatite (959 and 1038 cm-1) bands in the Raman spectrum and fluorescence lifetime shortened by 0.4 ns compared to uncalcified regions. FLIm-guided Raman imaging could rapidly identify degrees of cross-linking and detected calcified regions with high chemical specificity, an ability that can be used to monitor tissue engineering processes for applications in regenerative medicine.

Development of a Robust Autofluorescence Lifetime Sensing Method for Use in an Endoscopic Application

Endoscopic autofluorescence lifetime imaging is a promising technique for making quantitative and non-invasive diagnoses of abnormal tissue. However, motion artifacts caused by vibration in the direction perpendicular to the tissue surface in a body makes clinical diagnosis difficult. Thus, this paper proposes a robust autofluorescence lifetime sensing technique with a lens tracking system based on a laser beam spot analysis. Our optical setup can be easily mounted on the head of an endoscope. The variation in distance between the optical system and the target surface is tracked by the change in the spot size of the laser beam captured by the camera, and the lens actuator is feedback-controlled to suppress motion artifacts. The experimental results show that, when using a lens tracking system, the standard deviation of fluorescence lifetime is dramatically reduced. Furthermore, the validity of the proposed method is experimentally confirmed by using a bio-mimicking phantom that replicates the shape, optical parameters, and chemical component distribution of the cancerous tissue.

Real-time diagnosis and visualization of tumor margins in excised breast specimens using fluorescence lifetime imaging and machine learning

Tumor-free surgical margins are critical in breast-conserving surgery. In up to 38% of the cases, however, patients undergo a second surgery since malignant cells are found at the margins of the excised resection specimen. Thus, advanced imaging tools are needed to ensure clear margins at the time of surgery. The objective of this study was to evaluate a random forest classifier that makes use of parameters derived from point-scanning label-free fluorescence lifetime imaging (FLIm) measurements of breast specimens as a means to diagnose tumor at the resection margins and to enable an intuitive visualization of a probabilistic classifier on tissue specimen. FLIm data from fresh lumpectomy and mastectomy specimens from 18 patients were used in this study. The supervised training was based on a previously developed registration technique between autofluorescence imaging data and cross-sectional histology slides. A pathologist's histology annotations provide the ground truth to distinguish between adipose, fibrous, and tumor tissue. Current results demonstrate the ability of this approach to classify the tumor with 89% sensitivity and 93% specificity and to rapidly (∼ 20 frames per second) overlay the probabilistic classifier overlaid on excised breast specimens using an intuitive color scheme. Furthermore, we show an iterative imaging refinement that allows surgeons to switch between rapid scans with a customized, low spatial resolution to quickly cover the specimen and slower scans with enhanced resolution (400 μm per point measurement) in suspicious regions where more details are required. In summary, this technique provides high diagnostic prediction accuracy, rapid acquisition, adaptive resolution, nondestructive probing, and facile interpretation of images, thus holding potential for clinical breast imaging based on label-free FLIm.

Fluorescence lifetime imaging for intraoperative cancer delineation in transoral robotic surgery

This study evaluates the potential for fluorescence lifetime imaging (FLIm) to enhance intraoperative decisionmaking during robotic-assisted surgery of oropharyngeal cancer. Using a custom built FLIm instrument integrated with the da Vinci robotic surgical platform, we first demonstrate that cancer in epithelial tissue diagnosed by histopathology can be differentiated from surrounding healthy epithelial tissue imaged in vivo prior to cancer resection and ex vivo on the excised specimen. Second, we study the fluorescence properties of tissue imaged in vivo at surgical resection margins (tumor bed). Fluorescence lifetimes and spectral intensity ratios were calculated for three spectral channels, producing a set of six FLIm parameters. Current results from 10 patients undergoing TORS procedures demonstrate that healthy epithelium can be resolved from cancer (P < .001) for at least one FLIm parameter. We also showed that a multiparameter linear discriminant analysis approach provides superior discrimination to individual FLIm parameters for tissue imaged both in vivo and ex vivo. Overall, this study highlights the potential for FLIm to be developed into a diagnostic tool for clinical cancer applications of the oropharynx. This technique could help to circumvent the issues posed by the lack of tactile feedback associated with robotic surgical platforms to better enable cancer delineation.

Method for coregistration of optical measurements of breast tissue with histopathology: the importance of accounting for tissue deformations

For the validation of optical diagnostic technologies, experimental results need to be benchmarked against the gold standard. Currently, the gold standard for tissue characterization is assessment of hematoxylin and eosin (H&E)-stained sections by a pathologist. When processing tissue into H&E sections, the shape of the tissue deforms with respect to the initial shape when it was optically measured. We demonstrate the importance of accounting for these tissue deformations when correlating optical measurement with routinely acquired histopathology. We propose a method to register the tissue in the H&E sections to the optical measurements, which corrects for these tissue deformations. We compare the registered H&E sections to H&E sections that were registered with an algorithm that does not account for tissue deformations by evaluating both the shape and the composition of the tissue and using microcomputer tomography data as an independent measure. The proposed method, which did account for tissue deformations, was more accurate than the method that did not account for tissue deformations. These results emphasize the need for a registration method that accounts for tissue deformations, such as the method presented in this study, which can aid in validating optical techniques for clinical use.

Towards mm-wave spectroscopy for dielectric characterization of breast surgical margins

PURPOSE

The evaluation of the surgical margin in breast conservative surgery is a matter of general interest as such treatments are subject to the critical issue of margin status as positive surgical margins can undermine the effectiveness of the procedure. The relatively unexplored ability of millimeter-wave (mm-wave) spectroscopy to provide insight into the dielectric properties of breast tissues was investigated as a precursor to their possible use in assessment of surgical margins.

METHODS

We assessed the ability of a mm-wave system with a roughly hemispherical sensitive volume of ∼3 mm radius to distinguish malignant breast lesions in prospectively and consecutively collected tumoral and non-tumoral ex-vivo breast tissue samples from 91 patients. We characterized the dielectric properties of 346 sites in these samples, encompassing malignant, fibrocystic disease and normal breast tissues. An expert pathologist subsequently evaluated all measurement sites.

RESULTS

At multivariate analysis, mm-wave dielectric properties were significantly correlated to histologic diagnosis and fat content. Further, using 5-fold cross-validation in a Bayesian logistic mixed model that considered the patient as a random effect, the mm-wave dielectric properties of neoplastic tissues were significantly different from normal breast tissues, but not from fibrocystic tissue.

CONCLUSION

Reliable discrimination of malignant from normal, fat-rich breast tissue to a depth compatible with surgical margin assessment requirements was achieved with mm-wave spectroscopy.

Detection of glycosaminoglycan loss in articular cartilage by fluorescence lifetime imaging

Glycosaminoglycan (GAG) loss is an early marker of osteoarthritis, which is a clinical late stage disease that affects millions of people worldwide. The goal of our study was to evaluate the ability of a fiber-based fluorescence lifetime imaging (FLIm) technique to detect GAG loss in articular cartilage. Native bovine cartilage explants (n  =  20) were exposed to 0 (control), 0.5 (low), or 1  U  /  mL (high) concentrations of chondroitinase ABC (cABC) to create samples with different levels of GAG loss. FLIm assessment (excitation: 355 nm; detection: channel 1: 375 to 410 nm, channel 2: 450 to 485 nm, channel 3: 530 to 565 nm) was conducted on depth-resolved cross-sections of the cartilage sample. FLIm images, validated with histology, revealed that loss of GAG resulted in a decrease of fluorescence lifetime values in channel 2 (Δ  =  0.44  ns, p  <  0.05) and channel 3 (Δ  =  0.75  ns, p  <  0.01) compared to control samples (channel 2: 6.34 ns; channel 3: 5.22 ns). Fluorescence intensity ratio values were lower in channel 1 (37%, p  <  0.0001) and channel 2 (31% decrease, p  <  0.0001) and higher in channel 3 (23%, p  <  0.0001) relative to control samples. These results show that FLIm can detect the loss of GAG in articular cartilage and support further investigation into the feasibility of in vivo FLIm arthroscopy.

Review of methods for intraoperative margin detection for breast conserving surgery

Breast conserving surgery (BCS) is an effective treatment for early-stage cancers as long as the margins of the resected tissue are free of disease according to consensus guidelines for patient management. However, 15% to 35% of patients undergo a second surgery since malignant cells are found close to or at the margins of the original resection specimen. This review highlights imaging approaches being investigated to reduce the rate of positive margins, and they are reviewed with the assumption that a new system would need high sensitivity near 95% and specificity near 85%. The problem appears to be twofold. The first is for complete, fast surface scanning for cellular, structural, and/or molecular features of cancer, in a lumpectomy volume, which is variable in size, but can be large, irregular, and amorphous. A second is for full, volumetric imaging of the specimen at high spatial resolution, to better guide internal radiologic decision-making about the spiculations and duct tracks, which may inform that surfaces are involved. These two demands are not easily solved by a single tool. Optical methods that scan large surfaces quickly are needed with cellular/molecular sensitivity to solve the first problem, but volumetric imaging with high spatial resolution for soft tissues is largely outside of the optical realm and requires x-ray, micro-CT, or magnetic resonance imaging if they can be achieved efficiently. In summary, it appears that a combination of systems into hybrid platforms may be the optimal solution for these two very different problems. This concept must be cost-effective, image specimens within minutes and be coupled to decision-making tools that help a surgeon without adding to the procedure. The potential for optical systems to be involved in this problem is emerging and clinical trials are underway in several of these technologies to see if they could reduce positive margin rates in BCS.

Fiber-based fluorescence lifetime imaging of recellularization processes on vascular tissue constructs

New techniques able to monitor the maturation of tissue engineered constructs over time are needed for a more efficient control of developmental parameters. Here, a label-free fluorescence lifetime imaging (FLIm) approach implemented through a single fiber-optic interface is reported for nondestructive in situ assessment of vascular biomaterials. Recellularization processes of antigen removed bovine pericardium scaffolds with endothelial cells and mesenchymal stem cells were evaluated on the serous and the fibrous sides of the scaffolds, 2 distinct extracellular matrix niches, over the course of a 7 day culture period. Results indicated that fluorescence lifetime successfully report cell presence resolved from extracellular matrix fluorescence. The recellularization process was more rapid on the serous side than on the fibrous side for both cell types, and endothelial cells expanded faster than mesenchymal stem cells on antigen-removed bovine pericardium. Fiber-based FLIm has the potential to become a nondestructive tool for the assessment of tissue maturation by allowing in situ imaging of intraluminal vascular biomaterials.

Intra-operative spectroscopic assessment of surgical margins during breast conserving surgery

Background

In over 20% of breast conserving operations, postoperative pathological assessment of the excised tissue reveals positive margins, requiring additional surgery. Current techniques for intra-operative assessment of tumor margins are insufficient in accuracy or resolution to reliably detect small tumors. There is a distinct need for a fast technique to accurately identify tumors smaller than 1 mm² in large tissue surfaces within 30 min.

Methods

Multi-modal spectral histopathology (MSH), a multimodal imaging technique combining tissue auto-fluorescence and Raman spectroscopy was used to detect microscopic residual tumor at the surface of the excised breast tissue. New algorithms were developed to optimally utilize auto-fluorescence images to guide Raman measurements and achieve the required detection accuracy over large tissue surfaces (up to 4 × 6.5 cm²). Algorithms were trained on 91 breast tissue samples from 65 patients.

Results

Independent tests on 121 samples from 107 patients - including 51 fresh, whole excision specimens - detected breast carcinoma on the tissue surface with 95% sensitivity and 82% specificity. One surface of each uncut excision specimen was measured in 12–24 min. The combination of high spatial-resolution auto-fluorescence with specific diagnosis by Raman spectroscopy allows reliable detection even for invasive carcinoma or ductal carcinoma in situ smaller than 1 mm².

Conclusions

This study provides evidence that this multimodal approach could provide an objective tool for intra-operative assessment of breast conserving surgery margins, reducing the risk for unnecessary second operations.

Electronic supplementary material

The online version of this article (10.1186/s13058-018-1002-2) contains supplementary material, which is available to authorized users.