Structured light imaging for breast-conserving surgery, part II: texture analysis and classification

Intraoperative near infrared functional imaging of rectal cancer using artificial intelligence methods - now and near future state of the art

Colorectal cancer remains a major cause of cancer death and morbidity worldwide. Surgery is a major treatment modality for primary and, increasingly, secondary curative therapy. However, with more patients being diagnosed with early stage and premalignant disease manifesting as large polyps, greater accuracy in diagnostic and therapeutic precision is needed right from the time of first endoscopic encounter. Rapid advancements in the field of artificial intelligence (AI), coupled with widespread availability of near infrared imaging (currently based around indocyanine green (ICG)) can enable colonoscopic tissue classification and prognostic stratification for significant polyps, in a similar manner to contemporary dynamic radiological perfusion imaging but with the advantage of being able to do so directly within interventional procedural time frames. It can provide an explainable method for immediate digital biopsies that could guide or even replace traditional forceps biopsies and provide guidance re margins (both areas where current practice is only approximately 80% accurate prior to definitive excision). Here, we discuss the concept and practice of AI enhanced ICG perfusion analysis for rectal cancer surgery while highlighting recent and essential near-future advancements. These include breakthrough developments in computer vision and time series analysis that allow for real-time quantification and classification of fluorescent perfusion signals of rectal cancer tissue intraoperatively that accurately distinguish between normal, benign, and malignant tissues in situ endoscopically, which are now undergoing international prospective validation (the Horizon Europe CLASSICA study). Next stage advancements may include detailed digital characterisation of small rectal malignancy based on intraoperative assessment of specific intratumoral fluorescent signal pattern. This could include T staging and intratumoral molecular process profiling (e.g. regarding angiogenesis, differentiation, inflammatory component, and tumour to stroma ratio) with the potential to accurately predict the microscopic local response to nonsurgical treatment enabling personalised therapy via decision support tools. Such advancements are also applicable to the next generation fluorophores and imaging agents currently emerging from clinical trials. In addition, by providing an understandable, applicable method for detailed tissue characterisation visually, such technology paves the way for acceptance of other AI methodology during surgery including, potentially, deep learning methods based on whole screen/video detailing.

Evaluating Receptor-Specific Fresh Specimen Staining for Tumor Margin Detection in Clinical Breast Specimens

PURPOSE

Reliable and rapid identification of tumor in the margins of breast specimens during breast-conserving surgery to reduce repeat surgery rates is an active area of investigation. Dual-stain difference imaging (DDSI) is one of many approaches under evaluation for this application. This technique aims to topically apply fluorescent stain pairs (one targeted to a receptor-of-interest and the other a spectrally distinct isotype), image both stains, and compute a normalized difference image between the two channels. Prior evaluation and optimization in a variety of preclinical models produced encouraging diagnostic performance. Herein, we report on a pilot clinical study which evaluated HER2-targeted DDSI on 11 human breast specimens.

PROCEDURES

Gross sections from 11 freshly excised mastectomy specimens were processed using a HER2-receptor-targeted DDSI protocol shortly after resection. After staining with the dual-probe protocol, specimens were imaged on a fluorescence scanner, followed by tissue fixation for hematoxylin and eosin and anti-HER2 immunohistochemical staining. Receiver operator characteristic curves and area under the curve (AUC) analysis were used to assess diagnostic performance of the resulting images. Performance values were also compared to expression level determined from IHC staining.

RESULTS

Eight of the 11 specimens presented with distinguishable invasive ductal carcinoma and/or were not affected by an imaging artifact. In these specimens, the DDSI technique provided an AUC = 0.90 ± 0.07 for tumor-to-adipose tissue and 0.81 ± 0.15 for tumor-to-glandular tissue, which was significantly higher than AUC values recovered from images of the targeted probe alone. DDSI values and diagnostic performance did not correlate with HER2 expression level, and tumors with low HER2 expression often produced high AUC, suggesting that even the low expression levels were enough to help distinguish tumor.

CONCLUSIONS

The results from this preliminary study of rapid receptor-specific staining in human specimens were consistent with prior preclinical results and demonstrated promising diagnostic potential.

Automated assessment of breast margins in deep ultraviolet fluorescence images using texture analysis

Microscopy with ultraviolet surface excitation (MUSE) is increasingly studied for intraoperative assessment of tumor margins during breast-conserving surgery to reduce the re-excision rate. Here we report a two-step classification approach using texture analysis of MUSE images to automate the margin detection. A study dataset consisting of MUSE images from 66 human breast tissues was constructed for model training and validation. Features extracted using six texture analysis methods were investigated for tissue characterization, and a support vector machine was trained for binary classification of image patches within a full image based on selected feature subsets. A weighted majority voting strategy classified a sample as tumor or normal. Using the eight most predictive features ranked by the maximum relevance minimum redundancy and Laplacian scores methods has achieved a sample classification accuracy of 92.4% and 93.0%, respectively. Local binary pattern alone has achieved an accuracy of 90.3%.

Non-invasive visualization of amyloid-beta deposits in Alzheimer amyloidosis mice using magnetic resonance imaging and fluorescence molecular tomography

Abnormal cerebral accumulation of amyloid-beta peptide (Aβ) is a major hallmark of Alzheimer's disease. Non-invasive monitoring of Aβ deposits enables assessing the disease burden in patients and animal models mimicking aspects of the human disease as well as evaluating the efficacy of Aβ-modulating therapies. Previous in vivo assessments of plaque load have been predominantly based on macroscopic fluorescence reflectance imaging (FRI) and confocal or two-photon microscopy using Aβ-specific imaging agents. However, the former method lacks depth resolution, whereas the latter is restricted by the limited field of view preventing a full coverage of the large brain region. Here, we utilized a fluorescence molecular tomography (FMT)-magnetic resonance imaging (MRI) pipeline with the curcumin derivative fluorescent probe CRANAD-2 to achieve full 3D brain coverage for detecting Aβ accumulation in the arcAβ mouse model of cerebral amyloidosis. A homebuilt FMT system was used for data acquisition, whereas a customized software platform enabled the integration of MRI-derived anatomical information as prior information for FMT image reconstruction. The results obtained from the FMT-MRI study were compared to those from conventional planar FRI recorded under similar physiological conditions, yielding comparable time courses of the fluorescence intensity following intravenous injection of CRANAD-2 in a region-of-interest comprising the brain. In conclusion, we have demonstrated the feasibility of visualizing Aβ deposition in 3D using a multimodal FMT-MRI strategy. This hybrid imaging method provides complementary anatomical, physiological and molecular information, thereby enabling the detailed characterization of the disease status in arcAβ mouse models, which can also facilitate monitoring the efficacy of putative treatments targeting Aβ.

Indocyanine green fluorescence image processing techniques for breast cancer macroscopic demarcation

Re-operation due to disease being inadvertently close to the resection margin is a major challenge in breast conserving surgery (BCS). Indocyanine green (ICG) fluorescence imaging could be used to visualize the tumor boundaries and help surgeons resect disease more efficiently. In this work, ICG fluorescence and color images were acquired with a custom-built camera system from 40 patients treated with BCS. Images were acquired from the tumor in-situ, surgical cavity post-excision, freshly excised tumor and histopathology tumour grossing. Fluorescence image intensity and texture were used as individual or combined predictors in both logistic regression (LR) and support vector machine models to predict the tumor extent. ICG fluorescence spectra in formalin-fixed histopathology grossing tumor were acquired and analyzed. Our results showed that ICG remains in the tissue after formalin fixation. Therefore, tissue imaging could be validated in freshly excised and in formalin-fixed grossing tumor. The trained LR model with combined fluorescence intensity (pixel values) and texture (slope of power spectral density curve) identified the tumor's extent in the grossing images with pixel-level resolution and sensitivity, specificity of 0.75 ± 0.3, 0.89 ± 0.2.This model was applied on tumor in-situ and surgical cavity (post-excision) images to predict tumor presence.

Developing diagnostic assessment of breast lumpectomy tissues using radiomic and optical signatures

High positive margin rates in oncologic breast-conserving surgery are a pressing clinical problem. Volumetric X-ray scanning is emerging as a powerful ex vivo specimen imaging technique for analyzing resection margins, but X-rays lack contrast between non-malignant and malignant fibrous tissues. In this study, combined micro-CT and wide-field optical image radiomics were developed to classify malignancy of breast cancer tissues, demonstrating that X-ray/optical radiomics improve malignancy classification. Ninety-two standardized features were extracted from co-registered micro-CT and optical spatial frequency domain imaging samples extracted from 54 breast tumors exhibiting seven tissue subtypes confirmed by microscopic histological analysis. Multimodal feature sets improved classification performance versus micro-CT alone when adipose samples were included (AUC = 0.88 vs. 0.90; p-value = 3.65e-11) and excluded, focusing the classification task on exclusively non-malignant fibrous versus malignant tissues (AUC = 0.78 vs. 0.85; p-value = 9.33e-14). Extending the radiomics approach to high-dimensional optical data-termed "optomics" in this study-offers a promising optical image analysis technique for cancer detection. Radiomic feature data and classification source code are publicly available.

Modeling and Synthesis of Breast Cancer Optical Property Signatures With Generative Models

Is it possible to find deterministic relationships between optical measurements and pathophysiology in an unsupervised manner and based on data alone? Optical property quantification is a rapidly growing biomedical imaging technique for characterizing biological tissues that shows promise in a range of clinical applications, such as intraoperative breast-conserving surgery margin assessment. However, translating tissue optical properties to clinical pathology information is still a cumbersome problem due to, amongst other things, inter- and intrapatient variability, calibration, and ultimately the nonlinear behavior of light in turbid media. These challenges limit the ability of standard statistical methods to generate a simple model of pathology, requiring more advanced algorithms. We present a data-driven, nonlinear model of breast cancer pathology for real-time margin assessment of resected samples using optical properties derived from spatial frequency domain imaging data. A series of deep neural network models are employed to obtain sets of latent embeddings that relate optical data signatures to the underlying tissue pathology in a tractable manner. These self-explanatory models can translate absorption and scattering properties measured from pathology, while also being able to synthesize new data. The method was tested on a total of 70 resected breast tissue samples containing 137 regions of interest, achieving rapid optical property modeling with errors only limited by current semi-empirical models, allowing for mass sample synthesis and providing a systematic understanding of dataset properties, paving the way for deep automated margin assessment algorithms using structured light imaging or, in principle, any other optical imaging technique seeking modeling. Code is available.

Optical scatter imaging of resected breast tumor structures matches the patterns of micro-computed tomography

In patients undergoing breast-conserving surgery (BCS), the rate of re-excision procedures to remove residual tumor left behind after initial resection can be high. Projection radiography, and recently, volumetric x-ray imaging are used to assess margin adequacy, but x-ray imaging lacks contrast between healthy, abnormal benign, and malignant fibrous tissues important for surgical decision making. The purpose of this study was to compare micro-CT and optical scatter imagery of surgical breast specimens and to demonstrate enhanced contrast-to intra-tumoral morphologies and tumor boundary features revealed by optical scatter imaging. A total of 57 breast tumor slices from 57 patients were imagedex vivoby spatially co-registered micro-CT and optical scatter scanning. Optical scatter exhibited greater similarity with micro-CT in 89% (51/57) of specimens versus diffuse white light (DWL) luminance using mutual information (mean ± standard deviation of 0.48 ± 0.21 versus 0.24 ± 0.12;p < 0.001) and in 81% (46/57) of specimens using the Sørensen-Dice coefficient (0.48 ± 0.21 versus 0.33 ± 0.18;p < 0.001). The coefficient of variation (CV) quantified the feature content in each image. Optical scatter exhibited the highest CV in every specimen (optical scatter: 0.70 ± 0.17; diffuse luminance: 0.24 ± 01; micro-CT: 0.15 ± 0.03 for micro-CT;p < 0.001). Optical scatter also exhibited the highest contrast ratios across representative tumor boundaries with adjacent healthy/benign fibrous tissues (1.5-3.7 for optical scatter; 1.0-1.1 for diffuse luminance; 1.0-1.1 for micro-CT). The two main findings from this study were: first, optical scatter contrast was in general similar to the radiological view of the tissue relative to DWL imaging; and second, optical scatter revealed additional features associated with fibrous tissue structures of similar radiodensity that may be relevant to diagnosis. The value of micro-CT lies in its rapid three-dimensional scanning of specimen morphology, and combined with optical scatter imaging with sensitivity to fibrous surface tissues, may be an attractive solution for margin assessment during BCS.

Biophotonic technologies for assessment of breast tumor surgical margins-A review

Breast conserving surgery (BCS) offering similar surgical outcomes as mastectomy while retaining breast cosmesis is becoming increasingly popular for the management of early stage breast cancers. However, its association with reoperation rates of 20% to 40% following incomplete tumor removal warrants the need for a fast and accurate intraoperative surgical margin assessment tool that offers cellular, structural and molecular information of the whole specimen surface to a clinically relevant depth. Biophotonic technologies are evolving to qualify as such an intraoperative tool for clinical assessment of breast cancer surgical margins at the microscopic and macroscopic scale. Herein, we review the current research in the application of biophotonic technologies such as photoacoustic imaging, Raman spectroscopy, multimodal multiphoton imaging, diffuse optical imaging and fluorescence imaging using medically approved dyes for breast cancer detection and/or tumor subtype differentiation toward intraoperative assessment of surgical margins in BCS specimens, and possible challenges in their route to clinical translation.

Active line scan with spatial gating for sub-diffuse reflectance imaging of scatter microtexture

We examine the value of an active line scan with spatial gating for imaging sub-diffuse, wide-field reflectance microtexture. Line scanning combined with spatial gating and linear translation can be used for localized detection of features in the surface layer of a turbid target. The line scan provides broadband spatial frequency modulation, and the spatial gating effectively high-pass filters the reflectance. The major benefit of this approach is that of high dynamic range (70%-90%) signal preservation and high contrast to noise when imaging at high spatial frequencies. Alternative approaches, such as spatial frequency domain imaging, are degraded by low dynamic range in demodulated images, making it nearly impossible to image over a wide field of view at frequencies over 1.5mm-1 using commercial technology. As such, active line scanning with spatial gating presents as an inherently high sensitivity and high dynamic range method of imaging microscopic scattering features in only the surface layer of a turbid medium.

Structured light imaging for breast-conserving surgery, part I: optical scatter and color analysis

Structured light imaging (SLI) with high spatial frequency (HSF) illumination provides a method to amplify native tissue scatter contrast and better differentiate superficial tissues. This was investigated for margin analysis in breast-conserving surgery (BCS) and imaging gross clinical tissues from 70 BCS patients, and the SLI distinguishability was examined for six malignancy subtypes relative to three benign/normal breast tissue subtypes. Optical scattering images recovered were analyzed with five different color space representations of multispectral demodulated reflectance. Excluding rare combinations of invasive lobular carcinoma and fibrocystic disease, SLI was able to classify all subtypes of breast malignancy from surrounding benign tissues (p-value  <  0.05) based on scatter and color parameters. For color analysis, HSF illumination of the sample generated more statistically significant discrimination than regular uniform illumination. Pathological information about lesion subtype from a presurgical biopsy can inform the search for malignancy on the surfaces of specimens during BCS, motivating the focus on pairwise classification analysis. This SLI modality is of particular interest for its potential to differentiate tissue classes across a wide field-of-view (∼100  cm2) and for its ability to acquire images of macroscopic tissues rapidly but with microscopic-level sensitivity to structural and morphological tissue constituents.