Development of an integrated multimodal optical imaging system with real-time image analysis for the evaluation of oral premalignant lesions

Use of topical methylene blue to image nuclear morphometry with a low-cost scanning darkfield microendoscope

Significance

Fiber-optic microendoscopy is a promising approach to noninvasively visualize epithelial nuclear morphometry for early cancer and precancer detection. However, the broader clinical application of this approach is limited by a lack of topical contrast agents available for in vivo use.

Aim

The aim of this study was to evaluate the ability to image nuclear morphometry in vivo with a novel fiber-optic microendoscope used together with topical application of methylene blue (MB), a dye with FDA approval for use in chromoendoscopy in the gastrointestinal tract.

Approach

The low-cost, high-resolution microendoscope implements scanning darkfield imaging without complex optomechanical components by leveraging programmable illumination and the rolling shutter of the image sensor. We validate the integration of our system and MB staining for visualizing epithelial cell nuclei by performing ex vivo imaging on fresh animal specimens and in vivo imaging on healthy volunteers.

Results

The results indicate that scanning darkfield imaging significantly reduces specular reflection and resolves epithelial nuclei with enhanced image contrast and spatial resolution compared to non-scanning widefield imaging. The image quality of darkfield images with MB staining is comparable to that of fluorescence images with proflavine staining.

Conclusions

Our approach enables real-time microscopic evaluation of nuclear patterns and has the potential to be a powerful noninvasive tool for early cancer detection.

Single source CARS-based multimodal microscopy system for biological tissue imaging [Invited]

A coherent anti-Stokes Raman scattering (CARS)-based multimodality microscopy system was developed using a single Ti:sapphire femtosecond laser source for biological imaging. It provides three complementary and co-registered imaging modalities: CARS, MPM (multiphoton microscopy), and RCM (reflectance confocal microscopy). The imaging speed is about 1 frame-per-second (fps) with a digital resolution of 1024 × 1024 pixels. This microscopy system can provide clear 2-dimensional and 3-dimensional images of ex-vivo biological tissue samples. Its spectral selection initiates vibrational excitation in lipid cells (approximately 2850 cm-1) using two filters on the pump and Stokes beam paths. The excitation can be tuned over a wide spectral range with adjustable spectral filters. The imaging capability of this CARS-based multimodal microscopy system was demonstrated using porcine fat, murine skin, and murine liver tissue samples.

Automated In Vivo High-Resolution Imaging to Detect Human Papillomavirus-Associated Anal Precancer in Persons Living With HIV

INTRODUCTION

In the United States, the effectiveness of anal cancer screening programs has been limited by a lack of trained professionals proficient in high-resolution anoscopy (HRA) and a high patient lost-to-follow-up rate between diagnosis and treatment. Simplifying anal intraepithelial neoplasia grade 2 or more severe (AIN 2+) detection could radically improve the access and efficiency of anal cancer prevention. Novel optical imaging providing point-of-care diagnoses could substantially improve existing HRA and histology-based diagnosis. This work aims to demonstrate the potential of high-resolution microendoscopy (HRME) coupled with a novel machine learning algorithm for the automated, in vivo diagnosis of anal precancer.

METHODS

The HRME, a fiber-optic fluorescence microscope, was used to capture real-time images of anal squamous epithelial nuclei. Nuclear staining is achieved using 0.01% wt/vol proflavine, a topical contrast agent. HRME images were analyzed by a multitask deep learning network (MTN) that computed the probability of AIN 2+ for each HRME image.

RESULTS

The study accrued data from 77 people living with HIV. The MTN achieved an area under the receiver operating curve of 0.84 for detection of AIN 2+. At the AIN 2+ probability cutoff of 0.212, the MTN achieved comparable performance to expert HRA impression with a sensitivity of 0.92 ( P = 0.68) and specificity of 0.60 ( P = 0.48) when using histopathology as the gold standard.

DISCUSSION

When used in combination with HRA, this system could facilitate more selective biopsies and promote same-day AIN2+ treatment options by enabling real-time diagnosis.

Multimodal optical imaging with real-time projection of cancer risk and biopsy guidance maps for early oral cancer diagnosis and treatment

SIGNIFICANCE

Despite recent advances in multimodal optical imaging, oral imaging systems often do not provide real-time actionable guidance to the clinician who is making biopsy and treatment decisions.

AIM

We demonstrate a low-cost, portable active biopsy guidance system (ABGS) that uses multimodal optical imaging with deep learning to directly project cancer risk and biopsy guidance maps onto oral mucosa in real time.

APPROACH

Cancer risk maps are generated based on widefield autofluorescence images and projected onto the at-risk tissue using a digital light projector. Microendoscopy images are obtained from at-risk areas, and multimodal image data are used to calculate a biopsy guidance map, which is projected onto tissue.

RESULTS

Representative patient examples highlight clinically actionable visualizations provided in real time during an imaging procedure. Results show multimodal imaging with cancer risk and biopsy guidance map projection offers a versatile, quantitative, and precise tool to guide biopsy site selection and improve early detection of oral cancers.

CONCLUSIONS

The ABGS provides direct visible guidance to identify early lesions and locate appropriate sites to biopsy within those lesions. This represents an opportunity to translate multimodal imaging into real-time clinically actionable visualizations to help improve patient outcomes.

Mildly dysplastic oral lesions with optically-detectable abnormalities share genetic similarities with severely dysplastic lesions

OBJECTIVE

Optical imaging studies of oral premalignant lesions have shown that optical markers, including loss of autofluorescence and altered morphology of epithelial cell nuclei, are predictive of high-grade pathology. While these optical markers are consistently positive in lesions with moderate/severe dysplasia or cancer, they are positive only in a subset of lesions with mild dysplasia. This study compared the gene expression profiles of lesions with mild dysplasia (stratified by optical marker status) to lesions with severe dysplasia and without dysplasia.

MATERIALS AND METHODS

Forty oral lesions imaged in patients undergoing oral surgery were analyzed: nine without dysplasia, nine with severe dysplasia, and 22 with mild dysplasia. Samples were submitted for high throughput gene expression analysis.

RESULTS

The analysis revealed 116 genes differentially expressed among sites without dysplasia and sites with severe dysplasia; 50 were correlated with an optical marker quantifying altered nuclear morphology. Ten of 11 sites with mild dysplasia and positive optical markers (91%) had gene expression similar to sites with severe dysplasia. Nine of 11 sites with mild dysplasia and negative optical markers (82%) had similar gene expression as sites without dysplasia.

CONCLUSION

This study suggests that optical imaging may help identify patients with mild dysplasia who require more intensive clinical follow-up. If validated, this would represent a significant advance in patient care for patients with oral premalignant lesions.

Optical Imaging in the Diagnosis of OPMDs Malignant Transformation

Oral potentially malignant disorders (OPMDs) are a heterogeneous group of oral lesions with a variable risk of malignant transformation to oral squamous cell carcinoma. The current OPMDs malignant transformation screening depends on conventional oral examination (COE) and is confirmed by biopsy and histologic examination. However, early malignant lesions with subtle mucosal changes are easily unnoticed by COE based on visual inspection and palpation. Optical techniques have been used to determine the biological structure, composition, and function of cells and tissues noninvasively by analyzing the changes in their optical properties. The oral epithelium and stroma undergo persistent structural, functional, and biochemical alterations during malignant transformation, leading to variations in optical tissue properties; optical techniques are thus powerful tools for detecting OPMDs malignant transformation. The optical imaging methods already used to detect OPMDs malignant transformation in vivo include autofluorescence imaging, narrowband imaging, confocal reflectance microscopy, and optical coherence tomography. They exhibit advantages over COE in detecting biochemical or morphologic changes at the molecular or cellular level in vivo; however, limitations also exist. This article comprehensively reviews the various real-time in vivo optical imaging methods used in the adjunctive diagnosis of OPMDs malignant transformation. We focus on the principles of these techniques, review their clinical application, and compare and summarize their advantages and disadvantages. Finally, we conclude with a discussion of current challenges and future directions of this field.

Multimodal microscopy for the simultaneous visualization of five different imaging modalities using a single light source

Optical microscopy has been widely used in biomedical research as it provides photophysical and photochemical information of the target in subcellular spatial resolution without requiring physical contact with the specimen. To obtain a deeper understanding of biological phenomena, several efforts have been expended to combine such optical imaging modalities into a single microscope system. However, the use of multiple light sources and detectors through separated beam paths renders previous systems extremely complicated or slow for in vivo imaging. Herein, we propose a novel high-speed multimodal optical microscope system that simultaneously visualizes five different microscopic contrasts, i.e., two-photon excitation, second-harmonic generation, backscattered light, near-infrared fluorescence, and fluorescence lifetime, using a single femtosecond pulsed laser. Our proposed system can visualize five modal images with a frame rate of 3.7 fps in real-time, thereby providing complementary optical information that enhances both structural and functional contrasts. This highly photon-efficient multimodal microscope system enables various properties of biological tissues to be assessed.

Bimodal multispectral imaging system with cloud-based machine learning algorithm for real-time screening and detection of oral potentially malignant lesions and biopsy guidance

SIGNIFICANCE

Screening and early detection of oral potentially malignant lesions (OPMLs) are of great significance in reducing the mortality rates associated with head and neck malignancies. Intra-oral multispectral optical imaging of tissues in conjunction with cloud-based machine learning (CBML) can be used to detect oral precancers at the point-of-care (POC) and guide the clinician to the most malignant site for biopsy.

AIM

Develop a bimodal multispectral imaging system (BMIS) combining tissue autofluorescence and diffuse reflectance (DR) for mapping changes in oxygenated hemoglobin (HbO2) absorption in the oral mucosa, quantifying tissue abnormalities, and guiding biopsies.

APPROACH

The hand-held widefield BMIS consisting of LEDs emitting at 405, 545, 575, and 610 nm, 5MPx monochrome camera, and proprietary Windows-based software was developed for image capture, processing, and analytics. The DR image ratio (R610/R545) was compared with pathologic classification to develop a CBML algorithm for real-time assessment of tissue status at the POC.

RESULTS

Sensitivity of 97.5% and specificity of 92.5% were achieved for discrimination of OPML from patient normal in 40 sites, whereas 82% sensitivity and 96.6% specificity were obtained for discrimination of abnormal (OPML + SCC) in 89 sites. Site-specific algorithms derived for buccal mucosa (27 sites) showed improved sensitivity and specificity of 96.3% for discrimination of OPML from normal.

CONCLUSIONS

Assessment of oral cancer risk is possible by mapping of HbO2 absorption in tissues, and the BMIS system developed appears to be suitable for biopsy guidance and early detection of oral cancers.

High frame rate video mosaicking microendoscope to image large regions of intact tissue with subcellular resolution

High-resolution microendoscopy (HRME) is a low-cost strategy to acquire images of intact tissue with subcellular resolution at frame rates ranging from 11 to 18 fps. Current HRME imaging strategies are limited by the small microendoscope field of view (∼0.5 mm2); multiple images must be acquired and reliably registered to assess large regions of clinical interest. Image mosaics have been assembled from co-registered frames of video acquired as a microendoscope is slowly moved across the tissue surface, but the slow frame rate of previous HRME systems made this approach impractical for acquiring quality mosaicked images from large regions of interest. Here, we present a novel video mosaicking microendoscope incorporating a high frame rate CMOS sensor and optical probe holder to enable high-speed, high quality interrogation of large tissue regions of interest. Microendoscopy videos acquired at >90 fps are assembled into an image mosaic. We assessed registration accuracy and image sharpness across the mosaic for images acquired with a handheld probe over a range of translational speeds. This high frame rate video mosaicking microendoscope enables in vivo probe translation at >15 millimeters per second while preserving high image quality and accurate mosaicking, increasing the size of the region of interest that can be interrogated at high resolution from 0.5 mm2 to >30 mm2. Real-time deployment of this high-frame rate system is demonstrated in vivo and source code made publicly available.

Multimodal widefield fluorescence imaging with nonlinear optical microscopy workflow for noninvasive oral epithelial neoplasia detection: a preclinical study

SIGNIFICANCE

Early detection of epithelial cancers and precancers/neoplasia in the presence of benign lesions is challenging due to the lack of robust in vivo imaging and biopsy guidance techniques. Label-free nonlinear optical microscopy (NLOM) has shown promise for optical biopsy through the detection of cellular and extracellular signatures of neoplasia. Although in vivo microscopy techniques continue to be developed, the surface area imaged in microscopy is limited by the field of view. FDA-approved widefield fluorescence (WF) imaging systems that capture autofluorescence signatures of neoplasia provide molecular information at large fields of view, which may complement the cytologic and architectural information provided by NLOM.

AIM

A multimodal imaging approach with high-sensitivity WF and high-resolution NLOM was investigated to identify and distinguish image-based features of neoplasia from normal and benign lesions.

APPROACH

In vivo label-free WF imaging and NLOM was performed in preclinical hamster models of oral neoplasia and inflammation. Analyses of WF imaging, NLOM imaging, and dual modality (WF combined with NLOM) were performed.

RESULTS

WF imaging showed increased red-to-green autofluorescence ratio in neoplasia compared to inflammation and normal oral mucosa (p  <  0.01). In vivo assessment of the mucosal tissue with NLOM revealed subsurface cytologic (nuclear pleomorphism) and architectural (remodeling of extracellular matrix) atypia in histologically confirmed neoplastic tissue, which were not observed in inflammation or normal mucosa. Univariate and multivariate statistical analysis of macroscopic and microscopic image-based features indicated improved performance (94% sensitivity and 97% specificity) of a multiscale approach over WF alone, even in the presence of benign lesions (inflammation), a common confounding factor in diagnostics.

CONCLUSIONS

A multimodal imaging approach integrating strengths from WF and NLOM may be beneficial in identifying oral neoplasia. Our study could guide future studies on human oral neoplasia to further evaluate merits and limitations of multimodal workflows and inform the development of multiscale clinical imaging systems.

Algorithm to quantify nuclear features and confidence intervals for classification of oral neoplasia from high-resolution optical images

Purpose: In vivo optical imaging technologies like high-resolution microendoscopy (HRME) can image nuclei of the oral epithelium. In principle, automated algorithms can then calculate nuclear features to distinguish neoplastic from benign tissue. However, images frequently contain regions without visible nuclei, due to biological and technical factors, decreasing the data available to and accuracy of image analysis algorithms. Approach: We developed the nuclear density-confidence interval (ND-CI) algorithm to determine if an HRME image contains sufficient nuclei for classification, or if a better image is required. The algorithm uses a convolutional neural network to exclude image regions without visible nuclei. Then the remaining regions are used to estimate a confidence interval (CI) for the number of abnormal nuclei per   mm 2 , a feature used by a previously developed algorithm (called the ND algorithm), to classify images as benign or neoplastic. The range of the CI determines whether the ND-CI algorithm can classify an image with confidence, and if so, the predicted category. The ND and ND-CI algorithm were compared by calculating their positive predictive value (PPV) and negative predictive value (NPV) on 82 oral biopsies with histopathologically confirmed diagnoses. Results: After excluding the images that could not be classified with confidence, the ND-CI algorithm had higher PPV (65% versus 59%) and NPV (78% versus 75%) than the ND algorithm. Conclusions: The ND-CI algorithm could improve the real-time classification of HRME images of the oral epithelium by informing the user if an improved image is required for diagnosis.

Repurposing Molecular Imaging and Sensing for Cancer Image-Guided Surgery

Gone are the days when medical imaging was used primarily to visualize anatomic structures. The emergence of molecular imaging (MI), championed by radiolabeled ¹⁸F-FDG PET, has expanded the information content derived from imaging to include pathophysiologic and molecular processes. Cancer imaging, in particular, has leveraged advances in MI agents and technology to improve the accuracy of tumor detection, interrogate tumor heterogeneity, monitor treatment response, focus surgical resection, and enable image-guided biopsy. Surgeons are actively latching on to the incredible opportunities provided by medical imaging for preoperative planning, intraoperative guidance, and postoperative monitoring. From label-free techniques to enabling cancer-selective imaging agents, image-guided surgery provides surgical oncologists and interventional radiologists both macroscopic and microscopic views of cancer in the operating room. This review highlights the current state of MI and sensing approaches available for surgical guidance. Salient features of nuclear, optical, and multimodal approaches will be discussed, including their strengths, limitations, and clinical applications. To address the increasing complexity and diversity of methods available today, this review provides a framework to identify a contrast mechanism, suitable modality, and device. Emerging low-cost, portable, and user-friendly imaging systems make the case for adopting some of these technologies as the global standard of care in surgical practice.

Prospective evaluation of oral premalignant lesions using a multimodal imaging system: a pilot study

BACKGROUND

Multimodal optical imaging, incorporating reflectance and fluorescence modalities, is a promising tool to detect oral premalignant lesions in real-time.

METHODS

Images were acquired from 171 sites in 66 patient visits for clinical evaluation of oral lesions. An automated algorithm was used to classify lesions as high- or low-risk for neoplasia. Biopsies were acquired at clinically indicated sites and those classified as high-risk by imaging, at the surgeon's discretion.

RESULTS

Twenty sites were biopsied based on clinical examination or imaging. Of these, 12 were indicated clinically and by imaging; 58% were moderate dysplasia or worse. Four biopsies were indicated by imaging evaluation only; 75% were moderate dysplasia or worse. Finally, four biopsies were indicated by clinical evaluation only; 75% were moderate dysplasia or worse.

CONCLUSION

Multimodal imaging identified more cases of high-grade dysplasia than clinical evaluation, and can improve detection of high grade precancer in patients with oral lesions.

Autofluorescence Imaging to Monitor the Progression of Oral Potentially Malignant Disorders

Patients with oral potentially malignant disorders (OPMD) must undergo regular clinical surveillance to ensure that any progression to malignancy is detected promptly. Autofluorescence imaging (AFI) is an optical modality that can assist clinicians in detecting early cancers and high-grade dysplasia. Patients with OPMD undergoing surveillance for the development of oral cancer were examined using AFI at successive clinic visits. Autofluorescence images acquired at 133 clinical visits from sites in 15 patients who met inclusion criteria were analyzed quantitatively using an algorithm to calculate the red-to-green pixel intensity (RG ratio). A quantitative AFI threshold for high risk of progression was defined based on the RG ratio and was compared with expert clinical impression and with histopathology when available. Patients were divided into two groups based on their endpoint: surveillance (n = 6) or surgery (n = 9). In the surveillance group, 0 of 6 (0%) of patients were clinically identified as high risk for progression prior to the study endpoint, whereas 1 of 6 (17%) of patients were deemed at high risk for progression based on AFI during the same time period. In the surgery group, 9 of 9 (100%) of patients were clinically identified as high risk prior to the study endpoint, whereas 8 of 9 (89%) of patients were at high risk for progression based on AFI during the same time period. AFI results tracked over time were comparable with expert clinical impression in these patient groups. AFI has the potential to aid clinicians in noninvasively monitoring oral precancer and evaluating OPMDs that require increased surveillance.