Deep learning automated pathology in ex vivo microscopy

Artificial intelligence for nonmelanoma skin cancer

Nonmelanoma skin cancers (NMSCs) are among the top five most common cancers globally. NMSC is an area with great potential for novel application of diagnostic tools including artificial intelligence (AI). In this scoping review, we aimed to describe the applications of AI in the diagnosis and treatment of NMSC. Twenty-nine publications described AI applications to dermatopathology including lesion classification and margin assessment. Twenty-five publications discussed AI use in clinical image analysis, showing that algorithms are not superior to dermatologists and may rely on unbalanced, nonrepresentative, and nontransparent training data sets. Sixteen publications described the use of AI in cutaneous surgery for NMSC including use in margin assessment during excisions and Mohs surgery, as well as predicting procedural complexity. Eleven publications discussed spectroscopy, confocal microscopy, thermography, and the AI algorithms that analyze and interpret their data. Ten publications pertained to AI applications for the discovery and use of NMSC biomarkers. Eight publications discussed the use of smartphones and AI, specifically how they enable clinicians and patients to have increased access to instant dermatologic assessments but with varying accuracies. Five publications discussed large language models and NMSC, including how they may facilitate or hinder patient education and medical decision-making. Three publications pertaining to the skin of color and AI for NMSC discussed concerns regarding limited diverse data sets for the training of convolutional neural networks. AI demonstrates tremendous potential to improve diagnosis, patient and clinician education, and management of NMSC. Despite excitement regarding AI, data sets are often not transparently reported, may include low-quality images, and may not include diverse skin types, limiting generalizability. AI may serve as a tool to increase access to dermatology services for patients in rural areas and save health care dollars. These benefits can only be achieved, however, with consideration of potential ethical costs.

Artificial Intelligence for Mohs and Dermatologic Surgery: A Systematic Review and Meta-Analysis

BACKGROUND

Over the past decade, several studies have shown that potential of artificial intelligence (AI) in dermatology. However, there has yet to be a systematic review evaluating the usage of AI specifically within the field of Mohs micrographic surgery (MMS).

OBJECTIVE

In this review, we aimed to comprehensively evaluate the current state, efficacy, and future implications of AI when applied to MMS for the treatment of nonmelanoma skin cancers (NMSC).

MATERIALS AND METHODS

A systematic review and meta-analysis was conducted following PRISMA guidelines across several databases, including PubMed/MEDLINE, Embase, and Cochrane libraries. A predefined protocol was registered in PROSPERO, with literature search involving specific keywords related to AI and Mohs surgery for NMSC.

RESULTS

From 23 studies evaluated, our results find that AI shows promise as a prediction tool for precisely identifying NMSC in tissue sections during MMS. Furthermore, high AUC and concordance values were also found across the various usages of AI in MMS, including margin control, surgical recommendations, similarity metrics, and in the prediction of stage and construction complexity.

CONCLUSION

The findings of this review suggest promising potential for AI to enhance the accuracy and efficiency of Mohs surgery, particularly for NMSC.

Ex Vivo Fluorescence Confocal Microscopy for Intraoperative Examinations of Lung Tumors as Alternative to Frozen Sections-A Proof-of-Concept Study

BACKGROUND

Intraoperative frozen sections (FS) are frequently used to establish the diagnosis of lung cancer when preoperative examinations are not conclusive. The downside of FS is its resource-intensive nature and the risk of tissue depletion when small lesions are assessed. Ex vivo fluorescence confocal microscopy (FCM) is a novel microimaging method for loss-free examinations of native materials. We tested its suitability for the intraoperative diagnosis of lung tumors.

METHODS

Samples from 59 lung resection specimens containing 45 carcinomas were examined in the FCM. The diagnostic performance in the evaluation of malignancy and histological typing of lung tumors was evaluated in comparison with FS and the final diagnosis.

RESULTS

A total of 44/45 (98%) carcinomas were correctly identified as malignant in the FCM. A total of 33/44 (75%) carcinomas were correctly subtyped, which was comparable with the results of FS and conventional histology. Our tests documented the excellent visualization of cytological features of normal tissues and tumors. Compared to FS, FCM was technically less demanding and less personnel intensive.

CONCLUSIONS

The ex vivo FCM is a fast, effective, and safe method for diagnosing and subtyping lung cancer and is, therefore, a promising alternative to FS. The method preserves the tissue without loss for subsequent examinations, which is an advantage in the diagnosis of small tumors and for biobanking.

Breast tissue imaging atlas using ultra-fast confocal microscopy to identify cancer lesions

New generation ultra-fast fluorescence confocal microscopy (UFCM) allows to image histological architecture of fresh breast tissue and may be used for ex vivo intraoperative analysis for margin status. The criteria to identify breast tumoral and non-tumoral tissues in UFCM images are still objects of investigation. The objective of the study was to create an atlas of ex vivo UFCM images of breast tissues and breast carcinomas based on the first extensive collection of large field-of-view UFCM breast images. One hundred sixty patients who underwent conserving surgery for breast cancer were included. Their fresh surgical specimens were sliced, stained with acridine orange, and imaged at high resolution with large-field-of-view UFCM. The resulting images were digitally false colored to resemble frozen sections. Each UFCM image was correlated with the corresponding definitive histology. Representative images of normal tissue, inflammation, benign lesions, invasive carcinoma (IC), and ductal carcinoma in situ (DCIS) were collected. A total of 320 large-field images were recorded from 58 IC of no special type, 44 invasive lobular carcinomas, 1 invasive mucinous carcinoma, 47 DCIS, 2 lobular carcinomas in situ, and 8 specimens without cancer. Representative images of the main components of the normal breast and the main types of ICs and DCIS were annotated to establish an UFCM atlas. UFCM enables the imaging of the fresh breast tissue sections. Main morphological criteria defined in traditional histopathology such as tissue architecture and cell features can be applied to describe UFCM images content. The generated atlas of the main normal or tumoral tissue features will support the adoption of this optical technology for the intraoperative examination of breast specimens in clinical practice as it can be used to train physicians on UFCM images and develop artificial intelligence algorithms. Further studies are needed to document rare breast lesions.

High-resolution full-field optical coherence tomography microscope for the evaluation of freshly excised skin specimens during Mohs surgery: A feasibility study

Histopathology for tumor margin assessment is time-consuming and expensive. High-resolution full-field optical coherence tomography (FF-OCT) images fresh tissues rapidly at cellular resolution and potentially facilitates evaluation. Here, we define FF-OCT features of normal and neoplastic skin lesions in fresh ex vivo tissues and assess its diagnostic accuracy for malignancies. For this, normal and neoplastic tissues were obtained from Mohs surgery, imaged using FF-OCT, and their features were described. Two expert OCT readers conducted a blinded analysis to evaluate their diagnostic accuracies, using histopathology as the ground truth. A convolutional neural network was built to distinguish and outline normal structures and tumors. Of the 113 tissues imaged, 95 (84%) had a tumor (75 basal cell carcinomas [BCCs] and 17 squamous cell carcinomas [SCCs]). The average reader diagnostic accuracy was 88.1%, with a sensitivity of 93.7%, and a specificity of 58.3%. The artificial intelligence (AI) model achieved a diagnostic accuracy of 87.6 ± 5.9%, sensitivity of 93.2 ± 2.1%, and specificity of 81.2 ± 9.2%. A mean intersection-over-union of 60.3 ± 10.1% was achieved when delineating the nodular BCC from normal structures. Limitation of the study was the small sample size for all tumors, especially SCCs. However, based on our preliminary results, we envision FF-OCT to rapidly image fresh tissues, facilitating surgical margin assessment. AI algorithms can aid in automated tumor detection, enabling widespread adoption of this technique.

Harnessing the power of optical microscopy for visualization and analysis of histopathological images

Histopathology is the foundation and gold standard for identifying diseases, and precise quantification of histopathological images can provide the pathologist with objective clues to make a more convincing diagnosis. Optical microscopy (OM), an important branch of optical imaging technology that provides high-resolution images of tissue cytology and structural morphology, has been used in the diagnosis of histopathology and evolved into a new disciplinary direction of optical microscopic histopathology (OMH). There are a number of ex-vivo studies providing applicability of different OMH approaches, and a transfer of these techniques toward in vivo diagnosis is currently in progress. Furthermore, combined with advanced artificial intelligence algorithms, OMH allows for improved diagnostic reliability and convenience due to the complementarity of retrieval information. In this review, we cover recent advances in OMH, including the exploration of new techniques in OMH as well as their applications, and look ahead to new challenges in OMH. These typical application examples well demonstrate the application potential and clinical value of OMH techniques in histopathological diagnosis.

Novel method for predicting nonvisible symptoms using machine learning in cancer palliative care

End-of-life patients with cancer may find expressing their symptoms difficult if they can no longer communicate verbally because of deteriorating health. In this study, we assessed these symptoms using machine learning, which has excellent predictive capabilities and has recently been applied in healthcare. We performed a retrospective clinical survey involving 213 patients with cancer from August 2015 to August 2016. We divided the reported symptoms into two groups-visible and nonvisible symptoms. We used decision tree analysis, an analytical machine learning method that organizes and analyzes information in the form of a tree diagram to visually represent the information structure. Our machine learning model used patient background data and visible symptoms to predict nonvisible symptoms: pain, dyspnea, fatigue, drowsiness, anxiety, delirium, inadequate informed consent, and spiritual issues. The highest and/or lowest values for prediction accuracy, sensitivity, and specificity were 88.0%/55.5%, 84.9%/3.3%, and 96.7%/24.1%, respectively. This work will facilitate better assessment and management of symptoms in patients with cancer. This study was the first to predict nonvisible symptoms using decision tree analyses for patients with cancer receiving palliative care. Notably, applications based on our results may assess symptoms to the same extent as healthcare professionals.

Artificial Intelligence-Aided Optical Imaging for Cancer Theranostics

The use of artificial intelligence (AI) to assist biomedical imaging have demonstrated its high accuracy and high efficiency in medical decision-making for individualized cancer medicine. In particular, optical imaging methods are able to visualize both the structural and functional information of tumors tissues with high contrast, low cost, and noninvasive property. However, no systematic work has been performed to inspect the recent advances on AI-aided optical imaging for cancer theranostics. In this review, we demonstrated how AI can guide optical imaging methods to improve the accuracy on tumor detection, automated analysis and prediction of its histopathological section, its monitoring during treatment, and its prognosis by using computer vision, deep learning and natural language processing. By contrast, the optical imaging techniques involved mainly consisted of various tomography and microscopy imaging methods such as optical endoscopy imaging, optical coherence tomography, photoacoustic imaging, diffuse optical tomography, optical microscopy imaging, Raman imaging, and fluorescent imaging. Meanwhile, existing problems, possible challenges and future prospects for AI-aided optical imaging protocol for cancer theranostics were also discussed. It is expected that the present work can open a new avenue for precision oncology by using AI and optical imaging tools.

Confocal Microscopy for Diagnosis and Management of Cutaneous Malignancies: Clinical Impacts and Innovation

Cutaneous malignancies are common malignancies worldwide, with rising incidence. Most skin cancers, including melanoma, can be cured if diagnosed correctly at an early stage. Thus, millions of biopsies are performed annually, posing a major economic burden. Non-invasive skin imaging techniques can aid in early diagnosis and save unnecessary benign biopsies. In this review article, we will discuss in vivo and ex vivo confocal microscopy (CM) techniques that are currently being utilized in dermatology clinics for skin cancer diagnosis. We will discuss their current applications and clinical impact. Additionally, we will provide a comprehensive review of the advances in the field of CM, including multi-modal approaches, the integration of fluorescent targeted dyes, and the role of artificial intelligence for improved diagnosis and management.

Deep learning on reflectance confocal microscopy improves Raman spectral diagnosis of basal cell carcinoma

SIGNIFICANCE

Raman spectroscopy (RS) provides an automated approach for assisting Mohs micrographic surgery for skin cancer diagnosis; however, the specificity of RS is limited by the high spectral similarity between tumors and normal tissues structures. Reflectance confocal microscopy (RCM) provides morphological and cytological details by which many features of epidermis and hair follicles can be readily identified. Combining RS with deep-learning-aided RCM has the potential to improve the diagnostic accuracy of RS in an automated fashion, without requiring additional input from the clinician.

AIM

The aim of this study is to improve the specificity of RS for detecting basal cell carcinoma (BCC) using an artificial neural network trained on RCM images to identify false positive normal skin structures (hair follicles and epidermis).

APPROACH

Our approach was to build a two-step classification model. In the first step, a Raman biophysical model that was used in prior work classified BCC tumors from normal tissue structures with high sensitivity. In the second step, 191 RCM images were collected from the same site as the Raman data and served as inputs for two ResNet50 networks. The networks selected the hair structure and epidermis images, respectively, within all images corresponding to the positive predictions of the Raman biophysical model with high specificity. The specificity of the BCC biophysical model was improved by moving the Raman spectra corresponding to these selected images from false positive to true negative.

RESULTS

Deep-learning trained on RCM images removed 52% of false positive predictions from the Raman biophysical model result while maintaining a sensitivity of 100%. The specificity was improved from 84.2% using Raman spectra alone to 92.4% by integrating Raman spectra with RCM images.

CONCLUSIONS

Combining RS with deep-learning-aided RCM imaging is a promising tool for guiding tumor resection surgery.

Machine Learning Based Prediction of Squamous Cell Carcinoma in Ex Vivo Confocal Laser Scanning Microscopy

Image classification with convolutional neural networks (CNN) offers an unprecedented opportunity to medical imaging. Regulatory agencies in the USA and Europe have already cleared numerous deep learning/machine learning based medical devices and algorithms. While the field of radiology is on the forefront of artificial intelligence (AI) revolution, conventional pathology, which commonly relies on examination of tissue samples on a glass slide, is falling behind in leveraging this technology. On the other hand, ex vivo confocal laser scanning microscopy (ex vivo CLSM), owing to its digital workflow features, has a high potential to benefit from integrating AI tools into the assessment and decision-making process. Aim of this work was to explore a preliminary application of CNN in digitally stained ex vivo CLSM images of cutaneous squamous cell carcinoma (cSCC) for automated detection of tumor tissue. Thirty-four freshly excised tissue samples were prospectively collected and examined immediately after resection. After the histologically confirmed ex vivo CLSM diagnosis, the tumor tissue was annotated for segmentation by experts, in order to train the MobileNet CNN. The model was then trained and evaluated using cross validation. The overall sensitivity and specificity of the deep neural network for detecting cSCC and tumor free areas on ex vivo CLSM slides compared to expert evaluation were 0.76 and 0.91, respectively. The area under the ROC curve was equal to 0.90 and the area under the precision-recall curve was 0.85. The results demonstrate a high potential of deep learning models to detect cSCC regions on digitally stained ex vivo CLSM slides and to distinguish them from tumor-free skin.

[Artificial intelligence in ex vivo confocal laser scanning microscopy]

BACKGROUND

Visual data, such as clinical photographs or pictures from imaging examination methods, such as ex vivo confocal laser scanning microscopy (CLSM), are particularly suitable for machine learning techniques.

OBJECTIVES

The aim was to find out whether data have already been published on this innovative application in ex vivo CLSM and what potential challenges and limitations could arise.

MATERIAL AND METHODS

Review of the literature and summary of current knowledge and personal experience on the use of artificial intelligence (AI) in ex vivo CLSM.

RESULTS

Successful integration of digital hematoxylin-eosin-like staining has made ex vivo CLSM significantly more accessible for digital assessments. Several machine learning techniques have been developed to date in such a way that they have been able to identify malignant skin lesions on clinical photographs and pathological microscopic images with similar accuracy compared to experts, or even find visual patterns that have been overlooked by experts and that correlate with certain dermatological diseases. One study on the use of AI in ex vivo CLSM for automated tumor detection has been published to date. Several challenges and limitations can arise when using AI in ex vivo CLSM.

CONCLUSIONS

The already digitized ex vivo CLSM, which was established for rapid section examination purposes, is a predestined method for the development and use of new applications with machine learning in the healthcare sector. The results of further studies on this topic are anticipated with great hope.