Confocal laser microscopy as novel approach for real-time and in-vivo tissue examination during minimal-invasive surgery in colon cancer

Deep Learning-Enabled Technologies for Bioimage Analysis

Deep learning (DL) is a subfield of machine learning (ML), which has recently demonstrated its potency to significantly improve the quantification and classification workflows in biomedical and clinical applications. Among the end applications profoundly benefitting from DL, cellular morphology quantification is one of the pioneers. Here, we first briefly explain fundamental concepts in DL and then we review some of the emerging DL-enabled applications in cell morphology quantification in the fields of embryology, point-of-care ovulation testing, as a predictive tool for fetal heart pregnancy, cancer diagnostics via classification of cancer histology images, autosomal polycystic kidney disease, and chronic kidney diseases.

Endoscopic confocal laser-microscopy for the intraoperative nerve recognition: is it feasible?

Surgeons lose most of their tactile tissue information during minimal invasive surgery and need an additional tool of intraoperative tissue recognition. Confocal laser microscopy (CLM) is a well-established method of tissue investigation. The objective of this study was to analyze the feasibility and diagnostic accuracy of CLM nervous tissue recognition. Images taken with an endoscopic CLM system of sympathetic ganglions, nerve fibers and pleural tissue were characterized in terms of specific signal-patterns ex-vivo. No fluorescent dye was used. Diagnostic accuracy of tissue classification was evaluated by newly trained observers (sensitivity, specificity, PPV, NPV and interobserver variability). Although CLM images showed low CLM image contrast, assessment of nerve tissue was feasible without any fluorescent dye. Sensitivity and specificity ranged between 0.73 and 0.9 and 0.55-1.0, respectively. PPVs were 0.71-1.0 and the NPV range was between 0.58 and 0.86. The overall interobserver variability was 0.36. The eCLM enables to evaluate nervous tissue and to distinguish between nerve fibers, ganglions and pleural tissue based on backscattered light. However, the low image contrast and the heterogeneity in correct tissue diagnosis and a fair interobserver variability indicate the limit of CLM imaging without any fluorescent dye.

Confocal Laser Microscopy for in vivo Intraoperative Application: Diagnostic Accuracy of Investigator and Machine Learning Strategies

BACKGROUND

Confocal laser microscopy (CLM) is one of the optical techniques that are promising methods of intraoperative in vivo real-time tissue examination based on tissue fluorescence. However, surgeons might struggle interpreting CLM images intraoperatively due to different tissue characteristics of different tissue pathologies in clinical reality. Deep learning techniques enable fast and consistent image analysis and might support intraoperative image interpretation. The objective of this study was to analyze the diagnostic accuracy of newly trained observers in the evaluation of normal colon and peritoneal tissue and colon cancer and metastasis, respectively, and to compare it with that of convolutional neural networks (CNNs).

METHODS

Two hundred representative CLM images of the normal and malignant colon and peritoneal tissue were evaluated by newly trained observers (surgeons and pathologists) and CNNs (VGG-16 and Densenet121), respectively, based on tissue dignity. The primary endpoint was the correct detection of the normal and cancer/metastasis tissue measured by sensitivity and specificity of both groups. Additionally, positive predictive values (PPVs) and negative predictive values (NPVs) were calculated for the newly trained observer group. The interobserver variability of dignity evaluation was calculated using kappa statistic. The F1-score and area under the curve (AUC) were used to evaluate the performance of image recognition of the CNNs' training scenarios.

RESULTS

Sensitivity and specificity ranged between 0.55 and 1.0 (pathologists: 0.66-0.97; surgeons: 0.55-1.0) and between 0.65 and 0.96 (pathologists: 0.68-0.93; surgeons: 0.65-0.96), respectively. PPVs were 0.75 and 0.90 in the pathologists' group and 0.73-0.96 in the surgeons' group, respectively. NPVs were 0.73 and 0.96 for pathologists' and between 0.66 and 1.00 for surgeons' tissue analysis. The overall interobserver variability was 0.54. Depending on the training scenario, cancer/metastasis tissue was classified with an AUC of 0.77-0.88 by VGG-16 and 0.85-0.89 by Densenet121. Transfer learning improved performance over training from scratch.

CONCLUSIONS

Newly trained investigators are able to learn CLM images features and interpretation rapidly, regardless of their clinical experience. Heterogeneity in tissue diagnosis and a moderate interobserver variability reflect the clinical reality more realistic. CNNs provide comparable diagnostic results as clinical observers and could improve surgeons' intraoperative tissue assessment.

Confocal laser microscopy without fluorescent dye in minimal-invasive thoracic surgery: an ex-vivo pilot study in lung cancer

Cancer will be the leading cause of death in a few decades. In line with minimal invasive lung cancer surgery, surgeons loose most of their tactile tissue information and need an additional tool of intraoperative tissue navigation during surgery. Confocal laser microscopy is a well-established method of tissue investigation. In this ex-vivo pilot study, we evaluated an endoscopic confocal laser microscope (eCLM) that does not need any fluorescent dye as a diagnostic tool in non-malignant and malignant pulmonary tissue and distal stapler resection margins, respectively. In seven cases, an eCLM was used for examining pulmonary tissue ex-vivo. Images of non-malignant and non-small cell lung cancer tissue and distal stapler resection margins were characterized in terms of specific signal-patterns. No fluorescent dye was used. Correlations to findings in conventional histology were systematically recorded and described. Healthy lung tissue showed hyperreflectoric alveolar walls with dark alveolar spaces. Hyperreflective nets indicated the tumor stroma; whereas the hyperreflective areas indicated the tumor cell clusters. Compared to adenocarcinoma tissue, tissue from squamous cell carcinoma showed more distinctive hyperreflective stroma nets. eCLM characteristics seen in non-malignant and malignant tissue were also visible in distal stapler resection margins and so therefore it was feasible to distinguish between healthy lung tissue and lung cancer. This pilot study shows that the assessment of pulmonary tissue with this eCLM for minimally invasive surgical approach without any fluorescent dye is feasible. It enables to differentiate between benign and malignant tissue in pulmonary specimen by easy to evaluate and reproducible parameters.

Towards an Optical Biopsy during Visceral Surgical Interventions

Background

Cancer will replace cardiovascular diseases as the most frequent cause of death. Therefore, the goals of cancer treatment are prevention strategies and early detection by cancer screening and ideal stage therapy. From an oncological point of view, complete tumor resection is a significant prognostic factor. Optical coherence tomography (OCT) and confocal laser microscopy (CLM) are two techniques that have the potential to complement intraoperative frozen section analysis as in vivo and real-time optical biopsies.

Summary

In this review we present both procedures and review the progress of evaluation for intraoperative application in visceral surgery. For visceral surgery, there are promising studies evaluating OCT and CLM; however, application during routine visceral surgical interventions is still lacking.

Key Message

OCT and CLM are not competing but complementary approaches of tissue analysis to intraoperative frozen section analysis. Although intraoperative application of OCT and CLM is at an early stage, they are two promising techniques of intraoperative in vivo and real-time tissue examination. Additionally, deep learning strategies provide a significant supplement for automated tissue detection.

Deep transfer learning methods for colon cancer classification in confocal laser microscopy images

PURPOSE

The gold standard for colorectal cancer metastases detection in the peritoneum is histological evaluation of a removed tissue sample. For feedback during interventions, real-time in vivo imaging with confocal laser microscopy has been proposed for differentiation of benign and malignant tissue by manual expert evaluation. Automatic image classification could improve the surgical workflow further by providing immediate feedback.

METHODS

We analyze the feasibility of classifying tissue from confocal laser microscopy in the colon and peritoneum. For this purpose, we adopt both classical and state-of-the-art convolutional neural networks to directly learn from the images. As the available dataset is small, we investigate several transfer learning strategies including partial freezing variants and full fine-tuning. We address the distinction of different tissue types, as well as benign and malignant tissue.

RESULTS

We present a thorough analysis of transfer learning strategies for colorectal cancer with confocal laser microscopy. In the peritoneum, metastases are classified with an AUC of 97.1, and in the colon the primarius is classified with an AUC of 73.1. In general, transfer learning substantially improves performance over training from scratch. We find that the optimal transfer learning strategy differs for models and classification tasks.

CONCLUSIONS

We demonstrate that convolutional neural networks and transfer learning can be used to identify cancer tissue with confocal laser microscopy. We show that there is no generally optimal transfer learning strategy and model as well as task-specific engineering is required. Given the high performance for the peritoneum, even with a small dataset, application for intraoperative decision support could be feasible.