Microvessel prediction in H&E Stained Pathology Images using fully convolutional neural networks

Deep Learning Analysis for Predicting Tumor Spread through Air Space in Early-Stage Lung Adenocarcinoma Pathology Images

The presence of spread through air spaces (STASs) in early-stage lung adenocarcinoma is a significant prognostic factor associated with disease recurrence and poor outcomes. Although current STAS detection methods rely on pathological examinations, the advent of artificial intelligence (AI) offers opportunities for automated histopathological image analysis. This study developed a deep learning (DL) model for STAS prediction and investigated the correlation between the prediction results and patient outcomes. To develop the DL-based STAS prediction model, 1053 digital pathology whole-slide images (WSIs) from the competition dataset were enrolled in the training set, and 227 WSIs from the National Taiwan University Hospital were enrolled for external validation. A YOLOv5-based framework comprising preprocessing, candidate detection, false-positive reduction, and patient-based prediction was proposed for STAS prediction. The model achieved an area under the curve (AUC) of 0.83 in predicting STAS presence, with 72% accuracy, 81% sensitivity, and 63% specificity. Additionally, the DL model demonstrated a prognostic value in disease-free survival compared to that of pathological evaluation. These findings suggest that DL-based STAS prediction could serve as an adjunctive screening tool and facilitate clinical decision-making in patients with early-stage lung adenocarcinoma.

Deep learning-based automated pipeline for blood vessel detection and distribution analysis in multiplexed prostate cancer images

Introduction: Prostate cancer is a highly heterogeneous disease, presenting varying levels of aggressiveness and response to treatment. Angiogenesis is one of the hallmarks of cancer, providing oxygen and nutrient supply to tumors. Micro vessel density has previously been correlated with higher Gleason score and poor prognosis. Manual segmentation of blood vessels (BVs) In microscopy images is challenging, time consuming and may be prone to inter-rater variabilities. In this study, an automated pipeline is presented for BV detection and distribution analysis in multiplexed prostate cancer images. Methods: A deep learning model was trained to segment BVs by combining CD31, CD34 and collagen IV images. In addition, the trained model was used to analyze the size and distribution patterns of BVs in relation to disease progression in a cohort of prostate cancer patients (N = 215). Results: The model was capable of accurately detecting and segmenting BVs, as compared to ground truth annotations provided by two reviewers. The precision (P), recall (R) and dice similarity coefficient (DSC) were equal to 0.93 (SD 0.04), 0.97 (SD 0.02) and 0.71 (SD 0.07) with respect to reviewer 1, and 0.95 (SD 0.05), 0.94 (SD 0.07) and 0.70 (SD 0.08) with respect to reviewer 2, respectively. BV count was significantly associated with 5-year recurrence (adjusted p = 0.0042), while both count and area of blood vessel were significantly associated with Gleason grade (adjusted p = 0.032 and 0.003 respectively). Discussion: The proposed methodology is anticipated to streamline and standardize BV analysis, offering additional insights into the biology of prostate cancer, with broad applicability to other cancers.

Histopathology image classification: highlighting the gap between manual analysis and AI automation

The field of histopathological image analysis has evolved significantly with the advent of digital pathology, leading to the development of automated models capable of classifying tissues and structures within diverse pathological images. Artificial intelligence algorithms, such as convolutional neural networks, have shown remarkable capabilities in pathology image analysis tasks, including tumor identification, metastasis detection, and patient prognosis assessment. However, traditional manual analysis methods have generally shown low accuracy in diagnosing colorectal cancer using histopathological images. This study investigates the use of AI in image classification and image analytics using histopathological images using the histogram of oriented gradients method. The study develops an AI-based architecture for image classification using histopathological images, aiming to achieve high performance with less complexity through specific parameters and layers. In this study, we investigate the complicated state of histopathological image classification, explicitly focusing on categorizing nine distinct tissue types. Our research used open-source multi-centered image datasets that included records of 100.000 non-overlapping images from 86 patients for training and 7180 non-overlapping images from 50 patients for testing. The study compares two distinct approaches, training artificial intelligence-based algorithms and manual machine learning models, to automate tissue classification. This research comprises two primary classification tasks: binary classification, distinguishing between normal and tumor tissues, and multi-classification, encompassing nine tissue types, including adipose, background, debris, stroma, lymphocytes, mucus, smooth muscle, normal colon mucosa, and tumor. Our findings show that artificial intelligence-based systems can achieve 0.91 and 0.97 accuracy in binary and multi-class classifications. In comparison, the histogram of directed gradient features and the Random Forest classifier achieved accuracy rates of 0.75 and 0.44 in binary and multi-class classifications, respectively. Our artificial intelligence-based methods are generalizable, allowing them to be integrated into histopathology diagnostics procedures and improve diagnostic accuracy and efficiency. The CNN model outperforms existing machine learning techniques, demonstrating its potential to improve the precision and effectiveness of histopathology image analysis. This research emphasizes the importance of maintaining data consistency and applying normalization methods during the data preparation stage for analysis. It particularly highlights the potential of artificial intelligence to assess histopathological images.

Deep Learning for Predicting Effect of Neoadjuvant Therapies in Non-Small Cell Lung Carcinomas With Histologic Images

Neoadjuvant therapies are used for locally advanced non-small cell lung carcinomas, whereby pathologists histologically evaluate the effect using resected specimens. Major pathological response (MPR) has recently been used for treatment evaluation and as an economical survival surrogate; however, interobserver variability and poor reproducibility are often noted. The aim of this study was to develop a deep learning (DL) model to predict MPR from hematoxylin and eosin-stained tissue images and to validate its utility for clinical use. We collected data on 125 primary non-small cell lung carcinoma cases that were resected after neoadjuvant therapy. The cases were randomly divided into 55 for training/validation and 70 for testing. A total of 261 hematoxylin and eosin-stained slides were obtained from the maximum tumor beds, and whole slide images were prepared. We used a multiscale patch model that can adaptively weight multiple convolutional neural networks trained with different field-of-view images. We performed 3-fold cross-validation to evaluate the model. During testing, we compared the percentages of viable tumor evaluated by annotator pathologists (reviewed data), those evaluated by nonannotator pathologists (primary data), and those predicted by the DL-based model using 2-class confusion matrices and receiver operating characteristic curves and performed a survival analysis between MPR-achieved and non-MPR cases. In cross-validation, accuracy and mean F1 score were 0.859 and 0.805, respectively. During testing, accuracy and mean F1 score with reviewed data and those with primary data were 0.986, 0.985, 0.943, and 0.943, respectively. The areas under the receiver operating characteristic curve with reviewed and primary data were 0.999 and 0.978, respectively. The disease-free survival of MPR-achieved cases with reviewed and primary data was significantly better than that of the non-MPR cases (P<.001 and P=.001), and that predicted by the DL-based model was almost identical (P=.005). The DL model may support pathologist evaluations and can offer accurate determinations of MPR in patients.

Artificial Intelligence Assists in the Detection of Blood Vessels in Whole Slide Images: Practical Benefits for Oncological Pathology

The analysis of the microvasculature and the assessment of angiogenesis have significant prognostic value in various diseases, including cancer. The search for invasion into the blood and lymphatic vessels and the assessment of angiogenesis are important aspects of oncological diagnosis. These features determine the prognosis and aggressiveness of the tumor. Traditional manual evaluation methods are time consuming and subject to inter-observer variability. Blood vessel detection is a perfect task for artificial intelligence, which is capable of rapid analyzing thousands of tissue structures in whole slide images. The development of computer vision solutions requires the segmentation of tissue regions, the extraction of features and the training of machine learning models. In this review, we focus on the methodologies employed by researchers to identify blood vessels and vascular invasion across a range of tumor localizations, including breast, lung, colon, brain, renal, pancreatic, gastric and oral cavity cancers. Contemporary models herald a new era of computational pathology in morphological diagnostics.

A real-time computer-aided diagnosis method for hydatidiform mole recognition using deep neural network

BACKGROUND AND OBJECTIVE

Hydatidiform mole (HM) is one of the most common gestational trophoblastic diseases with malignant potential. Histopathological examination is the primary method for diagnosing HM. However, due to the obscure and confusing pathology features of HM, significant observer variability exists among pathologists, leading to over- and misdiagnosis in clinical practice. Efficient feature extraction can significantly improve the accuracy and speed of the diagnostic process. Deep neural network (DNN) has been proven to have excellent feature extraction and segmentation capabilities, which is widely used in clinical practice for many other diseases. We constructed a deep learning-based CAD method to recognize HM hydrops lesions under the microscopic view in real-time.

METHODS

To solve the challenge of lesion segmentation due to difficulties in extracting effective features from HM slide images, we proposed a hydrops lesion recognition module that employs DeepLabv3+ with our novel compound loss function and a stepwise training strategy to achieve great performance in recognizing hydrops lesions at both pixel and lesion level. Meanwhile, a Fourier transform-based image mosaic module and an edge extension module for image sequences were developed to make the recognition model more applicable to the case of moving slides in clinical practice. Such an approach also addresses the situation where the model has poor results for image edge recognition.

RESULTS

We evaluated our method using widely adopted DNNs on an HM dataset and chose DeepLabv3+ with our compound loss function as the segmentation model. The comparison experiments show that the edge extension module is able to improve the model performance by at most 3.4% regarding pixel-level IoU and 9.0% regarding lesion-level IoU. As for the final result, our method is able to achieve a pixel-level IoU of 77.0%, a precision of 86.0%, and a lesion-level recall of 86.2% while having a response time of 82 ms per frame. Experiments show that our method is able to display the full microscopic view with accurately labeled HM hydrops lesions following the movement of slides in real-time.

CONCLUSIONS

To the best of our knowledge, this is the first method to utilize deep neural networks in HM lesion recognition. This method provides a robust and accurate solution with powerful feature extraction and segmentation capabilities for auxiliary diagnosis of HM.

Proof of principle study: synchrotron X-ray fluorescence microscopy for identification of previously radioactive microparticles and elemental mapping of FFPE tissues

Biobanks containing formalin-fixed, paraffin-embedded (FFPE) tissues from animals and human atomic-bomb survivors exposed to radioactive particulates remain a vital resource for understanding the molecular effects of radiation exposure. These samples are often decades old and prepared using harsh fixation processes which limit sample imaging options. Optical imaging of hematoxylin and eosin (H&E) stained tissues may be the only feasible processing option, however, H&E images provide no information about radioactive microparticles or radioactive history. Synchrotron X-ray fluorescence microscopy (XFM) is a robust, non-destructive, semi-quantitative technique for elemental mapping and identifying candidate chemical element biomarkers in FFPE tissues. Still, XFM has never been used to uncover distribution of formerly radioactive micro-particulates in FFPE canine specimens collected more than 30 years ago. In this work, we demonstrate the first use of low-, medium-, and high-resolution XFM to generate 2D elemental maps of ~ 35-year-old, canine FFPE lung and lymph node specimens stored in the Northwestern University Radiobiology Archive documenting distribution of formerly radioactive micro-particulates. Additionally, we use XFM to identify individual microparticles and detect daughter products of radioactive decay. The results of this proof-of-principle study support the use of XFM to map chemical element composition in historic FFPE specimens and conduct radioactive micro-particulate forensics.

Low-grade mammary gland tumours in dogs have greater VEGF-A and BMP2 immunostaining and higher CD31 blood vessel density

Tumor angiogenesis is an important process in tumor growth, and different molecules are involved in its regulation including VEGF-A, BMP2, and CD31, which can be considered possible prognostic markers. The aim of this study was to verify whether the VEGF-A and BMP2 immunostaining area, and microvascular density (MVD) might be associated with the degree of malignancy in malignant mammary neoplasms of dogs. For this purpose, samples of mammary malignancies from female dogs embedded in wax were used and separated into four main histomorphological types: tubulopapillary carcinomas, solid, complex, and carcinosarcoma, which were separated based on high and low degrees of malignancy. Immunohistochemical analysis was performed on tissue microarray blocks using anti-CD31 antibodies for evaluation of MVD and vascular lumen area, and with anti-VEGF-A and anti-BMP2 to determine the immunostaining area using the DAKO EnVision™ FLEX+ kit. MVD and vascular lumen area were higher in tubulopapillary carcinomas as were the areas stained by VEGF-A and BMP2. Immunostaining for CD31 was higher in low-grade carcinomas as well as in areas immunostained by VEGF-A and BMP2. There was a positive correlation between VEGF and BMP2 in high (r = 0.556, p < 0.0001) and low-grade (r = 0.287, p<0.0001) carcinomas and between MVD and VEGF-A in low-grade carcinomas (r = 0.267, p = 0.0064). Thus, the markers evaluated showed greater immunostaining in canine mammary tumors with a lower degree of malignancy.

Artificial intelligence in lung cancer diagnosis and prognosis: Current application and future perspective

Lung cancer is one of the malignant tumors with the highest incidence and mortality in the world. The overall five-year survival rate of lung cancer is relatively lower than many leading cancers. Early diagnosis and prognosis of lung cancer are essential to improve the patient's survival rate. With artificial intelligence (AI) approaches widely applied in lung cancer, early diagnosis and prediction have achieved excellent performance in recent years. This review summarizes various types of AI algorithm applications in lung cancer, including natural language processing (NLP), machine learning and deep learning, and reinforcement learning. In addition, we provides evidence regarding the application of AI in lung cancer diagnostic and clinical prognosis. This review aims to elucidate the value of AI in lung cancer diagnosis and prognosis as the novel screening decision-making for the precise treatment of lung cancer patients.

Application of digital pathology and machine learning in the liver, kidney and lung diseases

The development of rapid and accurate Whole Slide Imaging (WSI) has paved the way for the application of Artificial Intelligence (AI) to digital pathology. The availability of WSI in the recent years allowed the rapid development of various AI technologies to blossom. WSI-based digital pathology combined with neural networks can automate arduous and time-consuming tasks of slide evaluation. Machine Learning (ML)-based AI has been demonstrated to outperform pathologists by eliminating inter- and intra-observer subjectivity, obtaining quantitative data from slide images, and extracting hidden image patterns that are relevant to disease subtype and progression. In this review, we outline the functionality of different AI technologies such as neural networks and deep learning and discover how aspects of different diseases make them benefit from the implementation of AI. AI has proven to be valuable in many different organs, with this review focusing on the liver, kidney, and lungs. We also discuss how AI and image analysis not only can grade diseases objectively but also discover aspects of diseases that have prognostic value. In the end, we review the current status of the integration of AI in pathology and share our vision on the future of digital pathology.

Further predictive value of lymphovascular invasion explored via supervised deep learning for lymph node metastases in breast cancer

Lymphovascular invasion, specifically lymph-blood vessel invasion (LBVI), is a risk factor for metastases in breast invasive ductal carcinoma (IDC) and is routinely screened using hematoxylin-eosin histopathological images. However, routine reports only describe whether LBVI is present and does not provide other potential prognostic information of LBVI. This study aims to evaluate the clinical significance of LBVI in 685 IDC cases and explore the added predictive value of LBVI on lymph node metastases (LNM) via supervised deep learning (DL), an expert-experience embedded knowledge transfer learning (EEKT) model in 40 LBVI-positive cases signed by the routine report. Multivariate logistic regression and propensity score matching analysis demonstrated that LBVI (OR 4.203, 95% CI 2.809-6.290, P < 0.001) was a significant risk factor for LNM. Then, the EEKT model trained on 5780 image patches automatically segmented LBVI with a patch-wise Dice similarity coefficient of 0.930 in the test set and output counts, location, and morphometric features of the LBVIs. Some morphometric features were beneficial for further stratification within the 40 LBVI-positive cases. The results showed that LBVI in cases with LNM had a higher short-to-long side ratio of the minimum rectangle (MR) (0.686 vs. 0.480, P = 0.001), LBVI-to-MR area ratio (0.774 vs. 0.702, P = 0.002), and solidity (0.983 vs. 0.934, P = 0.029) compared to LBVI in cases without LNM. The results highlight the potential of DL to assist pathologists in quantifying LBVI and, more importantly, in exploring added prognostic information from LBVI.

The relationship between radiomics and pathomics in Glioblastoma patients: Preliminary results from a cross-scale association study

Glioblastoma multiforme (GBM) typically exhibits substantial intratumoral heterogeneity at both microscopic and radiological resolution scales. Diffusion Weighted Imaging (DWI) and dynamic contrast-enhanced (DCE) magnetic resonance imaging (MRI) are two functional MRI techniques that are commonly employed in clinic for the assessment of GBM tumor characteristics. This work presents initial results aiming at determining if radiomics features extracted from preoperative ADC maps and post-contrast T1 (T1C) images are associated with pathomic features arising from H&E digitized pathology images. 48 patients from the public available CPTAC-GBM database, for which both radiology and pathology images were available, were involved in the study. 91 radiomics features were extracted from ADC maps and post-contrast T1 images using PyRadiomics. 65 pathomic features were extracted from cell detection measurements from H&E images. Moreover, 91 features were extracted from cell density maps of H&E images at four different resolutions. Radiopathomic associations were evaluated by means of Spearman's correlation (ρ) and factor analysis. p values were adjusted for multiple correlations by using a false discovery rate adjustment. Significant cross-scale associations were identified between pathomics and ADC, both considering features (n = 186, 0.45 < ρ < 0.74 in absolute value) and factors (n = 5, 0.48 < ρ < 0.54 in absolute value). Significant but fewer ρ values were found concerning the association between pathomics and radiomics features (n = 53, 0.5 < ρ < 0.65 in absolute value) and factors (n = 2, ρ = 0.63 and ρ = 0.53 in absolute value). The results of this study suggest that cross-scale associations may exist between digital pathology and ADC and T1C imaging. This can be useful not only to improve the knowledge concerning GBM intratumoral heterogeneity, but also to strengthen the role of radiomics approach and its validation in clinical practice as "virtual biopsy", introducing new insights for omics integration toward a personalized medicine approach.

Microvessel quantification by fully convolutional neural networks associated with type 2 inflammation in chronic rhinosinusitis

BACKGROUND

The pathogenesis of chronic rhinosinusitis (CRS) is still unclear, and little is known about angiogenesis in this disease. We utilized a fully convolutional network (FCN), which has been extensively used in image processing to study angiogenesis in CRS.

OBJECTIVE

To explore the tissue quantification of microvessels and their potential association with inflammation in CRS by using FCN to reflect the angiogenesis condition in CRS.

METHODS

For endotyping of CRS, tissue homogenates of 79 patients with CRS who had undergone functional endoscopic sinus surgery and 17 control subjects were analyzed for interferon gamma, transforming growth factor beta, interleukin (IL)-1β, IL-5, IL-6, IL-8, IL-10, IL-17, tumor necrosis factor alpha, eosinophilic cationic protein, immunoglobulin E, and Staphylococcus aureus-immunoglobulin E(SE-IgE). A total of 552 hematoxylin and eosin-stained images of 27 CRS tissue samples were used to develop an FCN, going through training, validation, and evaluation processes. An optimized FCN was applied to quantify the microvessels of tissue samples of all subjects. Correlation analysis between microvessel quantification with phenotype, endotype, clinical characteristics, and cytokine expression of CRS was carried out.

RESULTS

Quantification of microvessels in type 2 and non-type 2 CRS demonstrated considerable differences, with a higher expression in type 2 CRS. There was a strong negative correlation between the area ratio of microvessels with tissue tumor necrosis factor alpha and transforming growth factor beta levels and a mildly positive correlation with tissue IL-5 and eosinophilic cationic protein concentration.

CONCLUSION

FCN proved to facilitate the analysis of microvessels in airway tissue samples. This study elucidated the close association of angiogenesis with endotyping, suggesting that treatment aiming at antagonizing angiogenesis may assist to the therapy for the recrudescent and refractory CRS.

Advanced platelet-rich fibrin (A-PRF) has an impact on the initial healing of gingival regeneration after tooth extraction

OBJECTIVES

Platelet-rich fibrin (PRF) is widely used in wound healing because it contains several growth factors, including vascular endothelial growth factor (VEGF). In this study, we investigated the effects of advanced PRF (A-PRF) in early-stage gingival regeneration after tooth extraction.

METHODS

Blood sample was collected from females beagle dogs (age: 12 months) before tooth extraction for A-PRF preparation. All animals were sacrificed by perfusion-fixation on postoperative days 1, 3, and 7. The upper jaws were prepared for hematoxylin and eosin staining and immunostaining (for CD34 and VEGF). The lower jaw samples were prepared for scanning electron microscope observations. Blood flow in the gingiva before and after surgery was measured using laser Doppler flowmetry.

RESULTS

In the A-PRF group, a large number of microvessels were observed in the gingival tissue on postoperative day 1. The microvessels in the control group were fewer and sparse. Regarding the vascular resin cast, a large number of new blood vessels were observed on postoperative day 1 in the A-PRF group. A stronger CD34-positive signal was obtained around the blood vessels in the A-PRF group than in the control group. Further, a strong VEGF-positive signal was observed in the perivascular tissue in the A-PRF group. Gingival blood flow was significantly higher in the A-PRF group after surgery.

CONCLUSION

A-PRF had a positive impact on angiogenesis in the gingiva through the induction of VEGF expression. Thus, A-PRF may be beneficial for gingival tissue regeneration.

Optical clearing in cardiac imaging: A comparative study

The optical clearing of the cardiac tissue has always been a challenging goal to obtain successful three-dimensional reconstructions of entire hearts. Typically, the developed protocols are targeted at the clearing of the brain; cardiac tissue requires proper arrangements to the original protocols, which are usually tough and time-consuming to figure out. Here, we present the application of three different clearing methodologies on mouse hearts: uDISCO, CLARITY, and SHIELD. For each approach, we describe the required optimizations that we have developed to improve the outcome; in particular, we focus on comparing the features of the tissue after the application of each methodology, especially in terms of tissue preservation, transparency, and staining. We found that the uDISCO protocol induces strong fiber delamination of the cardiac tissue, thus reducing the reliability of structural analyses. The CLARITY protocol confers a high level of transparency to the heart and allows deep penetration of the fluorescent dyes; however, it requires long times for the clearing and the tissue loses its robustness. The SHIELD methodology, indeed, is very promising for tissue maintenance since it preserves its consistency and provides ideal transparency, but further approaches are needed to obtain homogeneous staining of the whole heart. Since the CLARITY procedure, despite the disadvantages in terms of tissue preservation and timings, is actually the most suitable approach to image labeled samples in depth, we optimized and performed the methodology also on human cardiac tissue from control hearts and hearts with hypertrophic cardiomyopathy.

DeepRePath: Identifying the Prognostic Features of Early-Stage Lung Adenocarcinoma Using Multi-Scale Pathology Images and Deep Convolutional Neural Networks

Simple Summary

Pathology images are vital for understanding solid cancers. In this study, we created DeepRePath using multi-scale pathology images with two-channel deep learning to predict the prognosis of patients with early-stage lung adenocarcinoma (LUAD). DeepRePath demonstrated that it could predict the recurrence of early-stage LUAD with average area under the curve scores of 0.77 and 0.76 in cohort I and cohort II (external validation set), respectively. Pathological features found to be associated with a high probability of recurrence included tumor necrosis, discohesive tumor cells, and atypical nuclei. In conclusion, DeepRePath can improve the treatment modality for patients with early-stage LUAD through recurrence prediction.

Abstract

The prognosis of patients with lung adenocarcinoma (LUAD), especially early-stage LUAD, is dependent on clinicopathological features. However, its predictive utility is limited. In this study, we developed and trained a DeepRePath model based on a deep convolutional neural network (CNN) using multi-scale pathology images to predict the prognosis of patients with early-stage LUAD. DeepRePath was pre-trained with 1067 hematoxylin and eosin-stained whole-slide images of LUAD from the Cancer Genome Atlas. DeepRePath was further trained and validated using two separate CNNs and multi-scale pathology images of 393 resected lung cancer specimens from patients with stage I and II LUAD. Of the 393 patients, 95 patients developed recurrence after surgical resection. The DeepRePath model showed average area under the curve (AUC) scores of 0.77 and 0.76 in cohort I and cohort II (external validation set), respectively. Owing to low performance, DeepRePath cannot be used as an automated tool in a clinical setting. When gradient-weighted class activation mapping was used, DeepRePath indicated the association between atypical nuclei, discohesive tumor cells, and tumor necrosis in pathology images showing recurrence. Despite the limitations associated with a relatively small number of patients, the DeepRePath model based on CNNs with transfer learning could predict recurrence after the curative resection of early-stage LUAD using multi-scale pathology images.

Medical image analysis based on deep learning approach

Medical imaging plays a significant role in different clinical applications such as medical procedures used for early detection, monitoring, diagnosis, and treatment evaluation of various medical conditions. Basicsof the principles and implementations of artificial neural networks and deep learning are essential for understanding medical image analysis in computer vision. Deep Learning Approach (DLA) in medical image analysis emerges as a fast-growing research field. DLA has been widely used in medical imaging to detect the presence or absence of the disease. This paper presents the development of artificial neural networks, comprehensive analysis of DLA, which delivers promising medical imaging applications. Most of the DLA implementations concentrate on the X-ray images, computerized tomography, mammography images, and digital histopathology images. It provides a systematic review of the articles for classification, detection, and segmentation of medical images based on DLA. This review guides the researchers to think of appropriate changes in medical image analysis based on DLA.

A narrative review of digital pathology and artificial intelligence: focusing on lung cancer

The emergence of whole slide imaging technology allows for pathology diagnosis on a computer screen. The applications of digital pathology are expanding, from supporting remote institutes suffering from a shortage of pathologists to routine use in daily diagnosis including that of lung cancer. Through practice and research large archival databases of digital pathology images have been developed that will facilitate the development of artificial intelligence (AI) methods for image analysis. Currently, several AI applications have been reported in the field of lung cancer; these include the segmentation of carcinoma foci, detection of lymph node metastasis, counting of tumor cells, and prediction of gene mutations. Although the integration of AI algorithms into clinical practice remains a significant challenge, we have implemented tumor cell count for genetic analysis, a helpful application for routine use. Our experience suggests that pathologists often overestimate the contents of tumor cells, and the use of AI-based analysis increases the accuracy and makes the tasks less tedious. However, there are several difficulties encountered in the practical use of AI in clinical diagnosis. These include the lack of sufficient annotated data for the development and validation of AI systems, the explainability of black box AI models, such as those based on deep learning that offer the most promising performance, and the difficulty in defining the ground truth data for training and validation owing to inherent ambiguity in most applications. All of these together present significant challenges in the development and clinical translation of AI methods in the practice of pathology. Additional research on these problems will help in resolving the barriers to the clinical use of AI. Helping pathologists in developing knowledge of the working and limitations of AI will benefit the use of AI in both diagnostics and research.

Histopathological distinction of non-invasive and invasive bladder cancers using machine learning approaches

Background

One of the most challenging tasks for bladder cancer diagnosis is to histologically differentiate two early stages, non-invasive Ta and superficially invasive T1, the latter of which is associated with a significantly higher risk of disease progression. Indeed, in a considerable number of cases, Ta and T1 tumors look very similar under microscope, making the distinction very difficult even for experienced pathologists. Thus, there is an urgent need for a favoring system based on machine learning (ML) to distinguish between the two stages of bladder cancer.

Methods

A total of 1177 images of bladder tumor tissues stained by hematoxylin and eosin were collected by pathologists at University of Rochester Medical Center, which included 460 non-invasive (stage Ta) and 717 invasive (stage T1) tumors. Automatic pipelines were developed to extract features for three invasive patterns characteristic to the T1 stage bladder cancer (i.e., desmoplastic reaction, retraction artifact, and abundant pinker cytoplasm), using imaging processing software ImageJ and CellProfiler. Features extracted from the images were analyzed by a suite of machine learning approaches.

Results

We extracted nearly 700 features from the Ta and T1 tumor images. Unsupervised clustering analysis failed to distinguish hematoxylin and eosin images of Ta vs. T1 tumors. With a reduced set of features, we successfully distinguished 1177 Ta or T1 images with an accuracy of 91–96% by six supervised learning methods. By contrast, convolutional neural network (CNN) models that automatically extract features from images produced an accuracy of 84%, indicating that feature extraction driven by domain knowledge outperforms CNN-based automatic feature extraction. Further analysis revealed that desmoplastic reaction was more important than the other two patterns, and the number and size of nuclei of tumor cells were the most predictive features.

Conclusions

We provide a ML-empowered, feature-centered, and interpretable diagnostic system to facilitate the accurate staging of Ta and T1 diseases, which has a potential to apply to other types of cancer.

Deep Learning Model as a New Trend in Computer-aided Diagnosis of Tumor Pathology for Lung Cancer

Lung cancer is one of the main causes of cancer-related death in the world. The identification and characteristics of malignant cells are essential for the diagnosis and treatment of primary or metastatic cancers. Deep learning is a new field of artificial intelligence, which can be used for computer aided diagnosis and scientific research of lung cancer pathology by analyzing and learning through establishment and simulation of human brain. In this review, we will introduce the application, progress and problems of deep learning in pathology of lung cancer and make prospects for its future development.

ConvPath: A software tool for lung adenocarcinoma digital pathological image analysis aided by a convolutional neural network

BACKGROUND

The spatial distributions of different types of cells could reveal a cancer cell's growth pattern, its relationships with the tumor microenvironment and the immune response of the body, all of which represent key "hallmarks of cancer". However, the process by which pathologists manually recognize and localize all the cells in pathology slides is extremely labor intensive and error prone.

METHODS

In this study, we developed an automated cell type classification pipeline, ConvPath, which includes nuclei segmentation, convolutional neural network-based tumor cell, stromal cell, and lymphocyte classification, and extraction of tumor microenvironment-related features for lung cancer pathology images. To facilitate users in leveraging this pipeline for their research, all source scripts for ConvPath software are available at https://qbrc.swmed.edu/projects/cnn/.

FINDINGS

The overall classification accuracy was 92.9% and 90.1% in training and independent testing datasets, respectively. By identifying cells and classifying cell types, this pipeline can convert a pathology image into a "spatial map" of tumor, stromal and lymphocyte cells. From this spatial map, we can extract features that characterize the tumor micro-environment. Based on these features, we developed an image feature-based prognostic model and validated the model in two independent cohorts. The predicted risk group serves as an independent prognostic factor, after adjusting for clinical variables that include age, gender, smoking status, and stage.

INTERPRETATION

The analysis pipeline developed in this study could convert the pathology image into a "spatial map" of tumor cells, stromal cells and lymphocytes. This could greatly facilitate and empower comprehensive analysis of the spatial organization of cells, as well as their roles in tumor progression and metastasis.

An automated method for segmentation and quantification of blood vessels in histology images

Alzheimer's disease (AD) is a chronic neuro-degenerative disease that adversely affect many people on a global scale. Despite different diagnostic and therapeutic treatment, there is no cure for AD. The brain is one of the most complex organ and researchers are still trying to understand so as to find a cure.

OBJECTIVE

To complement the efforts of clinical researchers engaged in research in alzheimer's disease, accurate segmentation and quantification of blood vessels in brain images is required.

METHOD

For robust segmentation of blood vessels even in the presence of colour variation, we introduce a fully automated morphological tool that can extract and quantify vessels from haematoxylin and diaminobenzidine stained histology brain image. The method, exploits saturation channel of stained image slides, ISODATA threshold method is applied to obtain a binary image. This helps in eliminating background and remaining with only blood vessels. A one-stage procedure that includes eliminating small artefacts is performed on the binary mask. The intensity of the image is transformed. Joining is performed to deal with fragmentation of intact blood vessels on the images, and artefactual appearance of the blood vessel structures. The artefactual fragments based on measured incoherence with neighbouring tissue are removed. The vessels are then labelled to facilitate quantification. Morphometric measurements are used during the vessel quantification assess both vessels with lumen and vessels without lumen. We have quantified the diameter of blood vessels.

RESULTS

The image processing technique is developed in close collaboration with neuroscientist experts to help clinician. We have evaluated our proposed approach qualitatively. The method was validated against their manual quantification results. Qualitative results show that the method can indeed segment the blood vessels in the presence of colour variations and artefacts. The quantitative method produces fairly better results.

Artificial Intelligence in Lung Cancer Pathology Image Analysis

OBJECTIVE

Accurate diagnosis and prognosis are essential in lung cancer treatment selection and planning. With the rapid advance of medical imaging technology, whole slide imaging (WSI) in pathology is becoming a routine clinical procedure. An interplay of needs and challenges exists for computer-aided diagnosis based on accurate and efficient analysis of pathology images. Recently, artificial intelligence, especially deep learning, has shown great potential in pathology image analysis tasks such as tumor region identification, prognosis prediction, tumor microenvironment characterization, and metastasis detection.

MATERIALS AND METHODS

In this review, we aim to provide an overview of current and potential applications for AI methods in pathology image analysis, with an emphasis on lung cancer.

RESULTS

We outlined the current challenges and opportunities in lung cancer pathology image analysis, discussed the recent deep learning developments that could potentially impact digital pathology in lung cancer, and summarized the existing applications of deep learning algorithms in lung cancer diagnosis and prognosis.

DISCUSSION AND CONCLUSION

With the advance of technology, digital pathology could have great potential impacts in lung cancer patient care. We point out some promising future directions for lung cancer pathology image analysis, including multi-task learning, transfer learning, and model interpretation.

Pathology Image Analysis Using Segmentation Deep Learning Algorithms

With the rapid development of image scanning techniques and visualization software, whole slide imaging (WSI) is becoming a routine diagnostic method. Accelerating clinical diagnosis from pathology images and automating image analysis efficiently and accurately remain significant challenges. Recently, deep learning algorithms have shown great promise in pathology image analysis, such as in tumor region identification, metastasis detection, and patient prognosis. Many machine learning algorithms, including convolutional neural networks, have been proposed to automatically segment pathology images. Among these algorithms, segmentation deep learning algorithms such as fully convolutional networks stand out for their accuracy, computational efficiency, and generalizability. Thus, deep learning-based pathology image segmentation has become an important tool in WSI analysis. In this review, the pathology image segmentation process using deep learning algorithms is described in detail. The goals are to provide quick guidance for implementing deep learning into pathology image analysis and to provide some potential ways of further improving segmentation performance. Although there have been previous reviews on using machine learning methods in digital pathology image analysis, this is the first in-depth review of the applications of deep learning algorithms for segmentation in WSI analysis.