Deep Learning Models for Histopathological Classification of Gastric and Colonic Epithelial Tumours

Computational pathology: A survey review and the way forward

Computational Pathology (CPath) is an interdisciplinary science that augments developments of computational approaches to analyze and model medical histopathology images. The main objective for CPath is to develop infrastructure and workflows of digital diagnostics as an assistive CAD system for clinical pathology, facilitating transformational changes in the diagnosis and treatment of cancer that are mainly address by CPath tools. With evergrowing developments in deep learning and computer vision algorithms, and the ease of the data flow from digital pathology, currently CPath is witnessing a paradigm shift. Despite the sheer volume of engineering and scientific works being introduced for cancer image analysis, there is still a considerable gap of adopting and integrating these algorithms in clinical practice. This raises a significant question regarding the direction and trends that are undertaken in CPath. In this article we provide a comprehensive review of more than 800 papers to address the challenges faced in problem design all-the-way to the application and implementation viewpoints. We have catalogued each paper into a model-card by examining the key works and challenges faced to layout the current landscape in CPath. We hope this helps the community to locate relevant works and facilitate understanding of the field's future directions. In a nutshell, we oversee the CPath developments in cycle of stages which are required to be cohesively linked together to address the challenges associated with such multidisciplinary science. We overview this cycle from different perspectives of data-centric, model-centric, and application-centric problems. We finally sketch remaining challenges and provide directions for future technical developments and clinical integration of CPath. For updated information on this survey review paper and accessing to the original model cards repository, please refer to GitHub. Updated version of this draft can also be found from arXiv.

Prediction of molecular subclasses of uveal melanoma by deep learning using routine haematoxylin-eosin-stained tissue slides

AIMS

Uveal melanoma has a high propensity to metastasize. Prognosis is associated with specific driver mutations and copy number variations, and these can only be obtained after genetic testing. In this study we evaluated the efficacy of patient outcome prediction using deep learning on haematoxylin and eosin (HE)-stained primary uveal melanoma slides in comparison to molecular testing.

METHODS

In this retrospective study of patients with uveal melanoma, 113 patients from the Erasmus Medical Centre who underwent enucleation had tumour tissue analysed for molecular classification between 1993 and 2020. Routine HE-stained slides were scanned to obtain whole-slide images (WSI). After annotation of regions of interest, tiles of 1024 × 1024 pixels were extracted at a magnification of 40×. An ablation study to select the best-performing deep-learning model was carried out using three state-of-the-art deep-learning models (EfficientNet, Vision Transformer, and Swin Transformer).

RESULTS

Deep-learning models were subjected to a training cohort (n = 40), followed by a validation cohort (n = 20), and finally underwent a test cohort (n = 48). A k-fold cross-validation (k = 3) of validation and test cohorts (n = 113 of three classes: BAP1, SF3B1, EIF1AX) demonstrated Swin Transformer as the best-performing deep-learning model to predict molecular subclasses based on HE stains. The model achieved an accuracy of 0.83 ± 0.09 on the validation cohort and 0.75 ± 0.04 on the test cohort. Within the subclasses, this model correctly predicted 70% BAP1-mutated, 61% SF3B1-mutated and 80% EIF1AX-mutated UM in the test set.

CONCLUSIONS

This study showcases the potential of the deep-learning methodology for predicting molecular subclasses in a multiclass manner using HE-stained WSI. This development holds promise for advanced prognostication of UM patients without the need of molecular or immunohistochemical testing. Additionally, this study suggests there are distinct histopathological features per subclass; mainly utilizing epithelioid cellular morphology for BAP1-classification, but an unknown feature distinguishes EIF1AX and SF3B1.

Study on an Automatic Classification Method for Determining the Malignancy Grade of Glioma Pathological Sections Based on Hyperspectral Multi-Scale Spatial-Spectral Fusion Features

This study describes a novel method for grading pathological sections of gliomas. Our own integrated hyperspectral imaging system was employed to characterize 270 bands of cancerous tissue samples from microarray slides of gliomas. These samples were then classified according to the guidelines developed by the World Health Organization, which define the subtypes and grades of diffuse gliomas. We explored a hyperspectral feature extraction model called SMLMER-ResNet using microscopic hyperspectral images of brain gliomas of different malignancy grades. The model combines the channel attention mechanism and multi-scale image features to automatically learn the pathological organization of gliomas and obtain hierarchical feature representations, effectively removing the interference of redundant information. It also completes multi-modal, multi-scale spatial-spectral feature extraction to improve the automatic classification of glioma subtypes. The proposed classification method demonstrated high average classification accuracy (>97.3%) and a Kappa coefficient (0.954), indicating its effectiveness in improving the automatic classification of hyperspectral gliomas. The method is readily applicable in a wide range of clinical settings, offering valuable assistance in alleviating the workload of clinical pathologists. Furthermore, the study contributes to the development of more personalized and refined treatment plans, as well as subsequent follow-up and treatment adjustment, by providing physicians with insights into the underlying pathological organization of gliomas.

VENet: Variational energy network for gland segmentation of pathological images and early gastric cancer diagnosis of whole slide images

BACKGROUND AND OBJECTIVE

Gland segmentation of pathological images is an essential but challenging step for adenocarcinoma diagnosis. Although deep learning methods have recently made tremendous progress in gland segmentation, they have not given satisfactory boundary and region segmentation results of adjacent glands. These glands usually have a large difference in glandular appearance, and the statistical distribution between the training and test sets in deep learning is inconsistent. These problems make networks not generalize well in the test dataset, bringing difficulties to gland segmentation and early cancer diagnosis.

METHODS

To address these problems, we propose a Variational Energy Network named VENet with a traditional variational energy Lv loss for gland segmentation of pathological images and early gastric cancer detection in whole slide images (WSIs). It effectively integrates the variational mathematical model and the data-adaptability of deep learning methods to balance boundary and region segmentation. Furthermore, it can effectively segment and classify glands in large-size WSIs with reliable nucleus width and nucleus-to-cytoplasm ratio features.

RESULTS

The VENet was evaluated on the 2015 MICCAI Gland Segmentation challenge (GlaS) dataset, the Colorectal Adenocarcinoma Glands (CRAG) dataset, and the self-collected Nanfang Hospital dataset. Compared with state-of-the-art methods, our method achieved excellent performance for GlaS Test A (object dice 0.9562, object F1 0.9271, object Hausdorff distance 73.13), GlaS Test B (object dice 94.95, object F1 95.60, object Hausdorff distance 59.63), and CRAG (object dice 95.08, object F1 92.94, object Hausdorff distance 28.01). For the Nanfang Hospital dataset, our method achieved a kappa of 0.78, an accuracy of 0.9, a sensitivity of 0.98, and a specificity of 0.80 on the classification task of test 69 WSIs.

CONCLUSIONS

The experimental results show that the proposed model accurately predicts boundaries and outperforms state-of-the-art methods. It can be applied to the early diagnosis of gastric cancer by detecting regions of high-grade gastric intraepithelial neoplasia in WSI, which can assist pathologists in analyzing large WSI and making accurate diagnostic decisions.

Investigation of the usefulness of a bile duct biopsy and bile cytology using a hyperspectral camera and machine learning

To improve the efficiency of pathological diagnoses, the development of automatic pathological diagnostic systems using artificial intelligence (AI) is progressing; however, problems include the low interpretability of AI technology and the need for large amounts of data. We herein report the usefulness of a general-purpose method that combines a hyperspectral camera with machine learning. As a result of analyzing bile duct biopsy and bile cytology specimens, which are especially difficult to determine as benign or malignant, using multiple machine learning models, both were able to identify benign or malignant cells with an accuracy rate of more than 80% (93.3% for bile duct biopsy specimens and 83.2% for bile cytology specimens). This method has the potential to contribute to the diagnosis and treatment of bile duct cancer and is expected to be widely applied and utilized in general pathological diagnoses.

Evaluation of the value of combined detection of tumor markers CA724, carcinoembryonic antigen, CA242, and CA19-9 in gastric cancer

BACKGROUND

Gastric cancer is a global health concern that poses a significant threat to human well-being.

AIM

To detecting serum changes in carcinoembryonic antigen (CEA), carbohydrate antigens (CA) 724, CA242, and CA19-9 expression among patients with gastric cancer.

METHODS

Eighty patients diagnosed with gastric cancer between January 2020 and January 2023 were included in the observation group, while 80 patients with benign gastric diseases were included in the control group. Both groups were tested for tumor markers (CA724, CEA, CA242, and CA19-9]. Tumor marker indicators (CA724, CEA, CA242, and CA19-9) were compared between the two groups, assessing positive rates of tumor markers across various stages in the observation group. Additionally, single and combined detection of various tumor markers were examined.

RESULTS

The sensitivity, specificity, accuracy, positive predictive value, and negative predictive value observed for the combined detection of CA724, CEA, CA242, and CA19-9 were higher than those of CA724, CEA, CA242, and CA19-9 individually. Therefore, the combined detection of CA724, CEA, CA242, and CA19-9 has a high diagnostic accuracy and could reduce the occurrence of missed or misdiagnosed cases, facilitating the early diagnosis and treatment of patients.

CONCLUSION

CA724, CEA, CA242, and CA19-9 serum levels in gastric cancer patients significantly surpassed those in non-gastric cancer patients (P < 0.05). Their combined detection can improve the diagnostic accuracy for gastric cancer, warranting clinical promotion.

Artificial intelligence in digital pathology: a systematic review and meta-analysis of diagnostic test accuracy

Ensuring diagnostic performance of artificial intelligence (AI) before introduction into clinical practice is essential. Growing numbers of studies using AI for digital pathology have been reported over recent years. The aim of this work is to examine the diagnostic accuracy of AI in digital pathology images for any disease. This systematic review and meta-analysis included diagnostic accuracy studies using any type of AI applied to whole slide images (WSIs) for any disease. The reference standard was diagnosis by histopathological assessment and/or immunohistochemistry. Searches were conducted in PubMed, EMBASE and CENTRAL in June 2022. Risk of bias and concerns of applicability were assessed using the QUADAS-2 tool. Data extraction was conducted by two investigators and meta-analysis was performed using a bivariate random effects model, with additional subgroup analyses also performed. Of 2976 identified studies, 100 were included in the review and 48 in the meta-analysis. Studies were from a range of countries, including over 152,000 whole slide images (WSIs), representing many diseases. These studies reported a mean sensitivity of 96.3% (CI 94.1-97.7) and mean specificity of 93.3% (CI 90.5-95.4). There was heterogeneity in study design and 99% of studies identified for inclusion had at least one area at high or unclear risk of bias or applicability concerns. Details on selection of cases, division of model development and validation data and raw performance data were frequently ambiguous or missing. AI is reported as having high diagnostic accuracy in the reported areas but requires more rigorous evaluation of its performance.

Advancing Artificial Intelligence Integration Into the Pathology Workflow: Exploring Opportunities in Gastrointestinal Tract Biopsies

This review aims to present a comprehensive overview of the current landscape of artificial intelligence (AI) applications in the analysis of tubular gastrointestinal biopsies. These publications cover a spectrum of conditions, ranging from inflammatory ailments to malignancies. Moving beyond the conventional diagnosis based on hematoxylin and eosin-stained whole-slide images, the review explores additional implications of AI, including its involvement in interpreting immunohistochemical results, molecular subtyping, and the identification of cellular spatial biomarkers. Furthermore, the review examines how AI can contribute to enhancing the quality and control of diagnostic processes, introducing new workflow options, and addressing the limitations and caveats associated with current AI platforms in this context.

Development and external validation of a transfer learning-based system for the pathological diagnosis of colorectal cancer: a large emulated prospective study

Background

The progress in Colorectal cancer (CRC) screening and management has resulted in an unprecedented caseload for histopathological diagnosis. While artificial intelligence (AI) presents a potential solution, the predominant emphasis on slide-level aggregation performance without thorough verification of cancer in each location, impedes both explainability and transparency. Effectively addressing these challenges is crucial to ensuring the reliability and efficacy of AI in histology applications.

Method

In this study, we created an innovative AI algorithm using transfer learning from a polyp segmentation model in endoscopy. The algorithm precisely localized CRC targets within 0.25 mm² grids from whole slide imaging (WSI). We assessed the CRC detection capabilities at this fine granularity and examined the influence of AI on the diagnostic behavior of pathologists. The evaluation utilized an extensive dataset comprising 858 consecutive patient cases with 1418 WSIs obtained from an external center.

Results

Our results underscore a notable sensitivity of 90.25% and specificity of 96.60% at the grid level, accompanied by a commendable area under the curve (AUC) of 0.962. This translates to an impressive 99.39% sensitivity at the slide level, coupled with a negative likelihood ratio of <0.01, signifying the dependability of the AI system to preclude diagnostic considerations. The positive likelihood ratio of 26.54, surpassing 10 at the grid level, underscores the imperative for meticulous scrutiny of any AI-generated highlights. Consequently, all four participating pathologists demonstrated statistically significant diagnostic improvements with AI assistance.

Conclusion

Our transfer learning approach has successfully yielded an algorithm that can be validated for CRC histological localizations in whole slide imaging. The outcome advocates for the integration of the AI system into histopathological diagnosis, serving either as a diagnostic exclusion application or a computer-aided detection (CADe) tool. This integration has the potential to alleviate the workload of pathologists and ultimately benefit patients.

Curcumin inhibits the growth and invasion of gastric cancer by regulating long noncoding RNA AC022424.2

BACKGROUND

Gastric cancer, characterized by a multifactorial etiology and high heterogeneity, continues to confound researchers in terms of its pathogenesis. Curcumin, a natural anticancer agent, exhibits therapeutic promise in gastric cancer. Its effects include promoting cell apoptosis, curtailing tumor angiogenesis, and enhancing sensitivity to radiation and chemotherapy. Long noncoding RNAs (lncRNAs) have garnered significant attention as biomarkers for early screening, diagnosis, treatment, and drug response because of their remarkable specificity and sensitivity. Recent investigations have revealed an association between aberrant lncRNA expression and early diagnosis, clinical staging, metastasis, drug sensitivity, and prognosis in gastric cancer. A profound understanding of the intricate mechanisms through which lncRNAs influence gastric cancer development can provide novel insights for precision treatment and tailored management of patients with gastric cancer. This study aimed to unravel the potential of curcumin in suppressing the malignant behavior of gastric cancer cells by upregulating specific lncRNAs and modulating gastric cancer onset and progression.

AIM

To identify lncRNAs associated with curcumin treatment and investigate the role of lncRNA AC022424.2 in the effects of curcumin on gastric cancer cell apoptosis, proliferation, and invasion. Furthermore, these findings were validated in clinical samples.

METHODS

The study employed CCK-8 assays to assess the impact of curcumin on gastric cancer cell proliferation, flow cytometry to investigate its effects on apoptosis, and scratch and Transwell assays to evaluate its influence on the migration and invasion of BGC-823 and MGC-803 cells. Western blotting was used to gauge changes in the protein expression levels of CDK6, CDK4, Bax, Bcl-2, caspase-3, P65, and the PI3K/Akt/mTOR pathway in gastric cancer cell lines after curcumin treatment. Differential expression of lncRNAs before and after curcumin treatment was assessed using lncRNA sequencing and validated using quantitative reverse transcription polymerase chain reaction (qRT-PCR) in BGC-823 and MGC-803 cells. AC022424.2-1 knockdown BGC-823 and MGC-803 cells were generated to scrutinize the impact of lncRNA AC022424.2 on apoptosis, proliferation, migration, and invasion of gastric cancer cells. Western blotting was performed to ascertain changes in the expression of proteins implicated in the PI3K/Akt/mTOR and NF-κB signaling pathways. RT-PCR was employed to measure lncRNA AC022424.2 expression in clinical gastric cancer tissues and to correlate its expression with clinical pathological characteristics.

RESULTS

Curcumin induced apoptosis and hindered proliferation, migration, and invasion of gastric cancer cells in a dose- and time-dependent manner. LncRNA AC022424.2 was upregulated after curcumin treatment, and its knockdown enhanced cancer cell aggressiveness. LncRNA AC022424.2 may have affected cancer cells via the PI3K/Akt/mTOR and NF-κB signaling pathways. LncRNA AC022424.2 downregulation was correlated with lymph node metastasis, making it a potential diagnostic and prognostic marker.

CONCLUSION

Curcumin has potential anticancer effects on gastric cancer cells by regulating lncRNA AC022424.2. This lncRNA plays a significant role in cancer cell behavior and may have clinical implications in diagnosis and prognosis evaluation. The results of this study enhance our understanding of gastric cancer development and precision treatment.

Predicting clinical endpoints and visual changes with quality-weighted tissue-based renal histological features

Two common obstacles limiting the performance of data-driven algorithms in digital histopathology classification tasks are the lack of expert annotations and the narrow diversity of datasets. Multi-instance learning (MIL) can address the former challenge for the analysis of whole slide images (WSI), but performance is often inferior to full supervision. We show that the inclusion of weak annotations can significantly enhance the effectiveness of MIL while keeping the approach scalable. An analysis framework was developed to process periodic acid-Schiff (PAS) and Sirius Red (SR) slides of renal biopsies. The workflow segments tissues into coarse tissue classes. Handcrafted and deep features were extracted from these tissues and combined using a soft attention model to predict several slide-level labels: delayed graft function (DGF), acute tubular injury (ATI), and Remuzzi grade components. A tissue segmentation quality metric was also developed to reduce the adverse impact of poorly segmented instances. The soft attention model was trained using 5-fold cross-validation on a mixed dataset and tested on the QUOD dataset containing n = 373 PAS and n = 195 SR biopsies. The average ROC-AUC over different prediction tasks was found to be 0.598 ± 0.011 , significantly higher than using only ResNet50 ( 0.545 ± 0.012 ), only handcrafted features ( 0.542 ± 0.011 ), and the baseline ( 0.532 ± 0.012 ) of state-of-the-art performance. In conjunction with soft attention, weighting tissues by segmentation quality has led to further improvement ( A U C = 0.618 ± 0.010 ) . Using an intuitive visualisation scheme, we show that our approach may also be used to support clinical decision making as it allows pinpointing individual tissues relevant to the predictions.

An effective colorectal polyp classification for histopathological images based on supervised contrastive learning

Early detection of colon adenomatous polyps is pivotal in reducing colon cancer risk. In this context, accurately distinguishing between adenomatous polyp subtypes, especially tubular and tubulovillous, from hyperplastic variants is crucial. This study introduces a cutting-edge computer-aided diagnosis system optimized for this task. Our system employs advanced Supervised Contrastive learning to ensure precise classification of colon histopathology images. Significantly, we have integrated the Big Transfer model, which has gained prominence for its exemplary adaptability to visual tasks in medical imaging. Our novel approach discerns between in-class and out-of-class images, thereby elevating its discriminatory power for polyp subtypes. We validated our system using two datasets: a specially curated one and the publicly accessible UniToPatho dataset. The results reveal that our model markedly surpasses traditional deep convolutional neural networks, registering classification accuracies of 87.1% and 70.3% for the custom and UniToPatho datasets, respectively. Such results emphasize the transformative potential of our model in polyp classification endeavors.

Utilizing nanotechnology and advanced machine learning for early detection of gastric cancer surgery

Nanotechnology has emerged as a promising frontier in revolutionizing the early diagnosis and surgical management of gastric cancers. The primary factors influencing curative efficacy in GIC patients are drug inefficacy and high surgical and pharmacological therapy recurrence rates. Due to its unique optical features, good biocompatibility, surface effects, and small size effects, nanotechnology is a developing and advanced area of study for detecting and treating cancer. Considering the limitations of GIC MRI and endoscopy and the complexity of gastric surgery, the early diagnosis and prompt treatment of gastric illnesses by nanotechnology has been a promising development. Nanoparticles directly target tumor cells, allowing their detection and removal. It also can be engineered to carry specific payloads, such as drugs or contrast agents, and enhance the efficacy and precision of cancer treatment. In this research, the boosting technique of machine learning was utilized to capture nonlinear interactions between a large number of input variables and outputs by using XGBoost and RNN-CNN as a classification method. The research sample included 350 patients, comprising 200 males and 150 females. The patients' mean ± SD was 50.34 ± 13.04 with a mean age of 50.34 ± 13.04. High-risk behaviors (P = 0.070), age at diagnosis (P = 0.034), distant metastasis (P = 0.004), and tumor stage (P = 0.014) were shown to have a statistically significant link with GC patient survival. AUC was 93.54%, Accuracy 93.54%, F1-score 93.57%, Precision 93.65%, and Recall 93.87% when analyzing stomach pictures. Integrating nanotechnology with advanced machine learning techniques holds promise for improving the diagnosis and treatment of gastric cancer, providing new avenues for precision medicine and better patient outcomes.

Enabling large-scale screening of Barrett's esophagus using weakly supervised deep learning in histopathology

Timely detection of Barrett's esophagus, the pre-malignant condition of esophageal adenocarcinoma, can improve patient survival rates. The Cytosponge-TFF3 test, a non-endoscopic minimally invasive procedure, has been used for diagnosing intestinal metaplasia in Barrett's. However, it depends on pathologist's assessment of two slides stained with H&E and the immunohistochemical biomarker TFF3. This resource-intensive clinical workflow limits large-scale screening in the at-risk population. To improve screening capacity, we propose a deep learning approach for detecting Barrett's from routinely stained H&E slides. The approach solely relies on diagnostic labels, eliminating the need for expensive localized expert annotations. We train and independently validate our approach on two clinical trial datasets, totaling 1866 patients. We achieve 91.4% and 87.3% AUROCs on discovery and external test datasets for the H&E model, comparable to the TFF3 model. Our proposed semi-automated clinical workflow can reduce pathologists' workload to 48% without sacrificing diagnostic performance, enabling pathologists to prioritize high risk cases.

An interpretable machine learning system for colorectal cancer diagnosis from pathology slides

Considering the profound transformation affecting pathology practice, we aimed to develop a scalable artificial intelligence (AI) system to diagnose colorectal cancer from whole-slide images (WSI). For this, we propose a deep learning (DL) system that learns from weak labels, a sampling strategy that reduces the number of training samples by a factor of six without compromising performance, an approach to leverage a small subset of fully annotated samples, and a prototype with explainable predictions, active learning features and parallelisation. Noting some problems in the literature, this study is conducted with one of the largest WSI colorectal samples dataset with approximately 10,500 WSIs. Of these samples, 900 are testing samples. Furthermore, the robustness of the proposed method is assessed with two additional external datasets (TCGA and PAIP) and a dataset of samples collected directly from the proposed prototype. Our proposed method predicts, for the patch-based tiles, a class based on the severity of the dysplasia and uses that information to classify the whole slide. It is trained with an interpretable mixed-supervision scheme to leverage the domain knowledge introduced by pathologists through spatial annotations. The mixed-supervision scheme allowed for an intelligent sampling strategy effectively evaluated in several different scenarios without compromising the performance. On the internal dataset, the method shows an accuracy of 93.44% and a sensitivity between positive (low-grade and high-grade dysplasia) and non-neoplastic samples of 0.996. On the external test samples varied with TCGA being the most challenging dataset with an overall accuracy of 84.91% and a sensitivity of 0.996.

The NCI Imaging Data Commons as a platform for reproducible research in computational pathology

BACKGROUND AND OBJECTIVES

Reproducibility is a major challenge in developing machine learning (ML)-based solutions in computational pathology (CompPath). The NCI Imaging Data Commons (IDC) provides >120 cancer image collections according to the FAIR principles and is designed to be used with cloud ML services. Here, we explore its potential to facilitate reproducibility in CompPath research.

METHODS

Using the IDC, we implemented two experiments in which a representative ML-based method for classifying lung tumor tissue was trained and/or evaluated on different datasets. To assess reproducibility, the experiments were run multiple times with separate but identically configured instances of common ML services.

RESULTS

The results of different runs of the same experiment were reproducible to a large extent. However, we observed occasional, small variations in AUC values, indicating a practical limit to reproducibility.

CONCLUSIONS

We conclude that the IDC facilitates approaching the reproducibility limit of CompPath research (i) by enabling researchers to reuse exactly the same datasets and (ii) by integrating with cloud ML services so that experiments can be run in identically configured computing environments.

MG-Trans: Multi-Scale Graph Transformer With Information Bottleneck for Whole Slide Image Classification

Multiple instance learning (MIL)-based methods have become the mainstream for processing the megapixel-sized whole slide image (WSI) with pyramid structure in the field of digital pathology. The current MIL-based methods usually crop a large number of patches from WSI at the highest magnification, resulting in a lot of redundancy in the input and feature space. Moreover, the spatial relations between patches can not be sufficiently modeled, which may weaken the model's discriminative ability on fine-grained features. To solve the above limitations, we propose a Multi-scale Graph Transformer (MG-Trans) with information bottleneck for whole slide image classification. MG-Trans is composed of three modules: patch anchoring module (PAM), dynamic structure information learning module (SILM), and multi-scale information bottleneck module (MIBM). Specifically, PAM utilizes the class attention map generated from the multi-head self-attention of vision Transformer to identify and sample the informative patches. SILM explicitly introduces the local tissue structure information into the Transformer block to sufficiently model the spatial relations between patches. MIBM effectively fuses the multi-scale patch features by utilizing the principle of information bottleneck to generate a robust and compact bag-level representation. Besides, we also propose a semantic consistency loss to stabilize the training of the whole model. Extensive studies on three subtyping datasets and seven gene mutation detection datasets demonstrate the superiority of MG-Trans.

Artificial Intelligence-Enabled Gastric Cancer Interpretations: Are We There yet?

The integration of digital pathology and artificial intelligence (AI) is revolutionizing pathology by providing pathologists with new tools to improve workflow, enhance diagnostic accuracy, and undertake novel discovery. The capability of AI to recognize patterns and features in digital images beyond human perception is particularly valuable, thereby providing additional information for prognostic and predictive purposes. AI-based tools diagnose gastric carcinoma in digital images, detect gastric carcinoma metastases in lymph nodes, automate Ki-67 scoring in gastric neuroendocrine tumors, and quantify tumor-infiltrating lymphocytes. This article provides an overview of all of these applications of AI pertaining to gastric cancer.

Computer image analysis with artificial intelligence: a practical introduction to convolutional neural networks for medical professionals

Artificial intelligence tools, particularly convolutional neural networks (CNNs), are transforming healthcare by enhancing predictive, diagnostic, and decision-making capabilities. This review provides an accessible and practical explanation of CNNs for clinicians and highlights their relevance in medical image analysis. CNNs have shown themselves to be exceptionally useful in computer vision, a field that enables machines to 'see' and interpret visual data. Understanding how these models work can help clinicians leverage their full potential, especially as artificial intelligence continues to evolve and integrate into healthcare. CNNs have already demonstrated their efficacy in diverse medical fields, including radiology, histopathology, and medical photography. In radiology, CNNs have been used to automate the assessment of conditions such as pneumonia, pulmonary embolism, and rectal cancer. In histopathology, CNNs have been used to assess and classify colorectal polyps, gastric epithelial tumours, as well as assist in the assessment of multiple malignancies. In medical photography, CNNs have been used to assess retinal diseases and skin conditions, and to detect gastric and colorectal polyps during endoscopic procedures. In surgical laparoscopy, they may provide intraoperative assistance to surgeons, helping interpret surgical anatomy and demonstrate safe dissection zones. The integration of CNNs into medical image analysis promises to enhance diagnostic accuracy, streamline workflow efficiency, and expand access to expert-level image analysis, contributing to the ultimate goal of delivering further improvements in patient and healthcare outcomes.

Explainable convolutional neural networks for assessing head and neck cancer histopathology

PURPOSE

Although neural networks have shown remarkable performance in medical image analysis, their translation into clinical practice remains difficult due to their lack of interpretability. An emerging field that addresses this problem is Explainable AI.

METHODS

Here, we aimed to investigate the ability of Convolutional Neural Networks (CNNs) to classify head and neck cancer histopathology. To this end, we manually annotated 101 histopathological slides of locally advanced head and neck squamous cell carcinoma. We trained a CNN to classify tumor and non-tumor tissue, and another CNN to semantically segment four classes - tumor, non-tumor, non-specified tissue, and background. We applied Explainable AI techniques, namely Grad-CAM and HR-CAM, to both networks and explored important features that contributed to their decisions.

RESULTS

The classification network achieved an accuracy of 89.9% on previously unseen data. Our segmentation network achieved a class-averaged Intersection over Union score of 0.690, and 0.782 for tumor tissue in particular. Explainable AI methods demonstrated that both networks rely on features agreeing with the pathologist's expert opinion.

CONCLUSION

Our work suggests that CNNs can predict head and neck cancer with high accuracy. Especially if accompanied by visual explanations, CNNs seem promising for assisting pathologists in the assessment of cancer sections.

Development and validation of artificial intelligence-based prescreening of large-bowel biopsies taken in the UK and Portugal: a retrospective cohort study

BACKGROUND

Histopathological examination is a crucial step in the diagnosis and treatment of many major diseases. Aiming to facilitate diagnostic decision making and improve the workload of pathologists, we developed an artificial intelligence (AI)-based prescreening tool that analyses whole-slide images (WSIs) of large-bowel biopsies to identify typical, non-neoplastic, and neoplastic biopsies.

METHODS

This retrospective cohort study was conducted with an internal development cohort of slides acquired from a hospital in the UK and three external validation cohorts of WSIs acquired from two hospitals in the UK and one clinical laboratory in Portugal. To learn the differential histological patterns from digitised WSIs of large-bowel biopsy slides, our proposed weakly supervised deep-learning model (Colorectal AI Model for Abnormality Detection [CAIMAN]) used slide-level diagnostic labels and no detailed cell or region-level annotations. The method was developed with an internal development cohort of 5054 biopsy slides from 2080 patients that were labelled with corresponding diagnostic categories assigned by pathologists. The three external validation cohorts, with a total of 1536 slides, were used for independent validation of CAIMAN. Each WSI was classified into one of three classes (ie, typical, atypical non-neoplastic, and atypical neoplastic). Prediction scores of image tiles were aggregated into three prediction scores for the whole slide, one for its likelihood of being typical, one for its likelihood of being non-neoplastic, and one for its likelihood of being neoplastic. The assessment of the external validation cohorts was conducted by the trained and frozen CAIMAN model. To evaluate model performance, we calculated area under the convex hull of the receiver operating characteristic curve (AUROC), area under the precision-recall curve, and specificity compared with our previously published iterative draw and rank sampling (IDaRS) algorithm. We also generated heat maps and saliency maps to analyse and visualise the relationship between the WSI diagnostic labels and spatial features of the tissue microenvironment. The main outcome of this study was the ability of CAIMAN to accurately identify typical and atypical WSIs of colon biopsies, which could potentially facilitate automatic removing of typical biopsies from the diagnostic workload in clinics.

FINDINGS

A randomly selected subset of all large bowel biopsies was obtained between Jan 1, 2012, and Dec 31, 2017. The AI training, validation, and assessments were done between Jan 1, 2021, and Sept 30, 2022. WSIs with diagnostic labels were collected between Jan 1 and Sept 30, 2022. Our analysis showed no statistically significant differences across prediction scores from CAIMAN for typical and atypical classes based on anatomical sites of the biopsy. At 0·99 sensitivity, CAIMAN (specificity 0·5592) was more accurate than an IDaRS-based weakly supervised WSI-classification pipeline (0·4629) in identifying typical and atypical biopsies on cross-validation in the internal development cohort (p<0·0001). At 0·99 sensitivity, CAIMAN was also more accurate than IDaRS for two external validation cohorts (p<0·0001), but not for a third external validation cohort (p=0·10). CAIMAN provided higher specificity than IDaRS at some high-sensitivity thresholds (0·7763 vs 0·6222 for 0·95 sensitivity, 0·7126 vs 0·5407 for 0·97 sensitivity, and 0·5615 vs 0·3970 for 0·99 sensitivity on one of the external validation cohorts) and showed high classification performance in distinguishing between neoplastic biopsies (AUROC 0·9928, 95% CI 0·9927-0·9929), inflammatory biopsies (0·9658, 0·9655-0·9661), and atypical biopsies (0·9789, 0·9786-0·9792). On the three external validation cohorts, CAIMAN had AUROC values of 0·9431 (95% CI 0·9165-0·9697), 0·9576 (0·9568-0·9584), and 0·9636 (0·9615-0·9657) for the detection of atypical biopsies. Saliency maps supported the representation of disease heterogeneity in model predictions and its association with relevant histological features.

INTERPRETATION

CAIMAN, with its high sensitivity in detecting atypical large-bowel biopsies, might be a promising improvement in clinical workflow efficiency and diagnostic decision making in prescreening of typical colorectal biopsies.

FUNDING

The Pathology Image Data Lake for Analytics, Knowledge and Education Centre of Excellence; the UK Government's Industrial Strategy Challenge Fund; and Innovate UK on behalf of UK Research and Innovation.

Lung Cancer Classification in Histopathology Images Using Multiresolution Efficient Nets

Histopathological images are very effective for investigating the status of various biological structures and diagnosing diseases like cancer. In addition, digital histopathology increases diagnosis precision and provides better image quality and more detail for the pathologist with multiple viewing options and team annotations. As a result of the benefits above, faster treatment is available, increasing therapy success rates and patient recovery and survival chances. However, the present manual examination of these images is tedious and time-consuming for pathologists. Therefore, reliable automated techniques are needed to effectively classify normal and malignant cancer images. This paper applied a deep learning approach, namely, EfficientNet and its variants from B0 to B7. We used different image resolutions for each model, from 224 × 224 pixels to 600 × 600 pixels. We also applied transfer learning and parameter tuning techniques to improve the results and overcome the overfitting problem. We collected the dataset from the Lung and Colon Cancer Histopathological Image LC25000 image dataset. The dataset acquisition consists of 25,000 histopathology images of five classes (lung adenocarcinoma, lung squamous cell carcinoma, benign lung tissue, colon adenocarcinoma, and colon benign tissue). Then, we performed preprocessing on the dataset to remove the noisy images and bring them into a standard format. The model's performance was evaluated in terms of classification accuracy and loss. We have achieved good accuracy results for all variants; however, the results of EfficientNetB2 stand excellent, with an accuracy of 97% for 260 × 260 pixels resolution images.

Preparing Data for Artificial Intelligence in Pathology with Clinical-Grade Performance

The pathology is decisive for disease diagnosis but relies heavily on experienced pathologists. In recent years, there has been growing interest in the use of artificial intelligence in pathology (AIP) to enhance diagnostic accuracy and efficiency. However, the impressive performance of deep learning-based AIP in laboratory settings often proves challenging to replicate in clinical practice. As the data preparation is important for AIP, the paper has reviewed AIP-related studies in the PubMed database published from January 2017 to February 2022, and 118 studies were included. An in-depth analysis of data preparation methods is conducted, encompassing the acquisition of pathological tissue slides, data cleaning, screening, and subsequent digitization. Expert review, image annotation, dataset division for model training and validation are also discussed. Furthermore, we delve into the reasons behind the challenges in reproducing the high performance of AIP in clinical settings and present effective strategies to enhance AIP's clinical performance. The robustness of AIP depends on a randomized collection of representative disease slides, incorporating rigorous quality control and screening, correction of digital discrepancies, reasonable annotation, and sufficient data volume. Digital pathology is fundamental in clinical-grade AIP, and the techniques of data standardization and weakly supervised learning methods based on whole slide image (WSI) are effective ways to overcome obstacles of performance reproduction. The key to performance reproducibility lies in having representative data, an adequate amount of labeling, and ensuring consistency across multiple centers. Digital pathology for clinical diagnosis, data standardization and the technique of WSI-based weakly supervised learning will hopefully build clinical-grade AIP.

A comprehensive survey of intestine histopathological image analysis using machine vision approaches

Colorectal Cancer (CRC) is currently one of the most common and deadly cancers. CRC is the third most common malignancy and the fourth leading cause of cancer death worldwide. It ranks as the second most frequent cause of cancer-related deaths in the United States and other developed countries. Histopathological images contain sufficient phenotypic information, they play an indispensable role in the diagnosis and treatment of CRC. In order to improve the objectivity and diagnostic efficiency for image analysis of intestinal histopathology, Computer-aided Diagnosis (CAD) methods based on machine learning (ML) are widely applied in image analysis of intestinal histopathology. In this investigation, we conduct a comprehensive study on recent ML-based methods for image analysis of intestinal histopathology. First, we discuss commonly used datasets from basic research studies with knowledge of intestinal histopathology relevant to medicine. Second, we introduce traditional ML methods commonly used in intestinal histopathology, as well as deep learning (DL) methods. Then, we provide a comprehensive review of the recent developments in ML methods for segmentation, classification, detection, and recognition, among others, for histopathological images of the intestine. Finally, the existing methods have been studied, and the application prospects of these methods in this field are given.

Applications of discriminative and deep learning feature extraction methods for whole slide image analysis: A survey

Digital pathology technologies, including whole slide imaging (WSI), have significantly improved modern clinical practices by facilitating storing, viewing, processing, and sharing digital scans of tissue glass slides. Researchers have proposed various artificial intelligence (AI) solutions for digital pathology applications, such as automated image analysis, to extract diagnostic information from WSI for improving pathology productivity, accuracy, and reproducibility. Feature extraction methods play a crucial role in transforming raw image data into meaningful representations for analysis, facilitating the characterization of tissue structures, cellular properties, and pathological patterns. These features have diverse applications in several digital pathology applications, such as cancer prognosis and diagnosis. Deep learning-based feature extraction methods have emerged as a promising approach to accurately represent WSI contents and have demonstrated superior performance in histology-related tasks. In this survey, we provide a comprehensive overview of feature extraction methods, including both manual and deep learning-based techniques, for the analysis of WSIs. We review relevant literature, analyze the discriminative and geometric features of WSIs (i.e., features suited to support the diagnostic process and extracted by "engineered" methods as opposed to AI), and explore predictive modeling techniques using AI and deep learning. This survey examines the advances, challenges, and opportunities in this rapidly evolving field, emphasizing the potential for accurate diagnosis, prognosis, and decision-making in digital pathology.

The Use of Deep Learning-Based Computer Diagnostic Algorithm for Detection of Lymph Node Metastases of Gastric Adenocarcinoma

Objectives. The diversifying modalities of treatment for gastric cancer raise urgent demands for the rapid and precise diagnosis of metastases in regional lymph nodes, thereby significantly impact the workload of pathologists. Meanwhile, the recent advent of whole-slide scanners and deep-learning techniques have enabled the computer-assisted analysis of histopathological images, which could help to alleviate this impact. Thus, we developed a deep learning-based diagnostic algorithm to detect lymph node metastases of gastric adenocarcinoma and evaluated its performance. Methods. We randomly selected 20 patients with gastric adenocarcinoma who underwent surgery as definitive treatment and were found to be node metastasis-positive. HEMATOXYLIN-eosin (HE) stained glass slides, including a total of 51 metastasis-positive nodes, were retrieved from the specimens of these cases. Other slides with 776 metastasis-negative nodes were also retrieved from other twenty cases with the same disease that were diagnosed as metastasis-negative by the final pathological examinations. All glass slides were digitized using a whole-slide scanner. A deep-learning algorithm to detect metastases was developed using the data in which metastasis-positive parts of the images were annotated by a well-trained pathologist, and its performance in detecting metastases was evaluated. Results. Cross-validation analysis indicated an area of 0.9994 under the receiver operating characteristic curve. Free-response receiver operating characteristic curve (FROC) analysis indicated a sensitivity of 1.00 with three false positives. Further evaluation using an independent dataset also showed similar level of accuracies. Conclusion. This deep learning-based diagnosis-aid system is a promising tool that can assist pathologists involved in gastric cancer care and reduce their workload.

ChampKit: A framework for rapid evaluation of deep neural networks for patch-based histopathology classification

BACKGROUND AND OBJECTIVE

Histopathology is the gold standard for diagnosis of many cancers. Recent advances in computer vision, specifically deep learning, have facilitated the analysis of histopathology images for many tasks, including the detection of immune cells and microsatellite instability. However, it remains difficult to identify optimal models and training configurations for different histopathology classification tasks due to the abundance of available architectures and the lack of systematic evaluations. Our objective in this work is to present a software tool that addresses this need and enables robust, systematic evaluation of neural network models for patch classification in histology in a light-weight, easy-to-use package for both algorithm developers and biomedical researchers.

METHODS

Here we present ChampKit (Comprehensive Histopathology Assessment of Model Predictions toolKit): an extensible, fully reproducible evaluation toolkit that is a one-stop-shop to train and evaluate deep neural networks for patch classification. ChampKit curates a broad range of public datasets. It enables training and evaluation of models supported by timm directly from the command line, without the need for users to write any code. External models are enabled through a straightforward API and minimal coding. As a result, Champkit facilitates the evaluation of existing and new models and deep learning architectures on pathology datasets, making it more accessible to the broader scientific community. To demonstrate the utility of ChampKit, we establish baseline performance for a subset of possible models that could be employed with ChampKit, focusing on several popular deep learning models, namely ResNet18, ResNet50, and R26-ViT, a hybrid vision transformer. In addition, we compare each model trained either from random weight initialization or with transfer learning from ImageNet pretrained models. For ResNet18, we also consider transfer learning from a self-supervised pretrained model.

RESULTS

The main result of this paper is the ChampKit software. Using ChampKit, we were able to systemically evaluate multiple neural networks across six datasets. We observed mixed results when evaluating the benefits of pretraining versus random intialization, with no clear benefit except in the low data regime, where transfer learning was found to be beneficial. Surprisingly, we found that transfer learning from self-supervised weights rarely improved performance, which is counter to other areas of computer vision.

CONCLUSIONS

Choosing the right model for a given digital pathology dataset is nontrivial. ChampKit provides a valuable tool to fill this gap by enabling the evaluation of hundreds of existing (or user-defined) deep learning models across a variety of pathology tasks. Source code and data for the tool are freely accessible at https://github.com/SBU-BMI/champkit.

Machine learning in computational histopathology: Challenges and opportunities

Digital histopathological images, high-resolution images of stained tissue samples, are a vital tool for clinicians to diagnose and stage cancers. The visual analysis of patient state based on these images are an important part of oncology workflow. Although pathology workflows have historically been conducted in laboratories under a microscope, the increasing digitization of histopathological images has led to their analysis on computers in the clinic. The last decade has seen the emergence of machine learning, and deep learning in particular, a powerful set of tools for the analysis of histopathological images. Machine learning models trained on large datasets of digitized histopathology slides have resulted in automated models for prediction and stratification of patient risk. In this review, we provide context for the rise of such models in computational histopathology, highlight the clinical tasks they have found success in automating, discuss the various machine learning techniques that have been applied to this domain, and underscore open problems and opportunities.

Classification of colorectal cancer consensus molecular subtypes using attention-based multi-instance learning network on whole-slide images

Colorectal cancer (CRC) is the third most common and second most lethal cancer globally. It is highly heterogeneous with different clinical-pathological characteristics, prognostic status, and therapy responses. Thus, the precise diagnosis of CRC subtypes is of great significance for improving the prognosis and survival of CRC patients. Nowadays, the most commonly used molecular-level CRC classification system is the Consensus Molecular Subtypes (CMSs). In this study, we applied a weakly supervised deep learning method, named attention-based multi-instance learning (MIL), on formalin-fixed paraffin-embedded (FFPE) whole-slide images (WSIs) to distinguish CMS1 subtype from CMS2, CMS3, and CMS4 subtypes, as well as distinguish CMS4 from CMS1, CMS2, and CMS3 subtypes. The advantage of MIL is training a bag of the tiled instance with bag-level labels only. Our experiment was performed on 1218 WSIs obtained from The Cancer Genome Atlas (TCGA). We constructed three convolutional neural network-based structures for model training and evaluated the ability of the max-pooling operator and mean-pooling operator on aggregating bag-level scores. The results showed that the 3-layer model achieved the best performance in both comparison groups. When compared CMS1 with CMS234, max-pooling reached the ACC of 83.86 % and the mean-pooling operator reached the AUC of 0.731. While comparing CMS4 with CMS123, mean-pooling reached the ACC of 74.26 % and max-pooling reached the AUC of 0.609. Our results implied that WSIs could be utilized to classify CMSs, and manual pixel-level annotation is not a necessity for computational pathology imaging analysis.

Artificial Intelligence in Minimally Invasive Adrenalectomy: Using Deep Learning to Identify the Left Adrenal Vein

BACKGROUND

Minimally invasive adrenalectomy is the main surgical treatment option for the resection of adrenal masses. Recognition and ligation of adrenal veins are critical parts of adrenal surgery. The utilization of artificial intelligence and deep learning algorithms to identify anatomic structures during laparoscopic and robot-assisted surgery can be used to provide real-time guidance.

METHODS

In this experimental feasibility study, intraoperative videos of patients who underwent minimally invasive transabdominal left adrenalectomy procedures between 2011 and 2022 in a tertiary endocrine referral center were retrospectively analyzed and used to develop an artificial intelligence model. Semantic segmentation of the left adrenal vein with deep learning was performed. To train a model, 50 random images per patient were captured during the identification and dissection of the left adrenal vein. A randomly selected 70% of data was used to train models while 15% for testing and 15% for validation with 3 efficient stage-wise feature pyramid networks (ESFPNet). Dice similarity coefficient (DSC) and intersection over union scores were used to evaluate segmentation accuracy.

RESULTS

A total of 40 videos were analyzed. Annotation of the left adrenal vein was performed in 2000 images. The segmentation network training on 1400 images was used to identify the left adrenal vein in 300 test images. The mean DSC and sensitivity for the highest scoring efficient stage-wise feature pyramid network B-2 network were 0.77 (±0.16 SD) and 0.82 (±0.15 SD), respectively, while the maximum DSC was 0.93, suggesting a successful prediction of anatomy.

CONCLUSIONS

Deep learning algorithms can predict the left adrenal vein anatomy with high performance and can potentially be utilized to identify critical anatomy during adrenal surgery and provide real-time guidance in the near future.

Multi-Scale Hybrid Vision Transformer for Learning Gastric Histology: AI-Based Decision Support System for Gastric Cancer Treatment

Gastric endoscopic screening is an effective way to decide appropriate gastric cancer treatment at an early stage, reducing gastric cancer-associated mortality rate. Although artificial intelligence has brought a great promise to assist pathologist to screen digitalized endoscopic biopsies, existing artificial intelligence systems are limited to be utilized in planning gastric cancer treatment. We propose a practical artificial intelligence-based decision support system that enables five subclassifications of gastric cancer pathology, which can be directly matched to general gastric cancer treatment guidance. The proposed framework is designed to efficiently differentiate multi-classes of gastric cancer through multiscale self-attention mechanism using 2-stage hybrid vision transformer networks, by mimicking the way how human pathologists understand histology. The proposed system demonstrates its reliable diagnostic performance by achieving class-average sensitivity of above 0.85 for multicentric cohort tests. Moreover, the proposed system demonstrates its great generalization capability on gastrointestinal track organ cancer by achieving the best class-average sensitivity among contemporary networks. Furthermore, in the observational study, artificial intelligence-assisted pathologists show significantly improved diagnostic sensitivity within saved screening time compared to human pathologists. Our results demonstrate that the proposed artificial intelligence system has a great potential for providing presumptive pathologic opinion and supporting decision of appropriate gastric cancer treatment in practical clinical settings.

Unleashing the potential of AI for pathology: challenges and recommendations

Computational pathology is currently witnessing a surge in the development of AI techniques, offering promise for achieving breakthroughs and significantly impacting the practices of pathology and oncology. These AI methods bring with them the potential to revolutionize diagnostic pipelines as well as treatment planning and overall patient care. Numerous peer-reviewed studies reporting remarkable performance across diverse tasks serve as a testimony to the potential of AI in the field. However, widespread adoption of these methods in clinical and pre-clinical settings still remains a challenge. In this review article, we present a detailed analysis of the major obstacles encountered during the development of effective models and their deployment in practice. We aim to provide readers with an overview of the latest developments, assist them with insights into identifying some specific challenges that may require resolution, and suggest recommendations and potential future research directions. © 2023 The Authors. The Journal of Pathology published by John Wiley & Sons Ltd on behalf of The Pathological Society of Great Britain and Ireland.

Neural network-based prognostic predictive tool for gastric cardiac cancer: the worldwide retrospective study

BACKGROUNDS

The incidence of gastric cardiac cancer (GCC) has obviously increased recently with poor prognosis. It's necessary to compare GCC prognosis with other gastric sites carcinoma and set up an effective prognostic model based on a neural network to predict the survival of GCC patients.

METHODS

In the population-based cohort study, we first enrolled the clinical features from the Surveillance, Epidemiology and End Results (SEER) data (n = 31,397) as well as the public Chinese data from different hospitals (n = 1049). Then according to the diagnostic time, the SEER data were then divided into two cohorts, the train cohort (patients were diagnosed as GCC in 2010-2014, n = 4414) and the test cohort (diagnosed in 2015, n = 957). Age, sex, pathology, tumor, node, and metastasis (TNM) stage, tumor size, surgery or not, radiotherapy or not, chemotherapy or not and history of malignancy were chosen as the predictive clinical features. The train cohort was utilized to conduct the neural network-based prognostic predictive model which validated by itself and the test cohort. Area under the receiver operating characteristics curve (AUC) was used to evaluate model performance.

RESULTS

The prognosis of GCC patients in SEER database was worse than that of non GCC (NGCC) patients, while it was not worse in the Chinese data. The total of 5371 patients were used to conduct the model, following inclusion and exclusion criteria. Neural network-based prognostic predictive model had a satisfactory performance for GCC overall survival (OS) prediction, which owned 0.7431 AUC in the train cohort (95% confidence intervals, CI, 0.7423-0.7439) and 0.7419 in the test cohort (95% CI, 0.7411-0.7428).

CONCLUSIONS

GCC patients indeed have different survival time compared with non GCC patients. And the neural network-based prognostic predictive tool developed in this study is a novel and promising software for the clinical outcome analysis of GCC patients.

A deep attention LSTM embedded aggregation network for multiple histopathological images

Recent advancements in computer vision and neural networks have facilitated the medical imaging survival analysis for various medical applications. However, challenges arise when patients have multiple images from multiple lesions, as current deep learning methods provide multiple survival predictions for each patient, complicating result interpretation. To address this issue, we developed a deep learning survival model that can provide accurate predictions at the patient level. We propose a deep attention long short-term memory embedded aggregation network (DALAN) for histopathology images, designed to simultaneously perform feature extraction and aggregation of lesion images. This design enables the model to efficiently learn imaging features from lesions and aggregate lesion-level information to the patient level. DALAN comprises a weight-shared CNN, attention layers, and LSTM layers. The attention layer calculates the significance of each lesion image, while the LSTM layer combines the weighted information to produce an all-encompassing representation of the patient's lesion data. Our proposed method performed better on both simulated and real data than other competing methods in terms of prediction accuracy. We evaluated DALAN against several naive aggregation methods on simulated and real datasets. Our results showed that DALAN outperformed the competing methods in terms of c-index on the MNIST and Cancer dataset simulations. On the real TCGA dataset, DALAN also achieved a higher c-index of 0.803±0.006 compared to the naive methods and the competing models. Our DALAN effectively aggregates multiple histopathology images, demonstrating a comprehensive survival model using attention and LSTM mechanisms.

Ensemble-based multi-tissue classification approach of colorectal cancer histology images using a novel hybrid deep learning framework

Colorectal cancer (CRC) is the second leading cause of cancer death in the world, so digital pathology is essential for assessing prognosis. Due to the increasing resolution and quantity of whole slide images (WSIs), as well as the lack of annotated information, previous methodologies cannot be generalized as effective decision-making systems. Since deep learning (DL) methods can handle large-scale applications, they can provide a viable alternative to histopathology image (HI) analysis. DL architectures, however, may not be sufficient to classify CRC tissues based on anatomical histopathology data. A dilated ResNet (dResNet) structure and attention module are used to generate deep feature maps in order to classify multiple tissues in HIs. In addition, neighborhood component analysis (NCA) overcomes the constraint of computational complexity. Data is fed into a deep support vector machine (SVM) based on an ensemble learning algorithm called DeepSVM after the features have been selected. CRC-5000 and NCT-CRC-HE-100 K datasets were analyzed to validate and test the hybrid procedure. We demonstrate that the hybrid model achieves 98.75% and 99.76% accuracy on CRC datasets. The results showed that only pathologists' labels could successfully classify unseen WSIs. Furthermore, the hybrid deep learning method outperforms state-of-the-art approaches in terms of computational efficiency and time. Using the proposed mechanism for tissue analysis, it will be possible to correctly predict CRC based on accurate pathology image classification.

Histopathological Gastric Cancer Detection on GasHisSDB Dataset Using Deep Ensemble Learning

Gastric cancer is a leading cause of cancer-related deaths worldwide, underscoring the need for early detection to improve patient survival rates. The current clinical gold standard for detection is histopathological image analysis, but this process is manual, laborious, and time-consuming. As a result, there has been growing interest in developing computer-aided diagnosis to assist pathologists. Deep learning has shown promise in this regard, but each model can only extract a limited number of image features for classification. To overcome this limitation and improve classification performance, this study proposes ensemble models that combine the decisions of several deep learning models. To evaluate the effectiveness of the proposed models, we tested their performance on the publicly available gastric cancer dataset, Gastric Histopathology Sub-size Image Database. Our experimental results showed that the top 5 ensemble model achieved state-of-the-art detection accuracy in all sub-databases, with the highest detection accuracy of 99.20% in the 160 × 160 pixels sub-database. These results demonstrated that ensemble models could extract important features from smaller patch sizes and achieve promising performance. Overall, our proposed work could assist pathologists in detecting gastric cancer through histopathological image analysis and contribute to early gastric cancer detection to improve patient survival rates.

Biased data, biased AI: deep networks predict the acquisition site of TCGA images

BACKGROUND

Deep learning models applied to healthcare applications including digital pathology have been increasing their scope and importance in recent years. Many of these models have been trained on The Cancer Genome Atlas (TCGA) atlas of digital images, or use it as a validation source. One crucial factor that seems to have been widely ignored is the internal bias that originates from the institutions that contributed WSIs to the TCGA dataset, and its effects on models trained on this dataset.

METHODS

8,579 paraffin-embedded, hematoxylin and eosin stained, digital slides were selected from the TCGA dataset. More than 140 medical institutions (acquisition sites) contributed to this dataset. Two deep neural networks (DenseNet121 and KimiaNet were used to extract deep features at 20× magnification. DenseNet was pre-trained on non-medical objects. KimiaNet has the same structure but trained for cancer type classification on TCGA images. The extracted deep features were later used to detect each slide's acquisition site, and also for slide representation in image search.

RESULTS

DenseNet's deep features could distinguish acquisition sites with 70% accuracy whereas KimiaNet's deep features could reveal acquisition sites with more than 86% accuracy. These findings suggest that there are acquisition site specific patterns that could be picked up by deep neural networks. It has also been shown that these medically irrelevant patterns can interfere with other applications of deep learning in digital pathology, namely image search. This study shows that there are acquisition site specific patterns that can be used to identify tissue acquisition sites without any explicit training. Furthermore, it was observed that a model trained for cancer subtype classification has exploited such medically irrelevant patterns to classify cancer types. Digital scanner configuration and noise, tissue stain variation and artifacts, and source site patient demographics are among factors that likely account for the observed bias. Therefore, researchers should be cautious of such bias when using histopathology datasets for developing and training deep networks.

Application of artificial neural network algorithm in pathological diagnosis and prognosis prediction of digestive tract malignant tumors

The application of artificial neural network algorithm in pathological diagnosis of gastrointestinal malignant tumors has become a research hotspot. In the previous studies, the algorithm research mainly focused on the model development based on convolutional neural networks, while only a few studies used the combination of convolutional neural networks and recurrent neural networks. The research contents included classical histopathological diagnosis and molecular typing of malignant tumors, and the prediction of patient prognosis by utilizing artificial neural networks. This article reviews the research progress on artificial neural network algorithm in the pathological diagnosis and prognosis prediction of digestive tract malignant tumors.

Using less annotation workload to establish a pathological auxiliary diagnosis system for gastric cancer

Pathological diagnosis of gastric cancer requires pathologists to have extensive clinical experience. To help pathologists improve diagnostic accuracy and efficiency, we collected 1,514 cases of stomach H&E-stained specimens with complete diagnostic information to establish a pathological auxiliary diagnosis system based on deep learning. At the slide level, our system achieves a specificity of 0.8878 while maintaining a high sensitivity close to 1.0 on 269 biopsy specimens (147 malignancies) and 163 surgical specimens (80 malignancies). The classified accuracy of our system is 0.9034 at the slide level for 352 biopsy specimens (201 malignancies) from 50 medical centers. With the help of our system, the pathologists' average false-negative rate and average false-positive rate on 100 biopsy specimens (50 malignancies) are reduced to 1/5 and 1/2 of the original rates, respectively. At the same time, the average uncertainty rate and the average diagnosis time are reduced by approximately 22% and 20%, respectively.

Improved classification of colorectal polyps on histopathological images with ensemble learning and stain normalization

BACKGROUND AND OBJECTIVE

Early detection of colon adenomatous polyps is critically important because correct detection of it significantly reduces the potential of developing colon cancers in the future. The key challenge in the detection of adenomatous polyps is differentiating it from its visually similar counterpart, non-adenomatous tissues. Currently, it solely depends on the experience of the pathologist. To assist the pathologists, the objective of this work is to provide a novel non-knowledge-based Clinical Decision Support System (CDSS) for improved detection of adenomatous polyps on colon histopathology images.

METHODS

The domain shift problem arises when the train and test data are coming from different distributions of diverse settings and unequal color levels. This problem, which can be tackled by stain normalization techniques, restricts the machine learning models to attain higher classification accuracies. In this work, the proposed method integrates stain normalization techniques with ensemble of competitively accurate, scalable and robust variants of CNNs, ConvNexts. The improvement is empirically analyzed for five widely employed stain normalization techniques. The classification performance of the proposed method is evaluated on three datasets comprising more than 10k colon histopathology images.

RESULTS

The comprehensive experiments demonstrate that the proposed method outperforms the state-of-the-art deep convolutional neural network based models by attaining 95% classification accuracy on the curated dataset, and 91.1% and 90% on EBHI and UniToPatho public datasets, respectively.

CONCLUSIONS

These results show that the proposed method can accurately classify colon adenomatous polyps on histopathology images. It retains remarkable performance scores even for different datasets coming from different distributions. This indicates that the model has a notable generalization ability.

Explainable synthetic image generation to improve risk assessment of rare pediatric heart transplant rejection

Expert microscopic analysis of cells obtained from frequent heart biopsies is vital for early detection of pediatric heart transplant rejection to prevent heart failure. Detection of this rare condition is prone to low levels of expert agreement due to the difficulty of identifying subtle rejection signs within biopsy samples. The rarity of pediatric heart transplant rejection also means that very few gold-standard images are available for developing machine learning models. To solve this urgent clinical challenge, we developed a deep learning model to automatically quantify rejection risk within digital images of biopsied tissue using an explainable synthetic data augmentation approach. We developed this explainable AI framework to illustrate how our progressive and inspirational generative adversarial network models distinguish between normal tissue images and those containing cellular rejection signs. To quantify biopsy-level rejection risk, we first detect local rejection features using a binary image classifier trained with expert-annotated and synthetic examples. We converted these local predictions into a biopsy-wide rejection score via an interpretable histogram-based approach. Our model significantly improves upon prior works with the same dataset with an area under the receiver operating curve (AUROC) of 98.84% for the local rejection detection task and 95.56% for the biopsy-rejection prediction task. A biopsy-level sensitivity of 83.33% makes our approach suitable for early screening of biopsies to prioritize expert analysis. Our framework provides a solution to rare medical imaging challenges currently limited by small datasets.

First passage time analysis of spatial mutation patterns reveals sub-clonal evolutionary dynamics in colorectal cancer

The signature of early cancer dynamics on the spatial arrangement of tumour cells is poorly understood, and yet could encode information about how sub-clones grew within the expanding tumour. Novel methods of quantifying spatial tumour data at the cellular scale are required to link evolutionary dynamics to the resulting spatial architecture of the tumour. Here, we propose a framework using first passage times of random walks to quantify the complex spatial patterns of tumour cell population mixing. First, using a simple model of cell mixing we demonstrate how first passage time statistics can distinguish between different pattern structures. We then apply our method to simulated patterns of mutated and non-mutated tumour cell population mixing, generated using an agent-based model of expanding tumours, to explore how first passage times reflect mutant cell replicative advantage, time of emergence and strength of cell pushing. Finally, we explore applications to experimentally measured human colorectal cancer, and estimate parameters of early sub-clonal dynamics using our spatial computational model. We infer a wide range of sub-clonal dynamics, with mutant cell division rates varying between 1 and 4 times the rate of non-mutated cells across our sample set. Some mutated sub-clones emerged after as few as 100 non-mutant cell divisions, and others only after 50,000 divisions. The majority were consistent with boundary driven growth or short-range cell pushing. By analysing multiple sub-sampled regions in a small number of samples, we explore how the distribution of inferred dynamics could inform about the initial mutational event. Our results demonstrate the efficacy of first passage time analysis as a new methodology in spatial analysis of solid tumour tissue, and suggest that patterns of sub-clonal mixing can provide insights into early cancer dynamics.

Oral biosciences: The annual review 2022

BACKGROUND

The Journal of Oral Biosciences is devoted to advancing and disseminating fundamental knowledge concerning every aspect of oral biosciences.

HIGHLIGHT

This review features review articles in the fields of "Bone Cell Biology," "Tooth Development & Regeneration," "Tooth Bleaching," "Adipokines," "Milk Thistle," "Epithelial-Mesenchymal Transition," "Periodontitis," "Diagnosis," "Salivary Glands," "Tooth Root," "Exosome," "New Perspectives of Tooth Identification," "Dental Pulp," and "Saliva" in addition to the review articles by the winner of the "Lion Dental Research Award" ("Plastic changes in nociceptive pathways contributing to persistent orofacial pain") presented by the Japanese Association for Oral Biology.

CONCLUSION

The review articles in the Journal of Oral Biosciences have inspired its readers to broaden their knowledge about various aspects of oral biosciences. The current editorial review introduces these exciting review articles.

Deep learning in digital pathology for personalized treatment plans of cancer patients

Over the past decade, many new cancer treatments have been developed and made available to patients. However, in most cases, these treatments only benefit a specific subgroup of patients, making the selection of treatment for a specific patient an essential but challenging task for oncologists. Although some biomarkers were found to associate with treatment response, manual assessment is time-consuming and subjective. With the rapid developments and expanded implementation of artificial intelligence (AI) in digital pathology, many biomarkers can be quantified automatically from histopathology images. This approach allows for a more efficient and objective assessment of biomarkers, aiding oncologists in formulating personalized treatment plans for cancer patients. This review presents an overview and summary of the recent studies on biomarker quantification and treatment response prediction using hematoxylin-eosin (H&E) stained pathology images. These studies have shown that an AI-based digital pathology approach can be practical and will become increasingly important in improving the selection of cancer treatments for patients.

Transfer Learning Approach and Nucleus Segmentation with MedCLNet Colon Cancer Database

Machine learning has been recently used especially in the medical field. In the diagnosis of serious diseases such as cancer, deep learning techniques can be used to reduce the workload of experts and to produce quick solutions. The nuclei found in the histopathology dataset are an essential parameter in disease detection. The nucleus segmentation was performed using the colorectal histology MNIST dataset for nucleus detection in this study. The graph theory, PSO, watershed, and random walker algorithms were used for the segmentation process. In addition, we present the 10-class MedCLNet visual dataset consisting of the NCT-CRC-HE-100 K dataset, LC25000 dataset, and GlaS dataset that can be used in transfer learning studies from deep learning techniques. The study proposes a transfer learning technique using the MedCLNet database. Deep neural networks pre-trained with the proposed transfer learning method were used in the classification with the colorectal histology MNIST dataset in the experimental process. DenseNet201, DenseNet169, InceptionResNetV2, InceptionV3, ResNet152V2, ResNet101V2, and Xception deep learning algorithms were used in transfer learning and the classification studies. The proposed approach was analyzed before and after transfer learning with different methods (DenseNet169 + SVM, DenseNet169 + GRU). In the performance measurement, using the colorectal histology MNIST dataset, 94.29% accuracy was obtained in the DenseNet169 model, which was initiated with random weights in the multi-classification study, and 95.00% accuracy after transfer learning was applied. In comparison with the results obtained from empirical studies, it was demonstrated that the proposed method produced satisfactory outcomes. The application is expected to provide a secondary evaluation for physicians in colon cancer detection and the segmentation.

Role of artificial intelligence in risk prediction, prognostication, and therapy response assessment in colorectal cancer: current state and future directions

Artificial Intelligence (AI) is a branch of computer science that utilizes optimization, probabilistic and statistical approaches to analyze and make predictions based on a vast amount of data. In recent years, AI has revolutionized the field of oncology and spearheaded novel approaches in the management of various cancers, including colorectal cancer (CRC). Notably, the applications of AI to diagnose, prognosticate, and predict response to therapy in CRC, is gaining traction and proving to be promising. There have also been several advancements in AI technologies to help predict metastases in CRC and in Computer-Aided Detection (CAD) Systems to improve miss rates for colorectal neoplasia. This article provides a comprehensive review of the role of AI in predicting risk, prognosis, and response to therapies among patients with CRC.

Implementing deep learning models for the classification of Echinococcus multilocularis infection in human liver tissue

BACKGROUND

The histological diagnosis of alveolar echinococcosis can be challenging. Decision support models based on deep learning (DL) are increasingly used to aid pathologists, but data on the histology of tissue-invasive parasitic infections are missing. The aim of this study was to implement DL methods to classify Echinococcus multilocularis liver lesions and normal liver tissue and assess which regions and structures play the most important role in classification decisions.

METHODS

We extracted 15,756 echinococcus tiles from 28 patients using 59 whole slide images (WSI); 11,602 tiles of normal liver parenchyma from 18 patients using 33 WSI served as a control group. Different pretrained model architectures were used with a 60-20-20% random splitting. We visualized the predictions using probability-thresholded heat maps of WSI. The area-under-the-curve (AUC) value and other performance metrics were calculated. The GradCAM method was used to calculate and visualize important spatial features.

RESULTS

The models achieved a high validation and test set accuracy. The calculated AUC values were 1.0 in all models. Pericystic fibrosis and necrotic areas, as well as germinative and laminated layers of the metacestodes played an important role in decision tasks according to the superimposed GradCAM heatmaps.

CONCLUSION

Deep learning models achieved a high predictive performance in classifying E. multilocularis liver lesions. A possible next step could be to validate the model using other datasets and test it against other pathologic entities as well, such as, for example, Echinococcus granulosus infection.

Inference of core needle biopsy whole slide images requiring definitive therapy for prostate cancer

BACKGROUND

Prostate cancer is often a slowly progressive indolent disease. Unnecessary treatments from overdiagnosis are a significant concern, particularly low-grade disease. Active surveillance has being considered as a risk management strategy to avoid potential side effects by unnecessary radical treatment. In 2016, American Society of Clinical Oncology (ASCO) endorsed the Cancer Care Ontario (CCO) Clinical Practice Guideline on active surveillance for the management of localized prostate cancer.

METHODS

Based on this guideline, we developed a deep learning model to classify prostate adenocarcinoma into indolent (applicable for active surveillance) and aggressive (necessary for definitive therapy) on core needle biopsy whole slide images (WSIs). In this study, we trained deep learning models using a combination of transfer, weakly supervised, and fully supervised learning approaches using a dataset of core needle biopsy WSIs (n=1300). In addition, we performed an inter-rater reliability evaluation on the WSI classification.

RESULTS

We evaluated the models on a test set (n=645), achieving ROC-AUCs of 0.846 for indolent and 0.980 for aggressive. The inter-rater reliability evaluation showed s-scores in the range of 0.10 to 0.95, with the lowest being on the WSIs with both indolent and aggressive classification by the model, and the highest on benign WSIs.

CONCLUSION

The results demonstrate the promising potential of deployment in a practical prostate adenocarcinoma histopathological diagnostic workflow system.

Artificial intelligence assists precision medicine in cancer treatment

Cancer is a major medical problem worldwide. Due to its high heterogeneity, the use of the same drugs or surgical methods in patients with the same tumor may have different curative effects, leading to the need for more accurate treatment methods for tumors and personalized treatments for patients. The precise treatment of tumors is essential, which renders obtaining an in-depth understanding of the changes that tumors undergo urgent, including changes in their genes, proteins and cancer cell phenotypes, in order to develop targeted treatment strategies for patients. Artificial intelligence (AI) based on big data can extract the hidden patterns, important information, and corresponding knowledge behind the enormous amount of data. For example, the ML and deep learning of subsets of AI can be used to mine the deep-level information in genomics, transcriptomics, proteomics, radiomics, digital pathological images, and other data, which can make clinicians synthetically and comprehensively understand tumors. In addition, AI can find new biomarkers from data to assist tumor screening, detection, diagnosis, treatment and prognosis prediction, so as to providing the best treatment for individual patients and improving their clinical outcomes.

Application of artificial intelligence in diagnosis and treatment of colorectal cancer: A novel Prospect

In the past few decades, according to the rapid development of information technology, artificial intelligence (AI) has also made significant progress in the medical field. Colorectal cancer (CRC) is the third most diagnosed cancer worldwide, and its incidence and mortality rates are increasing yearly, especially in developing countries. This article reviews the latest progress in AI in diagnosing and treating CRC based on a systematic collection of previous literature. Most CRCs transform from polyp mutations. The computer-aided detection systems can significantly improve the polyp and adenoma detection rate by early colonoscopy screening, thereby lowering the possibility of mutating into CRC. Machine learning and bioinformatics analysis can help screen and identify more CRC biomarkers to provide the basis for non-invasive screening. The Convolutional neural networks can assist in reading histopathologic tissue images, reducing the experience difference among doctors. Various studies have shown that AI-based high-level auxiliary diagnostic systems can significantly improve the readability of medical images and help clinicians make more accurate diagnostic and therapeutic decisions. Moreover, Robotic surgery systems such as da Vinci have been more and more commonly used to treat CRC patients, according to their precise operating performance. The application of AI in neoadjuvant chemoradiotherapy has further improved the treatment and efficacy evaluation of CRC. In addition, AI represented by deep learning in gene sequencing research offers a new treatment option. All of these things have seen that AI has a promising prospect in the era of precision medicine.