Automated Detection of Celiac Disease on Duodenal Biopsy Slides: A Deep Learning Approach

Computational pathology: A survey review and the way forward

Computational Pathology (CPath) is an interdisciplinary science that augments developments of computational approaches to analyze and model medical histopathology images. The main objective for CPath is to develop infrastructure and workflows of digital diagnostics as an assistive CAD system for clinical pathology, facilitating transformational changes in the diagnosis and treatment of cancer that are mainly address by CPath tools. With evergrowing developments in deep learning and computer vision algorithms, and the ease of the data flow from digital pathology, currently CPath is witnessing a paradigm shift. Despite the sheer volume of engineering and scientific works being introduced for cancer image analysis, there is still a considerable gap of adopting and integrating these algorithms in clinical practice. This raises a significant question regarding the direction and trends that are undertaken in CPath. In this article we provide a comprehensive review of more than 800 papers to address the challenges faced in problem design all-the-way to the application and implementation viewpoints. We have catalogued each paper into a model-card by examining the key works and challenges faced to layout the current landscape in CPath. We hope this helps the community to locate relevant works and facilitate understanding of the field's future directions. In a nutshell, we oversee the CPath developments in cycle of stages which are required to be cohesively linked together to address the challenges associated with such multidisciplinary science. We overview this cycle from different perspectives of data-centric, model-centric, and application-centric problems. We finally sketch remaining challenges and provide directions for future technical developments and clinical integration of CPath. For updated information on this survey review paper and accessing to the original model cards repository, please refer to GitHub. Updated version of this draft can also be found from arXiv.

Cellular and molecular basis of proximal small intestine disorders

The proximal part of the small intestine, including duodenum and jejunum, is not only dedicated to nutrient digestion and absorption but is also a highly regulated immune site exposed to environmental factors. Host-protective responses against pathogens and tolerance to food antigens are essential functions in the small intestine. The cellular ecology and molecular pathways to maintain those functions are complex. Maladaptation is highlighted by common immune-mediated diseases such as coeliac disease, environmental enteric dysfunction or duodenal Crohn's disease. An expanding spectrum of more than 100 rare monogenic disorders inform on causative molecular mechanisms of nutrient absorption, epithelial homeostasis and barrier function, as well as inflammatory immune responses and immune regulation. Here, after summarizing the architectural and cellular traits that underlie the functions of the proximal intestine, we discuss how the integration of tissue immunopathology and molecular mechanisms can contribute towards our understanding of disease and guide diagnosis. We propose an integrated mechanism-based taxonomy and discuss the latest experimental approaches to gain new mechanistic insight into these disorders with large disease burden worldwide as well as implications for therapeutic interventions.

Optimising the Diagnosis of Adult Coeliac Disease: Current Evidence and Future Directions

Coeliac disease is a common autoimmune disorder that affects nearly 1% of the general population. Current diagnostic strategies involve active case finding, serological tests, and endoscopy with biopsies. However, many patients with coeliac disease remain undiagnosed due to a wide gap between clinical guidelines and real-world practice in the diagnosis of adult coeliac disease. This highlights the need for increased education, training, and targeted quality-improvement interventions to optimise the diagnosis of coeliac disease.

Diagnostic accuracy of deep learning in detection and prognostication of renal cell carcinoma: a systematic review and meta-analysis

Introduction

The prevalence of Renal cell carcinoma (RCC) is increasing among adults. Histopathologic samples obtained after surgical resection or from biopsies of a renal mass require subtype classification for diagnosis, prognosis, and to determine surveillance. Deep learning in artificial intelligence (AI) and pathomics are rapidly advancing, leading to numerous applications such as histopathological diagnosis. In our meta-analysis, we assessed the pooled diagnostic performances of deep neural network (DNN) frameworks in detecting RCC subtypes and to predicting survival.

Methods

A systematic search was done in PubMed, Google Scholar, Embase, and Scopus from inception to November 2023. The random effects model was used to calculate the pooled percentages, mean, and 95% confidence interval. Accuracy was defined as the number of cases identified by AI out of the total number of cases, i.e. (True Positive + True Negative)/(True Positive + True Negative + False Positive + False Negative). The heterogeneity between study-specific estimates was assessed by the I 2 statistic. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines were used to conduct and report the analysis.

Results

The search retrieved 347 studies; 13 retrospective studies evaluating 5340 patients were included in the final analysis. The pooled performance of the DNN was as follows: accuracy 92.3% (95% CI: 85.8-95.9; I 2 = 98.3%), sensitivity 97.5% (95% CI: 83.2-99.7; I 2 = 92%), specificity 89.2% (95% CI: 29.9-99.4; I 2 = 99.6%) and area under the curve 0.91 (95% CI: 0.85-0.97.3; I 2 = 99.6%). Specifically, their accuracy in RCC subtype detection was 93.5% (95% CI: 88.7-96.3; I 2 = 92%), and the accuracy in survival analysis prediction was 81% (95% CI: 67.8-89.6; I 2 = 94.4%).

Discussion

The DNN showed excellent pooled diagnostic accuracy rates to classify RCC into subtypes and grade them for prognostic purposes. Further studies are required to establish generalizability and validate these findings on a larger scale.

Celiac Disease Deep Learning Image Classification Using Convolutional Neural Networks

Celiac disease (CD) is a gluten-sensitive immune-mediated enteropathy. This proof-of-concept study used a convolutional neural network (CNN) to classify hematoxylin and eosin (H&E) CD histological images, normal small intestine control, and non-specified duodenal inflammation (7294, 11,642, and 5966 images, respectively). The trained network classified CD with high performance (accuracy 99.7%, precision 99.6%, recall 99.3%, F1-score 99.5%, and specificity 99.8%). Interestingly, when the same network (already trained for the 3 class images), analyzed duodenal adenocarcinoma (3723 images), the new images were classified as duodenal inflammation in 63.65%, small intestine control in 34.73%, and CD in 1.61% of the cases; and when the network was retrained using the 4 histological subtypes, the performance was above 99% for CD and 97% for adenocarcinoma. Finally, the model added 13,043 images of Crohn's disease to include other inflammatory bowel diseases; a comparison between different CNN architectures was performed, and the gradient-weighted class activation mapping (Grad-CAM) technique was used to understand why the deep learning network made its classification decisions. In conclusion, the CNN-based deep neural system classified 5 diagnoses with high performance. Narrow artificial intelligence (AI) is designed to perform tasks that typically require human intelligence, but it operates within limited constraints and is task-specific.

Expanding role and scope of artificial intelligence in the field of gastrointestinal pathology

Digital pathology (DP) and its subsidiaries including artificial intelligence (AI) are rapidly making inroads into the area of diagnostic anatomic pathology (AP) including gastrointestinal (GI) pathology. It is poised to revolutionize the field of diagnostic AP. Historically, AP has been slow to adopt digital technology, but this is changing rapidly, with many centers worldwide transitioning to DP. Coupled with advanced techniques of AI such as deep learning and machine learning, DP is likely to transform histopathology from a subjective field to an objective, efficient, and transparent discipline. AI is increasingly integrated into GI pathology, offering numerous advancements and improvements in overall diagnostic accuracy, efficiency, and patient care. Specifically, AI in GI pathology enhances diagnostic accuracy, streamlines workflows, provides predictive insights, integrates multimodal data, supports research, and aids in education and training, ultimately improving patient care and outcomes. This review summarized the latest developments in the role and scope of AI in AP with a focus on GI pathology. The main aim was to provide updates and create awareness among the pathology community.

Advancing Automatic Gastritis Diagnosis: An Interpretable Multilabel Deep Learning Framework for the Simultaneous Assessment of Multiple Indicators

The evaluation of morphologic features, such as inflammation, gastric atrophy, and intestinal metaplasia, is crucial for diagnosing gastritis. However, artificial intelligence analysis for nontumor diseases like gastritis is limited. Previous deep learning models have omitted important morphologic indicators and cannot simultaneously diagnose gastritis indicators or provide interpretable labels. To address this, an attention-based multi-instance multilabel learning network (AMMNet) was developed to simultaneously achieve the multilabel diagnosis of activity, atrophy, and intestinal metaplasia with only slide-level weak labels. To evaluate AMMNet's real-world performance, a diagnostic test was designed to observe improvements in junior pathologists' diagnostic accuracy and efficiency with and without AMMNet assistance. In this study of 1096 patients from seven independent medical centers, AMMNet performed well in assessing activity [area under the curve (AUC), 0.93], atrophy (AUC, 0.97), and intestinal metaplasia (AUC, 0.93). The false-negative rates of these indicators were only 0.04, 0.08, and 0.18, respectively, and junior pathologists had lower false-negative rates with model assistance (0.15 versus 0.10). Furthermore, AMMNet reduced the time required per whole slide image from 5.46 to 2.85 minutes, enhancing diagnostic efficiency. In block-level clustering analysis, AMMNet effectively visualized task-related patches within whole slide images, improving interpretability. These findings highlight AMMNet's effectiveness in accurately evaluating gastritis morphologic indicators on multicenter data sets. Using multi-instance multilabel learning strategies to support routine diagnostic pathology deserves further evaluation.

Advancing Artificial Intelligence Integration Into the Pathology Workflow: Exploring Opportunities in Gastrointestinal Tract Biopsies

This review aims to present a comprehensive overview of the current landscape of artificial intelligence (AI) applications in the analysis of tubular gastrointestinal biopsies. These publications cover a spectrum of conditions, ranging from inflammatory ailments to malignancies. Moving beyond the conventional diagnosis based on hematoxylin and eosin-stained whole-slide images, the review explores additional implications of AI, including its involvement in interpreting immunohistochemical results, molecular subtyping, and the identification of cellular spatial biomarkers. Furthermore, the review examines how AI can contribute to enhancing the quality and control of diagnostic processes, introducing new workflow options, and addressing the limitations and caveats associated with current AI platforms in this context.

Application of automated machine learning for histological evaluation of feline endoscopic samples

Differentiating intestinal T-cell lymphoma from chronic enteropathy (CE) in endoscopic samples is often challenging. In the present study, automated machine learning systems were developed to distinguish between the two diseases, predict clonality, and detect prognostic factors of intestinal lymphoma in cats. Four models were created for four experimental conditions: experiment 1 to distinguish between intestinal T-cell lymphoma and CE; experiment 2 to distinguish large cell lymphoma, small cell lymphoma, and CE; experiment 3 to distinguish granzyme B+ lymphoma, granzyme B- lymphoma, and CE; and experiment 4 to distinguish between T-cell receptor (TCR) clonal population and TCR polyclonal population. After each experiment, a pathologist reviewed the test images and scored for lymphocytic infiltration, epitheliotropism, and epithelial injury. The models of experiments 1-4 achieved area under the receiver operating characteristic curve scores of 0.943 (precision, 87.59%; recall, 87.59%), 0.962 (precision, 86.30%; recall, 86.30%), 0.904 (precision, 82.86%; recall, 80%), and 0.904 (precision, 81.25%; recall, 81.25%), respectively. The images predicted as intestinal T-cell lymphoma showed significant infiltration of lymphocytes and epitheliotropism than CE. These models can provide evaluation tools to assist pathologists with differentiating between intestinal T-cell lymphoma and CE.

CD, or not CD, that is the question: a digital interobserver agreement study in coeliac disease

OBJECTIVE

Coeliac disease (CD) diagnosis generally depends on histological examination of duodenal biopsies. We present the first study analysing the concordance in examination of duodenal biopsies using digitised whole-slide images (WSIs). We further investigate whether the inclusion of immunoglobulin A tissue transglutaminase (IgA tTG) and haemoglobin (Hb) data improves the interobserver agreement of diagnosis.

DESIGN

We undertook a large study of the concordance in histological examination of duodenal biopsies using digitised WSIs in an entirely virtual reporting setting. Our study was organised in two phases: in phase 1, 13 pathologists independently classified 100 duodenal biopsies (40 normal; 40 CD; 20 indeterminate enteropathy) in the absence of any clinical or laboratory data. In phase 2, the same pathologists examined the (re-anonymised) WSIs with the inclusion of IgA tTG and Hb data.

RESULTS

We found the mean probability of two observers agreeing in the absence of additional data to be 0.73 (±0.08) with a corresponding Cohen's kappa of 0.59 (±0.11). We further showed that the inclusion of additional data increased the concordance to 0.80 (±0.06) with a Cohen's kappa coefficient of 0.67 (±0.09).

CONCLUSION

We showed that the addition of serological data significantly improves the quality of CD diagnosis. However, the limited interobserver agreement in CD diagnosis using digitised WSIs, even after the inclusion of IgA tTG and Hb data, indicates the importance of interpreting duodenal biopsy in the appropriate clinical context. It further highlights the unmet need for an objective means of reproducible duodenal biopsy diagnosis, such as the automated analysis of WSIs using artificial intelligence.

Pathologist-trained machine learning classifiers developed to quantitate celiac disease features differentiate endoscopic biopsies according to modified marsh score and dietary intervention response

BACKGROUND

Histologic evaluation of the mucosal changes associated with celiac disease is important for establishing an accurate diagnosis and monitoring the impact of investigational therapies. While the Marsh-Oberhuber classification has been used to categorize the histologic findings into discrete stages (i.e., Type 0-3c), significant variability has been documented between observers using this ordinal scoring system. Therefore, we evaluated whether pathologist-trained machine learning classifiers can be developed to objectively quantitate the pathological changes of villus blunting, intraepithelial lymphocytosis, and crypt hyperplasia in small intestine endoscopic biopsies.

METHODS

A convolutional neural network (CNN) was trained and combined with a secondary algorithm to quantitate intraepithelial lymphocytes (IEL) with 5 classes on CD3 immunohistochemistry whole slide images (WSI) and used to correlate feature outputs with ground truth modified Marsh scores in a total of 116 small intestine biopsies.

RESULTS

Across all samples, median %CD3 counts (positive cells/enterocytes) from villous epithelium (VE) increased with higher Marsh scores (Type 0%CD3 VE = 13.4; Type 1-3%CD3 VE = 41.9, p < 0.0001). Indicators of villus blunting and crypt hyperplasia were also observed (Type 0-2 villous epithelium/lamina propria area ratio = 0.81; Type 3a-3c villous epithelium/lamina propria area ratio = 0.29, p < 0.0001), and Type 0-1 crypt/villous epithelial area ratio = 0.59; Type 2-3 crypt/villous epithelial area ratio = 1.64, p < 0.0001). Using these individual features, a combined feature machine learning score (MLS) was created to evaluate a set of 28 matched pre- and post-intervention biopsies captured before and after dietary gluten restriction. The disposition of the continuous MLS paired biopsy result aligned with the Marsh score in 96.4% (27/28) of the cohort.

CONCLUSIONS

Machine learning classifiers can be developed to objectively quantify histologic features and capture additional data not achievable with manual scoring. Such approaches should be further investigated to improve biopsy evaluation, especially for clinical trials.

Stain normalization gives greater generalizability than stain jittering in neural network training for the classification of coeliac disease in duodenal biopsy whole slide images

Around 1% of the population of the UK and North America have a diagnosis of coeliac disease (CD), due to a damaging immune response to the small intestine. Assessing whether a patient has CD relies primarily on the examination of a duodenal biopsy, an unavoidably subjective process with poor inter-observer concordance. Wei et al. [11] developed a neural network-based method for diagnosing CD using a dataset of duodenal biopsy whole slide images (WSIs). As all training and validation data came from one source, there was no guarantee that their results would generalize to WSIs obtained from different scanners and laboratories. In this study, the effects of applying stain normalization and jittering to the training data were compared. We trained a deep neural network on 331 WSIs obtained with a Ventana scanner (WSIs; CD: n = 190 ; normal: n = 141 ) to classify presence of CD. In order to test the effects of stain processing when validating on WSIs scanned on varying scanners and from varying laboratories, the neural network was validated on 4 datasets: WSIs of slides scanned on a Ventana scanner (WSIs; CD: n = 48 ; normal: n = 35 ), WSIs of the same slides rescanned on a Hamamatsu scanner (WSIs; CD: n = 48 ; normal: n = 35 ), WSIs of the same slides rescanned on an Aperio scanner (WSIs; CD: n = 48 ; normal: n = 35 ), and WSIs of different slides scanned on an Aperio scanner (WSIs; CD: n = 38 ; normal: n = 37 ). Without stain processing, the F1 scores of the neural network were 0.947 , 0.619 , 0.746 , and 0.727 when validating on the Ventana validation WSIs, Hamamatsu and Aperio rescans of the Ventana validation WSIs, and Aperio WSIs from a different source respectively. With stain normalization, the performance of the neural network improved significantly with respective F1 scores 0.982 , 0.943 , 0.903 , and 0.847 . Stain jittering resulted in a better performance than stain normalization when validating on data from the same source F1 score 1.000 , but resulted in poorer performance than stain normalization when validating on WSIs from different scanners (F1 scores 0.939 , 0.814 , and 0.747 ). This study shows the importance of stain processing, in particular stain normalization, when training machine learning models on duodenal biopsy WSIs to ensure generalizability between different scanners and laboratories.

HistoPerm: A permutation-based view generation approach for improving histopathologic feature representation learning

Deep learning has been effective for histology image analysis in digital pathology. However, many current deep learning approaches require large, strongly- or weakly labeled images and regions of interest, which can be time-consuming and resource-intensive to obtain. To address this challenge, we present HistoPerm, a view generation method for representation learning using joint embedding architectures that enhances representation learning for histology images. HistoPerm permutes augmented views of patches extracted from whole-slide histology images to improve classification performance. We evaluated the effectiveness of HistoPerm on 2 histology image datasets for Celiac disease and Renal Cell Carcinoma, using 3 widely used joint embedding architecture-based representation learning methods: BYOL, SimCLR, and VICReg. Our results show that HistoPerm consistently improves patch- and slide-level classification performance in terms of accuracy, F1-score, and AUC. Specifically, for patch-level classification accuracy on the Celiac disease dataset, HistoPerm boosts BYOL and VICReg by 8% and SimCLR by 3%. On the Renal Cell Carcinoma dataset, patch-level classification accuracy is increased by 2% for BYOL and VICReg, and by 1% for SimCLR. In addition, on the Celiac disease dataset, models with HistoPerm outperform the fully supervised baseline model by 6%, 5%, and 2% for BYOL, SimCLR, and VICReg, respectively. For the Renal Cell Carcinoma dataset, HistoPerm lowers the classification accuracy gap for the models up to 10% relative to the fully supervised baseline. These findings suggest that HistoPerm can be a valuable tool for improving representation learning of histopathology features when access to labeled data is limited and can lead to whole-slide classification results that are comparable to or superior to fully supervised methods.

Detection of Colorectal Adenocarcinoma and Grading Dysplasia on Histopathologic Slides Using Deep Learning

Colorectal cancer (CRC) is one of the most common types of cancer among men and women. The grading of dysplasia and the detection of adenocarcinoma are important clinical tasks in the diagnosis of CRC and shape the patients' follow-up plans. This study evaluated the feasibility of deep learning models for the classification of colorectal lesions into four classes: benign, low-grade dysplasia, high-grade dysplasia, and adenocarcinoma. To this end, a deep neural network was developed on a training set of 655 whole slide images of digitized colorectal resection slides from a tertiary medical institution; and the network was evaluated on an internal test set of 234 slides, as well as on an external test set of 606 adenocarcinoma slides from The Cancer Genome Atlas database. The model achieved an overall accuracy, sensitivity, and specificity of 95.5%, 91.0%, and 97.1%, respectively, on the internal test set, and an accuracy and sensitivity of 98.5% for adenocarcinoma detection task on the external test set. Results suggest that such deep learning models can potentially assist pathologists in grading colorectal dysplasia, detecting adenocarcinoma, prescreening, and prioritizing the reviewing of suspicious cases to improve the turnaround time for patients with a high risk of CRC. Furthermore, the high sensitivity on the external test set suggests the model's generalizability in detecting colorectal adenocarcinoma on whole slide images across different institutions.

Automated analysis of small intestinal lamina propria to distinguish normal, Celiac Disease, and Non-Celiac Duodenitis biopsy images

BACKGROUND AND OBJECTIVE

Celiac Disease (CD) is characterized by gluten intolerance in genetically predisposed individuals. High disease prevalence, absence of a cure, and low diagnosis rates make this disease a public health problem. The diagnosis of CD predominantly relies on recognizing characteristic mucosal alterations of the small intestine, such as villous atrophy, crypt hyperplasia, and intraepithelial lymphocytosis. However, these changes are not entirely specific to CD and overlap with Non-Celiac Duodenitis (NCD) due to various etiologies. We investigated whether Artificial Intelligence (AI) models could assist in distinguishing normal, CD, and NCD (and unaffected individuals) based on the characteristics of small intestinal lamina propria (LP).

METHODS

Our method was developed using a dataset comprising high magnification biopsy images of the duodenal LP compartment of CD patients with different clinical stages of CD, those with NCD, and individuals lacking an intestinal inflammatory disorder (controls). A pre-processing step was used to standardize and enhance the acquired images.

RESULTS

For the normal controls versus CD use case, a Support Vector Machine (SVM) achieved an Accuracy (ACC) of 98.53%. For a second use case, we investigated the ability of the classification algorithm to differentiate between normal controls and NCD. In this use case, the SVM algorithm with linear kernel outperformed all the tested classifiers by achieving 98.55% ACC.

CONCLUSIONS

To the best of our knowledge, this is the first study that documents automated differentiation between normal, NCD, and CD biopsy images. These findings are a stepping stone toward automated biopsy image analysis that can significantly benefit patients and healthcare providers.

Multiple-instance-learning-based detection of coeliac disease in histological whole-slide images

We present a multiple-instance-learning-based scheme for detecting coeliac disease, an autoimmune disorder affecting the intestine, in histological whole-slide images (WSIs) of duodenal biopsies. We train our model to detect 2 distinct classes, normal tissue and coeliac disease, on the patch-level, and in turn leverage slide-level classifications. Using 5-fold cross-validation in a training set of 1841 (1163 normal; 680 coeliac disease) WSIs, our model classifies slides as normal with accuracy (96.7±0.6)%, precision (98.0±1.7)%, and recall (96.8±2.5)%, and as coeliac disease with accuracy (96.7±0.5)%, precision (94.9±3.7)%, and recall (96.5±2.9)% where the error bars are the cross-validation standard deviation. We apply our model to 2 test sets: one containing 191 WSIs (126 normal; 65 coeliac) from the same sources as the training data, and another from a completely independent source, containing 34 WSIs (17 normal; 17 coeliac), obtained with a scanner model not represented in the training data. Using the same-source test data, our model classifies slides as normal with accuracy 96.5%, precision 98.4% and recall 96.1%, and positive for coeliac disease with accuracy 96.5%, precision 93.5%, and recall 97.3%. Using the different-source test data the model classifies slides as normal with accuracy 94.1% (32/34), precision 89.5%, and recall 100%, and as positive for coeliac disease with accuracy 94.1%, precision 100%, and recall 88.2%. We discuss generalising our approach to screen for a range of pathologies.

Artificial Intelligence and the Future of Gastroenterology and Hepatology

The integration of artificial intelligence (AI) into gastroenterology and hepatology (GI) will inevitably transform the practice of GI in the coming decade. While the application of AI in health care is not new, advancements are occurring rapidly, and the future landscape of AI is beginning to come into focus. From endoscopic assistance via computer vision technology to the predictive capabilities of the vast information contained in the electronic health records, AI promises to optimize and expedite clinical and procedural practice and research in GI. The extensive body of literature already available on AI applications in gastroenterology may seem daunting at first; however, this review aims to provide a breakdown of the key studies conducted thus far and demonstrate the many potential ways this technology may impact the field. This review will also take a look into the future and imagine how GI can be transformed over the coming years, as well as potential limitations and pitfalls that must be overcome to realize this future.

Deep Transfer Learning for Automatic Prediction of Hemorrhagic Stroke on CT Images

Intracerebral hemorrhage (ICH) is the most common type of hemorrhagic stroke which occurs due to ruptures of weakened blood vessel in brain tissue. It is a serious medical emergency issues that needs immediate treatment. Large numbers of noncontrast-computed tomography (NCCT) brain images are analyzed manually by radiologists to diagnose the hemorrhagic stroke, which is a difficult and time-consuming process. In this study, we propose an automated transfer deep learning method that combines ResNet-50 and dense layer for accurate prediction of intracranial hemorrhage on NCCT brain images. A total of 1164 NCCT brain images were collected from 62 patients with hemorrhagic stroke from Kalinga Institute of Medical Science, Bhubaneswar and used for evaluating the model. The proposed model takes individual CT images as input and classifies them as hemorrhagic or normal. This deep transfer learning approach reached 99.6% accuracy, 99.7% specificity, and 99.4% sensitivity which are better results than that of ResNet-50 only. It is evident that the deep transfer learning model has advantages for automatic diagnosis of hemorrhagic stroke and has the potential to be used as a clinical decision support tool to assist radiologists in stroke diagnosis.

Automated detection of celiac disease using Machine Learning Algorithms

Celiac disease is a disorder of the immune system that mainly affects the small intestine but can also affect the skeletal system. The diagnosis relies on histological assessment of duodenal biopsies acquired by upper digestive endoscopy. Immunological tests involve collecting a blood sample to detect if the antibodies have been produced in the body. Endoscopy is invasive and histology is time-consuming. In recent years there have been various algorithms that use artificial intelligence (AI) and neural convolutions (CNN, Convolutional Neural Network) to process images from capsule endoscopy, a non-invasive endoscopy approach, that provides magnified, high qualitative images of the small bowel mucosa, to quickly establish a diagnosis. The proposed innovative approach do not use complex learning algorithms, instead it find some artefacts in the endoscopies using kernels and use classified machine learning algorithms. Each used artefacts have a psychical meaning: atrophies of the mucosa with a visible submucosal vascular pattern; the presence of cracks (depressions) that have an appearance similar to that of dry land; reduction or complete loss of folds in the duodenum; the presence of a submerged appearance at the Kerckring folds and a low number of villi. The results obtained for video capsule endoscopy images processing reveal an accuracy of 94.1% and F1 score of 94%, which is competitive with other complex algorithms. The main goal of the present research was to demonstrate that computer-aided diagnosis of celiac disease is possible even without the use of very complex algorithms, which require expensive hardware and a lot of processing time. The use of the proposed automated images processing acquired noninvasively by capsule endoscopy would be assistive in detecting the subtle presence of villous atrophy not evident by visual inspection. It may also be useful to assess the degree of improvement of celiac. Patients on a gluten-free diet, the main treatment method for stopping the autoimmune process and improving the state of the small intestinal villi. The novelty of the work is that the algorithm uses two modified filters to properly analyse the intestine wall texture. It is proved that using the right filters, the proper diagnostic can be obtained by image processing, without the use of a complicated machine learning algorithm.

Artificial intelligence as a tool for diagnosis in digital pathology whole slide images: A systematic review

Digital pathology had a recent growth, stimulated by the implementation of digital whole slide images (WSIs) in clinical practice, and the pathology field faces shortage of pathologists in the last few years. This scenario created fronts of research applying artificial intelligence (AI) to help pathologists. One of them is the automated diagnosis, helping in the clinical decision support, increasing efficiency and quality of diagnosis. However, the complexity nature of the WSIs requires special treatments to create a reliable AI model for diagnosis. Therefore, we systematically reviewed the literature to analyze and discuss all the methods and results in AI in digital pathology performed in WSIs on H&E stain, investigating the capacity of AI as a diagnostic support tool for the pathologist in the routine real-world scenario. This review analyzes 26 studies, reporting in detail all the best methods to apply AI as a diagnostic tool, as well as the main limitations, and suggests new ideas to improve the AI field in digital pathology as a whole. We hope that this study could lead to a better use of AI as a diagnostic tool in pathology, helping future researchers in the development of new studies and projects.

Application of Artificial Intelligence to Clinical Practice in Inflammatory Bowel Disease - What the Clinician Needs to Know

Artificial intelligence [AI] techniques are quickly spreading across medicine as an analytical method to tackle challenging clinical questions. What were previously thought of as highly complex data sources, such as images or free text, are now becoming manageable. Novel analytical methods merge the latest developments in information technology infrastructure with advances in computer science. Once primarily associated with Silicon Valley, AI techniques are now making their way into medicine, including in the field of inflammatory bowel diseases [IBD]. Understanding potential applications and limitations of these techniques can be difficult, in particular for busy clinicians. In this article, we explain the basic terminologies and provide a particular focus on the foundations behind state-of-the-art AI methodologies in both imaging and text. We explore the growing applications of AI in medicine, with a specific focus on IBD to inform the practising gastroenterologist and IBD specialist. Finally, we outline possible future uses of these technologies in daily clinical practice.

Resolution-based distillation for efficient histology image classification

Developing deep learning models to analyze histology images has been computationally challenging, as the massive size of the images causes excessive strain on all parts of the computing pipeline. This paper proposes a novel deep learning-based methodology for improving the computational efficiency of histology image classification. The proposed approach is robust when used with images that have reduced input resolution, and it can be trained effectively with limited labeled data. Moreover, our approach operates at either the tissue- or slide-level, removing the need for laborious patch-level labeling. Our method uses knowledge distillation to transfer knowledge from a teacher model pre-trained at high resolution to a student model trained on the same images at a considerably lower resolution. Also, to address the lack of large-scale labeled histology image datasets, we perform the knowledge distillation in a self-supervised fashion. We evaluate our approach on three distinct histology image datasets associated with celiac disease, lung adenocarcinoma, and renal cell carcinoma. Our results on these datasets demonstrate that a combination of knowledge distillation and self-supervision allows the student model to approach and, in some cases, surpass the teacher model's classification accuracy while being much more computationally efficient. Additionally, we observe an increase in student classification performance as the size of the unlabeled dataset increases, indicating that there is potential for this method to scale further with additional unlabeled data. Our model outperforms the high-resolution teacher model for celiac disease in accuracy, F1-score, precision, and recall while requiring 4 times fewer computations. For lung adenocarcinoma, our results at 1.25× magnification are within 1.5% of the results for the teacher model at 10× magnification, with a reduction in computational cost by a factor of 64. Our model on renal cell carcinoma at 1.25× magnification performs within 1% of the teacher model at 5× magnification while requiring 16 times fewer computations. Furthermore, our celiac disease outcomes benefit from additional performance scaling with the use of more unlabeled data. In the case of 0.625× magnification, using unlabeled data improves accuracy by 4% over the tissue-level baseline. Therefore, our approach can improve the feasibility of deep learning solutions for digital pathology on standard computational hardware and infrastructures.

Multiple abnormality classification in wireless capsule endoscopy images based on EfficientNet using attention mechanism

The wireless capsule endoscopy (WCE) procedure produces tens of thousands of images of the digestive tract, for which the use of the manual reading process is full of challenges. Convolutional neural networks are used to automatically detect lesions in WCE images. However, studies on clinical multilesion detection are scarce, and it is difficult to effectively balance the sensitivity to multiple lesions. A strategy for detecting multiple lesions is proposed, wherein common vascular and inflammatory lesions can be automatically and quickly detected on capsule endoscopic images. Based on weakly supervised learning, EfficientNet is fine-tuned to extract the endoscopic image features. Combining spatial features and channel features, the proposed attention network is then used as a classifier to obtain three classifications. The accuracy and speed of the model were compared with those of the ResNet121 and InceptionNetV4 models. It was tested on a public WCE image dataset obtained from 4143 subjects. On the computer-assisted diagnosis for capsule endoscopy database, the method gives a sensitivity of 96.67% for vascular lesions and 93.33% for inflammatory lesions. The precision for vascular lesions was 92.80%, and that for inflammatory lesions was 95.73%. The accuracy was 96.11%, which is 1.11% higher than that of the latest InceptionNetV4 network. Prediction for an image only requires 14 ms, which balances the accuracy and speed comparatively better. This strategy can be used as an auxiliary diagnostic method for specialists for the rapid reading of clinical capsule endoscopes.

Predicting prognosis and IDH mutation status for patients with lower-grade gliomas using whole slide images

We developed end-to-end deep learning models using whole slide images of adults diagnosed with diffusely infiltrating, World Health Organization (WHO) grade 2 gliomas to predict prognosis and the mutation status of a somatic biomarker, isocitrate dehydrogenase (IDH) 1/2. The models, which utilize ResNet-18 as a backbone, were developed and validated on 296 patients from The Cancer Genome Atlas (TCGA) database. To account for the small sample size, repeated random train/test splits were performed for hyperparameter tuning, and the out-of-sample predictions were pooled for evaluation. Our models achieved a concordance- (C-) index of 0.715 (95% CI: 0.569, 0.830) for predicting prognosis and an area under the curve (AUC) of 0.667 (0.532, 0.784) for predicting IDH mutations. When combined with additional clinical information, the performance metrics increased to 0.784 (95% CI: 0.655, 0.880) and 0.739 (95% CI: 0.613, 0.856), respectively. When evaluated on the WHO grade 3 gliomas from the TCGA dataset, which were not used for training, our models predicted survival with a C-index of 0.654 (95% CI: 0.537, 0.768) and IDH mutations with an AUC of 0.814 (95% CI: 0.721, 0.897). If validated in a prospective study, our method could potentially assist clinicians in managing and treating patients with diffusely infiltrating gliomas.

Deep-Learning-Driven Quantification of Interstitial Fibrosis in Digitized Kidney Biopsies

Interstitial fibrosis and tubular atrophy (IFTA) on a renal biopsy are strong indicators of disease chronicity and prognosis. Techniques that are typically used for IFTA grading remain manual, leading to variability among pathologists. Accurate IFTA estimation using computational techniques can reduce this variability and provide quantitative assessment. Using trichrome-stained whole-slide images (WSIs) processed from human renal biopsies, we developed a deep-learning framework that captured finer pathologic structures at high resolution and overall context at the WSI level to predict IFTA grade. WSIs (n = 67) were obtained from The Ohio State University Wexner Medical Center. Five nephropathologists independently reviewed them and provided fibrosis scores that were converted to IFTA grades: ≤10% (none or minimal), 11% to 25% (mild), 26% to 50% (moderate), and >50% (severe). The model was developed by associating the WSIs with the IFTA grade determined by majority voting (reference estimate). Model performance was evaluated on WSIs (n = 28) obtained from the Kidney Precision Medicine Project. There was good agreement on the IFTA grading between the pathologists and the reference estimate (κ = 0.622 ± 0.071). The accuracy of the deep-learning model was 71.8% ± 5.3% on The Ohio State University Wexner Medical Center and 65.0% ± 4.2% on Kidney Precision Medicine Project data sets. Our approach to analyzing microscopic- and WSI-level changes in renal biopsies attempts to mimic the pathologist and provides a regional and contextual estimation of IFTA. Such methods can assist clinicopathologic diagnosis.

Artificial intelligence in small intestinal diseases: Application and prospects

The small intestine is located in the middle of the gastrointestinal tract, so small intestinal diseases are more difficult to diagnose than other gastrointestinal diseases. However, with the extensive application of artificial intelligence in the field of small intestinal diseases, with its efficient learning capacities and computational power, artificial intelligence plays an important role in the auxiliary diagnosis and prognosis prediction based on the capsule endoscopy and other examination methods, which improves the accuracy of diagnosis and prediction and reduces the workload of doctors. In this review, a comprehensive retrieval was performed on articles published up to October 2020 from PubMed and other databases. Thereby the application status of artificial intelligence in small intestinal diseases was systematically introduced, and the challenges and prospects in this field were also analyzed.

Artificial intelligence in perioperative management of major gastrointestinal surgeries

Artificial intelligence (AI) demonstrated by machines is based on reinforcement learning and revolves around the usage of algorithms. The purpose of this review was to summarize concepts, the scope, applications, and limitations in major gastrointestinal surgery. This is a narrative review of the available literature on the key capabilities of AI to help anesthesiologists, surgeons, and other physicians to understand and critically evaluate ongoing and new AI applications in perioperative management. AI uses available databases called “big data” to formulate an algorithm. Analysis of other data based on these algorithms can help in early diagnosis, accurate risk assessment, intraoperative management, automated drug delivery, predicting anesthesia and surgical complications and postoperative outcomes and can thus lead to effective perioperative management as well as to reduce the cost of treatment. Perioperative physicians, anesthesiologists, and surgeons are well-positioned to help integrate AI into modern surgical practice. We all need to partner and collaborate with data scientists to collect and analyze data across all phases of perioperative care to provide clinical scenarios and context. Careful implementation and use of AI along with real-time human interpretation will revolutionize perioperative care, and is the way forward in future perioperative management of major surgery.

Artificial Intelligence-based Analytics for Diagnosis of Small Bowel Enteropathies and Black Box Feature Detection

OBJECTIVES

Striking histopathological overlap between distinct but related conditions poses a disease diagnostic challenge. There is a major clinical need to develop computational methods enabling clinicians to translate heterogeneous biomedical images into accurate and quantitative diagnostics. This need is particularly salient with small bowel enteropathies; environmental enteropathy (EE) and celiac disease (CD). We built upon our preliminary analysis by developing an artificial intelligence (AI)-based image analysis platform utilizing deep learning convolutional neural networks (CNNs) for these enteropathies.

METHODS

Data for the secondary analysis was obtained from three primary studies at different sites. The image analysis platform for EE and CD was developed using CNNs including one with multizoom architecture. Gradient-weighted class activation mappings (Grad-CAMs) were used to visualize the models' decision-making process for classifying each disease. A team of medical experts simultaneously reviewed the stain color normalized images done for bias reduction and Grad-CAMs to confirm structural preservation and biomedical relevance, respectively.

RESULTS

Four hundred and sixty-one high-resolution biopsy images from 150 children were acquired. Median age (interquartile range) was 37.5 (19.0-121.5) months with a roughly equal sex distribution; 77 males (51.3%). ResNet50 and shallow CNN demonstrated 98% and 96% case-detection accuracy, respectively, which increased to 98.3% with an ensemble. Grad-CAMs demonstrated models' ability to learn different microscopic morphological features for EE, CD, and controls.

CONCLUSIONS

Our AI-based image analysis platform demonstrated high classification accuracy for small bowel enteropathies which was capable of identifying biologically relevant microscopic features and emulating human pathologist decision-making process. Grad-CAMs illuminated the otherwise "black box" of deep learning in medicine, allowing for increased physician confidence in adopting these new technologies in clinical practice.

State of machine and deep learning in histopathological applications in digestive diseases

Machine learning (ML)- and deep learning (DL)-based imaging modalities have exhibited the capacity to handle extremely high dimensional data for a number of computer vision tasks. While these approaches have been applied to numerous data types, this capacity can be especially leveraged by application on histopathological images, which capture cellular and structural features with their high-resolution, microscopic perspectives. Already, these methodologies have demonstrated promising performance in a variety of applications like disease classification, cancer grading, structure and cellular localizations, and prognostic predictions. A wide range of pathologies requiring histopathological evaluation exist in gastroenterology and hepatology, indicating these as disciplines highly targetable for integration of these technologies. Gastroenterologists have also already been primed to consider the impact of these algorithms, as development of real-time endoscopic video analysis software has been an active and popular field of research. This heightened clinical awareness will likely be important for future integration of these methods and to drive interdisciplinary collaborations on emerging studies. To provide an overview on the application of these methodologies for gastrointestinal and hepatological histopathological slides, this review will discuss general ML and DL concepts, introduce recent and emerging literature using these methods, and cover challenges moving forward to further advance the field.

Automated interpretation of biopsy images for the detection of celiac disease using a machine learning approach

BACKGROUND AND OBJECTIVES

Celiac disease is an autoimmune disease occurring in about 1 in 100 people worldwide. Early diagnosis and efficient treatment are crucial in mitigating the complications that are associated with untreated celiac disease, such as intestinal lymphoma and malignancy, and the subsequent high morbidity. The current diagnostic methods using small intestinal biopsy histopathology, endoscopy, and video capsule endoscopy (VCE) involve manual interpretation of photomicrographs or images, which can be time-consuming and difficult, with inter-observer variability. In this paper, a machine learning technique was developed for the automation of biopsy image analysis to detect and classify villous atrophy based on modified Marsh scores. This is one of the first studies to employ conventional machine learning to automate the use of biopsy images for celiac disease detection and classification.

METHODS

The Steerable Pyramid Transform (SPT) method was used to obtain sub bands from which various types of entropy and nonlinear features were computed. All extracted features were automatically classified into two-class and multi-class, using six classifiers.

RESULTS

An accuracy of 88.89%, was achieved for the classification of two-class villous abnormalities based on analysis of Hematoxylin and Eosin (H&E) stained biopsy images. Similarly, an accuracy of 82.92% was achieved for the two-class classification of red-green-blue (RGB) biopsy images. Also, an accuracy of 72% was achieved in the classification of multi-class biopsy images.

CONCLUSION

The results obtained are promising, and demonstrate the possibility of automating biopsy image interpretation using machine learning. This can assist pathologists in accelerating the diagnostic process without bias, resulting in greater accuracy, and ultimately, earlier access to treatment.

A Deep Learning Convolutional Neural Network Can Differentiate Between Helicobacter Pylori Gastritis and Autoimmune Gastritis With Results Comparable to Gastrointestinal Pathologists

CONTEXT.—

Pathology studies using convolutional neural networks (CNNs) have focused on neoplasms, while studies in inflammatory pathology are rare. We previously demonstrated a CNN differentiates reactive gastropathy, Helicobacter pylori gastritis (HPG), and normal gastric mucosa.

OBJECTIVE.—

To determine whether a CNN can differentiate the following 2 gastric inflammatory patterns: autoimmune gastritis (AG) and HPG.

DESIGN.—

Gold standard diagnoses were blindly established by 2 gastrointestinal (GI) pathologists. One hundred eighty-seven cases were scanned for analysis by HALO-AI. All levels and tissue fragments per slide were included for analysis. The cases were randomized, 112 (60%; 60 HPG, 52 AG) in the training set and 75 (40%; 40 HPG, 35 AG) in the test set. A HALO-AI correct area distribution (AD) cutoff of 50% or more was required to credit the CNN with the correct diagnosis. The test set was blindly reviewed by pathologists with different levels of GI pathology expertise as follows: 2 GI pathologists, 2 general surgical pathologists, and 2 residents. Each pathologist rendered their preferred diagnosis, HPG or AG.

RESULTS.—

At the HALO-AI AD percentage cutoff of 50% or more, the CNN results were 100% concordant with the gold standard diagnoses. On average, autoimmune gastritis cases had 84.7% HALO-AI autoimmune gastritis AD and HP cases had 87.3% HALO-AI HP AD. The GI pathologists, general anatomic pathologists, and residents were on average, 100%, 86%, and 57% concordant with the gold standard diagnoses, respectively.

CONCLUSIONS.—

A CNN can distinguish between cases of HPG and autoimmune gastritis with accuracy equal to GI pathologists.

Development and evaluation of a deep neural network for histologic classification of renal cell carcinoma on biopsy and surgical resection slides

Renal cell carcinoma (RCC) is the most common renal cancer in adults. The histopathologic classification of RCC is essential for diagnosis, prognosis, and management of patients. Reorganization and classification of complex histologic patterns of RCC on biopsy and surgical resection slides under a microscope remains a heavily specialized, error-prone, and time-consuming task for pathologists. In this study, we developed a deep neural network model that can accurately classify digitized surgical resection slides and biopsy slides into five related classes: clear cell RCC, papillary RCC, chromophobe RCC, renal oncocytoma, and normal. In addition to the whole-slide classification pipeline, we visualized the identified indicative regions and features on slides for classification by reprocessing patch-level classification results to ensure the explainability of our diagnostic model. We evaluated our model on independent test sets of 78 surgical resection whole slides and 79 biopsy slides from our tertiary medical institution, and 917 surgical resection slides from The Cancer Genome Atlas (TCGA) database. The average area under the curve (AUC) of our classifier on the internal resection slides, internal biopsy slides, and external TCGA slides is 0.98 (95% confidence interval (CI): 0.97-1.00), 0.98 (95% CI: 0.96-1.00) and 0.97 (95% CI: 0.96-0.98), respectively. Our results suggest that the high generalizability of our approach across different data sources and specimen types. More importantly, our model has the potential to assist pathologists by (1) automatically pre-screening slides to reduce false-negative cases, (2) highlighting regions of importance on digitized slides to accelerate diagnosis, and (3) providing objective and accurate diagnosis as the second opinion.

Self-Attentive Adversarial Stain Normalization

Hematoxylin and Eosin (H&E) stained Whole Slide Images (WSIs) are utilized for biopsy visualization-based diagnostic and prognostic assessment of diseases. Variation in the H&E staining process across different lab sites can lead to significant variations in biopsy image appearance. These variations introduce an undesirable bias when the slides are examined by pathologists or used for training deep learning models. Traditionally proposed stain normalization and color augmentation strategies can handle the human level bias. But deep learning models can easily disentangle the linear transformation used in these approaches, resulting in undesirable bias and lack of generalization. To handle these limitations, we propose a Self-Attentive Adversarial Stain Normalization (SAASN) approach for the normalization of multiple stain appearances to a common domain. This unsupervised generative adversarial approach includes self-attention mechanism for synthesizing images with finer detail while preserving the structural consistency of the biopsy features during translation. SAASN demonstrates consistent and superior performance compared to other popular stain normalization techniques on H&E stained duodenal biopsy image data.

Artificial intelligence technology applications in the pathologic diagnosis of the gastrointestinal tract

Artificial intelligence (AI) is a complex technology with a steady flow of new applications, including in the pathology laboratory. Applications of AI in pathology are scarce but increasing; they are based on complex software-based machine learning with deep learning trained by pathologists. Their uses are based on tissue identification on histologic slides for classification into categories of normal, nonneoplastic and neoplastic conditions. Most AI applications are based on digital pathology. This commentary describes the role of AI in the pathological diagnosis of the gastrointestinal tract and provides insights into problems and future applications by answering four fundamental questions.

Digital histology in celiac disease: A practice changer

Artificial intelligence (AI) has grown tremendously in the last decades and is undoubtedly the future era in medicine. Concerning digestive diseases, applications of AI include clinical gastroenterology, gastrointestinal endoscopy and imaging, and not least pathological diagnosis. Several gastrointestinal pathologies require histological confirmation for a positive diagnosis. Among them, celiac disease (CD) diagnosis has been in the spotlight over time, but controversy is still ongoing with regard to the so-called celiac-type histology. Despite efforts to improve histological diagnosis in CD, there are still several issues and pitfalls associated with duodenal histology reading. Several papers have assessed the accuracy of AI techniques in detecting CD on duodenal biopsy images and have shown high diagnostic performance over standard histology reading. We discuss the role of computer-assisted histology in improving the assessment of mucosal architectural injury and inflammation in CD patients, both for diagnosis and follow-up.

Artificial intelligence as the next step towards precision pathology

Pathology is the cornerstone of cancer care. The need for accuracy in histopathologic diagnosis of cancer is increasing as personalized cancer therapy requires accurate biomarker assessment. The appearance of digital image analysis holds promise to improve both the volume and precision of histomorphological evaluation. Recently, machine learning, and particularly deep learning, has enabled rapid advances in computational pathology. The integration of machine learning into routine care will be a milestone for the healthcare sector in the next decade, and histopathology is right at the centre of this revolution. Examples of potential high-value machine learning applications include both model-based assessment of routine diagnostic features in pathology, and the ability to extract and identify novel features that provide insights into a disease. Recent groundbreaking results have demonstrated that applications of machine learning methods in pathology significantly improves metastases detection in lymph nodes, Ki67 scoring in breast cancer, Gleason grading in prostate cancer and tumour-infiltrating lymphocyte (TIL) scoring in melanoma. Furthermore, deep learning models have also been demonstrated to be able to predict status of some molecular markers in lung, prostate, gastric and colorectal cancer based on standard HE slides. Moreover, prognostic (survival outcomes) deep neural network models based on digitized HE slides have been demonstrated in several diseases, including lung cancer, melanoma and glioma. In this review, we aim to present and summarize the latest developments in digital image analysis and in the application of artificial intelligence in diagnostic pathology.

DeepSurvNet: deep survival convolutional network for brain cancer survival rate classification based on histopathological images

Histopathological whole slide images of haematoxylin and eosin (H&E)-stained biopsies contain valuable information with relation to cancer disease and its clinical outcomes. Still, there are no highly accurate automated methods to correlate histolopathological images with brain cancer patients' survival, which can help in scheduling patients therapeutic treatment and allocate time for preclinical studies to guide personalized treatments. We now propose a new classifier, namely, DeepSurvNet powered by deep convolutional neural networks, to accurately classify in 4 classes brain cancer patients' survival rate based on histopathological images (class I, 0-6 months; class II, 6-12 months; class III, 12-24 months; and class IV, >24 months survival after diagnosis). After training and testing of DeepSurvNet model on a public brain cancer dataset, The Cancer Genome Atlas, we have generalized it using independent testing on unseen samples. Using DeepSurvNet, we obtained precisions of 0.99 and 0.8 in the testing phases on the mentioned datasets, respectively, which shows DeepSurvNet is a reliable classifier for brain cancer patients' survival rate classification based on histopathological images. Finally, analysis of the frequency of mutations revealed differences in terms of frequency and type of genes associated to each class, supporting the idea of a different genetic fingerprint associated to patient survival. We conclude that DeepSurvNet constitutes a new artificial intelligence tool to assess the survival rate in brain cancer. Graphical abstract A DCNN model was generated to accurately predict survival rates of brain cancer patients (classified in 4 different classes) accurately. After training the model using images from H&E stained tissue biopsies from The Cancer Genome Atlas database (TCGA, left), the model can predict for each patient, based on a histological image (top right), its survival class accurately (bottom right).

CeliacNet: Celiac Disease Severity Diagnosis on Duodenal Histopathological Images Using Deep Residual Networks

Celiac Disease (CD) is a chronic autoimmune disease that affects the small intestine in genetically predisposed children and adults. Gluten exposure triggers an inflammatory cascade which leads to compromised intestinal barrier function. If this enteropathy is unrecognized, this can lead to anemia, decreased bone density, and, in longstanding cases, intestinal cancer. The prevalence of the disorder is 1% in the United States. An intestinal (duodenal) biopsy is considered the "gold standard" for diagnosis. The mild CD might go unnoticed due to non-specific clinical symptoms or mild histologic features. In our current work, we trained a model based on deep residual networks to diagnose CD severity using a histological scoring system called the modified Marsh score. The proposed model was evaluated using an independent set of 120 whole slide images from 15 CD patients and achieved an AUC greater than 0.96 in all classes. These results demonstrate the diagnostic power of the proposed model for CD severity classification using histological images.

Attention-Based Deep Neural Networks for Detection of Cancerous and Precancerous Esophagus Tissue on Histopathological Slides

Importance

Deep learning-based methods, such as the sliding window approach for cropped-image classification and heuristic aggregation for whole-slide inference, for analyzing histological patterns in high-resolution microscopy images have shown promising results. These approaches, however, require a laborious annotation process and are fragmented.

Objective

To evaluate a novel deep learning method that uses tissue-level annotations for high-resolution histological image analysis for Barrett esophagus (BE) and esophageal adenocarcinoma detection.

Design, Setting, and Participants

This diagnostic study collected deidentified high-resolution histological images (N = 379) for training a new model composed of a convolutional neural network and a grid-based attention network. Histological images of patients who underwent endoscopic esophagus and gastroesophageal junction mucosal biopsy between January 1, 2016, and December 31, 2018, at Dartmouth-Hitchcock Medical Center (Lebanon, New Hampshire) were collected.

Main Outcomes and Measures

The model was evaluated on an independent testing set of 123 histological images with 4 classes: normal, BE-no-dysplasia, BE-with-dysplasia, and adenocarcinoma. Performance of this model was measured and compared with that of the current state-of-the-art sliding window approach using the following standard machine learning metrics: accuracy, recall, precision, and F1 score.

Results

Of the independent testing set of 123 histological images, 30 (24.4%) were in the BE-no-dysplasia class, 14 (11.4%) in the BE-with-dysplasia class, 21 (17.1%) in the adenocarcinoma class, and 58 (47.2%) in the normal class. Classification accuracies of the proposed model were 0.85 (95% CI, 0.81-0.90) for the BE-no-dysplasia class, 0.89 (95% CI, 0.84-0.92) for the BE-with-dysplasia class, and 0.88 (95% CI, 0.84-0.92) for the adenocarcinoma class. The proposed model achieved a mean accuracy of 0.83 (95% CI, 0.80-0.86) and marginally outperformed the sliding window approach on the same testing set. The F1 scores of the attention-based model were at least 8% higher for each class compared with the sliding window approach: 0.68 (95% CI, 0.61-0.75) vs 0.61 (95% CI, 0.53-0.68) for the normal class, 0.72 (95% CI, 0.63-0.80) vs 0.58 (95% CI, 0.45-0.69) for the BE-no-dysplasia class, 0.30 (95% CI, 0.11-0.48) vs 0.22 (95% CI, 0.11-0.33) for the BE-with-dysplasia class, and 0.67 (95% CI, 0.54-0.77) vs 0.58 (95% CI, 0.44-0.70) for the adenocarcinoma class. However, this outperformance was not statistically significant.

Conclusions and Relevance

Results of this study suggest that the proposed attention-based deep neural network framework for BE and esophageal adenocarcinoma detection is important because it is based solely on tissue-level annotations, unlike existing methods that are based on regions of interest. This new model is expected to open avenues for applying deep learning to digital pathology.