Deep learning for digital pathology image analysis: A comprehensive tutorial with selected use cases

Evaluation of the precision and accuracy in the classification of breast histopathology images using the MobileNetV3 model

Accurate surgical pathological assessment of breast biopsies is essential to the proper management of breast lesions. Identifying histological features, such as nuclear pleomorphism, increased mitotic activity, cellular atypia, patterns of architectural disruption, as well as invasion through basement membranes into surrounding stroma and normal structures, including invasion of vascular and lymphatic spaces, help to classify lesions as malignant. This visual assessment is repeated on numerous slides taken at various sections through the resected tumor, each at different magnifications. Computer vision models have been proposed to assist human pathologists in classification tasks such as these. Using MobileNetV3, a convolutional architecture designed to achieve high accuracy with a compact parameter footprint, we attempted to classify breast cancer images in the BreakHis_v1 breast pathology dataset to determine the performance of this model out-of-the-box. Using transfer learning to take advantage of ImageNet embeddings without special feature extraction, we were able to correctly classify histopathology images broadly as benign or malignant with 0.98 precision, 0.97 recall, and an F1 score of 0.98. The ability to classify into histological subcategories was varied, with the greatest success being with classifying ductal carcinoma (accuracy 0.95), and the lowest success being with lobular carcinoma (accuracy 0.59). Multiclass ROC assessment of performance as a multiclass classifier yielded AUC values ≥0.97 in both benign and malignant subsets. In comparison with previous efforts, using older and larger convolutional network architectures with feature extraction pre-processing, our work highlights that modern, resource-efficient architectures can classify histopathological images with accuracy that at least matches that of previous efforts, without the need for labor-intensive feature extraction protocols. Suggestions to further refine the model are discussed.

SlideTiler: A dataset creator software for boosting deep learning on histological whole slide images

The introduction of deep learning caused a significant breakthrough in digital pathology. Thanks to its capability of mining hidden data patterns in digitised histological slides to resolve diagnostic tasks and extract prognostic and predictive information. However, the high performance achieved in classification tasks depends on the availability of large datasets, whose collection and preprocessing are still time-consuming processes. Therefore, strategies to make these steps more efficient are worth investigation. This work introduces SlideTiler, an open-source software with a user-friendly graphical interface. SlideTiler can manage several image preprocessing phases through an intuitive workflow that does not require specific coding skills. The software was designed to provide direct access to virtual slides, allowing custom tiling of specific regions of interest drawn by the user, tile labelling, quality assessment, and direct export to dataset directories. To illustrate the functions and the scalability of SlideTiler, a deep learning-based classifier was implemented to classify 4 different tumour histotypes available in the TCGA repository. The results demonstrate the effectiveness of SlideTiler in facilitating data preprocessing and promoting accessibility to digitised pathology images for research purposes. Considering the increasing interest in deep learning applications of digital pathology, SlideTiler has a positive impact on this field. Moreover, SlideTiler has been conceived as a dynamic tool in constant evolution, and more updated and efficient versions will be released in the future.

Computational pathology: A survey review and the way forward

Computational Pathology (CPath) is an interdisciplinary science that augments developments of computational approaches to analyze and model medical histopathology images. The main objective for CPath is to develop infrastructure and workflows of digital diagnostics as an assistive CAD system for clinical pathology, facilitating transformational changes in the diagnosis and treatment of cancer that are mainly address by CPath tools. With evergrowing developments in deep learning and computer vision algorithms, and the ease of the data flow from digital pathology, currently CPath is witnessing a paradigm shift. Despite the sheer volume of engineering and scientific works being introduced for cancer image analysis, there is still a considerable gap of adopting and integrating these algorithms in clinical practice. This raises a significant question regarding the direction and trends that are undertaken in CPath. In this article we provide a comprehensive review of more than 800 papers to address the challenges faced in problem design all-the-way to the application and implementation viewpoints. We have catalogued each paper into a model-card by examining the key works and challenges faced to layout the current landscape in CPath. We hope this helps the community to locate relevant works and facilitate understanding of the field's future directions. In a nutshell, we oversee the CPath developments in cycle of stages which are required to be cohesively linked together to address the challenges associated with such multidisciplinary science. We overview this cycle from different perspectives of data-centric, model-centric, and application-centric problems. We finally sketch remaining challenges and provide directions for future technical developments and clinical integration of CPath. For updated information on this survey review paper and accessing to the original model cards repository, please refer to GitHub. Updated version of this draft can also be found from arXiv.

Computer-assisted urine cytology: Faster, cheaper, better?

Recent advancements in computer-assisted diagnosis (CAD) have catalysed significant progress in pathology, particularly in the realm of urine cytopathology. This review synthesizes the latest developments and challenges in CAD for diagnosing urothelial carcinomas, addressing the limitations of traditional urinary cytology. Through a literature review, we identify and analyse CAD models and algorithms developed for urine cytopathology, highlighting their methodologies and performance metrics. We discuss the potential of CAD to improve diagnostic accuracy, efficiency and patient outcomes, emphasizing its role in streamlining workflow and reducing errors. Furthermore, CAD tools have shown potential in exploring pathological conditions, uncovering novel biomarkers and prognostic/predictive features previously unknown or unseen. Finally, we examine the practical issues surrounding the integration of CAD into clinical practice, including regulatory approval, validation and training for pathologists. Despite the promising results, challenges remain, necessitating further research and validation efforts. Overall, CAD presents a transformative opportunity to revolutionize diagnostic practices in urine cytopathology, paving the way for enhanced patient care and outcomes.

A machine learning algorithm improves the diagnostic accuracy of the histologic component of antibody mediated rejection (AMR-H) in cardiac transplant endomyocardial biopsies

BACKGROUND

Pathologic antibody mediated rejection (pAMR) remains a major driver of graft failure in cardiac transplant patients. The endomyocardial biopsy remains the primary diagnostic tool but presents with challenges, particularly in distinguishing the histologic component (pAMR-H) defined by 1) intravascular macrophage accumulation in capillaries and 2) activated endothelial cells that expand the cytoplasm to narrow or occlude the vascular lumen. Frequently, pAMR-H is difficult to distinguish from acute cellular rejection (ACR) and healing injury. With the advent of digital slide scanning and advances in machine deep learning, artificial intelligence technology is widely under investigation in the areas of oncologic pathology, but in its infancy in transplant pathology. For the first time, we determined if a machine learning algorithm could distinguish pAMR-H from normal myocardium, healing injury and ACR.

MATERIALS AND METHODS

A total of 4,212 annotations (1,053 regions of normal, 1,053 pAMR-H, 1,053 healing injury and 1,053 ACR) were completed from 300 hematoxylin and eosin slides scanned using a Leica Aperio GT450 digital whole slide scanner at 40X magnification. All regions of pAMR-H were annotated from patients confirmed with a previous diagnosis of pAMR2 (>50% positive C4d immunofluorescence and/or >10% CD68 positive intravascular macrophages). Annotations were imported into a Python 3.7 development environment using the OpenSlide™ package and a convolutional neural network approach utilizing transfer learning was performed.

RESULTS

The machine learning algorithm showed 98% overall validation accuracy and pAMR-H was correctly distinguished from specific categories with the following accuracies: normal myocardium (99.2%), healing injury (99.5%) and ACR (99.5%).

CONCLUSION

Our novel deep learning algorithm can reach acceptable, and possibly surpass, performance of current diagnostic standards of identifying pAMR-H. Such a tool may serve as an adjunct diagnostic aid for improving the pathologist's accuracy and reproducibility, especially in difficult cases with high inter-observer variability. This is one of the first studies that provides evidence that an artificial intelligence machine learning algorithm can be trained and validated to diagnose pAMR-H in cardiac transplant patients. Ongoing studies include multi-institutional verification testing to ensure generalizability.

Deep learning based approach: automated gingival inflammation grading model using gingival removal strategy

Gingival inflammation grade serves as a well-established index in periodontitis. The aim of this study was to develop a deep learning network utilizing a novel feature extraction method for the automatic assessment of gingival inflammation. T-distributed Stochastic Neighbor Embedding (t-SNE) was utilized for dimensionality reduction. A convolutional neural network (CNN) model based on DenseNet was developed for the identification and evaluation of gingival inflammation. To enhance the performance of the deep learning (DL) model, a novel teeth removal algorithm was implemented. Additionally, a Grad-CAM +  + encoder was applied to generate heatmaps for computer visual attention analysis. The mean Intersection over Union (MIoU) for the identification of gingivitis was 0.727 ± 0.117. The accuracy rates for the five inflammatory degrees were 77.09%, 77.25%, 74.38%, 73.68% and 79.22%. The Area Under the Receiver Operating Characteristic (AUROC) values were 0.83, 0.80, 0.81, 0.81 and 0.84, respectively. The attention ratio towards gingival tissue increased from 37.73% to 62.20%, and within 8 mm of the gingival margin, it rose from 21.11% to 38.23%. On the gingiva, the overall attention ratio increased from 51.82% to 78.21%. The proposed DL model with novel feature extraction method provides high accuracy and sensitivity for identifying and grading gingival inflammation.

Enhancing colorectal cancer histology diagnosis using modified deep neural networks optimizer

Optimizers are the bottleneck of the training process of any Convolutionolution neural networks (CNN) model. One of the critical steps when work on CNN model is choosing the optimal optimizer to solve a specific problem. Recent challenge in nowadays researches is building new versions of traditional CNN optimizers that can work more efficient than the traditional optimizers. Therefore, this work proposes a novel enhanced version of Adagrad optimizer called SAdagrad that avoids the drawbacks of Adagrad optimizer in dealing with tuning the learning rate value for each step of the training process. In order to evaluate SAdagrad, this paper builds a CNN model that combines a fine- tuning technique and a weight decay technique together. It trains the proposed CNN model on Kather colorectal cancer histology dataset which is one of the most challenging datasets in recent researches of Diagnose of Colorectal Cancer (CRC). In fact, recently, there have been plenty of deep learning models achieving successful results with regard to CRC classification experiments. However, the enhancement of these models remains challenging. To train our proposed model, a learning transfer process, which is adopted from a pre-complicated defined model is applied to the proposed model and combined it with a regularization technique that helps in avoiding overfitting. The experimental results show that SAdagrad reaches a remarkable accuracy (98%), when compared with Adaptive momentum optimizer (Adam) and Adagrad optimizer. The experiments also reveal that the proposed model has a more stable training and testing processes, can reduce the overfitting problem in multiple epochs and can achieve a higher accuracy compared with previous researches on Diagnosis CRC using the same Kather colorectal cancer histology dataset.

The Performance and Clinical Applicability of HER2 Digital Image Analysis in Breast Cancer: A Systematic Review

This systematic review aims to address the research gap in the performance of computational algorithms for the digital image analysis of HER2 images in clinical settings. While numerous studies have explored various aspects of these algorithms, there is a lack of comprehensive evaluation regarding their effectiveness in real-world clinical applications. We conducted a search of the Web of Science and PubMed databases for studies published from 31 December 2013 to 30 June 2024, focusing on performance effectiveness and components such as dataset size, diversity and source, ground truth, annotation, and validation methods. The study was registered with PROSPERO (CRD42024525404). Key questions guiding this review include the following: How effective are current computational algorithms at detecting HER2 status in digital images? What are the common validation methods and dataset characteristics used in these studies? Is there standardization of algorithm evaluations of clinical applications that can improve the clinical utility and reliability of computational tools for HER2 detection in digital image analysis? We identified 6833 publications, with 25 meeting the inclusion criteria. The accuracy rate with clinical datasets varied from 84.19% to 97.9%. The highest accuracy was achieved on the publicly available Warwick dataset at 98.8% in synthesized datasets. Only 12% of studies used separate datasets for external validation; 64% of studies used a combination of accuracy, precision, recall, and F1 as a set of performance measures. Despite the high accuracy rates reported in these studies, there is a notable absence of direct evidence supporting their clinical application. To facilitate the integration of these technologies into clinical practice, there is an urgent need to address real-world challenges and overreliance on internal validation. Standardizing study designs on real clinical datasets can enhance the reliability and clinical applicability of computational algorithms in improving the detection of HER2 cancer.

OralEpitheliumDB: A Dataset for Oral Epithelial Dysplasia Image Segmentation and Classification

Early diagnosis of potentially malignant disorders, such as oral epithelial dysplasia, is the most reliable way to prevent oral cancer. Computational algorithms have been used as an auxiliary tool to aid specialists in this process. Usually, experiments are performed on private data, making it difficult to reproduce the results. There are several public datasets of histological images, but studies focused on oral dysplasia images use inaccessible datasets. This prevents the improvement of algorithms aimed at this lesion. This study introduces an annotated public dataset of oral epithelial dysplasia tissue images. The dataset includes 456 images acquired from 30 mouse tongues. The images were categorized among the lesion grades, with nuclear structures manually marked by a trained specialist and validated by a pathologist. Also, experiments were carried out in order to illustrate the potential of the proposed dataset in classification and segmentation processes commonly explored in the literature. Convolutional neural network (CNN) models for semantic and instance segmentation were employed on the images, which were pre-processed with stain normalization methods. Then, the segmented and non-segmented images were classified with CNN architectures and machine learning algorithms. The data obtained through these processes is available in the dataset. The segmentation stage showed the F1-score value of 0.83, obtained with the U-Net model using the ResNet-50 as a backbone. At the classification stage, the most expressive result was achieved with the Random Forest method, with an accuracy value of 94.22%. The results show that the segmentation contributed to the classification results, but studies are needed for the improvement of these stages of automated diagnosis. The original, gold standard, normalized, and segmented images are publicly available and may be used for the improvement of clinical applications of CAD methods on oral epithelial dysplasia tissue images.

Computer-Based Evaluation of α-Synuclein Pathology in Multiple System Atrophy as a Novel Tool to Recognize Disease Subtypes

Multiple system atrophy (MSA) is a neurodegenerative disorder with variable disease course and distinct constellations of clinical (cerebellar [MSA-C] or parkinsonism [MSA-P]) and pathological phenotypes, suggestive of distinct α-synuclein (αSyn) strains. Neuropathologically, MSA is characterized by the accumulation of αSyn in oligodendrocytic glial cytoplasmic inclusions (GCI). Using a novel computer-based method, this study quantified the size of GCIs, density of all αSyn pathology, density of only the GCIs, and number of GCIs in MSA cases (n = 20). The putamen and cerebellar white matter were immunostained with the disease-associated 5G4 anti-αSyn antibody. Following digital scanning and image processing, total 5G4-immunoreactive pathology (ie, neuronal, neuritic, and glial) and GCIs were optically dissected for inclusion size and density measurement and then evaluated applying a novel computer-based method using ImageJ. GCI size varied between cases and brain regions (P < .0001), and heterogeneity in the density of all αSyn pathology including the density and number of GCIs were observed between regions and across cases, where MSA-C cases had a significantly higher density of all αSyn pathology in the cerebellar white matter (P = .049). Some region-specific morphologic variables inversely correlated with the age of onset and death, suggestive of an underlying aging-related cellular mechanism. Unsupervised K-means cluster analysis classified MSA cases into 3 distinct groups based on region-specific morphologic variables. In conclusion, we developed a novel computer-based method that is easily accessible, providing a first step to developing artificial intelligence-based evaluation strategies for large scale comparative studies. Our observations on the variability of morphologic variables between brain regions and cases highlight (1) the importance of computer-based approaches to detect features not considered in the routine diagnostic practice, and (2) novel aspects for the identification of previously unrecognized MSA subtypes that do not necessarily reflect the current clinical classification of MSA-C or MSA-P.

Contrastive and uncertainty-aware nuclei segmentation and classification

Nuclei segmentation and classification play a crucial role in pathology diagnosis, enabling pathologists to analyze cellular characteristics accurately. Overlapping cluster nuclei, misdetection of small-scale nuclei, and pleomorphic nuclei-induced misclassification have always been major challenges in the nuclei segmentation and classification tasks. To this end, we introduce an auxiliary task of nuclei boundary-guided contrastive learning to enhance the representativeness and discriminative power of visual features, particularly for addressing the challenge posed by the unclear contours of adherent nuclei and small nuclei. In addition, misclassifications resulting from pleomorphic nuclei often exhibit low classification confidence, indicating a high level of uncertainty. To mitigate misclassification, we capitalize on the characteristic clustering of similar cells to propose a locality-aware class embedding module, offering a regional perspective to capture category information. Moreover, we address uncertain classification in densely aggregated nuclei by designing a top-k uncertainty attention module that leverages deep features to enhance shallow features, thereby improving the learning of contextual semantic information. We demonstrate that the proposed network outperforms the off-the-shelf methods in both nuclei segmentation and classification experiments, achieving the state-of-the-art performance.

Unveiling the landscape of pathomics in personalized immunotherapy for lung cancer: a bibliometric analysis

Background

Pathomics has emerged as a promising biomarker that could facilitate personalized immunotherapy in lung cancer. It is essential to elucidate the global research trends and emerging prospects in this domain.

Methods

The annual distribution, journals, authors, countries, institutions, and keywords of articles published between 2018 and 2023 were visualized and analyzed using CiteSpace and other bibliometric tools.

Results

A total of 109 relevant articles or reviews were included, demonstrating an overall upward trend; The terms "deep learning", "tumor microenvironment", "biomarkers", "image analysis", "immunotherapy", and "survival prediction", etc. are hot keywords in this field.

Conclusion

In future research endeavors, advanced methodologies involving artificial intelligence and pathomics will be deployed for the digital analysis of tumor tissues and the tumor microenvironment in lung cancer patients, leveraging histopathological tissue sections. Through the integration of comprehensive multi-omics data, this strategy aims to enhance the depth of assessment, characterization, and understanding of the tumor microenvironment, thereby elucidating a broader spectrum of tumor features. Consequently, the development of a multimodal fusion model will ensue, enabling precise evaluation of personalized immunotherapy efficacy and prognosis for lung cancer patients, potentially establishing a pivotal frontier in this domain of investigation.

Deep learning-enabled classification of kidney allograft rejection on whole slide histopathologic images

Background

Diagnosis of kidney transplant rejection currently relies on manual histopathological assessment, which is subjective and susceptible to inter-observer variability, leading to limited reproducibility. We aim to develop a deep learning system for automated assessment of whole-slide images (WSIs) from kidney allograft biopsies to enable detection and subtyping of rejection and to predict the prognosis of rejection.

Method

We collected H&E-stained WSIs of kidney allograft biopsies at 400x magnification from January 2015 to September 2023 at two hospitals. These biopsy specimens were classified as T cell-mediated rejection, antibody-mediated rejection, and other lesions based on the consensus reached by two experienced transplant pathologists. To achieve feature extraction, feature aggregation, and global classification, we employed multi-instance learning and common convolution neural networks (CNNs). The performance of the developed models was evaluated using various metrics, including confusion matrix, receiver operating characteristic curves, the area under the curve (AUC), classification map, heat map, and pathologist-machine confrontations.

Results

In total, 906 WSIs from 302 kidney allograft biopsies were included for analysis. The model based on multi-instance learning enables detection and subtyping of rejection, named renal rejection artificial intelligence model (RRAIM), with the overall 3-category AUC of 0.798 in the independent test set, which is superior to that of three transplant pathologists under nearly routine assessment conditions. Moreover, the prognosis models accurately predicted graft loss within 1 year following rejection and treatment response for rejection, achieving AUC of 0.936 and 0.756, respectively.

Conclusion

We first developed deep-learning models utilizing multi-instance learning for the detection and subtyping of rejection and prediction of rejection prognosis in kidney allograft biopsies. These models performed well and may be useful in assisting the pathological diagnosis.

Swiss digital pathology recommendations: results from a Delphi process conducted by the Swiss Digital Pathology Consortium of the Swiss Society of Pathology

Integration of digital pathology (DP) into clinical diagnostic workflows is increasingly receiving attention as new hardware and software become available. To facilitate the adoption of DP, the Swiss Digital Pathology Consortium (SDiPath) organized a Delphi process to produce a series of recommendations for DP integration within Swiss clinical environments. This process saw the creation of 4 working groups, focusing on the various components of a DP system (1) scanners, quality assurance and validation of scans, (2) integration of Whole Slide Image (WSI)-scanners and DP systems into the Pathology Laboratory Information System, (3) digital workflow-compliance with general quality guidelines, and (4) image analysis (IA)/artificial intelligence (AI), with topic experts for each recruited for discussion and statement generation. The work product of the Delphi process is 83 consensus statements presented here, forming the basis for "SDiPath Recommendations for Digital Pathology". They represent an up-to-date resource for national and international hospitals, researchers, device manufacturers, algorithm developers, and all supporting fields, with the intent of providing expectations and best practices to help ensure safe and efficient DP usage.

Artificial intelligence for cardiovascular disease risk assessment in personalised framework: a scoping review

Background

The field of precision medicine endeavors to transform the healthcare industry by advancing individualised strategies for diagnosis, treatment modalities, and predictive assessments. This is achieved by utilizing extensive multidimensional biological datasets encompassing diverse components, such as an individual's genetic makeup, functional attributes, and environmental influences. Artificial intelligence (AI) systems, namely machine learning (ML) and deep learning (DL), have exhibited remarkable efficacy in predicting the potential occurrence of specific cancers and cardiovascular diseases (CVD).

Methods

We conducted a comprehensive scoping review guided by the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) framework. Our search strategy involved combining key terms related to CVD and AI using the Boolean operator AND. In August 2023, we conducted an extensive search across reputable scholarly databases including Google Scholar, PubMed, IEEE Xplore, ScienceDirect, Web of Science, and arXiv to gather relevant academic literature on personalised medicine for CVD. Subsequently, in January 2024, we extended our search to include internet search engines such as Google and various CVD websites. These searches were further updated in March 2024. Additionally, we reviewed the reference lists of the final selected research articles to identify any additional relevant literature.

Findings

A total of 2307 records were identified during the process of conducting the study, consisting of 564 entries from external sites like arXiv and 1743 records found through database searching. After 430 duplicate articles were eliminated, 1877 items that remained were screened for relevancy. In this stage, 1241 articles remained for additional review after 158 irrelevant articles and 478 articles with insufficient data were removed. 355 articles were eliminated for being inaccessible, 726 for being written in a language other than English, and 281 for not having undergone peer review. Consequently, 121 studies were deemed suitable for inclusion in the qualitative synthesis. At the intersection of CVD, AI, and precision medicine, we found important scientific findings in our scoping review. Intricate pattern extraction from large, complicated genetic datasets is a skill that AI algorithms excel at, allowing for accurate disease diagnosis and CVD risk prediction. Furthermore, these investigations have uncovered unique genetic biomarkers linked to CVD, providing insight into the workings of the disease and possible treatment avenues. The construction of more precise predictive models and personalised treatment plans based on the genetic profiles of individual patients has been made possible by the revolutionary advancement of CVD risk assessment through the integration of AI and genomics.

Interpretation

The systematic methodology employed ensured the thorough examination of available literature and the inclusion of relevant studies, contributing to the robustness and reliability of the study's findings. Our analysis stresses a crucial point in terms of the adaptability and versatility of AI solutions. AI algorithms designed in non-CVD domains such as in oncology, often include ideas and tactics that might be modified to address cardiovascular problems.

Funding

No funding received.

Study on an Automatic Classification Method for Determining the Malignancy Grade of Glioma Pathological Sections Based on Hyperspectral Multi-Scale Spatial-Spectral Fusion Features

This study describes a novel method for grading pathological sections of gliomas. Our own integrated hyperspectral imaging system was employed to characterize 270 bands of cancerous tissue samples from microarray slides of gliomas. These samples were then classified according to the guidelines developed by the World Health Organization, which define the subtypes and grades of diffuse gliomas. We explored a hyperspectral feature extraction model called SMLMER-ResNet using microscopic hyperspectral images of brain gliomas of different malignancy grades. The model combines the channel attention mechanism and multi-scale image features to automatically learn the pathological organization of gliomas and obtain hierarchical feature representations, effectively removing the interference of redundant information. It also completes multi-modal, multi-scale spatial-spectral feature extraction to improve the automatic classification of glioma subtypes. The proposed classification method demonstrated high average classification accuracy (>97.3%) and a Kappa coefficient (0.954), indicating its effectiveness in improving the automatic classification of hyperspectral gliomas. The method is readily applicable in a wide range of clinical settings, offering valuable assistance in alleviating the workload of clinical pathologists. Furthermore, the study contributes to the development of more personalized and refined treatment plans, as well as subsequent follow-up and treatment adjustment, by providing physicians with insights into the underlying pathological organization of gliomas.

Enhancing Histopathological Image Classification Performance through Synthetic Data Generation with Generative Adversarial Networks

Breast cancer is the second most common cancer worldwide, primarily affecting women, while histopathological image analysis is one of the possibile methods used to determine tumor malignancy. Regarding image analysis, the application of deep learning has become increasingly prevalent in recent years. However, a significant issue is the unbalanced nature of available datasets, with some classes having more images than others, which may impact the performance of the models due to poorer generalizability. A possible strategy to avoid this problem is downsampling the class with the most images to create a balanced dataset. Nevertheless, this approach is not recommended for small datasets as it can lead to poor model performance. Instead, techniques such as data augmentation are traditionally used to address this issue. These techniques apply simple transformations such as translation or rotation to the images to increase variability in the dataset. Another possibility is using generative adversarial networks (GANs), which can generate images from a relatively small training set. This work aims to enhance model performance in classifying histopathological images by applying data augmentation using GANs instead of traditional techniques.

Deep Learning Analysis for Predicting Tumor Spread through Air Space in Early-Stage Lung Adenocarcinoma Pathology Images

The presence of spread through air spaces (STASs) in early-stage lung adenocarcinoma is a significant prognostic factor associated with disease recurrence and poor outcomes. Although current STAS detection methods rely on pathological examinations, the advent of artificial intelligence (AI) offers opportunities for automated histopathological image analysis. This study developed a deep learning (DL) model for STAS prediction and investigated the correlation between the prediction results and patient outcomes. To develop the DL-based STAS prediction model, 1053 digital pathology whole-slide images (WSIs) from the competition dataset were enrolled in the training set, and 227 WSIs from the National Taiwan University Hospital were enrolled for external validation. A YOLOv5-based framework comprising preprocessing, candidate detection, false-positive reduction, and patient-based prediction was proposed for STAS prediction. The model achieved an area under the curve (AUC) of 0.83 in predicting STAS presence, with 72% accuracy, 81% sensitivity, and 63% specificity. Additionally, the DL model demonstrated a prognostic value in disease-free survival compared to that of pathological evaluation. These findings suggest that DL-based STAS prediction could serve as an adjunctive screening tool and facilitate clinical decision-making in patients with early-stage lung adenocarcinoma.

Evaluation of Dental Plaque Area with Artificial Intelligence Model

OBJECTIVES

This study aims to assess the diagnostic accuracy of an artificial intelligence (AI) system employing deep learning for identifying dental plaque, utilizing a dataset comprising photographs of permanent teeth.

MATERIALS AND METHODS

In this study, photographs of 168 teeth belonging to 20 patients aged between 10 and 15 years, who met our criteria, were included. Intraoral photographs were taken of the patients in two stages, before and after the application of the plaque staining agent. To train the AI system to identify plaque on teeth with dental plaque that is not discolored, plaque and teeth were marked on photos with exposed dental plaque. One hundred forty teeth were used to construct the training group, while 28 teeth were used to create the test group. Another dentist reviewed images of teeth with dental plaque that was not discolored, and the effectiveness of AI in detecting plaque was evaluated using pertinent performance indicators. To compare the AI model and the dentist's evaluation outcomes, the mean intersection over union (IoU) values were evaluated by the Wilcoxon test.

RESULTS

The AI system showed higher performance in our study with a precision of 82% accuracy, 84% sensitivity, 83% F1 score, 87% accuracy, and 89% specificity in plaque detection. The area under the curve (AUC) value was found to be 0.922, and the IoU value was 76%. Subsequently, the dentist's plaque diagnosis performance was also evaluated. The IoU value was 0.71, and the AUC was 0.833. The AI model showed statistically significantly higher performance than the dentist (P < 0.05).

CONCLUSIONS

The AI algorithm that we developed has achieved promising results and demonstrated clinically acceptable performance in detecting dental plaque compared to a dentist.

Digital pathology, deep learning, and cancer: a narrative review

Background and Objective

Cancer is a leading cause of morbidity and mortality worldwide. The emergence of digital pathology and deep learning technologies signifies a transformative era in healthcare. These technologies can enhance cancer detection, streamline operations, and bolster patient care. A substantial gap exists between the development phase of deep learning models in controlled laboratory environments and their translations into clinical practice. This narrative review evaluates the current landscape of deep learning and digital pathology, analyzing the factors influencing model development and implementation into clinical practice.

Methods

We searched multiple databases, including Web of Science, Arxiv, MedRxiv, BioRxiv, Embase, PubMed, DBLP, Google Scholar, IEEE Xplore, Semantic Scholar, and Cochrane, targeting articles on whole slide imaging and deep learning published from 2014 and 2023. Out of 776 articles identified based on inclusion criteria, we selected 36 papers for the analysis.

Key Content and Findings

Most articles in this review focus on the in-laboratory phase of deep learning model development, a critical stage in the deep learning lifecycle. Challenges arise during model development and their integration into clinical practice. Notably, lab performance metrics may not always match real-world clinical outcomes. As technology advances and regulations evolve, we expect more clinical trials to bridge this performance gap and validate deep learning models' effectiveness in clinical care. High clinical accuracy is vital for informed decision-making throughout a patient's cancer care.

Conclusions

Deep learning technology can enhance cancer detection, clinical workflows, and patient care. Challenges may arise during model development. The deep learning lifecycle involves data preprocessing, model development, and clinical implementation. Achieving health equity requires including diverse patient groups and eliminating bias during implementation. While model development is integral, most articles focus on the pre-deployment phase. Future longitudinal studies are crucial for validating models in real-world settings post-deployment. A collaborative approach among computational pathologists, technologists, industry, and healthcare providers is essential for driving adoption in clinical settings.

Enhancing AI Research for Breast Cancer: A Comprehensive Review of Tumor-Infiltrating Lymphocyte Datasets

The field of immunology is fundamental to our understanding of the intricate dynamics of the tumor microenvironment. In particular, tumor-infiltrating lymphocyte (TIL) assessment emerges as essential aspect in breast cancer cases. To gain comprehensive insights, the quantification of TILs through computer-assisted pathology (CAP) tools has become a prominent approach, employing advanced artificial intelligence models based on deep learning techniques. The successful recognition of TILs requires the models to be trained, a process that demands access to annotated datasets. Unfortunately, this task is hampered not only by the scarcity of such datasets, but also by the time-consuming nature of the annotation phase required to create them. Our review endeavors to examine publicly accessible datasets pertaining to the TIL domain and thereby become a valuable resource for the TIL community. The overall aim of the present review is thus to make it easier to train and validate current and upcoming CAP tools for TIL assessment by inspecting and evaluating existing publicly available online datasets.

Integrating spatial transcriptomics and bulk RNA-seq: predicting gene expression with enhanced resolution through graph attention networks

Spatial transcriptomics data play a crucial role in cancer research, providing a nuanced understanding of the spatial organization of gene expression within tumor tissues. Unraveling the spatial dynamics of gene expression can unveil key insights into tumor heterogeneity and aid in identifying potential therapeutic targets. However, in many large-scale cancer studies, spatial transcriptomics data are limited, with bulk RNA-seq and corresponding Whole Slide Image (WSI) data being more common (e.g. TCGA project). To address this gap, there is a critical need to develop methodologies that can estimate gene expression at near-cell (spot) level resolution from existing WSI and bulk RNA-seq data. This approach is essential for reanalyzing expansive cohort studies and uncovering novel biomarkers that have been overlooked in the initial assessments. In this study, we present STGAT (Spatial Transcriptomics Graph Attention Network), a novel approach leveraging Graph Attention Networks (GAT) to discern spatial dependencies among spots. Trained on spatial transcriptomics data, STGAT is designed to estimate gene expression profiles at spot-level resolution and predict whether each spot represents tumor or non-tumor tissue, especially in patient samples where only WSI and bulk RNA-seq data are available. Comprehensive tests on two breast cancer spatial transcriptomics datasets demonstrated that STGAT outperformed existing methods in accurately predicting gene expression. Further analyses using the TCGA breast cancer dataset revealed that gene expression estimated from tumor-only spots (predicted by STGAT) provides more accurate molecular signatures for breast cancer sub-type and tumor stage prediction, and also leading to improved patient survival and disease-free analysis. Availability: Code is available at https://github.com/compbiolabucf/STGAT.

Prognostic stratification of glioblastoma patients by unsupervised clustering of morphology patterns on whole slide images furthering our disease understanding

Introduction

Glioblastoma (GBM) is a highly aggressive malignant tumor of the central nervous system that displays varying molecular and morphological profiles, leading to challenging prognostic assessments. Stratifying GBM patients according to overall survival (OS) from H&E-stained whole slide images (WSI) using advanced computational methods is challenging, but with direct clinical implications.

Methods

This work is focusing on GBM (IDH-wildtype, CNS WHO Gr.4) cases, identified from the TCGA-GBM and TCGA-LGG collections after considering the 2021 WHO classification criteria. The proposed approach starts with patch extraction in each WSI, followed by comprehensive patch-level curation to discard artifactual content, i.e., glass reflections, pen markings, dust on the slide, and tissue tearing. Each patch is then computationally described as a feature vector defined by a pre-trained VGG16 convolutional neural network. Principal component analysis provides a feature representation of reduced dimensionality, further facilitating identification of distinct groups of morphology patterns, via unsupervised k-means clustering.

Results

The optimal number of clusters, according to cluster reproducibility and separability, is automatically determined based on the rand index and silhouette coefficient, respectively. Our proposed approach achieved prognostic stratification accuracy of 83.33% on a multi-institutional independent unseen hold-out test set with sensitivity and specificity of 83.33%.

Discussion

We hypothesize that the quantification of these clusters of morphology patterns, reflect the tumor's spatial heterogeneity and yield prognostic relevant information to distinguish between short and long survivors using a decision tree classifier. The interpretability analysis of the obtained results can contribute to furthering and quantifying our understanding of GBM and potentially improving our diagnostic and prognostic predictions.

Predicting meningioma grades and pathologic marker expression via deep learning

OBJECTIVES

To establish a deep learning (DL) model for predicting tumor grades and expression of pathologic markers of meningioma.

METHODS

A total of 1192 meningioma patients from two centers who underwent surgical resection between September 2018 and December 2021 were retrospectively included. The pathological data and post-contrast T1-weight images for each patient were collected. The patients from institute I were subdivided into training, validation, and testing sets, while the patients from institute II served as the external testing cohort. The fine-tuned ResNet50 model based on transfer learning was adopted to classify WHO grade in the whole cohort and predict Ki-67 index, H3K27me3, and progesterone receptor (PR) status of grade 1 meningiomas. The predictive performance was evaluated by the accuracy and loss curve, confusion matrix, receiver operating characteristic curve (ROC), and area under curve (AUC).

RESULTS

The DL prediction model for each label achieved high predictive performance in two cohorts. For WHO grade prediction, the area under the curve (AUC) was 0.966 (95%CI 0.957-0.975) in the internal testing set and 0.669 (95%CI 0.643-0.695) in the external validation cohort. The AUC in predicting Ki-67 index, H3K27me3, and PR status were 0.905 (95%CI 0.895-0.915), 0.773 (95%CI 0.760-0.786), and 0.771 (95%CI 0.750-0.792) in the internal testing set and 0.591 (95%CI 0.562-0.620), 0.658 (95%CI 0.648-0.668), and 0.703 (95%CI 0.674-0.732) in the external validation cohort, respectively.

CONCLUSION

DL models can preoperatively predict meningioma grades and pathologic marker expression with favorable predictive performance.

CLINICAL RELEVANCE STATEMENT

Our DL model could predict meningioma grades and expression of pathologic markers and identify high-risk patients with WHO grade 1 meningioma, which would suggest a more aggressive operative intervention preoperatively and a more frequent follow-up schedule postoperatively.

KEY POINTS

WHO grades and some pathologic markers of meningioma were associated with therapeutic strategies and clinical outcomes. A deep learning-based approach was employed to develop a model for predicting meningioma grades and the expression of pathologic markers. Preoperative prediction of meningioma grades and the expression of pathologic markers was beneficial for clinical decision-making.

A Histopathologic Image Analysis for the Classification of Endocervical Adenocarcinoma Silva Patterns Depend on Weakly Supervised Deep Learning

Twenty-five percent of cervical cancers are classified as endocervical adenocarcinomas (EACs), which comprise a highly heterogeneous group of tumors. A histopathologic risk stratification system known as the Silva pattern system was developed based on morphology. However, accurately classifying such patterns can be challenging. The study objective was to develop a deep learning pipeline (Silva3-AI) that automatically analyzes whole slide image-based histopathologic images and identifies Silva patterns with high accuracy. Initially, a total of 202 patients with EACs and histopathologic slides were obtained from Qilu Hospital of Shandong University for developing and internally testing the Silva3-AI model. Subsequently, an additional 161 patients and slides were collected from seven other medical centers for independent testing. The Silva3-AI model was developed using a vision transformer and recurrent neural network architecture, utilizing multi-magnification patches, and its performance was evaluated based on a class-specific area under the receiver-operating characteristic curve. Silva3-AI achieved a class-specific area under the receiver-operating characteristic curve of 0.947 for Silva A, 0.908 for Silva B, and 0.947 for Silva C on the independent test set. Notably, the performance of Silva3-AI was consistent with that of professional pathologists with 10 years' diagnostic experience. Furthermore, the visualization of prediction heatmaps facilitated the identification of tumor microenvironment heterogeneity, which is known to contribute to variations in Silva patterns.

Model-Agnostic Binary Patch Grouping for Bone Marrow Whole Slide Image Representation

Histopathology is the reference standard for pathology diagnosis, and has evolved with the digitization of glass slides [ie, whole slide images (WSIs)]. While trained histopathologists are able to diagnose diseases by examining WSIs visually, this process is time consuming and prone to variability. To address these issues, artificial intelligence models are being developed to generate slide-level representations of WSIs, summarizing the entire slide as a single vector. This enables various computational pathology applications, including interslide search, multimodal training, and slide-level classification. Achieving expressive and robust slide-level representations hinges on patch feature extraction and aggregation steps. This study proposed an additional binary patch grouping (BPG) step, a plugin that can be integrated into various slide-level representation pipelines, to enhance the quality of slide-level representation in bone marrow histopathology. BPG excludes patches with less clinical relevance through minimal interaction with the pathologist; a one-time human intervention for the entire process. This study further investigated domain-general versus domain-specific feature extraction models based on convolution and attention and examined two different feature aggregation methods, with and without BPG, showing BPG's generalizability. The results showed that using BPG boosts the performance of WSI retrieval (mean average precision at 10) by 4% and improves WSI classification (weighted-F1) by 5% compared to not using BPG. Additionally, domain-general large models and parameterized pooling produced the best-quality slide-level representations.

Optimizing Vision Transformers for Histopathology: Pretraining and Normalization in Breast Cancer Classification

This paper introduces a self-attention Vision Transformer model specifically developed for classifying breast cancer in histology images. We examine various training strategies and configurations, including pretraining, dimension resizing, data augmentation and color normalization strategies, patch overlap, and patch size configurations, in order to evaluate their impact on the effectiveness of the histology image classification. Additionally, we provide evidence for the increase in effectiveness gathered through geometric and color data augmentation techniques. We primarily utilize the BACH dataset to train and validate our methods and models, but we also test them on two additional datasets, BRACS and AIDPATH, to verify their generalization capabilities. Our model, developed from a transformer pretrained on ImageNet, achieves an accuracy rate of 0.91 on the BACH dataset, 0.74 on the BRACS dataset, and 0.92 on the AIDPATH dataset. Using a model based on the prostate small and prostate medium HistoEncoder models, we achieve accuracy rates of 0.89 and 0.86, respectively. Our results suggest that pretraining on large-scale general datasets like ImageNet is advantageous. We also show the potential benefits of using domain-specific pretraining datasets, such as extensive histopathological image collections as in HistoEncoder, though not yet with clear advantages.

Synergizing Deep Learning-Enabled Preprocessing and Human-AI Integration for Efficient Automatic Ground Truth Generation

The progress of incorporating deep learning in the field of medical image interpretation has been greatly hindered due to the tremendous cost and time associated with generating ground truth for supervised machine learning, alongside concerns about the inconsistent quality of images acquired. Active learning offers a potential solution to these problems of expanding dataset ground truth by algorithmically choosing the most informative samples for ground truth labeling. Still, this effort incurs the costs of human labeling, which needs minimization. Furthermore, automatic labeling approaches employing active learning often exhibit overfitting tendencies while selecting samples closely aligned with the training set distribution and excluding out-of-distribution samples, which could potentially improve the model's effectiveness. We propose that the majority of out-of-distribution instances can be attributed to inconsistent cross images. Since the FDA approved the first whole-slide image system for medical diagnosis in 2017, whole-slide images have provided enriched critical information to advance the field of automated histopathology. Here, we exemplify the benefits of a novel deep learning strategy that utilizes high-resolution whole-slide microscopic images. We quantitatively assess and visually highlight the inconsistencies within the whole-slide image dataset employed in this study. Accordingly, we introduce a deep learning-based preprocessing algorithm designed to normalize unknown samples to the training set distribution, effectively mitigating the overfitting issue. Consequently, our approach significantly increases the amount of automatic region-of-interest ground truth labeling on high-resolution whole-slide images using active deep learning. We accept 92% of the automatic labels generated for our unlabeled data cohort, expanding the labeled dataset by 845%. Additionally, we demonstrate expert time savings of 96% relative to manual expert ground-truth labeling.

DRA-Net: Medical image segmentation based on adaptive feature extraction and region-level information fusion

Medical image segmentation is a key task in computer aided diagnosis. In recent years, convolutional neural network (CNN) has made some achievements in medical image segmentation. However, the convolution operation can only extract features in a fixed size region at a time, which leads to the loss of some key features. The recently popular Transformer has global modeling capabilities, but it does not pay enough attention to local information and cannot accurately segment the edge details of the target area. Given these issues, we proposed dynamic regional attention network (DRA-Net). Different from the above methods, it first measures the similarity of features and concentrates attention on different dynamic regions. In this way, the network can adaptively select different modeling scopes for feature extraction, reducing information loss. Then, regional feature interaction is carried out to better learn local edge details. At the same time, we also design ordered shift multilayer perceptron (MLP) blocks to enhance communication within different regions, further enhancing the network's ability to learn local edge details. After several experiments, the results indicate that our network produces more accurate segmentation performance compared to other CNN and Transformer based networks.

Registered multi-device/staining histology image dataset for domain-agnostic machine learning models

Variations in color and texture of histopathology images are caused by differences in staining conditions and imaging devices between hospitals. These biases decrease the robustness of machine learning models exposed to out-of-domain data. To address this issue, we introduce a comprehensive histopathology image dataset named PathoLogy Images of Scanners and Mobile phones (PLISM). The dataset consisted of 46 human tissue types stained using 13 hematoxylin and eosin conditions and captured using 13 imaging devices. Precisely aligned image patches from different domains allowed for an accurate evaluation of color and texture properties in each domain. Variation in PLISM was assessed and found to be significantly diverse across various domains, particularly between whole-slide images and smartphones. Furthermore, we assessed the improvement in domain shift using a convolutional neural network pre-trained on PLISM. PLISM is a valuable resource that facilitates the precise evaluation of domain shifts in digital pathology and makes significant contributions towards the development of robust machine learning models that can effectively address challenges of domain shift in histological image analysis.

The Evaluation of Artificial Intelligence Technology for the Differentiation of Fresh Human Blood Cells From Other Species' Blood in the Investigation of Crime Scenes

OBJECTIVES

The current study used the deep machine learning approach to differentiate human blood specimens from cow, goat, and chicken blood stains based on cell morphology.

METHODS

A total of 1,955 known Giemsa-stained digitized images were acquired from the blood of humans, cows, goats, and chickens. To train the deep learning models, the well-known VGG16, Resnet18, and Resnet34 algorithms were used. Based on the image analysis, confusion matrices were generated.

RESULTS

Findings showed that the F1 score for the chicken, cow, goat, and human classes were all equal to 1.0 for each of the three algorithms. The Matthews correlation coefficient (MCC) was 1 for chickens, cows, and humans in all three algorithms, while the MCC score was 0.989 for goats by ResNet18, and it was 0.994 for both ResNet34 and VGG16 algorithms. The three algorithms showed 100% sensitivity, specificity, and positive and negative predictive values for the human, cow, and chicken cells. For the goat cells, the data showed 100% sensitivity and negative predictive values with specificity and positive predictive values ranging from 98.5% to 99.6%.

CONCLUSION

These data showed the importance of deep learning as a potential tool for the differentiation of the species of origin of fresh crime scene blood stains.

Social network analysis of cell networks improves deep learning for prediction of molecular pathways and key mutations in colorectal cancer

Colorectal cancer (CRC) is a primary global health concern, and identifying the molecular pathways, genetic subtypes, and mutations associated with CRC is crucial for precision medicine. However, traditional measurement techniques such as gene sequencing are costly and time-consuming, while most deep learning methods proposed for this task lack interpretability. This study offers a new approach to enhance the state-of-the-art deep learning methods for molecular pathways and key mutation prediction by incorporating cell network information. We build cell graphs with nuclei as nodes and nuclei connections as edges of the network and leverage Social Network Analysis (SNA) measures to extract abstract, perceivable, and interpretable features that explicitly describe the cell network characteristics in an image. Our approach does not rely on precise nuclei segmentation or feature extraction, is computationally efficient, and is easily scalable. In this study, we utilize the TCGA-CRC-DX dataset, comprising 499 patients and 502 diagnostic slides from primary colorectal tumours, sourced from 36 distinct medical centres in the United States. By incorporating the SNA features alongside deep features in two multiple instance learning frameworks, we demonstrate improved performance for chromosomal instability (CIN), hypermutated tumour (HM), TP53 gene, BRAF gene, and Microsatellite instability (MSI) status prediction tasks (2.4%-4% and 7-8.8% improvement in AUROC and AUPRC on average). Additionally, our method achieves outstanding performance on MSI prediction in an external PAIP dataset (99% AUROC and 98% AUPRC), demonstrating its generalizability. Our findings highlight the discrimination power of SNA features and how they can be beneficial to deep learning models' performance and provide insights into the correlation of cell network profiles with molecular pathways and key mutations.

Smartphone-based device for point-of-care diagnostics of pulmonary inflammation using convolutional neural networks (CNNs)

In pulmonary inflammation diseases, like COVID-19, lung involvement and inflammation determine the treatment regime. Respiratory inflammation is typically arisen due to the cytokine storm and the leakage of the vessels for immune cells recruitment. Currently, such a situation is detected by the clinical judgment of a specialist or precisely by a chest CT scan. However, the lack of accessibility to the CT machines in many poor medical centers as well as its expensive service, demands more accessible methods for fast and cheap detection of lung inflammation. Here, we have introduced a novel method for tracing the inflammation and lung involvement in patients with pulmonary inflammation, such as COVID-19, by a simple electrolyte detection in their sputum samples. The presence of the electrolyte in the sputum sample results in the fern-like structures after air-drying. These fern patterns are different in the CT positive and negative cases that are detected by an AI application on a smartphone and using a low-cost and portable mini-microscope. Evaluating 160 patient-derived sputum sample images, this method demonstrated an interesting accuracy of 95%, as confirmed by CT-scan results. This finding suggests that the method has the potential to serve as a promising and reliable approach for recognizing lung inflammatory diseases, such as COVID-19.

DiagSet: a dataset for prostate cancer histopathological image classification

Cancer diseases constitute one of the most significant societal challenges. In this paper, we introduce a novel histopathological dataset for prostate cancer detection. The proposed dataset, consisting of over 2.6 million tissue patches extracted from 430 fully annotated scans, 4675 scans with assigned binary diagnoses, and 46 scans with diagnoses independently provided by a group of histopathologists can be found at https://github.com/michalkoziarski/DiagSet . Furthermore, we propose a machine learning framework for detection of cancerous tissue regions and prediction of scan-level diagnosis, utilizing thresholding to abstain from the decision in uncertain cases. The proposed approach, composed of ensembles of deep neural networks operating on the histopathological scans at different scales, achieves 94.6% accuracy in patch-level recognition and is compared in a scan-level diagnosis with 9 human histopathologists showing high statistical agreement.

NuInsSeg: A fully annotated dataset for nuclei instance segmentation in H&E-stained histological images

In computational pathology, automatic nuclei instance segmentation plays an essential role in whole slide image analysis. While many computerized approaches have been proposed for this task, supervised deep learning (DL) methods have shown superior segmentation performances compared to classical machine learning and image processing techniques. However, these models need fully annotated datasets for training which is challenging to acquire, especially in the medical domain. In this work, we release one of the biggest fully manually annotated datasets of nuclei in Hematoxylin and Eosin (H&E)-stained histological images, called NuInsSeg. This dataset contains 665 image patches with more than 30,000 manually segmented nuclei from 31 human and mouse organs. Moreover, for the first time, we provide additional ambiguous area masks for the entire dataset. These vague areas represent the parts of the images where precise and deterministic manual annotations are impossible, even for human experts. The dataset and detailed step-by-step instructions to generate related segmentation masks are publicly available on the respective repositories.

The Detection of Neoplastic Cells Using Objective Cytomorphologic Parameters in Malignant Lymphoma

Pathologic evaluation is the most crucial method for diagnosing malignant lymphomas. However, there are no established diagnostic criteria for evaluating pathologic morphology. We manually circled cell nuclei in the lesions of 10 patients with diffuse large B-cell lymphoma (DLBCL), follicular lymphoma, and reactive lymphadenitis. Seventeen parameters related to nuclear shape, color, and other characteristics were measured. We attempted to compare the statistical differences between these subtypes and extract distinctive disease-specific populations on the basis of these parameters. Statistically significant differences were observed between the different types of lymphoma for many of the 17 parameters. Through t-distributed stochastic neighbor embedding analysis, we extracted a cluster of cells that showed distinctive features of DLBCL and were not found in follicular lymphoma or reactive lymphadenitis. We created a decision tree to identify the characteristics of the cells within that cluster. Based on a 5-fold cross-validation study, the average sensitivity, specificity, and accuracy obtained were 84.1%, 98.4%, and 97.3%, respectively. A similar result was achieved using a validation experiment. Important parameters that indicate the features of DLBCL include Area, ConcaveCount, MaxGray, and ModeGray. By quantifying pathologic morphology, it was possible to objectively represent the cell morphology specific to each lymphoma subtype using quantitative indicators. The quantified morphologic information has the potential to serve as a reproducible and flexible diagnostic tool.

PrecisionLymphoNet: Advancing Malignant Lymphoma Diagnosis via Ensemble Transfer Learning with CNNs

Malignant lymphoma, which impacts the lymphatic system, presents diverse challenges in accurate diagnosis due to its varied subtypes-chronic lymphocytic leukemia (CLL), follicular lymphoma (FL), and mantle cell lymphoma (MCL). Lymphoma is a form of cancer that begins in the lymphatic system, impacting lymphocytes, which are a specific type of white blood cell. This research addresses these challenges by proposing ensemble and non-ensemble transfer learning models employing pre-trained weights from VGG16, VGG19, DenseNet201, InceptionV3, and Xception. For the ensemble technique, this paper adopts a stack-based ensemble approach. It is a two-level classification approach and best suited for accuracy improvement. Testing on a multiclass dataset of CLL, FL, and MCL reveals exceptional diagnostic accuracy, with DenseNet201, InceptionV3, and Xception exceeding 90% accuracy. The proposed ensemble model, leveraging InceptionV3 and Xception, achieves an outstanding 99% accuracy over 300 epochs, surpassing previous prediction methods. This study demonstrates the feasibility and efficiency of the proposed approach, showcasing its potential in real-world medical applications for precise lymphoma diagnosis.

Transfer Learning with Pretrained Convolutional Neural Network for Automated Gleason Grading of Prostate Cancer Tissue Microarrays

Background

The Gleason grading system has been the most effective prediction for prostate cancer patients. This grading system provides this possibility to assess prostate cancer's aggressiveness and then constitutes an important factor for stratification and therapeutic decisions. However, determining Gleason grade requires highly-trained pathologists and is time-consuming and tedious, and suffers from inter-pathologist variability. To remedy these limitations, this paper introduces an automatic methodology based on transfer learning with pretrained convolutional neural networks (CNNs) for automatic Gleason grading of prostate cancer tissue microarray (TMA).

Methods

Fifteen pretrained (CNNs): Efficient Nets (B0-B5), NasNetLarge, NasNetMobile, InceptionV3, ResNet-50, SeResnet 50, Xception, DenseNet121, ResNext50, and inception_resnet_v2 were fine-tuned on a dataset of prostate carcinoma TMA images. Six pathologists separately identified benign and cancerous areas for each prostate TMA image by allocating benign, 3, 4, or 5 Gleason grade for 244 patients. The dataset was labeled by these pathologists and majority vote was applied on pixel-wise annotations to obtain a unified label.

Results

Results showed the NasnetLarge architecture is the best model among them in the classification of prostate TMA images of 244 patients with accuracy of 0.93 and area under the curve of 0.98.

Conclusion

Our study can act as a highly trained pathologist to categorize the prostate cancer stages with more objective and reproducible results.

Center-Focused Affinity Loss for Class Imbalance Histology Image Classification

Early-stage cancer diagnosis potentially improves the chances of survival for many cancer patients worldwide. Manual examination of Whole Slide Images (WSIs) is a time-consuming task for analyzing tumor-microenvironment. To overcome this limitation, the conjunction of deep learning with computational pathology has been proposed to assist pathologists in efficiently prognosing the cancerous spread. Nevertheless, the existing deep learning methods are ill-equipped to handle fine-grained histopathology datasets. This is because these models are constrained via conventional softmax loss function, which cannot expose them to learn distinct representational embeddings of the similarly textured WSIs containing an imbalanced data distribution. To address this problem, we propose a novel center-focused affinity loss (CFAL) function that exhibits 1) constructing uniformly distributed class prototypes in the feature space, 2) penalizing difficult samples, 3) minimizing intra-class variations, and 4) placing greater emphasis on learning minority class features. We evaluated the performance of the proposed CFAL loss function on two publicly available breast and colon cancer datasets having varying levels of imbalanced classes. The proposed CFAL function shows better discrimination abilities as compared to the popular loss functions such as ArcFace, CosFace, and Focal loss. Moreover, it outperforms several SOTA methods for histology image classification across both datasets.

Artificial intelligence-assisted digital pathology for non-alcoholic steatohepatitis: current status and future directions

The worldwide prevalence of non-alcoholic steatohepatitis (NASH) is increasing, causing a significant medical burden, but no approved therapeutics are currently available. NASH drug development requires histological analysis of liver biopsies by expert pathologists for trial enrolment and efficacy assessment, which can be hindered by multiple issues including sample heterogeneity, inter-reader and intra-reader variability, and ordinal scoring systems. Consequently, there is a high unmet need for accurate, reproducible, quantitative, and automated methods to assist pathologists with histological analysis to improve the precision around treatment and efficacy assessment. Digital pathology (DP) workflows in combination with artificial intelligence (AI) have been established in other areas of medicine and are being actively investigated in NASH to assist pathologists in the evaluation and scoring of NASH histology. DP/AI models can be used to automatically detect, localise, quantify, and score histological parameters and have the potential to reduce the impact of scoring variability in NASH clinical trials. This narrative review provides an overview of DP/AI tools in development for NASH, highlights key regulatory considerations, and discusses how these advances may impact the future of NASH clinical management and drug development. This should be a high priority in the NASH field, particularly to improve the development of safe and effective therapeutics.

Few-shot learning based oral cancer diagnosis using a dual feature extractor prototypical network

A large global health issue is cancer, wherein early diagnosis and treatment have proven to be life-saving. This holds true for oral cancer, thus emphasizing the significance of timely intervention. Deep learning techniques have gained traction in early cancer detection, exhibiting promising outcomes in accurate diagnosis. However, collecting a substantial amount of training data poses a challenge for deep learning models in cancer diagnosis. To address this limitation, this study proposes an oral cancer diagnosis approach based on a few-shot learning framework that circumvents the need for extensive training data. Specifically, a prototypical network is employed to construct a diagnostic model, wherein two feature extractors are utilized to extract prototypical features and query features respectively, departing from the conventional use of a single feature extraction function in prototypical networks. Moreover, a customized loss function is designed for the proposed method. Rigorous experimentation using a histopathological image dataset demonstrates the superior performance of our proposed approach over comparison methods.

Improving Representation Learning for Histopathologic Images with Cluster Constraints

Recent advances in whole-slide image (WSI) scanners and computational capabilities have significantly propelled the application of artificial intelligence in histopathology slide analysis. While these strides are promising, current supervised learning approaches for WSI analysis come with the challenge of exhaustively labeling high-resolution slides-a process that is both labor-intensive and timeconsuming. In contrast, self-supervised learning (SSL) pretraining strategies are emerging as a viable alternative, given that they don't rely on explicit data annotations. These SSL strategies are quickly bridging the performance disparity with their supervised counterparts. In this context, we introduce an SSL framework. This framework aims for transferable representation learning and semantically meaningful clustering by synergizing invariance loss and clustering loss in WSI analysis. Notably, our approach outperforms common SSL methods in downstream classification and clustering tasks, as evidenced by tests on the Camelyon16 and a pancreatic cancer dataset. The code and additional details are accessible at https://github.com/wwyi1828/CluSiam.

Use of deep learning for the classification of hyperplastic lymph node and common subtypes of canine lymphomas: a preliminary study

Artificial Intelligence has observed significant growth in its ability to classify different types of tumors in humans due to advancements in digital pathology technology. Among these tumors, lymphomas are quite common in dogs, despite studies on the application of AI in domestic species are scarce. This research aims to employ deep learning (DL) through convolutional neural networks (CNNs) to distinguish between normal lymph nodes and 3 WHO common subtypes of canine lymphomas. To train and validate the CNN, 1,530 high-resolution microscopic images derived from whole slide scans (WSIs) were used, including those of background areas, hyperplastic lymph nodes (n = 4), and three different lymphoma subtypes: diffuse large B cell lymphoma (DLBCL; n = 5), lymphoblastic (LBL; n = 5), and marginal zone lymphoma (MZL; n = 3). The CNN was able to correctly identify 456 images of the possible 457 test sets, achieving a maximum accuracy of 99.34%. The results of this study have demonstrated the feasibility of using deep learning to differentiate between hyperplastic lymph nodes and lymphomas, as well as to classify common WHO subtypes. Further research is required to explore the implications of these findings and validate the ability of the network to classify a broader range of lymphomas.

Improving Tumor-Infiltrating Lymphocytes Score Prediction in Breast Cancer with Self-Supervised Learning

Tumor microenvironment (TME) plays a pivotal role in immuno-oncology, which investigates the intricate interactions between tumors and the human immune system. Specifically, tumor-infiltrating lymphocytes (TILs) are crucial biomarkers for evaluating the prognosis of breast cancer patients and have the potential to refine immunotherapy precision and accurately identify tumor cells in specific cancer types. In this study, we conducted tissue segmentation and lymphocyte detection tasks to predict TIL scores by employing self-supervised learning (SSL) model-based approaches capable of addressing limited labeling data issues. Our experiments showed a 1.9% improvement in tissue segmentation and a 2% improvement in lymphocyte detection over the ImageNet pre-training model. Using these SSL-based models, we achieved a TIL score of 0.718 with a 4.4% improvement. In particular, when trained with only 10% of the entire dataset, the SwAV pre-trained model exhibited a superior performance over other models. Our work highlights improved tissue segmentation and lymphocyte detection using the SSL model with less labeled data for TIL score prediction.

Artificial intelligence in the neonatal intensive care unit: the time is now

Artificial intelligence (AI) has the potential to revolutionize the neonatal intensive care unit (NICU) care by leveraging the large-scale, high-dimensional data that are generated by NICU patients. There is an emerging recognition that the confluence of technological progress, commercialization pathways, and rich data sets provides a unique opportunity for AI to make a lasting impact on the NICU. In this perspective article, we discuss four broad categories of AI applications in the NICU: imaging interpretation, prediction modeling of electronic health record data, integration of real-time monitoring data, and documentation and billing. By enhancing decision-making, streamlining processes, and improving patient outcomes, AI holds the potential to transform the quality of care for vulnerable newborns, making the excitement surrounding AI advancements well-founded and the potential for significant positive change stronger than ever before.

Toward interpretable and generalized mitosis detection in digital pathology using deep learning

Objective

The mitotic activity index is an important prognostic factor in the diagnosis of cancer. The task of mitosis detection is difficult as the nuclei are microscopic in size and partially labeled, and there are many more non-mitotic nuclei compared to mitotic ones. In this paper, we highlight the challenges of current mitosis detection pipelines and propose a method to tackle these challenges.

Methods

Our proposed methodology is inspired from recent research on deep learning and an extensive analysis on the dataset and training pipeline. We first used the MiDoG'22 dataset for training, validation, and testing. We then tested the methodology without fine-tuning on the TUPAC'16 dataset and on a real-time case from Shaukat Khanum Memorial Cancer Hospital and Research Centre.

Results

Our methodology has shown promising results both quantitatively and qualitatively. Quantitatively, our methodology achieved an F1-score of 0.87 on the MiDoG'22 dataset and an F1-score of 0.83 on the TUPAC dataset. Qualitatively, our methodology is generalizable and interpretable across various datasets and clinical settings.

Conclusion

In this paper, we highlight the challenges of current mitosis detection pipelines and propose a method that can accurately predict mitotic nuclei. We illustrate the accuracy, generalizability, and interpretability of our approach across various datasets and clinical settings. Our methodology can speed up the adoption of computer-aided digital pathology in clinical settings.

Digital pathology in cardiac transplant diagnostics: from biopsies to algorithms

In the field of heart transplantation, the ability to accurately and promptly diagnose cardiac allograft rejection is crucial. This comprehensive review explores the transformative role of digital pathology and computational pathology, especially through machine learning, in this critical domain. These methodologies harness large datasets to extract subtle patterns and valuable information that extend beyond human perceptual capabilities, potentially enhancing diagnostic outcomes. Current research indicates that these computer-based systems could offer accuracy and performance matching, or even exceeding, that of expert pathologists, thereby introducing more objectivity and reducing observer variability. Despite promising results, several challenges such as limited sample sizes, diverse data sources, and the absence of standardized protocols pose significant barriers to the widespread adoption of these techniques. The future of digital pathology in heart transplantation diagnostics depends on utilizing larger, more diverse patient cohorts, standardizing data collection, processing, and evaluation protocols, and fostering collaborative research efforts. The integration of various data types, including clinical, demographic, and imaging information, could further refine diagnostic precision. As researchers address these challenges and promote collaborative efforts, digital pathology has the potential to become an integral part of clinical practice, ultimately improving patient care in heart transplantation.

Novel methodology for detecting and localizing cancer area in histopathological images based on overlapping patches

Cancer disease is one of the most important pathologies in the world, as it causes the death of millions of people, and the cure of this disease is limited in most cases. Rapid spread is one of the most important features of this disease, so many efforts are focused on its early-stage detection and localization. Medicine has made numerous advances in the recent decades with the help of artificial intelligence (AI), reducing costs and saving time. In this paper, deep learning models (DL) are used to present a novel method for detecting and localizing cancerous zones in WSI images, using tissue patch overlay to improve performance results. A novel overlapping methodology is proposed and discussed, together with different alternatives to evaluate the labels of the patches overlapping in the same zone to improve detection performance. The goal is to strengthen the labeling of different areas of an image with multiple overlapping patch testing. The results show that the proposed method improves the traditional framework and provides a different approach to cancer detection. The proposed method, based on applying 3x3 step 2 average pooling filters on overlapping patch labels, provides a better result with a 12.9% correction percentage for misclassified patches on the HUP dataset and 15.8% on the CINIJ dataset. In addition, a filter is implemented to correct isolated patches that were also misclassified. Finally, a CNN decision threshold study is performed to analyze the impact of the threshold value on the accuracy of the model. The alteration of the threshold decision along with the filter for isolated patches and the proposed method for overlapping patches, corrects about 20% of the patches that are mislabeled in the traditional method. As a whole, the proposed method achieves an accuracy rate of 94.6%. The code is available at https://github.com/sergioortiz26/Cancer_overlapping_filter_WSI_images.

Predicting nodal metastases in squamous cell carcinoma of the oral tongue using artificial intelligence

OBJECTIVE

The presence of occult nodal metastases in patients with squamous cell carcinoma (SCC) of the oral tongue has implications for treatment. Upwards of 30% of patients will have occult nodal metastases, yet a significant number of patients undergo unnecessary neck dissection to confirm nodal status. This study sought to predict the presence of nodal metastases in patients with SCC of the oral tongue using a convolutional neural network (CNN) that analyzed visual histopathology from the primary tumor alone.

METHODS

Cases of SCC of the oral tongue were identified from the records of a single institution. Only patients with complete pathology data were included in the study. The primary tumors were randomized into 2 groups for training and testing, which was performed at 2 different levels of supervision. Board-certified pathologists annotated each slide. HALO-AI convolutional neural network and image software was used to perform training and testing. Receiver operator characteristic (ROC) curves and the Youden J statistic were used for primary analysis.

RESULTS

Eighty-nine cases of SCC of the oral tongue were included in the study. The best performing algorithm had a high level of supervision and a sensitivity of 65% and specificity of 86% when identifying nodal metastases. The area under the curve (AUC) of the ROC curve for this algorithm was 0.729.

CONCLUSION

A CNN can produce an algorithm that is able to predict nodal metastases in patients with squamous cell carcinoma of the oral tongue by analyzing the visual histopathology of the primary tumor alone.