High-accuracy prostate cancer pathology using deep learning

Revolutionizing prostate cancer therapy: Artificial intelligence - Based nanocarriers for precision diagnosis and treatment

Prostate cancer is one of the major health challenges in the world and needs novel therapeutic approaches to overcome the limitations of conventional treatment. This review delineates the transformative potential of artificial intelligence (AL) in enhancing nanocarrier-based drug delivery systems for prostate cancer therapy. With its ability to optimize nanocarrier design and predict drug delivery kinetics, AI has revolutionized personalized treatment planning in oncology. We discuss how AI can be integrated with nanotechnology to address challenges related to tumor heterogeneity, drug resistance, and systemic toxicity. Emphasis is placed on strong AI-driven advancements in the design of nanocarriers, structural optimization, targeting of ligands, and pharmacokinetics. We also give an overview of how AI can better predict toxicity, reduce costs, and enable personalized medicine. While challenges persist in the way of data accessibility, regulatory hurdles, and interactions with the immune system, future directions based on explainable AI (XAI) models, integration of multimodal data, and green nanocarrier designs promise to move the field forward. Convergence between AI and nanotechnology has been one key step toward safer, more effective, and patient-tailored cancer therapy.

Self-interactive learning: Fusion and evolution of multi-scale histomorphology features for molecular traits prediction in computational pathology

Predicting disease-related molecular traits from histomorphology brings great opportunities for precision medicine. Despite the rich information present in histopathological images, extracting fine-grained molecular features from standard whole slide images (WSI) is non-trivial. The task is further complicated by the lack of annotations for subtyping and contextual histomorphological features that might span multiple scales. This work proposes a novel multiple-instance learning (MIL) framework capable of WSI-based cancer morpho-molecular subtyping by fusion of different-scale features. Our method, debuting as Inter-MIL, follows a weakly-supervised scheme. It enables the training of the patch-level encoder for WSI in a task-aware optimisation procedure, a step normally not modelled in most existing MIL-based WSI analysis frameworks. We demonstrate that optimising the patch-level encoder is crucial to achieving high-quality fine-grained and tissue-level subtyping results and offers a significant improvement over task-agnostic encoders. Our approach deploys a pseudo-label propagation strategy to update the patch encoder iteratively, allowing discriminative subtype features to be learned. This mechanism also empowers extracting fine-grained attention within image tiles (the small patches), a task largely ignored in most existing weakly supervised-based frameworks. With Inter-MIL, we carried out four challenging cancer molecular subtyping tasks in the context of ovarian, colorectal, lung, and breast cancer. Extensive evaluation results show that Inter-MIL is a robust framework for cancer morpho-molecular subtyping with superior performance compared to several recently proposed methods, in small dataset scenarios where the number of available training slides is less than 100. The iterative optimisation mechanism of Inter-MIL significantly improves the quality of the image features learned by the patch embedded and generally directs the attention map to areas that better align with experts' interpretation, leading to the identification of more reliable histopathology biomarkers. Moreover, an external validation cohort is used to verify the robustness of Inter-MIL on molecular trait prediction.

Comprehensive testing of large language models for extraction of structured data in pathology

BACKGROUND

Pathology departments generate large volumes of unstructured data as free-text diagnostic reports. Converting these reports into structured formats for analytics or artificial intelligence projects requires substantial manual effort by specialized personnel. While recent studies show promise in using advanced language models for structuring pathology data, they primarily rely on proprietary models, raising cost and privacy concerns. Additionally, important aspects such as prompt engineering and model quantization for deployment on consumer-grade hardware remain unaddressed.

METHODS

We created a dataset of 579 annotated pathology reports in German and English versions. Six language models (proprietary: GPT-4; open-source: Llama2 13B, Llama2 70B, Llama3 8B, Llama3 70B, and Qwen2.5 7B) were evaluated for their ability to extract eleven key parameters from these reports. Additionally, we investigated model performance across different prompt engineering strategies and model quantization techniques to assess practical deployment scenarios.

RESULTS

Here we show that open-source language models extract structured data from pathology reports with high precision, matching the accuracy of proprietary GPT-4 model. The precision varies significantly across different models and configurations. These variations depend on specific prompt engineering strategies and quantization methods used during model deployment.

CONCLUSIONS

Open-source language models demonstrate comparable performance to proprietary solutions in structuring pathology report data. This finding has significant implications for healthcare institutions seeking cost-effective, privacy-preserving data structuring solutions. The variations in model performance across different configurations provide valuable insights for practical deployment in pathology departments. Our publicly available bilingual dataset serves as both a benchmark and a resource for future research.

Colorectal cancer classification using weakly annotated whole slide images: Multiple instance learning optimization study

Colorectal cancer (CRC) is considered one of the most deadly cancer types nowadays. It is rapidly increasing due to many factors, such as unhealthy lifestyles, water and food pollution, aging, and medical diagnosis development. Detecting CRC in its early stages can help stop its growth by providing the necessary treatments, thereby saving many people's lives. There are various tests that doctors can perform to diagnose CRC; however, biopsy using histopathological images is considered the "gold standard" for CRC diagnosis. Deep learning techniques can now be leveraged to build computer-aided diagnosis (CAD) systems that can affirm if an input sample shows any symptoms of cancer and determine its stage and location with an acceptable degree of confidence. In this research, we utilize deep learning to study the CRC classification problem using weakly annotated histopathological whole slide images (WSIs). We relax the constraints of the multiple instance learning (MIL) algorithm and primarily propose WSI-label prediction functions to be integrated with MIL, which significantly enhances the performance of WSI-level classification. We also applied efficient preprocessing techniques that output a computationally power-efficient dataset representation and performed multiple experiments to compose the most efficient CAD system. Our study introduces a notable improvement over the results obtained by the baseline research where we achieved an accuracy of 93.05% compared to 84.17%. Furthermore, our results using only the weakly annotated WSIs outperformed the baseline results that are based on performing initial pre-training using a strongly annotated part of the dataset.

Context-guided segmentation for histopathologic cancer segmentation

Microscopic inspection of histologically stained tissue is considered as the gold standard for cancer diagnosis. This research is inspired by the practices of pathologists who analyze diagnostic samples by zooming in and out. We propose a dual-encoder model that simultaneously evaluates two views of the tissue at different levels of magnification. The lower magnification view provides contextual information for a target area, while the higher magnification view provides detailed information. The model consists of two encoder branches that consider both detail and context resolutions of the target area concurrently for binary pixel-wise segmentation. We introduce a unique weight initialization for the cross-attention between the context and detail feature tensors, allowing the model to incorporate contextual information. Our design is evaluated using the Camelyon16 dataset of sentinel lymph node tissue and cancer. The results demonstrate the benefit of including context regions when segmenting for cancer, with an improvement in AUC ranging from 0.31 to 0.92% and an improvement in cancer Dice score ranging from 4.09% to 6.81% compared to single detailed input models.

Stress testing deep learning models for prostate cancer detection on biopsies and surgical specimens

The presence, location, and extent of prostate cancer is assessed by pathologists using H&E-stained tissue slides. Machine learning approaches can accomplish these tasks for both biopsies and radical prostatectomies. Deep learning approaches using convolutional neural networks (CNNs) have been shown to identify cancer in pathologic slides, some securing regulatory approval for clinical use. However, differences in sample processing can subtly alter the morphology between sample types, making it unclear whether deep learning algorithms will consistently work on both types of slide images. Our goal was to investigate whether morphological differences between sample types affected the performance of biopsy-trained cancer detection CNN models when applied to radical prostatectomies and vice versa using multiple cohorts (N = 1,000). Radical prostatectomies (N = 100) and biopsies (N = 50) were acquired from The University of Pennsylvania to train (80%) and validate (20%) a DenseNet CNN for biopsies (MB), radical prostatectomies (MR), and a combined dataset (MB+R). On a tile level, MB and MR achieved F1 scores greater than 0.88 when applied to their own sample type but less than 0.65 when applied across sample types. On a whole-slide level, models achieved significantly better performance on their own sample type compared to the alternative model (p < 0.05) for all metrics. This was confirmed by external validation using digitized biopsy slide images from a clinical trial [NRG Radiation Therapy Oncology Group (RTOG)] (NRG/RTOG 0521, N = 750) via both qualitative and quantitative analyses (p < 0.05). A comprehensive review of model outputs revealed morphologically driven decision making that adversely affected model performance. MB appeared to be challenged with the analysis of open gland structures, whereas MR appeared to be challenged with closed gland structures, indicating potential morphological variation between the training sets. These findings suggest that differences in morphology and heterogeneity necessitate the need for more tailored, sample-specific (i.e. biopsy and surgical) machine learning models. © 2024 The Author(s). The Journal of Pathology published by John Wiley & Sons Ltd on behalf of The Pathological Society of Great Britain and Ireland.

Identification of genomic alteration and prognosis using pathomics-based artificial intelligence in oral leukoplakia and head and neck squamous cell carcinoma: a multicenter experimental study

BACKGROUND

Loss of chromosome 9p is an important biomarker in the malignant transformation of oral leukoplakia (OLK) to head and neck squamous cell carcinoma (HNSCC), and is associated with the prognosis of HNSCC patients. However, various challenges have prevented 9p loss from being assessed in clinical practice. The objective of this study was to develop a pathomics-based artificial intelligence (AI) model for the rapid and cost-effective prediction of 9p loss (9PLP).

MATERIALS AND METHODS

Three hundred thirty-three OLK cases were retrospectively collected with hematoxylin and eosin (H&E)-stained whole slide images and genomic alteration data from multicenter cohorts to develop the genomic alteration prediction AI model. They were divided into a training dataset ( n =217), a validation dataset ( n =93), and an external testing dataset ( n =23). The latest Transformer method and XGBoost algorithm were combined to develop the 9PLP model. The AI model was further applied and validated in two multicenter HNSCC datasets ( n =42 and n =365, respectively). Moreover, the combination of 9PLP with clinicopathological parameters was used to develop a nomogram model for assessing HNSCC patient prognosis.

RESULTS

9PLP could predict chromosome 9p loss rapidly and effectively using both OLK and HNSCC images, with the area under the curve achieving 0.890 and 0.825, respectively. Furthermore, the predictive model showed high accuracy in HNSCC patient prognosis assessment (the area under the curve was 0.739 for 1-year prediction, 0.705 for 3-year prediction, and 0.691 for 5-year prediction).

CONCLUSION

To the best of our knowledge, this study developed the first genomic alteration prediction deep learning model in OLK and HNSCC. This novel AI model could predict 9p loss and assess patient prognosis by identifying pathomics features in H&E-stained images with good performance. In the future, the 9PLP model may potentially contribute to better clinical management of OLK and HNSCC.

GrandQC: A comprehensive solution to quality control problem in digital pathology

Histological slides contain numerous artifacts that can significantly deteriorate the performance of image analysis algorithms. Here we develop the GrandQC tool for tissue and multi-class artifact segmentation. GrandQC allows for high-precision tissue segmentation (Dice score 0.957) and segmentation of tissue without artifacts (Dice score 0.919-0.938 dependent on magnification). Slides from 19 international pathology departments digitized with the most common scanning systems and from The Cancer Genome Atlas dataset were used to establish a QC benchmark, analyzing inter-institutional, intra-institutional, temporal, and inter-scanner slide quality variations. GrandQC improves the performance of downstream image analysis algorithms. We open-source the GrandQC tool, our large manually annotated test dataset, and all QC masks for the entire TCGA cohort to address the problem of QC in digital/computational pathology. GrandQC can be used as a tool to monitor sample preparation and scanning quality in pathology departments and help to track and eliminate major artifact sources.

Artificial intelligence-aided data mining of medical records for cancer detection and screening

The application of artificial intelligence methods to electronic patient records paves the way for large-scale analysis of multimodal data. Such population-wide data describing deep phenotypes composed of thousands of features are now being leveraged to create data-driven algorithms, which in turn has led to improved methods for early cancer detection and screening. Remaining challenges include establishment of infrastructures for prospective testing of such methods, ways to assess biases given the data, and gathering of sufficiently large and diverse datasets that reflect disease heterogeneities across populations. This Review provides an overview of artificial intelligence methods designed to detect cancer early, including key aspects of concern (eg, the problem of data drift-when the underlying health-care data change over time), ethical aspects, and discrepancies between access to cancer screening in high-income countries versus low-income and middle-income countries.

Computational pathology: A survey review and the way forward

Computational Pathology (CPath) is an interdisciplinary science that augments developments of computational approaches to analyze and model medical histopathology images. The main objective for CPath is to develop infrastructure and workflows of digital diagnostics as an assistive CAD system for clinical pathology, facilitating transformational changes in the diagnosis and treatment of cancer that are mainly address by CPath tools. With evergrowing developments in deep learning and computer vision algorithms, and the ease of the data flow from digital pathology, currently CPath is witnessing a paradigm shift. Despite the sheer volume of engineering and scientific works being introduced for cancer image analysis, there is still a considerable gap of adopting and integrating these algorithms in clinical practice. This raises a significant question regarding the direction and trends that are undertaken in CPath. In this article we provide a comprehensive review of more than 800 papers to address the challenges faced in problem design all-the-way to the application and implementation viewpoints. We have catalogued each paper into a model-card by examining the key works and challenges faced to layout the current landscape in CPath. We hope this helps the community to locate relevant works and facilitate understanding of the field's future directions. In a nutshell, we oversee the CPath developments in cycle of stages which are required to be cohesively linked together to address the challenges associated with such multidisciplinary science. We overview this cycle from different perspectives of data-centric, model-centric, and application-centric problems. We finally sketch remaining challenges and provide directions for future technical developments and clinical integration of CPath. For updated information on this survey review paper and accessing to the original model cards repository, please refer to GitHub. Updated version of this draft can also be found from arXiv.

Tumor architecture and emergence of strong genetic alterations are bottlenecks for clonal evolution in primary prostate cancer

Prostate cancer (PCA) exhibits high levels of intratumoral heterogeneity. In this study, we developed a mathematical model to study the growth and genetic evolution of PCA. We explored the possible evolutionary patterns and demonstrated that tumor architecture represents a major bottleneck for divergent clonal evolution. Early consecutive acquisition of strong genetic alterations serves as a proxy for the formation of aggressive tumors. A limited number of clonal hierarchy patterns were identified. A biopsy study of synthetic tumors shows complex spatial intermixing of clones and delineates the importance of biopsy extent. Deep whole-exome multiregional next-generation DNA sequencing of the primary tumors from five patients was performed to validate the results, supporting our main findings from mathematical modeling. In conclusion, our model provides qualitatively realistic predictions of PCA genomic evolution, closely aligned with the evidence available from patient samples. We share the code of the model for further studies. A record of this paper's transparent peer review process is included in the supplemental information.

Multiclass classification of Autism Spectrum Disorder, attention deficit hyperactivity disorder, and typically developed individuals using fMRI functional connectivity analysis

Neurodevelopmental conditions, such as Autism Spectrum Disorder (ASD) and Attention Deficit Hyperactivity Disorder (ADHD), present unique challenges due to overlapping symptoms, making an accurate diagnosis and targeted intervention difficult. Our study employs advanced machine learning techniques to analyze functional magnetic resonance imaging (fMRI) data from individuals with ASD, ADHD, and typically developed (TD) controls, totaling 120 subjects in the study. Leveraging multiclass classification (ML) algorithms, we achieve superior accuracy in distinguishing between ASD, ADHD, and TD groups, surpassing existing benchmarks with an area under the ROC curve near 98%. Our analysis reveals distinct neural signatures associated with ASD and ADHD: individuals with ADHD exhibit altered connectivity patterns of regions involved in attention and impulse control, whereas those with ASD show disruptions in brain regions critical for social and cognitive functions. The observed connectivity patterns, on which the ML classification rests, agree with established diagnostic approaches based on clinical symptoms. Furthermore, complex network analyses highlight differences in brain network integration and segregation among the three groups. Our findings pave the way for refined, ML-enhanced diagnostics in accordance with established practices, offering a promising avenue for developing trustworthy clinical decision-support systems.

Enhancing Prostate Cancer Diagnosis: Artificial Intelligence-Driven Virtual Biopsy for Optimal Magnetic Resonance Imaging-Targeted Biopsy Approach and Gleason Grading Strategy

An optimal approach to magnetic resonance imaging fusion targeted prostate biopsy (PBx) remains unclear (number of cores, intercore distance, Gleason grading [GG] principle). The aim of this study was to develop a precise pixel-wise segmentation diagnostic artificial intelligence (AI) algorithm for tumor detection and GG as well as an algorithm for virtual prostate biopsy that are used together to systematically investigate and find an optimal approach to targeted PBx. Pixel-wise AI algorithms for tumor detection and GG were developed using a high-quality, manually annotated data set (slides n = 442) after fast-track annotation transfer into segmentation style. To this end, a virtual biopsy algorithm was developed that can perform random biopsies from tumor regions in whole-mount whole-slide images with predefined parameters. A cohort of 115 radical prostatectomy (RP) patient cases with clinically significant, magnetic resonance imaging-visible tumors (n = 121) was used for systematic studies of the optimal biopsy approach. Three expert genitourinary (GU) pathologists (Y.T., A.P., A.Q.) participated in the validation. The tumor detection algorithm (aware version sensitivity/specificity 0.99/0.90, balanced version 0.97/0.97) and GG algorithm (quadratic kappa range vs pathologists 0.77-0.78) perform on par with expert GU pathologists. In total, 65,340 virtual biopsies were performed to study different biopsy approaches with the following results: (1) 4 biopsy cores is the optimal number for a targeted PBx, (2) cumulative GG strategy is superior to using maximal Gleason score for single cores, (3) controlling for minimal intercore distance does not improve the predictive accuracy for the RP Gleason score, (4) using tertiary Gleason pattern principle (for AI tool) in cumulative GG strategy might allow better predictions of final RP Gleason score. The AI algorithm (based on cumulative GG strategy) predicted the RP Gleason score of the tumor better than 2 of the 3 expert GU pathologists. In this study, using an original approach of virtual prostate biopsy on the real cohort of patient cases, we find the optimal approach to the biopsy procedure and the subsequent GG of a targeted PBx. We publicly release 2 large data sets with associated expert pathologists' GG and our virtual biopsy algorithm.

Diagnostic classification based on DNA methylation profiles using sequential machine learning approaches

Aberrant methylation patterns in human DNA have great potential for the discovery of novel diagnostic and disease progression biomarkers. In this paper we used machine learning algorithms to identify promising methylation sites for diagnosing cancerous tissue and to classify patients based on methylation values at these sites. We used genome-wide DNA methylation patterns from both cancerous and normal tissue samples, obtained from the Genomic Data Commons consortium and trialled our methods on three types of urological cancer. A decision tree was used to identify the methylation sites most useful for diagnosis. The identified locations were then used to train a neural network to classify samples as either cancerous or non-cancerous. Using this two-step approach we found strong indicative biomarker panels for each of the three cancer types. These methods could likely be translated to other cancers and improved by using non-invasive liquid methods such as blood instead of biopsy tissue.

Development and validation of a deep learning model for detecting signs of tuberculosis on chest radiographs among US-bound immigrants and refugees

Immigrants and refugees seeking admission to the United States must first undergo an overseas medical exam, overseen by the US Centers for Disease Control and Prevention (CDC), during which all persons ≥15 years old receive a chest x-ray to look for signs of tuberculosis. Although individual screening sites often implement quality control (QC) programs to ensure radiographs are interpreted correctly, the CDC does not currently have a method for conducting similar QC reviews at scale. We obtained digitized chest radiographs collected as part of the overseas immigration medical exam. Using radiographs from applicants 15 years old and older, we trained deep learning models to perform three tasks: identifying abnormal radiographs; identifying abnormal radiographs suggestive of tuberculosis; and identifying the specific findings (e.g., cavities or infiltrates) in abnormal radiographs. We then evaluated the models on both internal and external testing datasets, focusing on two classes of performance metrics: individual-level metrics, like sensitivity and specificity, and sample-level metrics, like accuracy in predicting the prevalence of abnormal radiographs. A total of 152,012 images (one image per applicant; mean applicant age 39 years) were used for model training. On our internal test dataset, our models performed well both in identifying abnormalities suggestive of TB (area under the curve [AUC] of 0.97; 95% confidence interval [CI]: 0.95, 0.98) and in estimating sample-level counts of the same (-2% absolute percentage error; 95% CIC: -8%, 6%). On the external test datasets, our models performed similarly well in identifying both generic abnormalities (AUCs ranging from 0.89 to 0.92) and those suggestive of TB (AUCs from 0.94 to 0.99). This performance was consistent across metrics, including those based on thresholded class predictions, like sensitivity, specificity, and F1 score. Strong performance relative to high-quality radiological reference standards across a variety of datasets suggests our models may make reliable tools for supporting chest radiography QC activities at CDC.

Learning generalizable AI models for multi-center histopathology image classification

Investigation of histopathology slides by pathologists is an indispensable component of the routine diagnosis of cancer. Artificial intelligence (AI) has the potential to enhance diagnostic accuracy, improve efficiency, and patient outcomes in clinical pathology. However, variations in tissue preparation, staining protocols, and histopathology slide digitization could result in over-fitting of deep learning models when trained on the data from only one center, thereby underscoring the necessity to generalize deep learning networks for multi-center use. Several techniques, including the use of grayscale images, color normalization techniques, and Adversarial Domain Adaptation (ADA) have been suggested to generalize deep learning algorithms, but there are limitations to their effectiveness and discriminability. Convolutional Neural Networks (CNNs) exhibit higher sensitivity to variations in the amplitude spectrum, whereas humans predominantly rely on phase-related components for object recognition. As such, we propose Adversarial fourIer-based Domain Adaptation (AIDA) which applies the advantages of a Fourier transform in adversarial domain adaptation. We conducted a comprehensive examination of subtype classification tasks in four cancers, incorporating cases from multiple medical centers. Specifically, the datasets included multi-center data for 1113 ovarian cancer cases, 247 pleural cancer cases, 422 bladder cancer cases, and 482 breast cancer cases. Our proposed approach significantly improved performance, achieving superior classification results in the target domain, surpassing the baseline, color augmentation and normalization techniques, and ADA. Furthermore, extensive pathologist reviews suggested that our proposed approach, AIDA, successfully identifies known histotype-specific features. This superior performance highlights AIDA's potential in addressing generalization challenges in deep learning models for multi-center histopathology datasets.

Combining Artificial Intelligence and Simplified Image Processing for the Automatic Detection of Mycobacterium tuberculosis in Acid-fast Stain : A Cross-institute Training and Validation Study

Tuberculosis (TB) poses a significant health threat in Taiwan, necessitating efficient detection methods. Traditional screening for acid-fast positive bacilli in acid-fast stain is time-consuming and prone to human error due to staining artifacts. To address this, we present an automated TB detection platform leveraging deep learning and image processing. Whole slide images from 2 hospitals were collected and processed on a high-performance system. The system utilizes an image processing technique to highlight red, rod-like regions and a modified EfficientNet model for binary classification of TB-positive regions. Our approach achieves a 97% accuracy in tile-based TB image classification, with minimal loss during the image processing step. By setting a 0.99 threshold, false positives are significantly reduced, resulting in a 94% detection rate when assisting pathologists, compared with 68% without artificial intelligence assistance. Notably, our system efficiently identifies artifacts and contaminants, addressing challenges in digital slide interpretation. Cross-hospital validation demonstrates the system's adaptability. The proposed artificial intelligence-assisted pipeline improves both detection rates and time efficiency, making it a promising tool for routine pathology work in TB detection.

A systematic comparison of deep learning methods for Gleason grading and scoring

Prostate cancer is the second most frequent cancer in men worldwide after lung cancer. Its diagnosis is based on the identification of the Gleason score that evaluates the abnormality of cells in glands through the analysis of the different Gleason patterns within tissue samples. The recent advancements in computational pathology, a domain aiming at developing algorithms to automatically analyze digitized histopathology images, lead to a large variety and availability of datasets and algorithms for Gleason grading and scoring. However, there is no clear consensus on which methods are best suited for each problem in relation to the characteristics of data and labels. This paper provides a systematic comparison on nine datasets with state-of-the-art training approaches for deep neural networks (including fully-supervised learning, weakly-supervised learning, semi-supervised learning, Additive-MIL, Attention-Based MIL, Dual-Stream MIL, TransMIL and CLAM) applied to Gleason grading and scoring tasks. The nine datasets are collected from pathology institutes and openly accessible repositories. The results show that the best methods for Gleason grading and Gleason scoring tasks are fully supervised learning and CLAM, respectively, guiding researchers to the best practice to adopt depending on the task to solve and the labels that are available.

A comprehensive survey on deep active learning in medical image analysis

Deep learning has achieved widespread success in medical image analysis, leading to an increasing demand for large-scale expert-annotated medical image datasets. Yet, the high cost of annotating medical images severely hampers the development of deep learning in this field. To reduce annotation costs, active learning aims to select the most informative samples for annotation and train high-performance models with as few labeled samples as possible. In this survey, we review the core methods of active learning, including the evaluation of informativeness and sampling strategy. For the first time, we provide a detailed summary of the integration of active learning with other label-efficient techniques, such as semi-supervised, self-supervised learning, and so on. We also summarize active learning works that are specifically tailored to medical image analysis. Additionally, we conduct a thorough comparative analysis of the performance of different AL methods in medical image analysis with experiments. In the end, we offer our perspectives on the future trends and challenges of active learning and its applications in medical image analysis. An accompanying paper list and code for the comparative analysis is available in https://github.com/LightersWang/Awesome-Active-Learning-for-Medical-Image-Analysis.

Multi-Scale Digital Pathology Patch-Level Prostate Cancer Grading Using Deep Learning: Use Case Evaluation of DiagSet Dataset

Prostate cancer remains a prevalent health concern, emphasizing the critical need for early diagnosis and precise treatment strategies to mitigate mortality rates. The accurate prediction of cancer grade is paramount for timely interventions. This paper introduces an approach to prostate cancer grading, framing it as a classification problem. Leveraging ResNet models on multi-scale patch-level digital pathology and the Diagset dataset, the proposed method demonstrates notable success, achieving an accuracy of 0.999 in identifying clinically significant prostate cancer. The study contributes to the evolving landscape of cancer diagnostics, offering a promising avenue for improved grading accuracy and, consequently, more effective treatment planning. By integrating innovative deep learning techniques with comprehensive datasets, our approach represents a step forward in the pursuit of personalized and targeted cancer care.

Fast-Track Development and Multi-Institutional Clinical Validation of an Artificial Intelligence Algorithm for Detection of Lymph Node Metastasis in Colorectal Cancer

Lymph node metastasis (LNM) detection can be automated using artificial intelligence (AI)-based diagnostic tools. Only limited studies have addressed this task for colorectal cancer (CRC). This study aimed to develop of a clinical-grade digital pathology tool for LNM detection in CRC using the original fast-track framework. The training cohort included 432 slides from one department. A segmentation algorithm detecting 8 relevant tissue classes was trained. The test cohorts consisted of materials from 5 pathology departments digitized by 4 different scanning systems. A high-quality, large training data set was generated within 7 days and a minimal amount of annotation work using fast-track principles. The AI tool showed very high accuracy for LNM detection in all cohorts, with sensitivity, negative predictive value, and specificity ranges of 0.980 to 1.000, 0.997 to 1.000, and 0.913 to 0.990, correspondingly. Only 5 of 14,460 analyzed test slides with tumor cells over all cohorts were classified as false negative (3/5 representing clusters of tumor cells in lymphatic vessels). A clinical-grade tool was trained in a short time using fast-track development principles and validated using the largest international, multi-institutional, multiscanner cohort of cases to date, showing very high precision for LNM detection in CRC. We are releasing a part of the test data sets to facilitate academic research.

A comprehensive AI model development framework for consistent Gleason grading

BACKGROUND

Artificial Intelligence(AI)-based solutions for Gleason grading hold promise for pathologists, while image quality inconsistency, continuous data integration needs, and limited generalizability hinder their adoption and scalability.

METHODS

We present a comprehensive digital pathology workflow for AI-assisted Gleason grading. It incorporates A!MagQC (image quality control), A!HistoClouds (cloud-based annotation), Pathologist-AI Interaction (PAI) for continuous model improvement, Trained on Akoya-scanned images only, the model utilizes color augmentation and image appearance migration to address scanner variations. We evaluate it on Whole Slide Images (WSI) from another five scanners and conduct validations with pathologists to assess AI efficacy and PAI.

RESULTS

Our model achieves an average F1 score of 0.80 on annotations and 0.71 Quadratic Weighted Kappa on WSIs for Akoya-scanned images. Applying our generalization solution increases the average F1 score for Gleason pattern detection from 0.73 to 0.88 on images from other scanners. The model accelerates Gleason scoring time by 43% while maintaining accuracy. Additionally, PAI improve annotation efficiency by 2.5 times and led to further improvements in model performance.

CONCLUSIONS

This pipeline represents a notable advancement in AI-assisted Gleason grading for improved consistency, accuracy, and efficiency. Unlike previous methods limited by scanner specificity, our model achieves outstanding performance across diverse scanners. This improvement paves the way for its seamless integration into clinical workflows.

Clinically Applicable Pan-Origin Cancer Detection for Lymph Nodes via Artificial Intelligence-Based Pathology

INTRODUCTION

Lymph node metastasis is one of the most common ways of tumour metastasis. The presence or absence of lymph node involvement influences the cancer's stage, therapy, and prognosis. The integration of artificial intelligence systems in the histopathological diagnosis of lymph nodes after surgery is urgent.

METHODS

Here, we propose a pan-origin lymph node cancer metastasis detection system. The system is trained by over 700 whole-slide images (WSIs) and is composed of two deep learning models to locate the lymph nodes and detect cancers.

RESULTS

It achieved an area under the receiver operating characteristic curve (AUC) of 0.958, with a 95.2% sensitivity and 72.2% specificity, on 1,402 WSIs from 49 organs at the National Cancer Center, China. Moreover, we demonstrated that the system could perform robustly with 1,051 WSIs from 52 organs from another medical centre, with an AUC of 0.925.

CONCLUSION

Our research represents a step forward in a pan-origin lymph node metastasis detection system, providing accurate pathological guidance by reducing the probability of missed diagnosis in routine clinical practice.

Don't Fear the Artificial Intelligence: A Systematic Review of Machine Learning for Prostate Cancer Detection in Pathology

CONTEXT

Automated prostate cancer detection using machine learning technology has led to speculation that pathologists will soon be replaced by algorithms. This review covers the development of machine learning algorithms and their reported effectiveness specific to prostate cancer detection and Gleason grading.

OBJECTIVE

To examine current algorithms regarding their accuracy and classification abilities. We provide a general explanation of the technology and how it is being used in clinical practice. The challenges to the application of machine learning algorithms in clinical practice are also discussed.

DATA SOURCES

The literature for this review was identified and collected using a systematic search. Criteria were established prior to the sorting process to effectively direct the selection of studies. A 4-point system was implemented to rank the papers according to their relevancy. For papers accepted as relevant to our metrics, all cited and citing studies were also reviewed. Studies were then categorized based on whether they implemented binary or multi-class classification methods. Data were extracted from papers that contained accuracy, area under the curve (AUC), or κ values in the context of prostate cancer detection. The results were visually summarized to present accuracy trends between classification abilities.

CONCLUSIONS

It is more difficult to achieve high accuracy metrics for multiclassification tasks than for binary tasks. The clinical implementation of an algorithm that can assign a Gleason grade to clinical whole slide images (WSIs) remains elusive. Machine learning technology is currently not able to replace pathologists but can serve as an important safeguard against misdiagnosis.

Harnessing artificial intelligence for prostate cancer management

Prostate cancer (PCa) is a common malignancy in males. The pathology review of PCa is crucial for clinical decision-making, but traditional pathology review is labor intensive and subjective to some extent. Digital pathology and whole-slide imaging enable the application of artificial intelligence (AI) in pathology. This review highlights the success of AI in detecting and grading PCa, predicting patient outcomes, and identifying molecular subtypes. We propose that AI-based methods could collaborate with pathologists to reduce workload and assist clinicians in formulating treatment recommendations. We also introduce the general process and challenges in developing AI pathology models for PCa. Importantly, we summarize publicly available datasets and open-source codes to facilitate the utilization of existing data and the comparison of the performance of different models to improve future studies.

Identification of agricultural surface source pollution in plain river network areas based on 3D-EEMs and convolutional neural networks

Agricultural non-point sources, as major sources of organic pollution, continue to flow into the river network area of the Jiangnan Plain, posing a serious threat to the quality of water bodies, the ecological environment, and human health. Therefore, there is an urgent need for a method that can accurately identify various types of agricultural organic pollution to prevent the water ecosystems in the region from significant organic pollution. In this study, a network model called RA-GoogLeNet is proposed for accurately identifying agricultural organic pollution in the river network area of the Jiangnan Plain. RA-GoogLeNet uses fluorescence spectral data of agricultural non-point source water quality in Changzhou Changdang Lake Basin, based on GoogLeNet architecture, and adds an efficient channel attention (ECA) mechanism to its A-Inception module, which enables the model to automatically learn the importance of independent channel features. ResNet are used to connect each A-Reception module. The experimental results show that RA-GoogLeNet performs well in fluorescence spectral classification of water quality, with an accuracy of 96.3%, which is 1.2% higher than the baseline model, and has good recall and F1 score. This study provides powerful technical support for the traceability of agricultural organic pollution.

Center-Focused Affinity Loss for Class Imbalance Histology Image Classification

Early-stage cancer diagnosis potentially improves the chances of survival for many cancer patients worldwide. Manual examination of Whole Slide Images (WSIs) is a time-consuming task for analyzing tumor-microenvironment. To overcome this limitation, the conjunction of deep learning with computational pathology has been proposed to assist pathologists in efficiently prognosing the cancerous spread. Nevertheless, the existing deep learning methods are ill-equipped to handle fine-grained histopathology datasets. This is because these models are constrained via conventional softmax loss function, which cannot expose them to learn distinct representational embeddings of the similarly textured WSIs containing an imbalanced data distribution. To address this problem, we propose a novel center-focused affinity loss (CFAL) function that exhibits 1) constructing uniformly distributed class prototypes in the feature space, 2) penalizing difficult samples, 3) minimizing intra-class variations, and 4) placing greater emphasis on learning minority class features. We evaluated the performance of the proposed CFAL loss function on two publicly available breast and colon cancer datasets having varying levels of imbalanced classes. The proposed CFAL function shows better discrimination abilities as compared to the popular loss functions such as ArcFace, CosFace, and Focal loss. Moreover, it outperforms several SOTA methods for histology image classification across both datasets.

Complete characterization of RNA biomarker fingerprints using a multi-modal ATR-FTIR and SERS approach for label-free early breast cancer diagnosis

Breast cancer is a prevalent form of cancer worldwide, and the current standard screening method, mammography, often requires invasive biopsy procedures for further assessment. Recent research has explored microRNAs (miRNAs) in circulating blood as potential biomarkers for early breast cancer diagnosis. In this study, we employed a multi-modal spectroscopy approach, combining attenuated total reflection Fourier transform infrared (ATR-FTIR) and surface-enhanced Raman scattering (SERS) to comprehensively characterize the full-spectrum fingerprints of RNA biomarkers in the blood serum of breast cancer patients. The sensitivity of conventional FTIR and Raman spectroscopy was enhanced by ATR-FTIR and SERS through the utilization of a diamond ATR crystal and silver-coated silicon nanopillars, respectively. Moreover, a wider measurement wavelength range was achieved with the multi-modal approach than with a single spectroscopic method alone. We have shown the results on 91 clinical samples, which comprised 44 malignant and 47 benign cases. Principal component analysis (PCA) was performed on the ATR-FTIR, SERS, and multi-modal data. From the peak analysis, we gained insights into biomolecular absorption and scattering-related features, which aid in the differentiation of malignant and benign samples. Applying 32 machine learning algorithms to the PCA results, we identified key molecular fingerprints and demonstrated that the multi-modal approach outperforms individual techniques, achieving higher average validation accuracy (95.1%), blind test accuracy (91.6%), specificity (94.7%), sensitivity (95.5%), and F-score (94.8%). The support vector machine (SVM) model showed the best area under the curve (AUC) characterization value of 0.9979, indicating excellent performance. These findings highlight the potential of the multi-modal spectroscopy approach as an accurate, reliable, and rapid method for distinguishing between malignant and benign breast tumors in women. Such a label-free approach holds promise for improving early breast cancer diagnosis and patient outcomes.

PRADclass: Hybrid Gleason Grade-Informed Computational Strategy Identifies Consensus Biomarker Features Predictive of Aggressive Prostate Adenocarcinoma

BACKGROUND

Prostate adenocarcinoma (PRAD) is a common cancer diagnosis among men globally, yet large gaps in our knowledge persist with respect to the molecular bases of its progression and aggression. It is mostly indolent and slow-growing, but aggressive prostate cancers need to be recognized early for optimising treatment, with a view to reducing mortality.

METHODS

Based on TCGA transcriptomic data pertaining to PRAD and the associated clinical metadata, we determined the sample Gleason grade, and used it to execute: (i) Gleason-grade wise linear modeling, followed by five contrasts against controls and ten contrasts between grades; and (ii) Gleason-grade wise network modeling via weighted gene correlation network analysis (WGCNA). Candidate biomarkers were obtained from the above analysis and the consensus found. The consensus biomarkers were used as the feature space to train ML models for classifying a sample as benign, indolent or aggressive.

RESULTS

The statistical modeling yielded 77 Gleason grade-salient genes while the WGCNA algorithm yielded 1003 trait-specific key genes in grade-wise significant modules. Consensus analysis of the two approaches identified two genes in Grade-1 (SLC43A1 and PHGR1), 26 genes in Grade-4 (including LOC100128675, PPP1R3C, NECAB1, UBXN10, SERPINA5, CLU, RASL12, DGKG, FHL1, NCAM1, and CEND1), and seven genes in Grade-5 (CBX2, DPYS, FAM72B, SHCBP1, TMEM132A, TPX2, UBE2C). A RandomForest model trained and optimized on these 35 biomarkers for the ternary classification problem yielded a balanced accuracy ∼ 86% on external validation.

CONCLUSIONS

The consensus of multiple parallel computational strategies has unmasked candidate Gleason grade-specific biomarkers. PRADclass, a validated AI model featurizing these biomarkers achieved good performance, and could be trialed to predict the differentiation of prostate cancers. PRADclass is available for academic use at: https://apalania.shinyapps.io/pradclass (online) and https://github.com/apalania/pradclass (command-line interface).

Artificial Intelligence-Based Tool for Tumor Detection and Quantitative Tissue Analysis in Colorectal Specimens

Digital pathology adoption allows for applying computational algorithms to routine pathology tasks. Our study aimed to develop a clinical-grade artificial intelligence (AI) tool for precise multiclass tissue segmentation in colorectal specimens (resections and biopsies) and clinically validate the tool for tumor detection in biopsy specimens. The training data set included 241 precisely manually annotated whole-slide images (WSIs) from multiple institutions. The algorithm was trained for semantic segmentation of 11 tissue classes with an additional module for biopsy WSI classification. Six case cohorts from 5 pathology departments (4 countries) were used for formal and clinical validation, digitized by 4 different scanning systems. The developed algorithm showed high precision of segmentation of different tissue classes in colorectal specimens with composite multiclass Dice score of up to 0.895 and pixel-wise tumor detection specificity and sensitivity of up to 0.958 and 0.987, respectively. In the clinical validation study on multiple external cohorts, the AI tool reached sensitivity of 1.0 and specificity of up to 0.969 for tumor detection in biopsy WSI. The AI tool analyzes most biopsy cases in less than 1 minute, allowing effective integration into clinical routine. We developed and extensively validated a highly accurate, clinical-grade tool for assistive diagnostic processing of colorectal specimens. This tool allows for quantitative deciphering of colorectal cancer tissue for development of prognostic and predictive biomarkers and personalization of oncologic care. This study is a foundation for a SemiCOL computational challenge. We open-source multiple manually annotated and weakly labeled test data sets, representing a significant contribution to the colorectal cancer computational pathology field.

Deep Learning Can Predict Bevacizumab Therapeutic Effect and Microsatellite Instability Directly from Histology in Epithelial Ovarian Cancer

Epithelial ovarian cancer (EOC) remains a significant cause of mortality among gynecologic cancers, with the majority of cases being diagnosed at an advanced stage. Before targeted therapies were available, EOC treatment relied largely on debulking surgery and platinum-based chemotherapy. Vascular endothelial growth factors have been identified as inducing tumor angiogenesis. According to several clinical trials, anti-vascular endothelial growth factor-targeted therapy with bevacizumab was effective in all phases of EOC treatment. However, there are currently no biomarkers accessible for regular therapeutic use despite the importance of patient selection. Microsatellite instability (MSI), caused by a deficiency of the DNA mismatch repair system, is a molecular abnormality observed in EOC associated with Lynch syndrome. Recent evidence suggests that angiogenesis and MSI are interconnected. Developing predictive biomarkers, which enable the selection of patients who might benefit from bevacizumab-targeted therapy or immunotherapy, is critical for realizing personalized precision medicine. In this study, we developed 2 improved deep learning methods that eliminate the need for laborious detailed image-wise annotations by pathologists and compared them with 3 state-of-the-art methods to not only predict the efficacy of bevacizumab in patients with EOC using mismatch repair protein immunostained tissue microarrays but also predict MSI status directly from histopathologic images. In prediction of therapeutic outcomes, the 2 proposed methods achieved excellent performance by obtaining the highest mean sensitivity and specificity score using MSH2 or MSH6 markers and outperformed 3 state-of-the-art deep learning methods. Moreover, both statistical analysis results, using Cox proportional hazards model analysis and Kaplan-Meier progression-free survival analysis, confirm that the 2 proposed methods successfully differentiate patients with positive therapeutic effects and lower cancer recurrence rates from patients experiencing disease progression after treatment (P < .01). In prediction of MSI status directly from histopathology images, our proposed method also achieved a decent performance in terms of mean sensitivity and specificity score even for imbalanced data sets for both internal validation using tissue microarrays from the local hospital and external validation using whole section slides from The Cancer Genome Atlas archive.

A Prostate-Specific Membrane Antigen PET-Based Approach for Improved Diagnosis of Prostate Cancer in Gleason Grade Group 1: A Multicenter Retrospective Study

The preoperative Gleason grade group (GG) from transrectal ultrasound-guided prostate biopsy is crucial for treatment decisions but may underestimate the postoperative GG and miss clinically significant prostate cancer (csPCa), particularly in patients with biopsy GG1. In such patients, an SUVmax of at least 12 has 100% specificity for detecting csPCa. In patients with an SUVmax of less than 12, we aimed to develop a model to improve the diagnostic accuracy of csPCa. Methods: The study retrospectively included 56 prostate cancer patients with transrectal ultrasound-guided biopsy GG1 and an SUVmax of less than 12 from 2 tertiary hospitals. All [68Ga]Ga-PSMA-HBED-CC PET scans were centrally reviewed in Xijing Hospital. A deep learning model was used to evaluate the overlap of SUVmax (size scale, 3 cm) and the level of Gleason pattern (size scale, 500 μm). A diagnostic model was developed using the PRIMARY score and SUVmax, and its discriminative performance and clinical utility were compared with other methods. The 5-fold cross-validation (repeated 1,000 times) was used for internal validation. Results: In patients with GG1 and an SUVmax of less than 12, significant prostate-specific membrane antigen (PSMA) histochemical score (H-score) H-score overlap occurred among benign gland, Gleason pattern 3, and Gleason pattern 4 lesions, causing SUVmax overlap between csPCa and non-csPCa. The model of 10 × PRIMARY score + 2 × SUVmax exhibited a higher area under the curve (AUC, 0.8359; 95% CI, 0.7233-0.9484) than that found using only the SUVmax (AUC, 0.7353; P = 0.048) or PRIMARY score (AUC, 0.7257; P = 0.009) for the cohort and a higher AUC (0.8364; 95% CI, 0.7114-0.9614) than that found using only the Prostate Imaging Reporting and Data System (PI-RADS) score of 5-4 versus 3-1 (AUC, 0.7036; P = 0.149) and the PI-RADS score of 5-3 versus 2-1 (AUC, 0.6373; P = 0.014) for a subgroup. The model reduced the misdiagnosis of the PI-RADS score of 5-4 versus 3-1 by 78.57% (11/14) and the PI-RADS score of 5-3 versus 2-1 by 77.78% (14/18). The internal validation showed that the mean 5-fold cross-validated AUC was 0.8357 (95% CI, 0.8357-0.8358). Conclusion: We preliminarily suggest that the model of 10 × PRIMARY score + 2 × SUVmax may enhance the diagnostic accuracy of csPCa in patients with biopsy GG1 and an SUVmax of less than 12 by maximizing PSMA information use, reducing the misdiagnosis of the PI-RADS score, and thereby aiding in making appropriate treatment decisions.

Analysis of functional connectivity using machine learning and deep learning in different data modalities from individuals with schizophrenia

Objective. Schizophrenia(SCZ) is a severe mental disorder associated with persistent or recurrent psychosis, hallucinations, delusions, and thought disorders that affect approximately 26 million people worldwide, according to the World Health Organization. Several studies encompass machine learning (ML) and deep learning algorithms to automate the diagnosis of this mental disorder. Others study SCZ brain networks to get new insights into the dynamics of information processing in individuals suffering from the condition. In this paper, we offer a rigorous approach with ML and deep learning techniques for evaluating connectivity matrices and measures of complex networks to establish an automated diagnosis and comprehend the topology and dynamics of brain networks in SCZ individuals.Approach.For this purpose, we employed an functional magnetic resonance imaging (fMRI) and electroencephalogram (EEG) dataset. In addition, we combined EEG measures, i.e. Hjorth mobility and complexity, with complex network measurements to be analyzed in our model for the first time in the literature.Main results.When comparing the SCZ group to the control group, we found a high positive correlation between the left superior parietal lobe and the left motor cortex and a positive correlation between the left dorsal posterior cingulate cortex and the left primary motor. Regarding complex network measures, the diameter, which corresponds to the longest shortest path length in a network, may be regarded as a biomarker because it is the most crucial measure in different data modalities. Furthermore, the SCZ brain networks exhibit less segregation and a lower distribution of information. As a result, EEG measures outperformed complex networks in capturing the brain alterations associated with SCZ.Significance. Our model achieved an area under receiver operating characteristic curve (AUC) of 100% and an accuracy of 98.5% for the fMRI, an AUC of 95%, and an accuracy of 95.4% for the EEG data set. These are excellent classification results. Furthermore, we investigated the impact of specific brain connections and network measures on these results, which helped us better describe changes in the diseased brain.

ChampKit: A framework for rapid evaluation of deep neural networks for patch-based histopathology classification

BACKGROUND AND OBJECTIVE

Histopathology is the gold standard for diagnosis of many cancers. Recent advances in computer vision, specifically deep learning, have facilitated the analysis of histopathology images for many tasks, including the detection of immune cells and microsatellite instability. However, it remains difficult to identify optimal models and training configurations for different histopathology classification tasks due to the abundance of available architectures and the lack of systematic evaluations. Our objective in this work is to present a software tool that addresses this need and enables robust, systematic evaluation of neural network models for patch classification in histology in a light-weight, easy-to-use package for both algorithm developers and biomedical researchers.

METHODS

Here we present ChampKit (Comprehensive Histopathology Assessment of Model Predictions toolKit): an extensible, fully reproducible evaluation toolkit that is a one-stop-shop to train and evaluate deep neural networks for patch classification. ChampKit curates a broad range of public datasets. It enables training and evaluation of models supported by timm directly from the command line, without the need for users to write any code. External models are enabled through a straightforward API and minimal coding. As a result, Champkit facilitates the evaluation of existing and new models and deep learning architectures on pathology datasets, making it more accessible to the broader scientific community. To demonstrate the utility of ChampKit, we establish baseline performance for a subset of possible models that could be employed with ChampKit, focusing on several popular deep learning models, namely ResNet18, ResNet50, and R26-ViT, a hybrid vision transformer. In addition, we compare each model trained either from random weight initialization or with transfer learning from ImageNet pretrained models. For ResNet18, we also consider transfer learning from a self-supervised pretrained model.

RESULTS

The main result of this paper is the ChampKit software. Using ChampKit, we were able to systemically evaluate multiple neural networks across six datasets. We observed mixed results when evaluating the benefits of pretraining versus random intialization, with no clear benefit except in the low data regime, where transfer learning was found to be beneficial. Surprisingly, we found that transfer learning from self-supervised weights rarely improved performance, which is counter to other areas of computer vision.

CONCLUSIONS

Choosing the right model for a given digital pathology dataset is nontrivial. ChampKit provides a valuable tool to fill this gap by enabling the evaluation of hundreds of existing (or user-defined) deep learning models across a variety of pathology tasks. Source code and data for the tool are freely accessible at https://github.com/SBU-BMI/champkit.

An international multi-institutional validation study of the algorithm for prostate cancer detection and Gleason grading

Pathologic examination of prostate biopsies is time consuming due to the large number of slides per case. In this retrospective study, we validate a deep learning-based classifier for prostate cancer (PCA) detection and Gleason grading (AI tool) in biopsy samples. Five external cohorts of patients with multifocal prostate biopsy were analyzed from high-volume pathology institutes. A total of 5922 H&E sections representing 7473 biopsy cores from 423 patient cases (digitized using three scanners) were assessed concerning tumor detection. Two tumor-bearing datasets (core n = 227 and 159) were graded by an international group of pathologists including expert urologic pathologists (n = 11) to validate the Gleason grading classifier. The sensitivity, specificity, and NPV for the detection of tumor-bearing biopsies was in a range of 0.971-1.000, 0.875-0.976, and 0.988-1.000, respectively, across the different test cohorts. In several biopsy slides tumor tissue was correctly detected by the AI tool that was initially missed by pathologists. Most false positive misclassifications represented lesions suspicious for carcinoma or cancer mimickers. The quadratically weighted kappa levels for Gleason grading agreement for single pathologists was 0.62-0.80 (0.77 for AI tool) and 0.64-0.76 (0.72 for AI tool) for the two grading datasets, respectively. In cases where consensus for grading was reached among pathologists, kappa levels for AI tool were 0.903 and 0.855. The PCA detection classifier showed high accuracy for PCA detection in biopsy cases during external validation, independent of the institute and scanner used. High levels of agreement for Gleason grading were indistinguishable between experienced genitourinary pathologists and the AI tool.

Artificial Intelligence Predictive Model for Hormone Therapy Use in Prostate Cancer

BACKGROUND: Androgen deprivation therapy (ADT) with radiotherapy can benefit patients with localized prostate cancer. However, ADT can negatively impact quality of life, and there remain no validated predictive models to guide its use. METHODS: We used digital pathology images from pretreatment prostate tissue and clinical data from 5727 patients enrolled in five phase 3 randomized trials, in which treatment was radiotherapy with or without ADT, as our data source to develop and validate an artificial intelligence (AI)–derived predictive patient-specific model that would determine which patients would develop the primary end point of distant metastasis. The model used baseline data to provide a binary output that a given patient will likely benefit from ADT or not. After the model was locked, validation was performed using data from NRG Oncology/Radiation Therapy Oncology Group (RTOG) 9408 (n=1594), a trial that randomly assigned men to radiotherapy plus or minus 4 months of ADT. Fine–Gray regression and restricted mean survival times were used to assess the interaction between treatment and the predictive model and within predictive model–positive, i.e., benefited from ADT, and –negative subgroup treatment effects. RESULTS: Overall, in the NRG/RTOG 9408 validation cohort (14.9 years of median follow-up), ADT significantly improved time to distant metastasis. Of these enrolled patients, 543 (34%) were model positive, and ADT significantly reduced the risk of distant metastasis compared with radiotherapy alone. Of 1051 patients who were model negative, ADT did not provide benefit. CONCLUSIONS: Our AI-based predictive model was able to identify patients with a predominantly intermediate risk for prostate cancer likely to benefit from short-term ADT. (Supported by a grant [U10CA180822] from NRG Oncology Statistical and Data Management Center, a grant [UG1CA189867] from NCI Community Oncology Research Program, a grant [U10CA180868] from NRG Oncology Operations, and a grant [U24CA196067] from NRG Specimen Bank from the National Cancer Institute and by Artera, Inc. ClinicalTrials.gov numbers NCT00767286, NCT00002597, NCT00769548, NCT00005044, and NCT00033631.)

Deep learning-based diagnosis of histopathological patterns for invasive non-mucinous lung adenocarcinoma using semantic segmentation

OBJECTIVES

The application of artificial intelligence (AI) to the field of pathology has facilitated the development of digital pathology, hence, making AI-assisted diagnosis possible. Due to the variety of lung cancers and the subjectivity of manual evaluation, invasive non-mucinous lung adenocarcinoma (ADC) is difficult to diagnose. We aim to offer a deep learning solution that automatically classifies invasive non-mucinous lung ADC histological subtypes.

DESIGN

For this investigation, 523 whole-slide images (WSIs) were obtained. We divided 376 of the WSIs at random for model training. According to WHO diagnostic criteria, six histological components of invasive non-mucinous lung ADC, comprising lepidic, papillary, acinar, solid, micropapillary and cribriform arrangements, were annotated at the pixel level and employed as the predicting target. We constructed the deep learning model using DeepLab v3, and used 27 WSIs for model validation and the remaining 120 WSIs for testing. The predictions were analysed by senior pathologists.

RESULTS

The model could accurately predict the predominant subtype and the majority of minor subtypes and has achieved good performance. Except for acinar, the area under the curve of the model was larger than 0.8 for all the subtypes. Meanwhile, the model was able to generate pathological reports. The NDCG scores were greater than 75%. Through the analysis of feature maps and incidents of model misdiagnosis, we discovered that the deep learning model was consistent with the thought process of pathologists and revealed better performance in recognising minor lesions.

CONCLUSIONS

The findings of the deep learning model for predicting the major and minor subtypes of invasive non-mucinous lung ADC are favourable. Its appearance and sensitivity to tiny lesions can be of great assistance to pathologists.

High accuracy epidermal growth factor receptor mutation prediction via histopathological deep learning

BACKGROUND

The detection of epidermal growth factor receptor (EGFR) mutations in patients with non-small cell lung cancer is critical for tyrosine kinase inhibitor therapy. EGFR detection requires tissue samples, which are difficult to obtain in some patients, costing them the opportunity for further treatment. To realize EGFR mutation prediction without molecular detection, we aimed to build a high-accuracy deep learning model with only haematoxylin and eosin (H&E)-stained slides.

METHODS

We collected 326 H&E-stained non-small cell lung cancer slides from Beijing Chest Hospital, China, and used 226 slides (88 with EGFR mutations) for model training. The remaining 100 images (50 with EGFR mutations) were used for testing. We trained a convolutional neural network based on ResNet-50 to classify EGFR mutation status on the slide level.

RESULTS

The sensitivity and specificity of the model were 76% and 74%, respectively, with an area under the curve of 0.82. When applying the double-threshold approach, 33% of the patients could be predicted by the deep learning model as EGFR positive or negative with a sensitivity and specificity of 100.0% and 87.5%. The remaining 67% of the patients got an uncertain result and will be recommenced to perform further examination. By incorporating adenocarcinoma subtype information, we achieved 100% sensitivity in predicting EGFR mutations in 37.3% of adenocarcinoma patients.

CONCLUSIONS

Our study demonstrates the potential of a deep learning-based EGFR mutation prediction model for rapid and cost-effective pre-screening. It could serve as a high-accuracy complement to current molecular detection methods and provide treatment opportunities for non-small cell lung cancer patients from whom limited samples are available.

Diagnosis of autism spectrum disorder based on functional brain networks and machine learning

Autism is a multifaceted neurodevelopmental condition whose accurate diagnosis may be challenging because the associated symptoms and severity vary considerably. The wrong diagnosis can affect families and the educational system, raising the risk of depression, eating disorders, and self-harm. Recently, many works have proposed new methods for the diagnosis of autism based on machine learning and brain data. However, these works focus on only one pairwise statistical metric, ignoring the brain network organization. In this paper, we propose a method for the automatic diagnosis of autism based on functional brain imaging data recorded from 500 subjects, where 242 present autism spectrum disorder considering the regions of interest throughout Bootstrap Analysis of Stable Cluster map. Our method can distinguish the control group from autism spectrum disorder patients with high accuracy. Indeed the best performance provides an AUC near 1.0, which is higher than that found in the literature. We verify that the left ventral posterior cingulate cortex region is less connected to an area in the cerebellum of patients with this neurodevelopment disorder, which agrees with previous studies. The functional brain networks of autism spectrum disorder patients show more segregation, less distribution of information across the network, and less connectivity compared to the control cases. Our workflow provides medical interpretability and can be used on other fMRI and EEG data, including small data sets.

Artificial intelligence for tumour tissue detection and histological regression grading in oesophageal adenocarcinomas: a retrospective algorithm development and validation study

BACKGROUND

Oesophageal adenocarcinoma and adenocarcinoma of the oesophagogastric junction are among the most common malignant epithelial tumours. Most patients receive neoadjuvant therapy before complete tumour resection. Histological assessment after resection includes identification of residual tumour tissue and areas of regressive tumour, data which are used to calculate a clinically relevant regression score. We developed an artificial intelligence (AI) algorithm for tumour tissue detection and tumour regression grading in surgical specimens from patients with oesophageal adenocarcinoma or adenocarcinoma of the oesophagogastric junction.

METHODS

We used one training cohort and four independent test cohorts to develop, train, and validate a deep learning tool. The material consisted of histological slides from surgically resected specimens from patients with oesophageal adenocarcinoma and adenocarcinoma of the oesophagogastric junction from three pathology institutes (two in Germany, one in Austria) and oesophageal cancer cohort of The Cancer Genome Atlas (TCGA). All slides were from neoadjuvantly treated patients except for those from the TCGA cohort, who were neoadjuvant-therapy naive. Data from training cohort and test cohort cases were extensively manually annotated for 11 tissue classes. A convolutional neural network was trained on the data using a supervised principle. First, the tool was formally validated using manually annotated test datasets. Next, tumour regression grading was assessed in a retrospective cohort of post-neoadjuvant therapy surgical specimens. The grading of the algorithm was compared with that of a group of 12 board-certified pathologists from one department. To further validate the tool, three pathologists processed whole resection cases with and without AI assistance.

FINDINGS

Of the four test cohorts, one included 22 manually annotated histological slides (n=20 patients), one included 62 sides (n=15), one included 214 slides (n=69), and the final one included 22 manually annotated histological slides (n=22). In the independent test cohorts the AI tool had high patch-level accuracy for identifying both tumour and regression tissue. When we validated the concordance of the AI tool against analyses by a group of pathologists (n=12), agreement was 63·6% (quadratic kappa 0·749; p<0·0001) at case level. The AI-based regression grading triggered true reclassification of resected tumour slides in seven cases (including six cases who had small tumour regions that were initially missed by pathologists). Use of the AI tool by three pathologists increased interobserver agreement and substantially reduced diagnostic time per case compared with working without AI assistance.

INTERPRETATION

Use of our AI tool in the diagnostics of oesophageal adenocarcinoma resection specimens by pathologists increased diagnostic accuracy, interobserver concordance, and significantly reduced assessment time. Prospective validation of the tool is required.

FUNDING

North Rhine-Westphalia state, Federal Ministry of Education and Research of Germany, and the Wilhelm Sander Foundation.

Artificial Intelligence Predictive Model for Hormone Therapy Use in Prostate Cancer

Background

Androgen deprivation therapy (ADT) with radiotherapy can benefit patients with localized prostate cancer. However, ADT can negatively impact quality of life and there remain no validated predictive models to guide its use.

Methods

Digital pathology image and clinical data from pre-treatment prostate tissue from 5,727 patients enrolled on five phase III randomized trials treated with radiotherapy +/- ADT were used to develop and validate an artificial intelligence (AI)-derived predictive model to assess ADT benefit with the primary endpoint of distant metastasis. After the model was locked, validation was performed on NRG/RTOG 9408 (n = 1,594) that randomized men to radiotherapy +/- 4 months of ADT. Fine-Gray regression and restricted mean survival times were used to assess the interaction between treatment and predictive model and within predictive model positive and negative subgroup treatment effects.

Results

In the NRG/RTOG 9408 validation cohort (14.9 years of median follow-up), ADT significantly improved time to distant metastasis (subdistribution hazard ratio [sHR] = 0.64, 95%CI [0.45-0.90], p = 0.01). The predictive model-treatment interaction was significant (p-interaction = 0.01). In predictive model positive patients (n = 543, 34%), ADT significantly reduced the risk of distant metastasis compared to radiotherapy alone (sHR = 0.34, 95%CI [0.19-0.63], p < 0.001). There were no significant differences between treatment arms in the predictive model negative subgroup (n = 1,051, 66%; sHR = 0.92, 95%CI [0.59-1.43], p = 0.71).

Conclusions

Our data, derived and validated from completed randomized phase III trials, show that an AI-based predictive model was able to identify prostate cancer patients, with predominately intermediate-risk disease, who are likely to benefit from short-term ADT.

A comparative study of the inter-observer variability on Gleason grading against Deep Learning-based approaches for prostate cancer

BACKGROUND

Among all the cancers known today, prostate cancer is one of the most commonly diagnosed in men. With modern advances in medicine, its mortality has been considerably reduced. However, it is still a leading type of cancer in terms of deaths. The diagnosis of prostate cancer is mainly conducted by biopsy test. From this test, Whole Slide Images are obtained, from which pathologists diagnose the cancer according to the Gleason scale. Within this scale from 1 to 5, grade 3 and above is considered malignant tissue. Several studies have shown an inter-observer discrepancy between pathologists in assigning the value of the Gleason scale. Due to the recent advances in artificial intelligence, its application to the computational pathology field with the aim of supporting and providing a second opinion to the professional is of great interest.

METHOD

In this work, the inter-observer variability of a local dataset of 80 whole-slide images annotated by a team of 5 pathologists from the same group was analyzed at both area and label level. Four approaches were followed to train six different Convolutional Neural Network architectures, which were evaluated on the same dataset on which the inter-observer variability was analyzed.

RESULTS

An inter-observer variability of 0.6946 κ was obtained, with 46% discrepancy in terms of area size of the annotations performed by the pathologists. The best trained models achieved 0.826±0.014κ on the test set when trained with data from the same source.

CONCLUSIONS

The obtained results show that deep learning-based automatic diagnosis systems could help reduce the widely-known inter-observer variability that is present among pathologists and support them in their decision, serving as a second opinion or as a triage tool for medical centers.

A review and comparative study of cancer detection using machine learning: SBERT and SimCSE application

BACKGROUND

Using visual, biological, and electronic health records data as the sole input source, pretrained convolutional neural networks and conventional machine learning methods have been heavily employed for the identification of various malignancies. Initially, a series of preprocessing steps and image segmentation steps are performed to extract region of interest features from noisy features. Then, the extracted features are applied to several machine learning and deep learning methods for the detection of cancer.

METHODS

In this work, a review of all the methods that have been applied to develop machine learning algorithms that detect cancer is provided. With more than 100 types of cancer, this study only examines research on the four most common and prevalent cancers worldwide: lung, breast, prostate, and colorectal cancer. Next, by using state-of-the-art sentence transformers namely: SBERT (2019) and the unsupervised SimCSE (2021), this study proposes a new methodology for detecting cancer. This method requires raw DNA sequences of matched tumor/normal pair as the only input. The learnt DNA representations retrieved from SBERT and SimCSE will then be sent to machine learning algorithms (XGBoost, Random Forest, LightGBM, and CNNs) for classification. As far as we are aware, SBERT and SimCSE transformers have not been applied to represent DNA sequences in cancer detection settings.

RESULTS

The XGBoost model, which had the highest overall accuracy of 73 ± 0.13 % using SBERT embeddings and 75 ± 0.12 % using SimCSE embeddings, was the best performing classifier. In light of these findings, it can be concluded that incorporating sentence representations from SimCSE's sentence transformer only marginally improved the performance of machine learning models.

Mansouri R. Archimedes Optimization Algorithm with Deep Learning-Based Prostate Cancer Classification on Magnetic Resonance Imaging

Prostate cancer (PCa) is becoming one of the most frequently occurring cancers among men and causes an even greater number of deaths. Due to the complexity of tumor masses, radiologists find it difficult to identify PCa accurately. Over the years, several PCa-detecting methods have been formulated, but these methods cannot identify cancer efficiently. Artificial Intelligence (AI) has both information technologies that simulate natural or biological phenomena and human intelligence in addressing issues. AI technologies have been broadly implemented in the healthcare domain, including 3D printing, disease diagnosis, health monitoring, hospital scheduling, clinical decision support, classification and prediction, and medical data analysis. These applications significantly boost the cost-effectiveness and accuracy of healthcare services. This article introduces an Archimedes Optimization Algorithm with Deep Learning-based Prostate Cancer Classification (AOADLB-P2C) model on MRI images. The presented AOADLB-P2C model examines MRI images for the identification of PCa. To accomplish this, the AOADLB-P2C model performs pre-processing in two stages: adaptive median filtering (AMF)-based noise removal and contrast enhancement. Additionally, the presented AOADLB-P2C model extracts features via a densely connected network (DenseNet-161) model with a root-mean-square propagation (RMSProp) optimizer. Finally, the presented AOADLB-P2C model classifies PCa using the AOA with a least-squares support vector machine (LS-SVM) method. The simulation values of the presented AOADLB-P2C model are tested using a benchmark MRI dataset. The comparative experimental results demonstrate the improvements of the AOADLB-P2C model over other recent approaches.

Annotation-Free Deep Learning-Based Prediction of Thyroid Molecular Cancer Biomarker BRAF (V600E) from Cytological Slides

Thyroid cancer is the most common endocrine cancer. Papillary thyroid cancer (PTC) is the most prevalent form of malignancy among all thyroid cancers arising from follicular cells. Fine needle aspiration cytology (FNAC) is a non-invasive method regarded as the most cost-effective and accurate diagnostic method of choice in diagnosing PTC. Identification of BRAF (V600E) mutation in thyroid neoplasia may be beneficial because it is specific for malignancy, implies a worse prognosis, and is the target for selective BRAF inhibitors. To the authors' best knowledge, this is the first automated precision oncology framework effectively predict BRAF (V600E) immunostaining result in thyroidectomy specimen directly from Papanicolaou-stained thyroid fine-needle aspiration cytology and ThinPrep cytological slides, which is helpful for novel targeted therapies and prognosis prediction. The proposed deep learning (DL) framework is evaluated on a dataset of 118 whole slide images. The results show that the proposed DL-based technique achieves an accuracy of 87%, a precision of 94%, a sensitivity of 91%, a specificity of 71% and a mean of sensitivity and specificity at 81% and outperformed three state-of-the-art deep learning approaches. This study demonstrates the feasibility of DL-based prediction of critical molecular features in cytological slides, which not only aid in accurate diagnosis but also provide useful information in guiding clinical decision-making in patients with thyroid cancer. With the accumulation of data and the continuous advancement of technology, the performance of DL systems is expected to be improved in the near future. Therefore, we expect that DL can provide a cost-effective and time-effective alternative tool for patients in the era of precision oncology.

An update on computational pathology tools for genitourinary pathology practice: A review paper from the Genitourinary Pathology Society (GUPS)

Machine learning has been leveraged for image analysis applications throughout a multitude of subspecialties. This position paper provides a perspective on the evolutionary trajectory of practical deep learning tools for genitourinary pathology through evaluating the most recent iterations of such algorithmic devices. Deep learning tools for genitourinary pathology demonstrate potential to enhance prognostic and predictive capacity for tumor assessment including grading, staging, and subtype identification, yet limitations in data availability, regulation, and standardization have stymied their implementation.

Automatic diagnosis and grading of Prostate Cancer with weakly supervised learning on whole slide images

BACKGROUND

The workflow of prostate cancer diagnosis and grading is cumbersome and the results suffer from substantial inter-observer variability. Recent trials have shown potential in using machine learning to develop automated systems to address this challenge. Most automated deep learning systems for prostate cancer Gleason grading focused on supervised learning requiring demanding fine-grained pixel-level annotations.

METHODS

A weakly-supervised deep learning model with slide-level labels is presented in this study for the diagnosis and grading of prostate cancer with whole slide image (WSI). WSIs are first cropped into small patches and then processed with a deep learning model to extract patch-level features. A graph convolution network (GCN) is used to aggregate the features for classifications. Throughout the training process, the noisy labels are progressively filtered out to reduce inter-observer variations in clinical reports. Finally, multi-center independent test cohorts with 6,174 slides are collected to evaluate the prostate cancer diagnosis and grading performance of our model.

RESULTS

The cancer diagnosis (2-level classification) results on two external test sets (n= 4,675, n= 844) show an area under the receiver operating characteristic curve (AUC) of 0.985 and 0.986. The Gleason grading (6-level classification) results reach 0.931 quadratic weighted kappa on the internal test set (n= 531). It generalizes well on the external test dataset (n= 844) with 0.801 quadratic weighted kappa with the reference standard set independently. The model enables pathological meaningful interpretability by visualizing the most attended lesions which are highly consistent with expert annotations.

CONCLUSION

The proposed model incorporates a graph network in weakly supervised learning with only slide-level reports. A robust learning strategy is also employed to correct the label noise. It is highly accurate (>0.985 AUC for diagnosis) and also interpretable with intuitive heatmap visualization. It can be unified with a digital pathology pipeline to deliver prostate cancer metrics for a pathology report.

Application of machine learning and complex network measures to an EEG dataset from ayahuasca experiments

Ayahuasca is a blend of Amazonian plants that has been used for traditional medicine by the inhabitants of this region for hundreds of years. Furthermore, this plant has been demonstrated to be a viable therapy for a variety of neurological and mental diseases. EEG experiments have found specific brain regions that changed significantly due to ayahuasca. Here, we used an EEG dataset to investigate the ability to automatically detect changes in brain activity using machine learning and complex networks. Machine learning was applied at three different levels of data abstraction: (A) the raw EEG time series, (B) the correlation of the EEG time series, and (C) the complex network measures calculated from (B). Further, at the abstraction level of (C), we developed new measures of complex networks relating to community detection. As a result, the machine learning method was able to automatically detect changes in brain activity, with case (B) showing the highest accuracy (92%), followed by (A) (88%) and (C) (83%), indicating that connectivity changes between brain regions are more important for the detection of ayahuasca. The most activated areas were the frontal and temporal lobe, which is consistent with the literature. F3 and PO4 were the most important brain connections, a significant new discovery for psychedelic literature. This connection may point to a cognitive process akin to face recognition in individuals during ayahuasca-mediated visual hallucinations. Furthermore, closeness centrality and assortativity were the most important complex network measures. These two measures are also associated with diseases such as Alzheimer's disease, indicating a possible therapeutic mechanism. Moreover, the new measures were crucial to the predictive model and suggested larger brain communities associated with the use of ayahuasca. This suggests that the dissemination of information in functional brain networks is slower when this drug is present. Overall, our methodology was able to automatically detect changes in brain activity during ayahuasca consumption and interpret how these psychedelics alter brain networks, as well as provide insights into their mechanisms of action.