Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer

Abnormality-aware multimodal learning for WSI classification

Whole slide images (WSIs) play a vital role in cancer diagnosis and prognosis. However, their gigapixel resolution, lack of pixel-level annotations, and reliance on unimodal visual data present challenges for accurate and efficient computational analysis. Existing methods typically divide WSIs into thousands of patches, which increases computational demands and makes it challenging to effectively focus on diagnostically relevant regions. Furthermore, these methods frequently rely on feature extractors pretrained on natural images, which are not optimized for pathology tasks, and overlook multimodal data sources such as cellular and textual information that can provide critical insights. To address these limitations, we propose the Abnormality-Aware MultiModal (AAMM) learning framework, which integrates abnormality detection and multimodal feature learning for WSI classification. AAMM incorporates a Gaussian Mixture Variational Autoencoder (GMVAE) to identify and select the most informative patches, reducing computational complexity while retaining critical diagnostic information. It further integrates multimodal features from pathology-specific foundation models, combining patch-level, cell-level, and text-level representations through cross-attention mechanisms. This approach enhances the ability to comprehensively analyze WSIs for cancer diagnosis and subtyping. Extensive experiments on normal-tumor classification and cancer subtyping demonstrate that AAMM achieves superior performance compared to state-of-the-art methods. By combining abnormal detection with multimodal feature integration, our framework offers an efficient and scalable solution for advancing computational pathology.

Finding Consensus on Trust in AI in Health Care: Recommendations From a Panel of International Experts

BACKGROUND

The integration of artificial intelligence (AI) into health care has become a crucial element in the digital transformation of health systems worldwide. Despite the potential benefits across diverse medical domains, a significant barrier to the successful adoption of AI systems in health care applications remains the prevailing low user trust in these technologies. Crucially, this challenge is exacerbated by the lack of consensus among experts from different disciplines on the definition of trust in AI within the health care sector.

OBJECTIVE

We aimed to provide the first consensus-based analysis of trust in AI in health care based on an interdisciplinary panel of experts from different domains. Our findings can be used to address the problem of defining trust in AI in health care applications, fostering the discussion of concrete real-world health care scenarios in which humans interact with AI systems explicitly.

METHODS

We used a combination of framework analysis and a 3-step consensus process involving 18 international experts from the fields of computer science, medicine, philosophy of technology, ethics, and social sciences. Our process consisted of a synchronous phase during an expert workshop where we discussed the notion of trust in AI in health care applications, defined an initial framework of important elements of trust to guide our analysis, and agreed on 5 case studies. This was followed by a 2-step iterative, asynchronous process in which the authors further developed, discussed, and refined notions of trust with respect to these specific cases.

RESULTS

Our consensus process identified key contextual factors of trust, namely, an AI system's environment, the actors involved, and framing factors, and analyzed causes and effects of trust in AI in health care. Our findings revealed that certain factors were applicable across all discussed cases yet also pointed to the need for a fine-grained, multidisciplinary analysis bridging human-centered and technology-centered approaches. While regulatory boundaries and technological design features are critical to successful AI implementation in health care, ultimately, communication and positive lived experiences with AI systems will be at the forefront of user trust. Our expert consensus allowed us to formulate concrete recommendations for future research on trust in AI in health care applications.

CONCLUSIONS

This paper advocates for a more refined and nuanced conceptual understanding of trust in the context of AI in health care. By synthesizing insights into commonalities and differences among specific case studies, this paper establishes a foundational basis for future debates and discussions on trusting AI in health care.

Context-guided segmentation for histopathologic cancer segmentation

Microscopic inspection of histologically stained tissue is considered as the gold standard for cancer diagnosis. This research is inspired by the practices of pathologists who analyze diagnostic samples by zooming in and out. We propose a dual-encoder model that simultaneously evaluates two views of the tissue at different levels of magnification. The lower magnification view provides contextual information for a target area, while the higher magnification view provides detailed information. The model consists of two encoder branches that consider both detail and context resolutions of the target area concurrently for binary pixel-wise segmentation. We introduce a unique weight initialization for the cross-attention between the context and detail feature tensors, allowing the model to incorporate contextual information. Our design is evaluated using the Camelyon16 dataset of sentinel lymph node tissue and cancer. The results demonstrate the benefit of including context regions when segmenting for cancer, with an improvement in AUC ranging from 0.31 to 0.92% and an improvement in cancer Dice score ranging from 4.09% to 6.81% compared to single detailed input models.

Likelihood Systems Can Improve Hit Rates in Low-Prevalence Visual Search Over Binary Systems

OBJECTIVE

To study the performance consequences of binary versus likelihood decision support systems in low-prevalence visual search.

BACKGROUND

Hit rates in visual search are often low if target prevalence is low, an issue that is relevant for numerous real-world visual search tasks (e.g., luggage screening and medical imaging). Given that binary decision support systems produce many false alarms at low prevalence, they have often been discounted as a solution to this low-prevalence problem. By offering additional information about the certainty of target-present indications through splitting these into warnings and alarms, likelihood-based systems could potentially boost hit rates without raising the number of false alarms.

METHOD

We used a simulated medical search task with low target prevalence in a paradigm where participants sequentially uncovered parts of the stimulus with their mouse. In two sessions, participants completed the task either while being supported by a binary or a likelihood system.

RESULTS

Hit rates were higher when interacting with the likelihood systems than with the binary system, at no cost of higher false alarms.

CONCLUSION

Likelihood systems are a promising way to tackle the low-prevalence problem, and might further be an effective means to make systems more transparent.

APPLICATION

Simple-to-process information about system certainty for each case might be a solution to low hit rates in domains with low target prevalence, such as radiology.

Invited commentary: deep learning-methods to amplify epidemiologic data collection and analyses

Deep learning is a subfield of artificial intelligence and machine learning, based mostly on neural networks and often combined with attention algorithms, that has been used to detect and identify objects in text, audio, images, and video. Serghiou and Rough (Am J Epidemiol. 2023;192(11):1904-1916) presented a primer for epidemiologists on deep learning models. These models provide substantial opportunities for epidemiologists to expand and amplify their research in both data collection and analyses by increasing the geographic reach of studies, including more research subjects, and working with large or high-dimensional data. The tools for implementing deep learning methods are not as straightforward or ubiquitous for epidemiologists as traditional regression methods found in standard statistical software, but there are exciting opportunities for interdisciplinary collaboration with deep learning experts, just as epidemiologists have with statisticians, health care providers, urban planners, and other professionals. Despite the novelty of these methods, epidemiologic principles of assessing bias, study design, interpretation, and others still apply when implementing deep learning methods or assessing the findings of studies that have used them.

CAMIL: channel attention-based multiple instance learning for whole slide image classification

MOTIVATION

The classification task based on whole-slide images (WSIs) is a classic problem in computational pathology. Multiple instance learning (MIL) provides a robust framework for analyzing whole slide images with slide-level labels at gigapixel resolution. However, existing MIL models typically focus on modeling the relationships between instances while neglecting the variability across the channel dimensions of instances, which prevents the model from fully capturing critical information in the channel dimension.

RESULTS

To address this issue, we propose a plug-and-play module called Multi-scale Channel Attention Block (MCAB), which models the interdependencies between channels by leveraging local features with different receptive fields. By alternately stacking four layers of Transformer and MCAB, we designed a channel attention-based MIL model (CAMIL) capable of simultaneously modeling both inter-instance relationships and intra-channel dependencies. To verify the performance of the proposed CAMIL in classification tasks, several comprehensive experiments were conducted across three datasets: Camelyon16, TCGA-NSCLC, and TCGA-RCC. Empirical results demonstrate that, whether the feature extractor is pretrained on natural images or on WSIs, our CAMIL surpasses current state-of-the-art MIL models across multiple evaluation metrics.

AVAILABILITY AND IMPLEMENTATION

All implementation code is available at https://github.com/maojy0914/CAMIL.

Annotation-free deep learning for predicting gene mutations from whole slide images of acute myeloid leukemia

The rapid development of deep learning has revolutionized medical image processing, including analyzing whole slide images (WSIs). Despite the demonstrated potential for characterizing gene mutations directly from WSIs in certain cancers, challenges remain due to image resolution and reliance on manual annotations for acute myeloid leukemia (AML). We, therefore, propose a deep learning model based on multiple instance learning (MIL) with ensemble techniques to predict gene mutations from AML WSIs. Our model predicts NPM1 mutations and FLT3-ITD without requiring patch-level or cell-level annotations. Using a dataset of 572 WSIs, the largest database with both WSI and genetic mutation information, our model achieved an AUC of 0.90 ± 0.08 for NPM1 and 0.80 ± 0.10 for FLT3-ITD in the testing cohort. Additionally, we found that blasts are pivotal indicators for gene mutation predictions, with their proportions varying between mutated and standard WSIs, highlighting the clinical potential of AML WSI analysis.

Unleash the Power of State Space Model for Whole Slide Image With Local Aware Scanning and Importance Resampling

Whole slide image (WSI) analysis is gaining prominence within the medical imaging field. However, previous methods often fall short of efficiently processing entire WSIs due to their gigapixel size. Inspired by recent developments in state space models, this paper introduces a new Pathology Mamba (PAM) for more accurate and robust WSI analysis. PAM includes three carefully designed components to tackle the challenges of enormous image size, the utilization of local and hierarchical information, and the mismatch between the feature distributions of training and testing during WSI analysis. Specifically, we design a Bi-directional Mamba Encoder to process the extensive patches present in WSIs effectively and efficiently, which can handle large-scale pathological images while achieving high performance and accuracy. To further harness the local information and inherent hierarchical structure of WSI, we introduce a novel Local-aware Scanning module, which employs a local-aware mechanism alongside hierarchical scanning to adeptly capture both the local information and the overarching structure within WSIs. Moreover, to alleviate the patch feature distribution misalignment between training and testing, we propose a Test-time Importance Resampling module to conduct testing patch resampling to ensure consistency of feature distribution between the training and testing phases, and thus enhance model prediction. Extensive evaluation on nine WSI datasets with cancer subtyping and survival prediction tasks demonstrates that PAM outperforms current state-of-the-art methods and also its enhanced capability in modeling discriminative areas within WSIs. The source code is available at https://github.com/HKU-MedAI/PAM.

Making Pathologists Ready for the New Artificial Intelligence Era: Changes in Required Competencies

In recent years, there has been an increasing interest in developing and using artificial intelligence (AI) models in pathology. Although pathologists generally have a positive attitude toward AI, they report a lack of knowledge and skills regarding how to use it in practice. Furthermore, it remains unclear what skills pathologists would require to use AI adequately and responsibly. However, adequate training of (future) pathologists is essential for successful AI use in pathology. In this paper, we assess which entrustable professional activities (EPAs) and associated competencies pathologists should acquire in order to use AI in their daily practice. We make use of the available academic literature, including literature in radiology, another image-based discipline, which is currently more advanced in terms of AI development and implementation. Although microscopy evaluation and reporting could be transferrable to AI in the future, most of the current pathologist EPAs and competencies will likely remain relevant when using AI techniques and interpreting and communicating results for individual patient cases. In addition, new competencies related to technology evaluation and implementation will likely be necessary, along with knowing one's own strengths and limitations in human-AI interactions. Because current EPAs do not sufficiently address the need to train pathologists in developing expertise related to technology evaluation and implementation, we propose a new EPA to enable pathology training programs to make pathologists fit for the new AI era "using AI in diagnostic pathology practice" and outline its associated competencies.

Leveraging domain knowledge for synthetic ultrasound image generation: a novel approach to rare disease AI detection

PURPOSE

This study explores the use of deep generative models to create synthetic ultrasound images for the detection of hemarthrosis in hemophilia patients. Addressing the challenge of sparse datasets in rare disease diagnostics, the study aims to enhance AI model robustness and accuracy through the integration of domain knowledge into the synthetic image generation process.

METHODS

The study employed two ultrasound datasets: a base dataset (Db) of knee recess distension images from non-hemophiliac patients and a target dataset (Dt) of hemarthrosis images from hemophiliac patients. The synthetic generation framework included a content generator (Gc) trained on Db and a context generator (Gs) to adapt these images to match Dt's context. This approach generated a synthetic target dataset (Ds), primed for AI training in rare disease research. The assessment of synthetic image generation involved expert evaluations, statistical analysis, and the use of domain-invariant perceptual distance and Fréchet inception distance for quality measurement.

RESULTS

Expert evaluation revealed that images produced by our synthetic generation framework were comparable to real ones, with no significant difference in overall quality or anatomical accuracy. Additionally, the use of synthetic data in training convolutional neural networks demonstrated robustness in detecting hemarthrosis, especially with limited sample sizes.

CONCLUSION

This study presents a novel approach for generating synthetic ultrasound images for rare disease detection, such as hemarthrosis in hemophiliac knees. By leveraging deep generative models and integrating domain knowledge, the proposed framework successfully addresses the limitations of sparse datasets and enhances AI model training and robustness. The synthetic images produced are of high quality and contribute significantly to AI-driven diagnostics in rare diseases, highlighting the potential of synthetic data in medical imaging.

Using digital pathology to standardize and automate histological evaluations of environmental samples

Histological evaluations of tissues are commonly used in environmental monitoring studies to assess the health and fitness status of populations or even whole ecosystems. Although traditional histology can be cost-effective, there is a shortage of proficient histopathologists and results can often be subjective between operators, leading to variance. Digital pathology is a powerful diagnostic tool that has already significantly transformed research in human health but has rarely been applied to environmental studies. Digital analyses of whole slide images introduce possibilities of highly standardized histopathological evaluations, as well as the use of artificial intelligence for novel analyses. Furthermore, incorporation of digital pathology into environmental monitoring studies using standardized bioindicator species or groups such as bivalves and fish can greatly improve the accuracy, reproducibility, and efficiency of the studies. This review aims to introduce readers to digital pathology and how it can be applied to environmental studies. This includes guidelines for sample preparation, potential sources of error, and comparisons to traditional histopathological analyses.

Deep Learning in Breast Cancer Imaging: A Decade of Progress and Future Directions

Breast cancer has reached the highest incidence rate worldwide among all malignancies since 2020. Breast imaging plays a significant role in early diagnosis and intervention to improve the outcome of breast cancer patients. In the past decade, deep learning has shown remarkable progress in breast cancer imaging analysis, holding great promise in interpreting the rich information and complex context of breast imaging modalities. Considering the rapid improvement in deep learning technology and the increasing severity of breast cancer, it is critical to summarize past progress and identify future challenges to be addressed. This paper provides an extensive review of deep learning-based breast cancer imaging research, covering studies on mammograms, ultrasound, magnetic resonance imaging, and digital pathology images over the past decade. The major deep learning methods and applications on imaging-based screening, diagnosis, treatment response prediction, and prognosis are elaborated and discussed. Drawn from the findings of this survey, we present a comprehensive discussion of the challenges and potential avenues for future research in deep learning-based breast cancer imaging.

Analysis and Validation of Image Search Engines in Histopathology

Searching for similar images in archives of histology and histopathology images is a crucial task that may aid in patient tissue comparison for various purposes, ranging from triaging and diagnosis to prognosis and prediction. Whole slide images (WSIs) are highly detailed digital representations of tissue specimens mounted on glass slides. Matching WSI to WSI can serve as the critical method for patient tissue comparison. In this paper, we report extensive analysis and validation of four search methods bag of visual words (BoVW), Yottixel, SISH, RetCCL, and some of their potential variants. We analyze their algorithms and structures and assess their performance. For this evaluation, we utilized four internal datasets (1269 patients) and three public datasets (1207 patients), totaling more than 200,000 patches from 38 different classes/subtypes across five primary sites. Certain search engines, for example, BoVW, exhibit notable efficiency and speed but suffer from low accuracy. Conversely, search engines like Yottixel demonstrate efficiency and speed, providing moderately accurate results. Recent proposals, including SISH, display inefficiency and yield inconsistent outcomes, while alternatives like RetCCL prove inadequate in both accuracy and efficiency. Further research is imperative to address the dual aspects of accuracy and minimal storage requirements in histopathological image search.

Revolutionizing public health through digital health technology

The aging population and increasing chronic diseases strain public health systems. Advancements in digital health promise to tackle these challenges and enhance public health outcomes. Digital health integrates digital health technology (DHT) across healthcare, including smart consumer devices. This article examines the application of DHT in public health and its significant impact on revolutionizing the field. Historically, DHT has not only enhanced the efficiency of disease prevention, diagnosis, and treatment but also facilitated the equitable distribution of global health resources. Looking ahead, DHT holds vast potential in areas such as personalized medicine, telemedicine, and intelligent health management. However, it also encounters challenges such as ethics, privacy, and data security. To further advance DHT, concerted efforts are essential, including policy support, investment in research and development, involvement of medical institutions, and improvement of public digital health literacy.

Clinical Applications of Artificial Intelligence (AI) in Human Cancer: Is It Time to Update the Diagnostic and Predictive Models in Managing Hepatocellular Carcinoma (HCC)?

In recent years, novel findings have progressively and promisingly supported the potential role of Artificial intelligence (AI) in transforming the management of various neoplasms, including hepatocellular carcinoma (HCC). HCC represents the most common primary liver cancer. Alarmingly, the HCC incidence is dramatically increasing worldwide due to the simultaneous "pandemic" spreading of metabolic dysfunction-associated steatotic liver disease (MASLD). MASLD currently constitutes the leading cause of chronic hepatic damage (steatosis and steatohepatitis), fibrosis, and liver cirrhosis, configuring a scenario where an HCC onset has been reported even in the early disease stage. On the other hand, HCC represents a serious plague, significantly burdening the outcomes of chronic hepatitis B (HBV) and hepatitis C (HCV) virus-infected patients. Despite the recent progress in the management of this cancer, the overall prognosis for advanced-stage HCC patients continues to be poor, suggesting the absolute need to develop personalized healthcare strategies further. In this "cold war", machine learning techniques and neural networks are emerging as weapons, able to identify the patterns and biomarkers that would have normally escaped human observation. Using advanced algorithms, AI can analyze large volumes of clinical data and medical images (including routinely obtained ultrasound data) with an elevated accuracy, facilitating early diagnosis, improving the performance of predictive models, and supporting the multidisciplinary (oncologist, gastroenterologist, surgeon, radiologist) team in opting for the best "tailored" individual treatment. Additionally, AI can significantly contribute to enhancing the effectiveness of metabolomics-radiomics-based models, promoting the identification of specific HCC-pathogenetic molecules as new targets for realizing novel therapeutic regimens. In the era of precision medicine, integrating AI into routine clinical practice appears as a promising frontier, opening new avenues for liver cancer research and treatment.

Optimized digital workflow for pathologist-grade evaluation in bleomycin-induced pulmonary fibrosis mouse model

Idiopathic pulmonary fibrosis (IPF) is a progressive and ultimately fatal disorder of unknown etiology, characterized by interstitial fibrosis of the lungs. Bleomycin-induced pulmonary fibrosis mouse model (BLM model) is a widely used animal model to evaluate therapeutic targets for IPF. Histopathological analysis of lung fibrosis is an important method for evaluating BLM model. However, this method requires expertise in recognizing complex visual patterns and is time-consuming, making the workflow difficult and inefficient. Therefore, we developed a new workflow for BLM model that reduces inter- and intra-observer variations and improves the evaluation process. We generated deep learning models for grading lung fibrosis that were able to achieve accuracy comparable to that of pathologists. These models incorporate complex image patterns and qualitative factors, such as collagen texture and distribution, potentially identifying drug candidates overlooked in evaluations based solely on simple area extraction. This deep learning-based fibrosis grade assessment has the potential to streamline drug development for pulmonary fibrosis by offering higher granularity and reproducibility in evaluating BLM model.

Reusable specimen-level inference in computational pathology

Foundation models for computational pathology have shown great promise for specimen-level tasks and are increasingly accessible to researchers. However, specimen-level models built on these foundation models remain largely unavailable, hindering their broader utility and impact. To address this gap, we developed SpinPath, a toolkit designed to democratize specimen-level deep learning by providing a zoo of pretrained specimen-level models, a Python-based inference engine, and a JavaScript-based inference platform. We demonstrate the utility of SpinPath in metastasis detection tasks across nine foundation models. SpinPath may foster reproducibility, simplify experimentation, and accelerate the adoption of specimen-level deep learning in computational pathology research.

Harnessing Artificial Intelligence to Enhance Global Breast Cancer Care: A Scoping Review of Applications, Outcomes, and Challenges

BACKGROUND

In recent years, Artificial Intelligence (AI) has shown transformative potential in advancing breast cancer care globally. This scoping review seeks to provide a comprehensive overview of AI applications in breast cancer care, examining how they could reshape diagnosis, treatment, and management on a worldwide scale and discussing both the benefits and challenges associated with their adoption.

METHODS

In accordance with PRISMA-ScR and ensuing guidelines on scoping reviews, PubMed, Web of Science, Cochrane Library, and Embase were systematically searched from inception to end of May 2024. Keywords included "Artificial Intelligence" and "Breast Cancer". Original studies were included based on their focus on AI applications in breast cancer care and narrative synthesis was employed for data extraction and interpretation, with the findings organized into coherent themes.

RESULTS

Finally, 84 articles were included. The majority were conducted in developed countries (n = 54). The majority of publications were in the last 10 years (n = 83). The six main themes for AI applications were AI for breast cancer screening (n = 32), AI for image detection of nodal status (n = 7), AI-assisted histopathology (n = 8), AI in assessing post-neoadjuvant chemotherapy (NACT) response (n = 23), AI in breast cancer margin assessment (n = 5), and AI as a clinical decision support tool (n = 9). AI has been used as clinical decision support tools to augment treatment decisions for breast cancer and in multidisciplinary tumor board settings. Overall, AI applications demonstrated improved accuracy and efficiency; however, most articles did not report patient-centric clinical outcomes.

CONCLUSIONS

AI applications in breast cancer care show promise in enhancing diagnostic accuracy and treatment planning. However, persistent challenges in AI adoption, such as data quality, algorithm transparency, and resource disparities, must be addressed to advance the field.

Advancing breast cancer diagnosis: token vision transformers for faster and accurate classification of histopathology images

The vision transformer (ViT) architecture, with its attention mechanism based on multi-head attention layers, has been widely adopted in various computer-aided diagnosis tasks due to its effectiveness in processing medical image information. ViTs are notably recognized for their complex architecture, which requires high-performance GPUs or CPUs for efficient model training and deployment in real-world medical diagnostic devices. This renders them more intricate than convolutional neural networks (CNNs). This difficulty is also challenging in the context of histopathology image analysis, where the images are both limited and complex. In response to these challenges, this study proposes a TokenMixer hybrid-architecture that combines the strengths of CNNs and ViTs. This hybrid architecture aims to enhance feature extraction and classification accuracy with shorter training time and fewer parameters by minimizing the number of input patches employed during training, while incorporating tokenization of input patches using convolutional layers and encoder transformer layers to process patches across all network layers for fast and accurate breast cancer tumor subtype classification. The TokenMixer mechanism is inspired by the ConvMixer and TokenLearner models. First, the ConvMixer model dynamically generates spatial attention maps using convolutional layers, enabling the extraction of patches from input images to minimize the number of input patches used in training. Second, the TokenLearner model extracts relevant regions from the selected input patches, tokenizes them to improve feature extraction, and trains all tokenized patches in an encoder transformer network. We evaluated the TokenMixer model on the BreakHis public dataset, comparing it with ViT-based and other state-of-the-art methods. Our approach achieved impressive results for both binary and multi-classification of breast cancer subtypes across various magnification levels (40×, 100×, 200×, 400×). The model demonstrated accuracies of 97.02% for binary classification and 93.29% for multi-classification, with decision times of 391.71 and 1173.56 s, respectively. These results highlight the potential of our hybrid deep ViT-CNN architecture for advancing tumor classification in histopathological images. The source code is accessible: https://github.com/abimouloud/TokenMixer .

Harnessing the potential of human induced pluripotent stem cells, functional assays and machine learning for neurodevelopmental disorders

Neurodevelopmental disorders (NDDs) affect 4.7% of the global population and are associated with delays in brain development and a spectrum of impairments that can lead to lifelong disability and even mortality. Identification of biomarkers for accurate diagnosis and medications for effective treatment are lacking, in part due to the historical use of preclinical model systems that do not translate well to the clinic for neurological disorders, such as rodents and heterologous cell lines. Human-induced pluripotent stem cells (hiPSCs) are a promising in vitro system for modeling NDDs, providing opportunities to understand mechanisms driving NDDs in human neurons. Functional assays, including patch clamping, multielectrode array, and imaging-based assays, are popular tools employed with hiPSC disease models for disease investigation. Recent progress in machine learning (ML) algorithms also presents unprecedented opportunities to advance the NDD research process. In this review, we compare two-dimensional and three-dimensional hiPSC formats for disease modeling, discuss the applications of functional assays, and offer insights on incorporating ML into hiPSC-based NDD research and drug screening.

Out-of-distribution generalization for segmentation of lymph node metastasis in breast cancer

Pathology provides the definitive diagnosis, and Artificial Intelligence (AI) tools are poised to improve accuracy, inter-rater agreement, and turn-around time (TAT) of pathologists, leading to improved quality of care. A high value clinical application is the grading of Lymph Node Metastasis (LNM) which is used for breast cancer staging and guides treatment decisions. A challenge of implementing AI tools widely for LNM classification is domain shift, where Out-of-Distribution (OOD) data has a different distribution than the In-Distribution (ID) data used to train the model, resulting in a drop in performance in OOD data. This work proposes a novel clustering and sampling method to automatically curate training datasets in an unsupervised manner with the aim of improving model generalization abilities. To evaluate the generalization performance of the proposed models, we applied a novel use of the Two One-sided Tests (TOST) method. This method examines whether the performance on ID and OOD data is equivalent, serving as a proxy for generalization. We provide the first evidence for computing equivalence margins that are data-dependent, which reduces subjectivity. The proposed framework shows the ensembled models constructed from models that generalized across both tumor and normal patches enhanced performance, achieving an F1 score of 0.81 for LNM classification on unseen ID and OOD samples. Interactive viewing of slide-level segmentations can be accessed on PathcoreFlow™ through https://web.pathcore.com/folder/18555?s=QTJVHJuhrfe5 . Segmentation models are available at https://github.com/IAMLAB-Ryerson/OOD-Generalization-LNM .

Self-Supervised Representation Distribution Learning for Reliable Data Augmentation in Histopathology WSI Classification

Multiple instance learning (MIL) based whole slide image (WSI) classification is often carried out on the representations of patches extracted from WSI with a pre-trained patch encoder. The performance of classification relies on both patch-level representation learning and MIL classifier training. Most MIL methods utilize a frozen model pre-trained on ImageNet or a model trained with self-supervised learning on histopathology image dataset to extract patch image representations and then fix these representations in the training of the MIL classifiers for efficiency consideration. However, the invariance of representations cannot meet the diversity requirement for training a robust MIL classifier, which has significantly limited the performance of the WSI classification. In this paper, we propose a Self-Supervised Representation Distribution Learning framework (SSRDL) for patch-level representation learning with an online representation sampling strategy (ORS) for both patch feature extraction and WSI-level data augmentation. The proposed method was evaluated on three datasets under three MIL frameworks. The experimental results have demonstrated that the proposed method achieves the best performance in histopathology image representation learning and data augmentation and outperforms state-of-the-art methods under different WSI classification frameworks. The code is available at https://github.com/lazytkm/SSRDL.

Computer-assisted diagnosis to improve diagnostic pathology: A review

ABSTRACT

With an increasing demand for accuracy and efficiency in diagnostic pathology, computer-assisted diagnosis (CAD) emerges as a prominent and transformative solution. This review aims to explore the practical applications, implications, strengths, and weaknesses of CAD applied to diagnostic pathology. A comprehensive literature search was conducted to include English-language studies focusing on CAD tools, digital pathology, and Artificial intelligence (AI) applications in pathology. The review underscores the transformative potential of CAD tools in pathology, particularly in streamlining diagnostic processes, reducing turnaround times, and augmenting diagnostic accuracy. It emphasizes the strides made in digital pathology, the integration of AI, and the promising prospects for prognostic biomarker discovery using computational methods. Additionally, ethical considerations regarding data privacy, equity, and trust in AI deployment are examined. CAD has the potential to revolutionize diagnostic pathology. The insights gleaned from this review offer a panoramic view of recent advancements. Ultimately, this review aims to guide future research, influence clinical practice, and inform policy-making by elucidating the promising horizons and potential pitfalls of integrating CAD tools in pathology.

ATEC23 Challenge: Automated prediction of treatment effectiveness in ovarian cancer using histopathological images

Ovarian cancer, predominantly epithelial ovarian cancer (EOC), is a global health concern due to its high mortality rate. Despite the progress made during the last two decades in the surgery and chemotherapy of ovarian cancer, more than 70% of advanced patients are with recurrent cancer and disease. Bevacizumab is a humanized monoclonal antibody, which blocks VEGF signaling in cancer, inhibits angiogenesis and causes tumor shrinkage, and has been recently approved by the FDA as a monotherapy for advanced ovarian cancer in combination with chemotherapy. Unfortunately, Bevacizumab may also induce harmful adverse effects, such as hypertension, bleeding, arterial thromboembolism, poor wound healing and gastrointestinal perforation. Given the expensive cost and unwanted toxicities, there is an urgent need for predictive methods to identify who could benefit from bevacizumab. Of the 18 (approved) requests from 5 countries, 6 teams using 284 whole section WSIs for training to develop fully automated systems submitted their predictions on a test set of 180 tissue core images, with the corresponding ground truth labels kept private. This paper summarizes the 5 qualified methods successfully submitted to the international challenge of automated prediction of treatment effectiveness in ovarian cancer using the histopathologic images (ATEC23) held at the 26th International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI) in 2023 and evaluates the methods in comparison with 5 state of the art deep learning approaches. This study further assesses the effectiveness of the presented prediction models as indicators for patient selection utilizing both Cox proportional hazards analysis and Kaplan-Meier survival analysis. A robust and cost-effective deep learning pipeline for digital histopathology tasks has become a necessity within the context of the medical community. This challenge highlights the limitations of current MIL methods, particularly within the context of prognosis-based classification tasks, and the importance of DCNNs like inception that has nonlinear convolutional modules at various resolutions to facilitate processing the data in multiple resolutions, which is a key feature required for pathology related prediction tasks. This further suggests the use of feature reuse at various scales to improve models for future research directions. In particular, this paper releases the labels of the testing set and provides applications for future research directions in precision oncology to predict ovarian cancer treatment effectiveness and facilitate patient selection via histopathological images.

Shapley Values-Enabled Progressive Pseudo Bag Augmentation for Whole-Slide Image Classification

In computational pathology, whole-slide image (WSI) classification presents a formidable challenge due to its gigapixel resolution and limited fine-grained annotations. Multiple-instance learning (MIL) offers a weakly supervised solution, yet refining instance-level information from bag-level labels remains challenging. While most of the conventional MIL methods use attention scores to estimate instance importance scores (IIS) which contribute to the prediction of the slide labels, these often lead to skewed attention distributions and inaccuracies in identifying crucial instances. To address these issues, we propose a new approach inspired by cooperative game theory: employing Shapley values to assess each instance's contribution, thereby improving IIS estimation. The computation of the Shapley value is then accelerated using attention, meanwhile retaining the enhanced instance identification and prioritization. We further introduce a framework for the progressive assignment of pseudo bags based on estimated IIS, encouraging more balanced attention distributions in MIL models. Our extensive experiments on CAMELYON-16, BRACS, TCGA-LUNG, and TCGA-BRCA datasets show our method's superiority over existing state-of-the-art approaches, offering enhanced interpretability and class-wise insights. Our source code is available at https://github.com/RenaoYan/PMIL.

Out-of-distribution detection in digital pathology: Do foundation models bring the end to reconstruction-based approaches?

Artificial intelligence (AI) has shown promising results for computational pathology tasks. However, one of the limitations in clinical practice is that these algorithms are optimised for the distribution represented by the training data. For out-of-distribution (OOD) data, they often deliver predictions with equal confidence, even though these often are incorrect. In the pursuit of OOD detection in digital pathology, this study evaluates the state-of-the-art (SOTA) in computational pathology OOD detection, based on diffusion probabilistic models, specifically by adapting the latent diffusion model (LDM) for this purpose (AnoLDM). We compare this against post-hoc methods based on the latent space of foundation models, which are SOTA in general computer vision research. The approaches are not only evaluated on data from the same medical centres as the training set, but also on several datasets with data distribution shifts. The results show that AnoLDM performs similarly well or better than diffusion model based approaches published in previous studies in computational pathology but with reduced computational costs. However, our optimal configuration of an approach based on foundation models (kang_residual) outperforms AnoLDM on OOD detection on data not experiencing any covariate shifts, with an AUROC of 96.17 versus 91.86. Interestingly, AnoLDM is more successful at handling the data distribution shifts investigated in this study. However, both AnoLDM and kang_residual suffer substantial loss in the performance under the data distribution shifts, hence future work should focus on improving the generalisation of OOD detection for computational pathology applications.

Deep learning model to diagnose cardiac amyloidosis from haematoxylin/eosin-stained myocardial tissue

Aims

Amyloid deposition in myocardial tissue is a definitive feature for diagnosing cardiac amyloidosis, though less invasive imaging modalities such as bone tracer cardiac scintigraphy and cardiac magnetic resonance imaging have been established as first steps for its diagnosis. This study aimed to develop a deep learning model to support the diagnosis of cardiac amyloidosis from haematoxylin/eosin (HE)-stained myocardial tissue.

Methods and results

This single-centre retrospective observational study enrolled 166 patients who underwent myocardial biopsies between 2008 and 2022, including 76 patients diagnosed with cardiac amyloidosis and 90 with other diagnoses. A deep learning model was developed to output the probabilities of cardiac amyloidosis for all the small patches cutout from each myocardial specimen. The developed model highlighted the area in the stained images as highly suspicious, corresponding to where Dylon staining marked amyloid deposition, and discriminated the patches in the evaluation dataset with an area under the curve of 0.965. Provided that the diagnostic criterion for cardiac amyloidosis was defined as a median probability of cardiac amyloidosis >50% in all patches, the model achieved perfect performance in discriminating patients with cardiac amyloidosis from those without it, with an area under the curve of 1.0.

Conclusion

A deep learning model was developed to diagnose cardiac amyloidosis from HE-stained myocardial tissue accurately. Although further prospective validation of this model using HE-stained myocardial tissues from multiple centres is needed, it may help minimize the risk of missing cardiac amyloidosis and maximize the utility of histological diagnosis in clinical practice.

Evaluation of a Task-Specific Self-Supervised Learning Framework in Digital Pathology Relative to Transfer Learning Approaches and Existing Foundation Models

An integral stage in typical digital pathology workflows involves deriving specific features from tiles extracted from a tessellated whole-slide image. Notably, various computer vision neural network architectures, particularly the ImageNet pretrained, have been extensively used in this domain. This study critically analyzes multiple strategies for encoding tiles to understand the extent of transfer learning and identify the most effective approach. The study categorizes neural network performance into 3 weight initialization methods: random, ImageNet-based, and self-supervised learning. Additionally, we propose a framework based on task-specific self-supervised learning, which introduces a shallow feature extraction method, employing a spatial-channel attention block to glean distinctive features optimized for histopathology intricacies. Across 2 different downstream classification tasks (patch classification and weakly supervised whole-slide image classification) with diverse classification data sets, including colorectal cancer histology, Patch Camelyon, prostate cancer detection, The Cancer Genome Atlas, and CIFAR-10, our task-specific self-supervised encoding approach consistently outperforms other convolutional neural network-based encoders. The better performances highlight the potential of task-specific attention-based self-supervised training in tailoring feature extraction for histopathology, indicating a shift from using pretrained models originating outside the histopathology domain. Our study supports the idea that task-specific self-supervised learning allows domain-specific feature extraction, encouraging a more focused analysis.

Role of artificial intelligence in haematolymphoid diagnostics

The advent of digital pathology and the deployment of high-throughput molecular techniques are generating an unprecedented mass of data. Thanks to advances in computational sciences, artificial intelligence (AI) approaches represent a promising avenue for extracting relevant information from complex data structures. From diagnostic assistance to powerful research tools, the potential fields of application of machine learning techniques in pathology are vast and constitute the subject of considerable research work. The aim of this article is to provide an overview of the potential applications of AI in the field of haematopathology and to define the role that these emerging technologies could play in our laboratories in the short to medium term.

Conformable Ultrasound Breast Patch - The Future of Breast Cancer Screening?

Breast cancer is the most common cancer type among women worldwide with an average lifetime risk of 12.9%. Early detection and screening are the most important factors for improved prognosis and mammography remains the main screening tool for the average risk patients. Ultrasound (US) is used in women with elevated breast cancer risk, younger patients and patients with extremely dense breasts. Conventional US has certain limitations including operator dependence and reported low specificity. We designed a conformable US device (cUSBr-Patch) which offers large-area, deep tissue scanning and multi-angle, repeatable breast imaging. It is able to detect lesions as small as 1mm with excellent accuracy and reliability validated by in vivo comparison with conventional US. This is a user-friendly, innovating device designed to be used by patients with the potential to reshape our approach to breast cancer screening.

Advancements in pathology: Digital transformation, precision medicine, and beyond

Pathology, a cornerstone of medical diagnostics and research, is undergoing a revolutionary transformation fueled by digital technology, molecular biology advancements, and big data analytics. Digital pathology converts conventional glass slides into high-resolution digital images, enhancing collaboration and efficiency among pathologists worldwide. Integrating artificial intelligence (AI) and machine learning (ML) algorithms with digital pathology improves diagnostic accuracy, particularly in complex diseases like cancer. Molecular pathology, facilitated by next-generation sequencing (NGS), provides comprehensive genomic, transcriptomic, and proteomic insights into disease mechanisms, guiding personalized therapies. Immunohistochemistry (IHC) plays a pivotal role in biomarker discovery, refining disease classification and prognostication. Precision medicine integrates pathology's molecular findings with individual genetic, environmental, and lifestyle factors to customize treatment strategies, optimizing patient outcomes. Telepathology extends diagnostic services to underserved areas through remote digital pathology. Pathomics leverages big data analytics to extract meaningful insights from pathology images, advancing our understanding of disease pathology and therapeutic targets. Virtual autopsies employ non-invasive imaging technologies to revolutionize forensic pathology. These innovations promise earlier diagnoses, tailored treatments, and enhanced patient care. Collaboration across disciplines is essential to fully realize the transformative potential of these advancements in medical practice and research.

RSDCNet: An efficient and lightweight deep learning model for benign and malignant pathology detection in breast cancer

Background

Breast cancer is a leading malignant tumor among women globally, with its pathological classification into benign or malignant directly influencing treatment strategies and prognosis. Traditional diagnostic methods, reliant on manual interpretation, are not only time-intensive and subjective but also susceptible to variability based on the pathologist's expertise and workload. Consequently, the development of an efficient, automated, and precise pathological detection method is crucial.

Methods

This study introduces RSDCNet, an enhanced lightweight neural network architecture designed for the automatic detection of benign and malignant breast cancer pathology. Utilizing the BreakHis dataset, which comprises 9109 microscopic images of breast tumors including various differentiation levels of benign and malignant samples, RSDCNet integrates depthwise separable convolution and SCSE modules. This integration aims to reduce model parameters while enhancing key feature extraction capabilities, thereby achieving both lightweight design and high efficiency.

Results

RSDCNet demonstrated superior performance across multiple evaluation metrics in the classification task. The model achieved an accuracy of 0.9903, a recall of 0.9897, an F1 score of 0.9888, and a precision of 0.9879, outperforming established deep learning models such as EfficientNet, RegNet, HRNet, and ViT. Notably, RSDCNet's parameter count stood at just 1,199,662, significantly lower than HRNet's 19,254,102 and ViT's 85,800,194, highlighting its enhanced resource efficiency.

Conclusion

The RSDCNet model presented in this study excels in the efficient and accurate classification of benign and malignant breast cancer pathology. Compared to traditional methods and other leading models, RSDCNet not only reduces computational resource consumption but also offers improved feature extraction and clinical interpretability. This advancement provides substantial technical support for the intelligent diagnosis of breast cancer, paving the way for more effective treatment planning and prognosis assessment.

Ultrasound-Based Deep Learning Radiomics Nomogram for Tumor and Axillary Lymph Node Status Prediction After Neoadjuvant Chemotherapy

RATIONALE AND OBJECTIVES

This study aims to explore the feasibility of the deep learning radiomics nomogram (DLRN) for predicting tumor status and axillary lymph node metastasis (ALNM) after neoadjuvant chemotherapy (NAC) in patients with breast cancer. Additionally, we employ a Cox regression model for survival analysis to validate the effectiveness of the fusion algorithm.

MATERIALS AND METHODS

A total of 243 patients who underwent NAC were retrospectively included between October 2014 and July 2022. The DLRN integrated clinical characteristics as well as radiomics and deep transfer learning features extracted from ultrasound (US) images. The diagnostic performance of DLRN was evaluated by constructing ROC curves, and the clinical usefulness of models was assessed using decision curve analysis (DCA). A survival model was developed to validate the effectiveness of the fusion algorithm.

RESULTS

In the training cohort, the DLRN yielded area under the receiver operating characteristic curve values of 0.984 and 0.985 for the tumor and LNM, while 0.892 and 0.870, respectively, in the test cohort. The consistency indices (C-index) of the nomogram were 0.761 and 0.731, respectively, in the training and test cohorts. The Kaplan-Meier survival curves showed that patients in the high-risk group had significantly poorer overall survival than patients in the low-risk group (P < 0.05).

CONCLUSION

The US-based DLRN model could hold promise as clinical guidance for predicting the status of tumors and LNM after NAC in patients with breast cancer. This fusion model can also predict the prognosis of patients, which could help clinicians make better clinical decisions.

Near-Infrared Autofluorescence Signature: A New Parameter for Intraoperative Assessment of Parathyroid Glands in Primary Hyperparathyroidism

BACKGROUND

The success of parathyroidectomy in primary hyperparathyroidism depends on the intraoperative differentiation of diseased from normal glands. Deep learning can potentially be applied to digitalize this subjective interpretation process that relies heavily on surgeon expertise. In this study, we aimed to investigate whether diseased vs normal parathyroid glands have different near-infrared autofluorescence (NIRAF) signatures and whether related deep learning models can predict normal vs diseased parathyroid glands based on intraoperative in vivo images.

STUDY DESIGN

This prospective study included patients who underwent parathyroidectomy for primary hyperparathyroidism or thyroidectomy using intraoperative NIRAF imaging at a single tertiary referral center from November 2019 to March 2024. Autofluorescence intensity and heterogeneity index of normal vs diseased parathyroid glands were compared, and a deep learning model was developed.

RESULTS

NIRAF images of a total of 1,506 normal and 597 diseased parathyroid glands from 797 patients were analyzed. Normal vs diseased glands exhibited a higher median normalized NIRAF intensity (2.68 [2.19 to 3.23] vs 2.09 [1.68 to 2.56] pixels, p < 0.0001) and lower heterogeneity index (0.11 [0.08 to 0.15] vs 0.18 [0.13 to 0.23], p < 0.0001). On receiver operating characteristics analysis, optimal thresholds to predict a diseased gland were 2.22 in pixel intensity and 0.14 in heterogeneity index. On deep learning, precision and recall of the model were 83.3% each, and area under the precision-recall curve was 0.908.

CONCLUSIONS

Normal and diseased parathyroid glands in primary hyperparathyroidism have different intraoperative NIRAF patterns that could be quantified with intensity and heterogeneity analyses. Visual deep learning models relying on these NIRAF signatures could be built to assist surgeons in differentiating normal from diseased parathyroid glands.

FR-MIL: Distribution Re-Calibration-Based Multiple Instance Learning With Transformer for Whole Slide Image Classification

In digital pathology, whole slide images (WSI) are crucial for cancer prognostication and treatment planning. WSI classification is generally addressed using multiple instance learning (MIL), alleviating the challenge of processing billions of pixels and curating rich annotations. Though recent MIL approaches leverage variants of the attention mechanism to learn better representations, they scarcely study the properties of the data distribution itself i.e., different staining and acquisition protocols resulting in intra-patch and inter-slide variations. In this work, we first introduce a distribution re-calibration strategy to shift the feature distribution of a WSI bag (instances) using the statistics of the max-instance (critical) feature. Second, we enforce class (bag) separation via a metric loss assuming that positive bags exhibit larger magnitudes than negatives. We also introduce a generative process leveraging Vector Quantization (VQ) for improved instance discrimination i.e., VQ helps model bag latent factors for improved classification. To model spatial and context information, a position encoding module (PEM) is employed with transformer-based pooling by multi-head self-attention (PMSA). Evaluation of popular WSI benchmark datasets reveals our approach improves over state-of-the-art MIL methods. Further, we validate the general applicability of our method on classic MIL benchmark tasks and for point cloud classification with limited points. https://github.com/PhilipChicco/FRMIL.

Multi-Task Adaptive Resolution Network for Lymph Node Metastasis Diagnosis From Whole Slide Images of Colorectal Cancer

Automated detection of lymph node metastasis (LNM) holds great potential to alleviate the workload of doctors and reduce misinterpretations. Despite the practical successes achieved, effectively addressing the highly complex and heterogeneous tumor microenvironment remains an open and challenging problem, especially when tumor subtypes intermingle and are difficult to delineate. In this paper, we propose a multi-task adaptive resolution network, named MAR-Net, for LNM detection and subtyping in complex mixed-type cancers. Specifically, we construct a resolution-aware module to mine heterogeneous diagnostic information, which exploits the multi-scale pyramid information and adaptively combines multi-resolution structured features for comprehensive representation. Additionally, we adopt a multi-task learning approach that simultaneously addresses LNM detection and subtyping, reducing model instability during optimization and improving performance across both tasks. More importantly, to rectify the potential misclassification of tumor subtypes, we elaborately design a hierarchical subtying refinement (HSR) algorithm that leverages a generic segmentation model informed by pathologists' prior knowledge. Evaluations have been conducted on three private and one public cancer datasets (554 WSIs, 4.8 million patches). Our experimental results demonstrate that the proposed method consistently achieves superior performance compared to the state-of-the-art methods, achieving 0.5% to 3.2% higher AUC in LNM detection and 3.8% to 4.4% higher AUC in LNM subtyping.

Machine learning methods for histopathological image analysis: Updates in 2024

The combination of artificial intelligence and digital pathology has emerged as a transformative force in healthcare and biomedical research. As an update to our 2018 review, this review presents comprehensive analysis of machine learning applications in histopathological image analysis, with focus on the developments since 2018. We highlight significant advances that have expanded the technical capabilities and practical applications of computational pathology. The review examines progress in addressing key challenges in the field as follows: processing of gigapixel whole slide images, insufficient labeled data, multidimensional analysis, domain shifts across institutions, and interpretability of machine learning models. We evaluate emerging trends, such as foundation models and multimodal integration, that are reshaping the field. Overall, our review highlights the potential of machine learning in enhancing both routine pathological analysis and scientific discovery in pathology. By providing this comprehensive overview, this review aims to guide researchers and clinicians in understanding the current state of the pathology image analysis field and its future trajectory.

Role of artificial intelligence in cancer detection using protein p53: A Review

Normal cell development and prevention of tumor formation rely on the tumor-suppressor protein p53. This crucial protein is produced from the Tp53 gene, which encodes the p53 protein. The p53 protein plays a vital role in regulating cell growth, DNA repair, and apoptosis (programmed cell death), thereby maintaining the integrity of the genome and preventing the formation of tumors. Since p53 was discovered 43 years ago, many researchers have clarified its functions in the development of tumors. With the support of the protein p53 and targeted artificial intelligence modeling, it will be possible to detect cancer and tumor activity at an early stage. This will open up new research opportunities. In this review article, a comprehensive analysis was conducted on different machine learning techniques utilized in conjunction with the protein p53 to predict and speculate cancer. The study examined the types of data incorporated and evaluated the performance of these techniques. The aim was to provide a thorough understanding of the effectiveness of machine learning in predicting and speculating cancer using the protein p53.

From pixels to patients: the evolution and future of deep learning in cancer diagnostics

Deep learning has revolutionized cancer diagnostics, shifting from pixel-based image analysis to more comprehensive, patient-centric care. This opinion article explores recent advancements in neural network architectures, highlighting their evolution in biomedical research and their impact on medical imaging interpretation and multimodal data integration. We emphasize the need for domain-specific artificial intelligence (AI) systems capable of handling complex clinical tasks, advocating for the development of multimodal large language models that can integrate diverse data sources. These models have the potential to significantly enhance the precision and efficiency of cancer diagnostics, transforming AI from a supplementary tool into a core component of clinical decision-making, ultimately improving patient outcomes and advancing cancer care.

Advancements in Artificial Intelligence-Enhanced Imaging Diagnostics for the Management of Liver Disease-Applications and Challenges in Personalized Care

Liver disease can significantly impact life expectancy, making early diagnosis and therapeutic intervention critical challenges in medical care. Imaging diagnostics play a crucial role in diagnosing and managing liver diseases. Recently, the application of artificial intelligence (AI) in medical imaging analysis has become indispensable in healthcare. AI, trained on vast datasets of medical images, has sometimes demonstrated diagnostic accuracy that surpasses that of human experts. AI-assisted imaging diagnostics are expected to contribute significantly to the standardization of diagnostic quality. Furthermore, AI has the potential to identify image features that are imperceptible to humans, thereby playing an essential role in clinical decision-making. This capability enables physicians to make more accurate diagnoses and develop effective treatment strategies, ultimately improving patient outcomes. Additionally, AI is anticipated to become a powerful tool in personalized medicine. By integrating individual patient imaging data with clinical information, AI can propose optimal plans for treatment, making it an essential component in the provision of the most appropriate care for each patient. Current reports highlight the advantages of AI in managing liver diseases. As AI technology continues to evolve, it is expected to advance personalized diagnostics and treatments and contribute to overall improvements in healthcare quality.

Intersection of Performance, Interpretability, and Fairness in Neural Prototype Tree for Chest X-Ray Pathology Detection: Algorithm Development and Validation Study

BACKGROUND

While deep learning classifiers have shown remarkable results in detecting chest X-ray (CXR) pathologies, their adoption in clinical settings is often hampered by the lack of transparency. To bridge this gap, this study introduces the neural prototype tree (NPT), an interpretable image classifier that combines the diagnostic capability of deep learning models and the interpretability of the decision tree for CXR pathology detection.

OBJECTIVE

This study aimed to investigate the utility of the NPT classifier in 3 dimensions, including performance, interpretability, and fairness, and subsequently examined the complex interaction between these dimensions. We highlight both local and global explanations of the NPT classifier and discuss its potential utility in clinical settings.

METHODS

This study used CXRs from the publicly available Chest X-ray 14, CheXpert, and MIMIC-CXR datasets. We trained 6 separate classifiers for each CXR pathology in all datasets, 1 baseline residual neural network (ResNet)-152, and 5 NPT classifiers with varying levels of interpretability. Performance, interpretability, and fairness were measured using the area under the receiver operating characteristic curve (ROC AUC), interpretation complexity (IC), and mean true positive rate (TPR) disparity, respectively. Linear regression analyses were performed to investigate the relationship between IC and ROC AUC, as well as between IC and mean TPR disparity.

RESULTS

The performance of the NPT classifier improved as the IC level increased, surpassing that of ResNet-152 at IC level 15 for the Chest X-ray 14 dataset and IC level 31 for the CheXpert and MIMIC-CXR datasets. The NPT classifier at IC level 1 exhibited the highest degree of unfairness, as indicated by the mean TPR disparity. The magnitude of unfairness, as measured by the mean TPR disparity, was more pronounced in groups differentiated by age (chest X-ray 14 0.112, SD 0.015; CheXpert 0.097, SD 0.010; MIMIC 0.093, SD 0.017) compared to sex (chest X-ray 14 0.054 SD 0.012; CheXpert 0.062, SD 0.008; MIMIC 0.066, SD 0.013). A significant positive relationship between interpretability (ie, IC level) and performance (ie, ROC AUC) was observed across all CXR pathologies (P<.001). Furthermore, linear regression analysis revealed a significant negative relationship between interpretability and fairness (ie, mean TPR disparity) across age and sex subgroups (P<.001).

CONCLUSIONS

By illuminating the intricate relationship between performance, interpretability, and fairness of the NPT classifier, this research offers insightful perspectives that could guide future developments in effective, interpretable, and equitable deep learning classifiers for CXR pathology detection.

Improved DeTraC Binary Coyote Net-Based Multiple Instance Learning for Predicting Lymph Node Metastasis of Breast Cancer From Whole-Slide Pathological Images

BACKGROUND

Early detection of lymph node metastasis in breast cancer is vital for improving treatment outcomes and prognosis.

METHODS

This study introduces an Improved Decompose, Transfer, and Compose Binary Coyote Net-based Multiple Instance Learning (ImDeTraC-BCNet-MIL) method for predicting lymph node metastasis from Whole Slide Images (WSIs) using multiple instance learning. The method involves segmenting WSIs into patches using Otsu and double-dimensional clustering techniques. The developed multiple instance learning approach introduces a paradigm into computational pathology by shaping pathological data and constructing features. ImDeTraC-BCNet-MIL was utilised for feature generation during both training and testing to differentiate lymph node metastasis in WSIs.

RESULTS

The proposed model achieves the highest accuracy of 95.3% and 99.8%, precision values of 98% and 99.8%, and recall rates of 92.9% and 99.8% on the Camelyon16 and Camelyon17 datasets.

CONCLUSIONS

These findings underscore the effectiveness of ImDeTraC-BCNet-MIL in enhancing the early detection of lymph node metastasis in breast cancer.

Evaluation of a Deep Learning Model for Metastatic Squamous Cell Carcinoma Prediction From Whole Slide Images

CONTEXT.—

Squamous cell carcinoma (SCC) is a histologic type of cancer that exhibits various degrees of keratinization. Identifying lymph node metastasis in SCC is crucial for prognosis and treatment strategies. Although artificial intelligence (AI) has shown promise in cancer prediction, applications specifically targeting SCC are limited.

OBJECTIVE.—

To design and validate a deep learning model tailored to predict metastatic SCC in radical lymph node dissection specimens using whole slide images (WSIs).

DESIGN.—

Using the EfficientNetB1 architecture, a model was trained on 6587 WSIs (2413 SCC and 4174 nonneoplastic) from several hospitals, encompassing esophagus, head and neck, lung, and skin specimens. The training exclusively relied on WSI-level labels without annotations. We evaluated the model on a test set consisting of 541 WSIs (41 SCC and 500 nonneoplastic) of radical lymph node dissection specimens.

RESULTS.—

The model exhibited high performance, with receiver operating characteristic curve areas under the curve between 0.880 and 0.987 in detecting SCC metastases in lymph nodes. Although true positives and negatives were accurately identified, certain limitations were observed. These included false positives due to germinal centers, dust cell aggregations, and specimen-handling artifacts, as well as false negatives due to poor differentiation.

CONCLUSIONS.—

The developed artificial intelligence model presents significant potential in enhancing SCC lymph node detection, offering workload reduction for pathologists and increasing diagnostic efficiency. Continuous refinement is needed to overcome existing challenges, making the model more robust and clinically relevant.

Multimodal Gated Mixture of Experts Using Whole Slide Image and Flow Cytometry for Multiple Instance Learning Classification of Lymphoma

In this study, we present a deep-learning-based multimodal classification method for lymphoma diagnosis in digital pathology, which utilizes a whole slide image (WSI) as the primary image data and flow cytometry (FCM) data as auxiliary information. In pathological diagnosis of malignant lymphoma, FCM serves as valuable auxiliary information during the diagnosis process, offering useful insights into predicting the major class (superclass) of subtypes. By incorporating both images and FCM data into the classification process, we can develop a method that mimics the diagnostic process of pathologists, enhancing the explainability. In order to incorporate the hierarchical structure between superclasses and their subclasses, the proposed method utilizes a network structure that effectively combines the mixture of experts (MoE) and multiple instance learning (MIL) techniques, where MIL is widely recognized for its effectiveness in handling WSIs in digital pathology. The MoE network in the proposed method consists of a gating network for superclass classification and multiple expert networks for (sub)class classification, specialized for each superclass. To evaluate the effectiveness of our method, we conducted experiments involving a six-class classification task using 600 lymphoma cases. The proposed method achieved a classification accuracy of 72.3%, surpassing the 69.5% obtained through the straightforward combination of FCM and images, as well as the 70.2% achieved by the method using only images. Moreover, the combination of multiple weights in the MoE and MIL allows for the visualization of specific cellular and tumor regions, resulting in a highly explanatory model that cannot be attained with conventional methods. It is anticipated that by targeting a larger number of classes and increasing the number of expert networks, the proposed method could be effectively applied to the real problem of lymphoma diagnosis.

A Deep Learning Approach for the Identification of the Molecular Subtypes of Pancreatic Ductal Adenocarcinoma Based on Whole Slide Pathology Images

Delayed diagnosis and treatment resistance result in high pancreatic ductal adenocarcinoma (PDAC) mortality rates. Identifying molecular subtypes can improve treatment, but current methods are costly and time-consuming. In this study, deep learning models were used to identify histologic features that classify PDAC molecular subtypes based on routine hematoxylin-eosin-stained histopathologic slides. A total of 97 histopathology slides associated with resectable PDAC from The Cancer Genome Atlas project were used to train a deep learning model and test the performance on 44 needle biopsy material (110 slides) from a local annotated patient cohort. The model achieved balanced accuracy of 96.19% and 83.03% in identifying the classical and basal subtypes of PDAC in The Cancer Genome Atlas and the local cohort, respectively. This study provides a promising method to cost-effectively and rapidly classify PDAC molecular subtypes based on routine hematoxylin-eosin-stained slides, potentially leading to more effective clinical management of this disease.

Publicly available datasets of breast histopathology H&E whole-slide images: A scoping review

Advancements in digital pathology and computing resources have made a significant impact in the field of computational pathology for breast cancer diagnosis and treatment. However, access to high-quality labeled histopathological images of breast cancer is a big challenge that limits the development of accurate and robust deep learning models. In this scoping review, we identified the publicly available datasets of breast H&E-stained whole-slide images (WSIs) that can be used to develop deep learning algorithms. We systematically searched 9 scientific literature databases and 9 research data repositories and found 17 publicly available datasets containing 10 385 H&E WSIs of breast cancer. Moreover, we reported image metadata and characteristics for each dataset to assist researchers in selecting proper datasets for specific tasks in breast cancer computational pathology. In addition, we compiled 2 lists of breast H&E patches and private datasets as supplementary resources for researchers. Notably, only 28% of the included articles utilized multiple datasets, and only 14% used an external validation set, suggesting that the performance of other developed models may be susceptible to overestimation. The TCGA-BRCA was used in 52% of the selected studies. This dataset has a considerable selection bias that can impact the robustness and generalizability of the trained algorithms. There is also a lack of consistent metadata reporting of breast WSI datasets that can be an issue in developing accurate deep learning models, indicating the necessity of establishing explicit guidelines for documenting breast WSI dataset characteristics and metadata.

Cross-Attention-Based Saliency Inference for Predicting Cancer Metastasis on Whole Slide Images

Although multiple instance learning (MIL) methods are widely used for automatic tumor detection on whole slide images (WSI), they suffer from the extreme class imbalance WSIs containing small tumors where the tumor may include only a few isolated cells. For early detection, it is important that MIL algorithms can identify small tumors. Existing studies have attempted to address this issue using attention-based architectures and instance selection-based methodologies but have not produced significant improvements. This paper proposes cross-attention-based salient instance inference MIL (CASiiMIL), which involves a novel saliency-informed attention mechanism to identify small tumors (e.g., breast cancer lymph node micro-metastasis) on WSIs without needing any annotations. In addition to this new attention mechanism, we introduce a negative representation learning algorithm to facilitate the learning of saliency-informed attention weights for improved sensitivity on tumor WSIs. The proposed model outperforms the state-of-the-art MIL methods on two popular tumor metastasis detection datasets. The proposed approach demonstrates great cross-center generalizability, high accuracy in classifying WSIs with small tumor lesions, and excellent interpretability attributed to the saliency-informed attention weights. We expect that the proposed method will pave the way for training algorithms for early tumor detection on large datasets where acquiring fine-grained annotations is is not practical.