Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer

Multimodal Gated Mixture of Experts Using Whole Slide Image and Flow Cytometry for Multiple Instance Learning Classification of Lymphoma

In this study, we present a deep-learning-based multimodal classification method for lymphoma diagnosis in digital pathology, which utilizes a whole slide image (WSI) as the primary image data and flow cytometry (FCM) data as auxiliary information. In pathological diagnosis of malignant lymphoma, FCM serves as valuable auxiliary information during the diagnosis process, offering useful insights into predicting the major class (superclass) of subtypes. By incorporating both images and FCM data into the classification process, we can develop a method that mimics the diagnostic process of pathologists, enhancing the explainability. In order to incorporate the hierarchical structure between superclasses and their subclasses, the proposed method utilizes a network structure that effectively combines the mixture of experts (MoE) and multiple instance learning (MIL) techniques, where MIL is widely recognized for its effectiveness in handling WSIs in digital pathology. The MoE network in the proposed method consists of a gating network for superclass classification and multiple expert networks for (sub)class classification, specialized for each superclass. To evaluate the effectiveness of our method, we conducted experiments involving a six-class classification task using 600 lymphoma cases. The proposed method achieved a classification accuracy of 72.3%, surpassing the 69.5% obtained through the straightforward combination of FCM and images, as well as the 70.2% achieved by the method using only images. Moreover, the combination of multiple weights in the MoE and MIL allows for the visualization of specific cellular and tumor regions, resulting in a highly explanatory model that cannot be attained with conventional methods. It is anticipated that by targeting a larger number of classes and increasing the number of expert networks, the proposed method could be effectively applied to the real problem of lymphoma diagnosis.

Publicly available datasets of breast histopathology H&E whole-slide images: A scoping review

Advancements in digital pathology and computing resources have made a significant impact in the field of computational pathology for breast cancer diagnosis and treatment. However, access to high-quality labeled histopathological images of breast cancer is a big challenge that limits the development of accurate and robust deep learning models. In this scoping review, we identified the publicly available datasets of breast H&E-stained whole-slide images (WSIs) that can be used to develop deep learning algorithms. We systematically searched 9 scientific literature databases and 9 research data repositories and found 17 publicly available datasets containing 10 385 H&E WSIs of breast cancer. Moreover, we reported image metadata and characteristics for each dataset to assist researchers in selecting proper datasets for specific tasks in breast cancer computational pathology. In addition, we compiled 2 lists of breast H&E patches and private datasets as supplementary resources for researchers. Notably, only 28% of the included articles utilized multiple datasets, and only 14% used an external validation set, suggesting that the performance of other developed models may be susceptible to overestimation. The TCGA-BRCA was used in 52% of the selected studies. This dataset has a considerable selection bias that can impact the robustness and generalizability of the trained algorithms. There is also a lack of consistent metadata reporting of breast WSI datasets that can be an issue in developing accurate deep learning models, indicating the necessity of establishing explicit guidelines for documenting breast WSI dataset characteristics and metadata.

Computational methods for metastasis detection in lymph nodes and characterization of the metastasis-free lymph node microarchitecture: A systematic-narrative hybrid review

Background

Histological examination of tumor draining lymph nodes (LNs) plays a vital role in cancer staging and prognostication. However, as soon as a LN is classed as metastasis-free, no further investigation will be performed and thus, potentially clinically relevant information detectable in tumor-free LNs is currently not captured.

Objective

To systematically study and critically assess methods for the analysis of digitized histological LN images described in published research.

Methods

A systematic search was conducted in several public databases up to December 2023 using relevant search terms. Studies using brightfield light microscopy images of hematoxylin and eosin or immunohistochemically stained LN tissue sections aiming to detect and/or segment LNs, their compartments or metastatic tumor using artificial intelligence (AI) were included. Dataset, AI methodology, cancer type, and study objective were compared between articles.

Results

A total of 7201 articles were collected and 73 articles remained for detailed analyses after article screening. Of the remaining articles, 86% aimed at LN metastasis identification, 8% aimed at LN compartment segmentation, and remaining focused on LN contouring. Furthermore, 78% of articles used patch classification and 22% used pixel segmentation models for analyses. Five out of six studies (83%) of metastasis-free LNs were performed on publicly unavailable datasets, making quantitative article comparison impossible.

Conclusions

Multi-scale models mimicking multiple microscopy zooms show promise for computational LN analysis. Large-scale datasets are needed to establish the clinical relevance of analyzing metastasis-free LN in detail. Further research is needed to identify clinically interpretable metrics for LN compartment characterization.

SPMLD: A skin pathological image dataset for non-melanoma with detailed lesion area annotation

Skin tumors are the most common tumors in humans and the clinical characteristics of three common non-melanoma tumors (IDN, SK, BCC) are similar, resulting in a high misdiagnosis rate. The accurate differential diagnosis of these tumors needs to be judged based on pathological images. However, a shortage of experienced dermatological pathologists leads to bias in the diagnostic accuracy of these skin tumors in China. In this paper, we establish a skin pathological image dataset, SPMLD, for three non-melanoma to achieve automatic and accurate intelligent identification for them. Meanwhile, we propose a lesion-area-based enhanced classification network with the KLS module and an attention module. Specifically, we first collect thousands of H&E-stained tissue sections from patients with clinically and pathologically confirmed IDN, SK, and BCC from a single-center hospital. Then, we scan them to construct a pathological image dataset of these three skin tumors. Furthermore, we mark the complete lesion area of the entire pathology image to better learn the pathologist's diagnosis process. In addition, we applied the proposed network for lesion classification prediction on the SPMLD dataset. Finally, we conduct a series of experiments to demonstrate that this annotation and our network can effectively improve the classification results of various networks. The source dataset and code are available at https://github.com/efss24/SPMLD.git.

Artificial intelligence enhances whole-slide interpretation of PD-L1 CPS in triple-negative breast cancer: A multi-institutional ring study

BACKGROUND AND AIMS

Evaluation of the programmed cell death ligand-1 (PD-L1) combined positive score (CPS) is vital to predict the efficacy of the immunotherapy in triple-negative breast cancer (TNBC), but pathologists show substantial variability in the consistency and accuracy of the interpretation. It is of great importance to establish an objective and effective method which is highly repeatable.

METHODS

We proposed a model in a deep learning-based framework, which at the patch level incorporated cell analysis and tissue region analysis, followed by the whole-slide level fusion of patch results. Three rounds of ring studies (RSs) were conducted. Twenty-one pathologists of different levels from four institutions evaluated the PD-L1 CPS in TNBC specimens as continuous scores by visual assessment and our artificial intelligence (AI)-assisted method.

RESULTS

In the visual assessment, the interpretation results of PD-L1 (Dako 22C3) CPS by different levels of pathologists have significant differences and showed weak consistency. Using AI-assisted interpretation, there were no significant differences between all pathologists (P = 0.43), and the intraclass correlation coefficient (ICC) value was increased from 0.618 [95% confidence interval (CI) = 0.524-0.719] to 0.931 (95% CI = 0.902-0.955). The accuracy of interpretation result is further improved to 0.919 (95% CI = 0.886-0.947). Acceptance of AI results by junior pathologists was the highest among all levels, and 80% of the AI results were accepted overall.

CONCLUSION

With the help of the AI-assisted diagnostic method, different levels of pathologists achieved excellent consistency and repeatability in the interpretation of PD-L1 (Dako 22C3) CPS. Our AI-assisted diagnostic approach was proved to strengthen the consistency and repeatability in clinical practice.

Advances of surgical robotics: image-guided classification and application

Surgical robotics application in the field of minimally invasive surgery has developed rapidly and has been attracting increasingly more research attention in recent years. A common consensus has been reached that surgical procedures are to become less traumatic and with the implementation of more intelligence and higher autonomy, which is a serious challenge faced by the environmental sensing capabilities of robotic systems. One of the main sources of environmental information for robots are images, which are the basis of robot vision. In this review article, we divide clinical image into direct and indirect based on the object of information acquisition, and into continuous, intermittent continuous, and discontinuous according to the target-tracking frequency. The characteristics and applications of the existing surgical robots in each category are introduced based on these two dimensions. Our purpose in conducting this review was to analyze, summarize, and discuss the current evidence on the general rules on the application of image technologies for medical purposes. Our analysis gives insight and provides guidance conducive to the development of more advanced surgical robotics systems in the future.

Antibody-Mediated Rejection in Lung Transplantation: Diagnosis and Therapeutic Armamentarium in a 21st Century Perspective

Humoral immunity is a major waypoint towards chronic allograft dysfunction in lung transplantation (LT) recipients. Though allo-immunization and antibody-mediated rejection (AMR) are well-known entities, some diagnostic gaps need to be addressed. Morphological analysis could be enhanced by digital pathology and artificial intelligence-based companion tools. Graft transcriptomics can help to identify graft failure phenotypes or endotypes. Donor-derived cell free DNA is being evaluated for graft-loss risk stratification and tailored surveillance. Preventative therapies should be tailored according to risk. The donor pool can be enlarged for candidates with HLA sensitization, with strategies combining plasma exchange, intravenous immunoglobulin and immune cell depletion, or with emerging or innovative therapies such as imlifidase or immunoadsorption. In cases of insufficient pre-transplant desensitization, the effects of antibodies on the allograft can be prevented by targeting the complement cascade, although evidence for this strategy in LT is limited. In LT recipients with a humoral response, strategies are combined, including depletion of immune cells (plasmapheresis or immunoadsorption), inhibition of immune pathways, or modulation of the inflammatory cascade, which can be achieved with photopheresis. Altogether, these innovative techniques offer promising perspectives for LT recipients and shape the 21st century's armamentarium against AMR.

Deep learning-based diagnosis and survival prediction of patients with renal cell carcinoma from primary whole slide images

There is an urgent clinical demand to explore novel diagnostic and prognostic biomarkers for renal cell carcinoma (RCC). We proposed deep learning-based artificial intelligence strategies. The study included 1752 whole slide images from multiple centres. Based on the pixel-level of RCC segmentation, the diagnosis diagnostic model achieved an area under the receiver operating characteristic curve (AUC) of 0.977 (95% CI 0.969-0.984) in the external validation cohort. In addition, our diagnostic model exhibited excellent performance in the differential diagnosis of RCC from renal oncocytoma, which achieved an AUC of 0.951 (0.922-0.972). The graderisk for the recognition of high-grade tumour achieved AUCs of 0.840 (0.805-0.871) in the Cancer Genome Atlas (TCGA) cohort, 0.857 (0.813-0.894) in the Shanghai General Hospital (General) cohort, and 0.894 (0.842-0.933) in the Clinical Proteomic Tumor Analysis Consortium (CPTAC) cohort, for the recognition of high-grade tumour. The OSrisk for predicting 5-year survival status achieved an AUC of 0.784 (0.746-0.819) in the TCGA cohort, which was further verified in the independent general cohort and the CPTAC cohort, with AUCs of 0.774 (0.723-0.820) and 0.702 (0.632-0.765), respectively. Moreover, the competing-risk nomogram (CRN) showed its potential to be a prognostic indicator, with a hazard ratio (HR) of 5.664 (3.893-8.239, p<0.0001), outperforming other traditional clinical prognostic indicators. Kaplan-Meier survival analysis further illustrated that our CRN could significantly distinguish patients with high survival risk. Deep learning-based artificial intelligence could be a useful tool for clinicians to diagnose and predict the prognosis of RCC patients, thus improving the process of individualised treatment.

The (Im)perfect Automation Schema: Who Is Trusted More, Automated or Human Decision Support?

OBJECTIVE

This study's purpose was to better understand the dynamics of trust attitude and behavior in human-agent interaction.

BACKGROUND

Whereas past research provided evidence for a perfect automation schema, more recent research has provided contradictory evidence.

METHOD

To disentangle these conflicting findings, we conducted an online experiment using a simulated medical X-ray task. We manipulated the framing of support agents (i.e., artificial intelligence (AI) versus expert versus novice) between-subjects and failure experience (i.e., perfect support, imperfect support, back-to-perfect support) within subjects. Trust attitude and behavior as well as perceived reliability served as dependent variables.

RESULTS

Trust attitude and perceived reliability were higher for the human expert than for the AI than for the human novice. Moreover, the results showed the typical pattern of trust formation, dissolution, and restoration for trust attitude and behavior as well as perceived reliability. Forgiveness after failure experience did not differ between agents.

CONCLUSION

The results strongly imply the existence of an imperfect automation schema. This illustrates the need to consider agent expertise for human-agent interaction.

APPLICATION

When replacing human experts with AI as support agents, the challenge of lower trust attitude towards the novel agent might arise.

Diagnosing glaucoma in primary eye care and the role of Artificial Intelligence applications for reducing the prevalence of undetected glaucoma in Australia

Glaucoma is the commonest cause of irreversible blindness worldwide, with over 70% of people affected remaining undiagnosed. Early detection is crucial for halting progressive visual impairment in glaucoma patients, as there is no cure available. This narrative review aims to: identify reasons for the significant under-diagnosis of glaucoma globally, particularly in Australia, elucidate the role of primary healthcare in glaucoma diagnosis using Australian healthcare as an example, and discuss how recent advances in artificial intelligence (AI) can be implemented to improve diagnostic outcomes. Glaucoma is a prevalent disease in ageing populations and can have improved visual outcomes through appropriate treatment, making it essential for general medical practice. In countries such as Australia, New Zealand, Canada, USA, and the UK, optometrists serve as the gatekeepers for primary eye care, and glaucoma detection often falls on their shoulders. However, there is significant variation in the capacity for glaucoma diagnosis among eye professionals. Automation with Artificial Intelligence (AI) analysis of optic nerve photos can help optometrists identify high-risk changes and mitigate the challenges of image interpretation rapidly and consistently. Despite its potential, there are significant barriers and challenges to address before AI can be deployed in primary healthcare settings, including external validation, high quality real-world implementation, protection of privacy and cybersecurity, and medico-legal implications. Overall, the incorporation of AI technology in primary healthcare has the potential to reduce the global prevalence of undiagnosed glaucoma cases by improving diagnostic accuracy and efficiency.

Knowledge and attitude of medical students towards artificial intelligence in ophthalmology in Riyadh, Saudi Arabia: a cross-sectional study

Background

The use of artificial intelligence (AI) in ophthalmology represents a transformative leap in healthcare. AI-powered technologies, such as machine learning and computer vision, enhance the accuracy and efficiency of ophthalmic diagnosis and treatment.

Objective

This study aimed to determine medical students' awareness and attitudes towards the use of artificial intelligence in ophthalmology.

Methods

This cross-sectional, questionnaire-based study was conducted between November 2022 and January 2023 using online questionnaires. Data collection was carried out using convenience sampling among medical students at the University. IBM SPSS version 23 was used to analyze the data.

Results

The current finding shows that most of the participants N=309 (89.6%) had heard of the use of AI in medicine, and N=294 (85.2%) heard of the use of AI in ophthalmology. 98.6% (n=340) of respondents believed AI would be a helpful tool in ophthalmology. Along this line of questioning, a significant majority of respondents, 332 (96.2%) selected screening, 332 (96.2%) selected diagnosis, and 293 (84.9%) selected prevention as a usage of AI ophthalmology. However, the majority, 76.5%) of students had little understanding of the development of AI in ophthalmology. In addition, a significant relationship between sex, academic year, cumulative GPA (cGPA), and awareness of AI in ophthalmology (P<0.001) was found in this study.

Conclusions

Overall, medical students in Saudi Arabia appear to have favorable thoughts about AI and positive perceptions towards AI in ophthalmology. However, the findings of this study emphasize the limited understanding and low confidence levels of medical students in Saudi Arabia regarding the use of AI in ophthalmology. As a result, early exposure to AI-related materials in medical curricula is crucial for addressing these challenges through comprehensive AI education and practical exposure to prepare future ophthalmologists.

A universal multiple instance learning framework for whole slide image analysis

BACKGROUND

The emergence of digital whole slide image (WSI) has driven the development of computational pathology. However, obtaining patch-level annotations is challenging and time-consuming due to the high resolution of WSI, which limits the applicability of fully supervised methods. We aim to address the challenges related to patch-level annotations.

METHODS

We propose a universal framework for weakly supervised WSI analysis based on Multiple Instance Learning (MIL). To achieve effective aggregation of instance features, we design a feature aggregation module from multiple dimensions by considering feature distribution, instances correlation and instance-level evaluation. First, we implement instance-level standardization layer and deep projection unit to improve the separation of instances in the feature space. Then, a self-attention mechanism is employed to explore dependencies between instances. Additionally, an instance-level pseudo-label evaluation method is introduced to enhance the available information during the weak supervision process. Finally, a bag-level classifier is used to obtain preliminary WSI classification results. To achieve even more accurate WSI label predictions, we have designed a key instance selection module that strengthens the learning of local features for instances. Combining the results from both modules leads to an improvement in WSI prediction accuracy.

RESULTS

Experiments conducted on Camelyon16, TCGA-NSCLC, SICAPv2, PANDA and classical MIL benchmark datasets demonstrate that our proposed method achieves a competitive performance compared to some recent methods, with maximum improvement of 14.6 % in terms of classification accuracy.

CONCLUSION

Our method can improve the classification accuracy of whole slide images in a weakly supervised way, and more accurately detect lesion areas.

Unsupervised mutual transformer learning for multi-gigapixel Whole Slide Image classification

The classification of gigapixel Whole Slide Images (WSIs) is an important task in the emerging area of computational pathology. There has been a surge of interest in deep learning models for WSI classification with clinical applications such as cancer detection or prediction of cellular mutations. Most supervised methods require expensive and labor-intensive manual annotations by expert pathologists. Weakly supervised Multiple Instance Learning (MIL) methods have recently demonstrated excellent performance; however, they still require large-scale slide-level labeled training datasets that require a careful inspection of each slide by an expert pathologist. In this work, we propose a fully unsupervised WSI classification algorithm based on mutual transformer learning. The instances (i.e., patches) from gigapixel WSIs are transformed into a latent space and then inverse-transformed to the original space. Using the transformation loss, pseudo labels are generated and cleaned using a transformer label cleaner. The proposed transformer-based pseudo-label generator and cleaner modules mutually train each other iteratively in an unsupervised manner. A discriminative learning mechanism is introduced to improve normal versus cancerous instance labeling. In addition to the unsupervised learning, we demonstrate the effectiveness of the proposed framework for weakly supervised learning and cancer subtype classification as downstream analysis. Extensive experiments on four publicly available datasets show better performance of the proposed algorithm compared to the existing state-of-the-art methods.

Breast Cancer Classification From Digital Pathology Images via Connectivity-Aware Graph Transformer

Automated classification of breast cancer subtypes from digital pathology images has been an extremely challenging task due to the complicated spatial patterns of cells in the tissue micro-environment. While newly proposed graph transformers are able to capture more long-range dependencies to enhance accuracy, they largely ignore the topological connectivity between graph nodes, which is nevertheless critical to extract more representative features to address this difficult task. In this paper, we propose a novel connectivity-aware graph transformer (CGT) for phenotyping the topology connectivity of the tissue graph constructed from digital pathology images for breast cancer classification. Our CGT seamlessly integrates connectivity embedding to node feature at every graph transformer layer by using local connectivity aggregation, in order to yield more comprehensive graph representations to distinguish different breast cancer subtypes. In light of the realistic intercellular communication mode, we then encode the spatial distance between two arbitrary nodes as connectivity bias in self-attention calculation, thereby allowing the CGT to distinctively harness the connectivity embedding based on the distance of two nodes. We extensively evaluate the proposed CGT on a large cohort of breast carcinoma digital pathology images stained by Haematoxylin & Eosin. Experimental results demonstrate the effectiveness of our CGT, which outperforms state-of-the-art methods by a large margin. Codes are released on https://github.com/wang-kang-6/CGT.

Prediction of the short-term efficacy and recurrence of photodynamic therapy in the treatment of oral leukoplakia based on deep learning

BACKGROUND

The treatment of oral leukoplakia (OLK) with aminolaevulinic acid photodynamic therapy (ALA-PDT) is widespread. Nonetheless, there is variation in efficacy. Therefore, this study constructed a model for predicting the short-term efficacy and recurrence of OLK after ALA-PDT.

METHODS

The short-term efficacy and recurrence of ALA-PDT were calculated by statistical analysis, and the relevant influencing factors were analyzed by Logistic regression and COX regression model. Finally, prediction models for total response (TR) rate, complete response (CR) rate and recurrence in OLK patients after ALA-PDT treatment were established. Features from pathology sections were extracted using deep learning autoencoder and combined with clinical variables to improve prediction performance of the model.

RESULTS

The logistic regression analysis showed that the non-homogeneous (OR: 4.911, P: 0.023) OLK and lesions with moderate to severe epithelial dysplasia (OR: 4.288, P: 0.042) had better short-term efficacy. The area under receiver operating characteristic curve (AUC) of CR, TR and recurrence predict models after the ALA-PDT treatment of OLK patients is 0.872, 0.718, and 0.564, respectively. Feature extraction revealed an association between inflammatory cell infiltration in the lamina propria and recurrence after PDT. Combining clinical variables and deep learning improved the performance of recurrence model by more than 30 %.

CONCLUSIONS

ALA-PDT has excellent short-term efficacy in the management of OLK but the recurrence rate was high. Prediction model based on clinicopathological characteristics has excellent predictive effect for short-term efficacy but limited effect for recurrence. The use of deep learning and pathology images greatly improves predictive value of the models.

Advancing Automatic Gastritis Diagnosis: An Interpretable Multilabel Deep Learning Framework for the Simultaneous Assessment of Multiple Indicators

The evaluation of morphologic features, such as inflammation, gastric atrophy, and intestinal metaplasia, is crucial for diagnosing gastritis. However, artificial intelligence analysis for nontumor diseases like gastritis is limited. Previous deep learning models have omitted important morphologic indicators and cannot simultaneously diagnose gastritis indicators or provide interpretable labels. To address this, an attention-based multi-instance multilabel learning network (AMMNet) was developed to simultaneously achieve the multilabel diagnosis of activity, atrophy, and intestinal metaplasia with only slide-level weak labels. To evaluate AMMNet's real-world performance, a diagnostic test was designed to observe improvements in junior pathologists' diagnostic accuracy and efficiency with and without AMMNet assistance. In this study of 1096 patients from seven independent medical centers, AMMNet performed well in assessing activity [area under the curve (AUC), 0.93], atrophy (AUC, 0.97), and intestinal metaplasia (AUC, 0.93). The false-negative rates of these indicators were only 0.04, 0.08, and 0.18, respectively, and junior pathologists had lower false-negative rates with model assistance (0.15 versus 0.10). Furthermore, AMMNet reduced the time required per whole slide image from 5.46 to 2.85 minutes, enhancing diagnostic efficiency. In block-level clustering analysis, AMMNet effectively visualized task-related patches within whole slide images, improving interpretability. These findings highlight AMMNet's effectiveness in accurately evaluating gastritis morphologic indicators on multicenter data sets. Using multi-instance multilabel learning strategies to support routine diagnostic pathology deserves further evaluation.

Disrupting prostate cancer research: Challenge accepted; report from the 2023 Coffey-Holden Prostate Cancer Academy Meeting

INTRODUCTION

The 2023 Coffey-Holden Prostate Cancer Academy (CHPCA) Meeting, themed "Disrupting Prostate Cancer Research: Challenge Accepted," was convened at the University of California, Los Angeles, Luskin Conference Center, in Los Angeles, CA, from June 22 to 25, 2023.

METHODS

The 2023 marked the 10th Annual CHPCA Meeting, a discussion-oriented scientific think-tank conference convened annually by the Prostate Cancer Foundation, which centers on innovative and emerging research topics deemed pivotal for advancing critical unmet needs in prostate cancer research and clinical care. The 2023 CHPCA Meeting was attended by 81 academic investigators and included 40 talks across 8 sessions.

RESULTS

The central topic areas covered at the meeting included: targeting transcription factor neo-enhancesomes in cancer, AR as a pro-differentiation and oncogenic transcription factor, why few are cured with androgen deprivation therapy and how to change dogma to cure metastatic prostate cancer without castration, reducing prostate cancer morbidity and mortality with genetics, opportunities for radiation to enhance therapeutic benefit in oligometastatic prostate cancer, novel immunotherapeutic approaches, and the new era of artificial intelligence-driven precision medicine.

DISCUSSION

This article provides an overview of the scientific presentations delivered at the 2023 CHPCA Meeting, such that this knowledge can help in facilitating the advancement of prostate cancer research worldwide.

A novel fully automatic segmentation and counting system for metastatic lymph nodes on multimodal magnetic resonance imaging: Evaluation and prognostic implications in nasopharyngeal carcinoma

BACKGROUND

The number of metastatic lymph nodes (MLNs) is crucial for the survival of nasopharyngeal carcinoma (NPC), but manual counting is laborious. This study aims to explore the feasibility and prognostic value of automatic MLNs segmentation and counting.

METHODS

We retrospectively enrolled 980 newly diagnosed patients in the primary cohort and 224 patients from two external cohorts. We utilized the nnUnet model for automatic MLNs segmentation on multimodal magnetic resonance imaging. MLNs counting methods, including manual delineation-assisted counting (MDAC) and fully automatic lymph node counting system (AMLNC), were compared with manual evaluation (Gold standard).

RESULTS

In the internal validation group, the MLNs segmentation results showed acceptable agreement with manual delineation, with a mean Dice coefficient of 0.771. The consistency among three counting methods was as follows 0.778 (Gold vs. AMLNC), 0.638 (Gold vs. MDAC), and 0.739 (AMLNC vs. MDAC). MLNs numbers were categorized into three-category variable (1-4, 5-9, > 9) and two-category variable (<4, ≥ 4) based on the gold standard and AMLNC. These categorical variables demonstrated acceptable discriminating abilities for 5-year overall survival (OS), progression-free, and distant metastasis-free survival. Compared with base prediction model, the model incorporating two-category AMLNC-counting numbers showed improved C-indexes for 5-year OS prediction (0.658 vs. 0.675, P = 0.045). All results have been successfully validated in the external cohort.

CONCLUSIONS

The AMLNC system offers a time- and labor-saving approach for fully automatic MLNs segmentation and counting in NPC. MLNs counting using AMLNC demonstrated non-inferior performance in survival discrimination compared to manual detection.

PathoDuet: Foundation models for pathological slide analysis of H&E and IHC stains

Large amounts of digitized histopathological data display a promising future for developing pathological foundation models via self-supervised learning methods. Foundation models pretrained with these methods serve as a good basis for downstream tasks. However, the gap between natural and histopathological images hinders the direct application of existing methods. In this work, we present PathoDuet, a series of pretrained models on histopathological images, and a new self-supervised learning framework in histopathology. The framework is featured by a newly-introduced pretext token and later task raisers to explicitly utilize certain relations between images, like multiple magnifications and multiple stains. Based on this, two pretext tasks, cross-scale positioning and cross-stain transferring, are designed to pretrain the model on Hematoxylin and Eosin (H&E) images and transfer the model to immunohistochemistry (IHC) images, respectively. To validate the efficacy of our models, we evaluate the performance over a wide variety of downstream tasks, including patch-level colorectal cancer subtyping and whole slide image (WSI)-level classification in H&E field, together with expression level prediction of IHC marker, tumor identification and slide-level qualitative analysis in IHC field. The experimental results show the superiority of our models over most tasks and the efficacy of proposed pretext tasks. The codes and models are available at https://github.com/openmedlab/PathoDuet.

Artificial intelligence in medical education - perception among medical students

BACKGROUND

As Artificial Intelligence (AI) becomes pervasive in healthcare, including applications like robotic surgery and image analysis, the World Medical Association emphasises integrating AI education into medical curricula. This study evaluates medical students' perceptions of 'AI in medicine', their preferences for AI training in education, and their grasp of AI's ethical implications in healthcare.

MATERIALS & METHODS

A cross-sectional study was conducted among 325 medical students in Kerala using a pre-validated, semi structured questionnaire. The survey collected demographic data, any past educational experience about AI, participants' self-evaluation of their knowledge and evaluated self-perceived understanding of applications of AI in medicine. Participants responded to twelve Likert-scale questions targeting perceptions and ethical aspects and their opinions on suggested topics on AI to be included in their curriculum.

RESULTS & DISCUSSION

AI was viewed as an assistive technology for reducing medical errors by 57.2% students and 54.2% believed AI could enhance medical decision accuracy. About 49% agreed that AI could potentially improve accessibility to healthcare. Concerns about AI replacing physicians were reported by 37.6% and 69.2% feared a reduction in the humanistic aspect of medicine. Students were worried about challenges to trust (52.9%), patient-physician relationships (54.5%) and breach of professional confidentiality (53.5%). Only 3.7% felttotally competent in informing patients about features and risks associated with AI applications. Strong demand for structured AI training was expressed, particularly on reducing medical errors (76.9%) and ethical issues (79.4%).

CONCLUSION

This study highlights medical students' demand for structured AI training in undergraduate curricula, emphasising its importance in addressing evolving healthcare needs and ethical considerations. Despite widespread ethical concerns, the majority perceive AI as an assistive technology in healthcare. These findings provide valuable insights for curriculum development and defining learning outcomes in AI education for medical students.

A foundation model for clinical-grade computational pathology and rare cancers detection

The analysis of histopathology images with artificial intelligence aims to enable clinical decision support systems and precision medicine. The success of such applications depends on the ability to model the diverse patterns observed in pathology images. To this end, we present Virchow, the largest foundation model for computational pathology to date. In addition to the evaluation of biomarker prediction and cell identification, we demonstrate that a large foundation model enables pan-cancer detection, achieving 0.95 specimen-level area under the (receiver operating characteristic) curve across nine common and seven rare cancers. Furthermore, we show that with less training data, the pan-cancer detector built on Virchow can achieve similar performance to tissue-specific clinical-grade models in production and outperform them on some rare variants of cancer. Virchow's performance gains highlight the value of a foundation model and open possibilities for many high-impact applications with limited amounts of labeled training data.

Artificial Intelligence-Based Histopathological Subtyping of High-Grade Serous Ovarian Cancer

Four subtypes of ovarian high-grade serous carcinoma (HGSC) have previously been identified, each with different prognoses and drug sensitivities. However, the accuracy of classification depended on the assessor's experience. This study aimed to develop a universal algorithm for HGSC-subtype classification using deep learning techniques. An artificial intelligence (AI)-based classification algorithm, which replicates the consensus diagnosis of pathologists, was formulated to analyze the morphological patterns and tumor-infiltrating lymphocyte counts for each tile extracted from whole slide images of ovarian HGSC available in The Cancer Genome Atlas (TCGA) data set. The accuracy of the algorithm was determined using the validation set from the Japanese Gynecologic Oncology Group 3022A1 (JGOG3022A1) and Kindai and Kyoto University (Kindai/Kyoto) cohorts. The algorithm classified the four HGSC-subtypes with mean accuracies of 0.933, 0.910, and 0.862 for the TCGA, JGOG3022A1, and Kindai/Kyoto cohorts, respectively. To compare mesenchymal transition (MT) with non-MT groups, overall survival analysis was performed in the TCGA data set. The AI-based prediction of HGSC-subtype classification in TCGA cases showed that the MT group had a worse prognosis than the non-MT group (P = 0.017). Furthermore, Cox proportional hazard regression analysis identified AI-based MT subtype classification prediction as a contributing factor along with residual disease after surgery, stage, and age. In conclusion, a robust AI-based HGSC-subtype classification algorithm was established using virtual slides of ovarian HGSC.

Deep Learning Auto-Segmentation Network for Pediatric Computed Tomography Data Sets: Can We Extrapolate From Adults?

PURPOSE

Artificial intelligence (AI)-based auto-segmentation models hold promise for enhanced efficiency and consistency in organ contouring for adaptive radiation therapy and radiation therapy planning. However, their performance on pediatric computed tomography (CT) data and cross-scanner compatibility remain unclear. This study aimed to evaluate the performance of AI-based auto-segmentation models trained on adult CT data when applied to pediatric data sets and explore the improvement in performance gained by including pediatric training data. It also examined their ability to accurately segment CT data acquired from different scanners.

METHODS AND MATERIALS

Using the nnU-Net framework, segmentation models were trained on data sets of adult, pediatric, and combined CT scans for 7 pelvic/thoracic organs. Each model was trained on 290 to 300 cases per category and organ. Training data sets included a combination of clinical data and several open repositories. The study incorporated a database of 459 pediatric (0-16 years) CT scans and 950 adults (>18 years), ensuring all scans had human expert ground-truth contours of the selected organs. Performance was evaluated based on Dice similarity coefficients (DSC) of the model-generated contours.

RESULTS

AI models trained exclusively on adult data underperformed on pediatric data, especially for the 0 to 2 age group: mean DSC was below 0.5 for the bladder and spleen. The addition of pediatric training data demonstrated significant improvement for all age groups, achieving a mean DSC of above 0.85 for all organs in every age group. Larger organs like the liver and kidneys maintained consistent performance for all models across age groups. No significant difference emerged in the cross-scanner performance evaluation, suggesting robust cross-scanner generalization.

CONCLUSIONS

For optimal segmentation across age groups, it is important to include pediatric data in the training of segmentation models. The successful cross-scanner generalization also supports the real-world clinical applicability of these AI models. This study emphasizes the significance of data set diversity in training robust AI systems for medical image interpretation tasks.

Latent Diffusion Models with Image-Derived Annotations for Enhanced AI-Assisted Cancer Diagnosis in Histopathology

Artificial Intelligence (AI)-based image analysis has immense potential to support diagnostic histopathology, including cancer diagnostics. However, developing supervised AI methods requires large-scale annotated datasets. A potentially powerful solution is to augment training data with synthetic data. Latent diffusion models, which can generate high-quality, diverse synthetic images, are promising. However, the most common implementations rely on detailed textual descriptions, which are not generally available in this domain. This work proposes a method that constructs structured textual prompts from automatically extracted image features. We experiment with the PCam dataset, composed of tissue patches only loosely annotated as healthy or cancerous. We show that including image-derived features in the prompt, as opposed to only healthy and cancerous labels, improves the Fréchet Inception Distance (FID) by 88.6. We also show that pathologists find it challenging to detect synthetic images, with a median sensitivity/specificity of 0.55/0.55. Finally, we show that synthetic data effectively train AI models.

Deep learning-enabled classification of kidney allograft rejection on whole slide histopathologic images

Background

Diagnosis of kidney transplant rejection currently relies on manual histopathological assessment, which is subjective and susceptible to inter-observer variability, leading to limited reproducibility. We aim to develop a deep learning system for automated assessment of whole-slide images (WSIs) from kidney allograft biopsies to enable detection and subtyping of rejection and to predict the prognosis of rejection.

Method

We collected H&E-stained WSIs of kidney allograft biopsies at 400x magnification from January 2015 to September 2023 at two hospitals. These biopsy specimens were classified as T cell-mediated rejection, antibody-mediated rejection, and other lesions based on the consensus reached by two experienced transplant pathologists. To achieve feature extraction, feature aggregation, and global classification, we employed multi-instance learning and common convolution neural networks (CNNs). The performance of the developed models was evaluated using various metrics, including confusion matrix, receiver operating characteristic curves, the area under the curve (AUC), classification map, heat map, and pathologist-machine confrontations.

Results

In total, 906 WSIs from 302 kidney allograft biopsies were included for analysis. The model based on multi-instance learning enables detection and subtyping of rejection, named renal rejection artificial intelligence model (RRAIM), with the overall 3-category AUC of 0.798 in the independent test set, which is superior to that of three transplant pathologists under nearly routine assessment conditions. Moreover, the prognosis models accurately predicted graft loss within 1 year following rejection and treatment response for rejection, achieving AUC of 0.936 and 0.756, respectively.

Conclusion

We first developed deep-learning models utilizing multi-instance learning for the detection and subtyping of rejection and prediction of rejection prognosis in kidney allograft biopsies. These models performed well and may be useful in assisting the pathological diagnosis.

Application of Photoactive Compounds in Cancer Theranostics: Review on Recent Trends from Photoactive Chemistry to Artificial Intelligence

According to the World Health Organization (WHO) and the International Agency for Research on Cancer (IARC), the number of cancer cases and deaths worldwide is predicted to nearly double by 2030, reaching 21.7 million cases and 13 million fatalities. The increase in cancer mortality is due to limitations in the diagnosis and treatment options that are currently available. The close relationship between diagnostics and medicine has made it possible for cancer patients to receive precise diagnoses and individualized care. This article discusses newly developed compounds with potential for photodynamic therapy and diagnostic applications, as well as those already in use. In addition, it discusses the use of artificial intelligence in the analysis of diagnostic images obtained using, among other things, theranostic agents.

Video-based diagnosis support system for pianists with Musician's dystonia

Background

Musician's dystonia is a task-specific movement disorder that deteriorates fine motor control of skilled movements in musical performance. Although this disorder threatens professional careers, its diagnosis is challenging for clinicians who have no specialized knowledge of musical performance.

Objectives

To support diagnostic evaluation, the present study proposes a novel approach using a machine learning-based algorithm to identify the symptomatic movements of Musician's dystonia.

Methods

We propose an algorithm that identifies the dystonic movements using the anomaly detection method with an autoencoder trained with the hand kinematics of healthy pianists. A unique feature of the algorithm is that it requires only the video image of the hand, which can be derived by a commercially available camera. We also measured the hand biomechanical functions to assess the contribution of peripheral factors and improve the identification of dystonic symptoms.

Results

The proposed algorithm successfully identified Musician's dystonia with an accuracy and specificity of 90% based only on video footages of the hands. In addition, we identified the degradation of biomechanical functions involved in controlling multiple fingers, which is not specific to musical performance. By contrast, there were no dystonia-specific malfunctions of hand biomechanics, including the strength and agility of individual digits.

Conclusion

These findings demonstrate the effectiveness of the present technique in aiding in the accurate diagnosis of Musician's dystonia.

A Pan-Cancer Patient-Derived Xenograft Histology Image Repository with Genomic and Pathologic Annotations Enables Deep Learning Analysis

Patient-derived xenografts (PDX) model human intra- and intertumoral heterogeneity in the context of the intact tissue of immunocompromised mice. Histologic imaging via hematoxylin and eosin (H&E) staining is routinely performed on PDX samples, which could be harnessed for computational analysis. Prior studies of large clinical H&E image repositories have shown that deep learning analysis can identify intercellular and morphologic signals correlated with disease phenotype and therapeutic response. In this study, we developed an extensive, pan-cancer repository of >1,000 PDX and paired parental tumor H&E images. These images, curated from the PDX Development and Trial Centers Research Network Consortium, had a range of associated genomic and transcriptomic data, clinical metadata, pathologic assessments of cell composition, and, in several cases, detailed pathologic annotations of neoplastic, stromal, and necrotic regions. The amenability of these images to deep learning was highlighted through three applications: (i) development of a classifier for neoplastic, stromal, and necrotic regions; (ii) development of a predictor of xenograft-transplant lymphoproliferative disorder; and (iii) application of a published predictor of microsatellite instability. Together, this PDX Development and Trial Centers Research Network image repository provides a valuable resource for controlled digital pathology analysis, both for the evaluation of technical issues and for the development of computational image-based methods that make clinical predictions based on PDX treatment studies. Significance: A pan-cancer repository of >1,000 patient-derived xenograft hematoxylin and eosin-stained images will facilitate cancer biology investigations through histopathologic analysis and contributes important model system data that expand existing human histology repositories.

Focused active learning for histopathological image classification

Active Learning (AL) has the potential to solve a major problem of digital pathology: the efficient acquisition of labeled data for machine learning algorithms. However, existing AL methods often struggle in realistic settings with artifacts, ambiguities, and class imbalances, as commonly seen in the medical field. The lack of precise uncertainty estimations leads to the acquisition of images with a low informative value. To address these challenges, we propose Focused Active Learning (FocAL), which combines a Bayesian Neural Network with Out-of-Distribution detection to estimate different uncertainties for the acquisition function. Specifically, the weighted epistemic uncertainty accounts for the class imbalance, aleatoric uncertainty for ambiguous images, and an OoD score for artifacts. We perform extensive experiments to validate our method on MNIST and the real-world Panda dataset for the classification of prostate cancer. The results confirm that other AL methods are 'distracted' by ambiguities and artifacts which harm the performance. FocAL effectively focuses on the most informative images, avoiding ambiguities and artifacts during acquisition. For both experiments, FocAL outperforms existing AL approaches, reaching a Cohen's kappa of 0.764 with only 0.69% of the labeled Panda data.

[Clinical Application of Artificial Intelligence-Based Detection Assistance Devices for Chest X-Ray Interpretation: Current Status and Practical Considerations]

Artificial intelligence (AI) technology is actively being applied for the interpretation of medical imaging, such as chest X-rays. AI-based software medical devices, which automatically detect various types of abnormal findings in chest X-ray images to assist physicians in their interpretation, are actively being commercialized and clinically implemented in Korea. Several important issues need to be considered for AI-based detection assistant tools to be applied in clinical practice: the evaluation of performance and efficacy prior to implementation; the determination of the target application, range, and method of delivering results; and monitoring after implementation and legal liability issues. Appropriate decision making regarding these devices based on the situation in each institution is necessary. Radiologists must be engaged as medical assessment experts using the software for these devices as well as in medical image interpretation to ensure the safe and efficient implementation and operation of AI-based detection assistant tools.

Opening the Black Box: Spatial Transcriptomics and the Relevance of Artificial Intelligence-Detected Prognostic Regions in High-Grade Serous Carcinoma

Image-based deep learning models are used to extract new information from standard hematoxylin and eosin pathology slides; however, biological interpretation of the features detected by artificial intelligence (AI) remains a challenge. High-grade serous carcinoma of the ovary (HGSC) is characterized by aggressive behavior and chemotherapy resistance, but also exhibits striking variability in outcome. Our understanding of this disease is limited, partly due to considerable tumor heterogeneity. We previously trained an AI model to identify HGSC tumor regions that are highly associated with outcome status but are indistinguishable by conventional morphologic methods. Here, we applied spatially resolved transcriptomics to further profile the AI-identified tumor regions in 16 patients (8 per outcome group) and identify molecular features related to disease outcome in patients who underwent primary debulking surgery and platinum-based chemotherapy. We examined formalin-fixed paraffin-embedded tissue from (1) regions identified by the AI model as highly associated with short or extended chemotherapy response, and (2) background tumor regions (not identified by the AI model as highly associated with outcome status) from the same tumors. We show that the transcriptomic profiles of AI-identified regions are more distinct than background regions from the same tumors, are superior in predicting outcome, and differ in several pathways including those associated with chemoresistance in HGSC. Further, we find that poor outcome and good outcome regions are enriched by different tumor subpopulations, suggesting distinctive interaction patterns. In summary, our work presents proof of concept that AI-guided spatial transcriptomic analysis improves recognition of biologic features relevant to patient outcomes.

Construction and validation of a deep learning prognostic model based on digital pathology images of stage III colorectal cancer

BACKGROUND

TNM staging is the main reference standard for prognostic prediction of colorectal cancer (CRC), but the prognosis heterogeneity of patients with the same stage is still large. This study aimed to classify the tumor microenvironment of patients with stage III CRC and quantify the classified tumor tissues based on deep learning to explore the prognostic value of the developed tumor risk signature (TRS).

METHODS

A tissue classification model was developed to identify nine tissues (adipose, background, debris, lymphocytes, mucus, smooth muscle, normal mucosa, stroma, and tumor) in whole-slide images (WSIs) of stage III CRC patients. This model was used to extract tumor tissues from WSIs of 265 stage III CRC patients from The Cancer Genome Atlas and 70 stage III CRC patients from the Sixth Affiliated Hospital of Sun Yat-sen University. We used three different deep learning models for tumor feature extraction and applied a Cox model to establish the TRS. Survival analysis was conducted to explore the prognostic performance of TRS.

RESULTS

The tissue classification model achieved 94.4 % accuracy in identifying nine tissue types. The TRS showed a Harrell's concordance index of 0.736, 0.716, and 0.711 in the internal training, internal validation, and external validation sets. Survival analysis showed that TRS had significant predictive ability (hazard ratio: 3.632, p = 0.03) for prognostic prediction.

CONCLUSION

The TRS is an independent and significant prognostic factor for PFS of stage III CRC patients and it contributes to risk stratification of patients with different clinical stages.

Artificial Intelligence Helps Pathologists Increase Diagnostic Accuracy and Efficiency in the Detection of Breast Cancer Lymph Node Metastases

The detection of lymph node metastases is essential for breast cancer staging, although it is a tedious and time-consuming task where the sensitivity of pathologists is suboptimal. Artificial intelligence (AI) can help pathologists detect lymph node metastases, which could help alleviate workload issues. We studied how pathologists' performance varied when aided by AI. An AI algorithm was trained using more than 32 000 breast sentinel lymph node whole slide images (WSIs) matched with their corresponding pathology reports from more than 8000 patients. The algorithm highlighted areas suspicious of harboring metastasis. Three pathologists were asked to review a dataset comprising 167 breast sentinel lymph node WSIs, of which 69 harbored cancer metastases of different sizes, enriched for challenging cases. Ninety-eight slides were benign. The pathologists read the dataset twice, both digitally, with and without AI assistance, randomized for slide and reading orders to reduce bias, separated by a 3-week washout period. Their slide-level diagnosis was recorded, and they were timed during their reads. The average reading time per slide was 129 seconds during the unassisted phase versus 58 seconds during the AI-assisted phase, resulting in an overall efficiency gain of 55% ( P <0.001). These efficiency gains are applied to both benign and malignant WSIs. Two of the 3 reading pathologists experienced significant sensitivity improvements, from 74.5% to 93.5% ( P ≤0.006). This study highlights that AI can help pathologists shorten their reading times by more than half and also improve their metastasis detection rate.

Value of CT quantification in progressive fibrosing interstitial lung disease: a deep learning approach

OBJECTIVES

To evaluate the relationship of changes in the deep learning-based CT quantification of interstitial lung disease (ILD) with changes in forced vital capacity (FVC) and visual assessments of ILD progression, and to investigate their prognostic implications.

METHODS

This study included ILD patients with CT scans at intervals of over 2 years between January 2015 and June 2021. Deep learning-based texture analysis software was used to segment ILD findings on CT images (fibrosis: reticular opacity + honeycombing cysts; total ILD extent: ground-glass opacity + fibrosis). Patients were grouped according to the absolute decline of predicted FVC (< 5%, 5-10%, and ≥ 10%) and ILD progression assessed by thoracic radiologists, and their quantification results were compared among these groups. The associations between quantification results and survival were evaluated using multivariable Cox regression analysis.

RESULTS

In total, 468 patients (239 men; 64 ± 9.5 years) were included. Fibrosis and total ILD extents more increased in patients with larger FVC decline (p < .001 in both). Patients with ILD progression had higher fibrosis and total ILD extent increases than those without ILD progression (p < .001 in both). Increases in fibrosis and total ILD extent were significant prognostic factors when adjusted for absolute FVC declines of ≥ 5% (hazard ratio [HR] 1.844, p = .01 for fibrosis; HR 2.484, p < .001 for total ILD extent) and ≥ 10% (HR 2.918, p < .001 for fibrosis; HR 3.125, p < .001 for total ILD extent).

CONCLUSION

Changes in ILD CT quantification correlated with changes in FVC and visual assessment of ILD progression, and they were independent prognostic factors in ILD patients.

CLINICAL RELEVANCE STATEMENT

Quantifying the CT features of interstitial lung disease using deep learning techniques could play a key role in defining and predicting the prognosis of progressive fibrosing interstitial lung disease.

KEY POINTS

• Radiologic findings on high-resolution CT are important in diagnosing progressive fibrosing interstitial lung disease. • Deep learning-based quantification results for fibrosis and total interstitial lung disease extents correlated with the decline in forced vital capacity and visual assessments of interstitial lung disease progression, and emerged as independent prognostic factors. • Deep learning-based interstitial lung disease CT quantification can play a key role in diagnosing and prognosticating progressive fibrosing interstitial lung disease.

Possible benefits, challenges, pitfalls, and future perspective of using ChatGPT in pathology

The much-hyped artificial intelligence (AI) model called ChatGPT developed by Open AI can have great benefits for physicians, especially pathologists, by saving time so that they can use their time for more significant work. Generative AI is a special class of AI model, which uses patterns and structures learned from existing data and can create new data. Utilizing ChatGPT in Pathology offers a multitude of benefits, encompassing the summarization of patient records and its promising prospects in Digital Pathology, as well as its valuable contributions to education and research in this field. However, certain roadblocks need to be dealt like integrating ChatGPT with image analysis which will act as a revolution in the field of pathology by increasing diagnostic accuracy and precision. The challenges with the use of ChatGPT encompass biases from its training data, the need for ample input data, potential risks related to bias and transparency, and the potential adverse outcomes arising from inaccurate content generation. Generation of meaningful insights from the textual information which will be efficient in processing different types of image data, such as medical images, and pathology slides. Due consideration should be given to ethical and legal issues including bias.

Enhancing Medical Imaging Segmentation with GB-SAM: A Novel Approach to Tissue Segmentation Using Granular Box Prompts

Recent advances in foundation models have revolutionized model development in digital pathology, reducing dependence on extensive manual annotations required by traditional methods. The ability of foundation models to generalize well with few-shot learning addresses critical barriers in adapting models to diverse medical imaging tasks. This work presents the Granular Box Prompt Segment Anything Model (GB-SAM), an improved version of the Segment Anything Model (SAM) fine-tuned using granular box prompts with limited training data. The GB-SAM aims to reduce the dependency on expert pathologist annotators by enhancing the efficiency of the automated annotation process. Granular box prompts are small box regions derived from ground truth masks, conceived to replace the conventional approach of using a single large box covering the entire H&E-stained image patch. This method allows a localized and detailed analysis of gland morphology, enhancing the segmentation accuracy of individual glands and reducing the ambiguity that larger boxes might introduce in morphologically complex regions. We compared the performance of our GB-SAM model against U-Net trained on different sizes of the CRAG dataset. We evaluated the models across histopathological datasets, including CRAG, GlaS, and Camelyon16. GB-SAM consistently outperformed U-Net, with reduced training data, showing less segmentation performance degradation. Specifically, on the CRAG dataset, GB-SAM achieved a Dice coefficient of 0.885 compared to U-Net's 0.857 when trained on 25% of the data. Additionally, GB-SAM demonstrated segmentation stability on the CRAG testing dataset and superior generalization across unseen datasets, including challenging lymph node segmentation in Camelyon16, which achieved a Dice coefficient of 0.740 versus U-Net's 0.491. Furthermore, compared to SAM-Path and Med-SAM, GB-SAM showed competitive performance. GB-SAM achieved a Dice score of 0.900 on the CRAG dataset, while SAM-Path achieved 0.884. On the GlaS dataset, Med-SAM reported a Dice score of 0.956, whereas GB-SAM achieved 0.885 with significantly less training data. These results highlight GB-SAM's advanced segmentation capabilities and reduced dependency on large datasets, indicating its potential for practical deployment in digital pathology, particularly in settings with limited annotated datasets.

Clinical implementation of artificial-intelligence-assisted detection of breast cancer metastases in sentinel lymph nodes: the CONFIDENT-B single-center, non-randomized clinical trial

Pathologists' assessment of sentinel lymph nodes (SNs) for breast cancer (BC) metastases is a treatment-guiding yet labor-intensive and costly task because of the performance of immunohistochemistry (IHC) in morphologically negative cases. This non-randomized, single-center clinical trial (International Standard Randomized Controlled Trial Number:14323711) assessed the efficacy of an artificial intelligence (AI)-assisted workflow for detecting BC metastases in SNs while maintaining diagnostic safety standards. From September 2022 to May 2023, 190 SN specimens were consecutively enrolled and allocated biweekly to the intervention arm (n = 100) or control arm (n = 90). In both arms, digital whole-slide images of hematoxylin-eosin sections of SN specimens were assessed by an expert pathologist, who was assisted by the 'Metastasis Detection' app (Visiopharm) in the intervention arm. Our primary endpoint showed a significantly reduced adjusted relative risk of IHC use (0.680, 95% confidence interval: 0.347-0.878) for AI-assisted pathologists, with subsequent cost savings of ~3,000 €. Secondary endpoints showed significant time reductions and up to 30% improved sensitivity for AI-assisted pathologists. This trial demonstrates the safety and potential for cost and time savings of AI assistance.

Advancements in computer vision and pathology: Unraveling the potential of artificial intelligence for precision diagnosis and beyond

The integration of computer vision into pathology through slide digitalization represents a transformative leap in the field's evolution. Traditional pathology methods, while reliable, are often time-consuming and susceptible to intra- and interobserver variability. In contrast, computer vision, empowered by artificial intelligence (AI) and machine learning (ML), promises revolutionary changes, offering consistent, reproducible, and objective results with ever-increasing speed and scalability. The applications of advanced algorithms and deep learning architectures like CNNs and U-Nets augment pathologists' diagnostic capabilities, opening new frontiers in automated image analysis. As these technologies mature and integrate into digital pathology workflows, they are poised to provide deeper insights into disease processes, quantify and standardize biomarkers, enhance patient outcomes, and automate routine tasks, reducing pathologists' workload. However, this transformative force calls for cross-disciplinary collaboration between pathologists, computer scientists, and industry innovators to drive research and development. While acknowledging its potential, this chapter addresses the limitations of AI in pathology, encompassing technical, practical, and ethical considerations during development and implementation.

AI-based histopathology image analysis reveals a distinct subset of endometrial cancers

Endometrial cancer (EC) has four molecular subtypes with strong prognostic value and therapeutic implications. The most common subtype (NSMP; No Specific Molecular Profile) is assigned after exclusion of the defining features of the other three molecular subtypes and includes patients with heterogeneous clinical outcomes. In this study, we employ artificial intelligence (AI)-powered histopathology image analysis to differentiate between p53abn and NSMP EC subtypes and consequently identify a sub-group of NSMP EC patients that has markedly inferior progression-free and disease-specific survival (termed 'p53abn-like NSMP'), in a discovery cohort of 368 patients and two independent validation cohorts of 290 and 614 from other centers. Shallow whole genome sequencing reveals a higher burden of copy number abnormalities in the 'p53abn-like NSMP' group compared to NSMP, suggesting that this group is biologically distinct compared to other NSMP ECs. Our work demonstrates the power of AI to detect prognostically different and otherwise unrecognizable subsets of EC where conventional and standard molecular or pathologic criteria fall short, refining image-based tumor classification. This study's findings are applicable exclusively to females.

MIST: Multi-instance selective transformer for histopathological subtype prediction

Accurate histopathological subtype prediction is clinically significant for cancer diagnosis and tumor microenvironment analysis. However, achieving accurate histopathological subtype prediction is a challenging task due to (1) instance-level discrimination of histopathological images, (2) low inter-class and large intra-class variances among histopathological images in their shape and chromatin texture, and (3) heterogeneous feature distribution over different images. In this paper, we formulate subtype prediction as fine-grained representation learning and propose a novel multi-instance selective transformer (MIST) framework, effectively achieving accurate histopathological subtype prediction. The proposed MIST designs an effective selective self-attention mechanism with multi-instance learning (MIL) and vision transformer (ViT) to adaptive identify informative instances for fine-grained representation. Innovatively, the MIST entrusts each instance with different contributions to the bag representation based on its interactions with instances and bags. Specifically, a SiT module with selective multi-head self-attention (S-MSA) is well-designed to identify the representative instances by modeling the instance-to-instance interactions. On the contrary, a MIFD module with the information bottleneck is proposed to learn the discriminative fine-grained representation for histopathological images by modeling instance-to-bag interactions with the selected instances. Substantial experiments on five clinical benchmarks demonstrate that the MIST achieves accurate histopathological subtype prediction and obtains state-of-the-art performance with an accuracy of 0.936. The MIST shows great potential to handle fine-grained medical image analysis, such as histopathological subtype prediction in clinical applications.

Dual-source dual-energy CT and deep learning for equivocal lymph nodes on CT images for thyroid cancer

OBJECTIVES

This study investigated the diagnostic performance of dual-energy computed tomography (CT) and deep learning for the preoperative classification of equivocal lymph nodes (LNs) on CT images in thyroid cancer patients.

METHODS

In this prospective study, from October 2020 to March 2021, 375 patients with thyroid disease underwent thin-section dual-energy thyroid CT at a small field of view (FOV) and thyroid surgery. The data of 183 patients with 281 LNs were analyzed. The targeted LNs were negative or equivocal on small FOV CT images. Six deep-learning models were used to classify the LNs on conventional CT images. The performance of all models was compared with pathology reports.

RESULTS

Of the 281 LNs, 65.5% had a short diameter of less than 4 mm. Multiple quantitative dual-energy CT parameters significantly differed between benign and malignant LNs. Multivariable logistic regression analyses showed that the best combination of parameters had an area under the curve (AUC) of 0.857, with excellent consistency and discrimination, and its diagnostic accuracy and sensitivity were 74.4% and 84.2%, respectively (p < 0.001). The visual geometry group 16 (VGG16) based model achieved the best accuracy (86%) and sensitivity (88%) in differentiating between benign and malignant LNs, with an AUC of 0.89.

CONCLUSIONS

The VGG16 model based on small FOV CT images showed better diagnostic accuracy and sensitivity than the spectral parameter model. Our study presents a noninvasive and convenient imaging biomarker to predict malignant LNs without suspicious CT features in thyroid cancer patients.

CLINICAL RELEVANCE STATEMENT

Our study presents a deep-learning-based model to predict malignant lymph nodes in thyroid cancer without suspicious features on conventional CT images, which shows better diagnostic accuracy and sensitivity than the regression model based on spectral parameters.

KEY POINTS

Many cervical lymph nodes (LNs) do not express suspicious features on conventional computed tomography (CT). Dual-energy CT parameters can distinguish between benign and malignant LNs. Visual geometry group 16 model shows superior diagnostic accuracy and sensitivity for malignant LNs.

Prospector Heads: Generalized Feature Attribution for Large Models & Data

Feature attribution, the ability to localize regions of the input data that are relevant for classification, is an important capability for ML models in scientific and biomedical domains. Current methods for feature attribution, which rely on "explaining" the predictions of end-to-end classifiers, suffer from imprecise feature localization and are inadequate for use with small sample sizes and high-dimensional datasets due to computational challenges. We introduce prospector heads, an efficient and interpretable alternative to explanation-based attribution methods that can be applied to any encoder and any data modality. Prospector heads generalize across modalities through experiments on sequences (text), images (pathology), and graphs (protein structures), outperforming baseline attribution methods by up to 26.3 points in mean localization AUPRC. We also demonstrate how prospector heads enable improved interpretation and discovery of class-specific patterns in input data. Through their high performance, flexibility, and generalizability, prospectors provide a framework for improving trust and transparency for ML models in complex domains.

Microscopy Image Dataset for Deep Learning-Based Quantitative Assessment of Pulmonary Vascular Changes

Pulmonary hypertension (PH) is a syndrome complex that accompanies a number of diseases of different etiologies, associated with basic mechanisms of structural and functional changes of the pulmonary circulation vessels and revealed pressure increasing in the pulmonary artery. The structural changes in the pulmonary circulation vessels are the main limiting factor determining the prognosis of patients with PH. Thickening and irreversible deposition of collagen in the pulmonary artery branches walls leads to rapid disease progression and a therapy effectiveness decreasing. In this regard, histological examination of the pulmonary circulation vessels is critical both in preclinical studies and clinical practice. However, measurements of quantitative parameters such as the average vessel outer diameter, the vessel walls area, and the hypertrophy index claimed significant time investment and the requirement for specialist training to analyze micrographs. A dataset of pulmonary circulation vessels for pathology assessment using semantic segmentation techniques based on deep-learning is presented in this work. 609 original microphotographs of vessels, numerical data from experts' measurements, and microphotographs with outlines of these measurements for each of the vessels are presented. Furthermore, here we cite an example of a deep learning pipeline using the U-Net semantic segmentation model to extract vascular regions. The presented database will be useful for the development of new software solutions for the analysis of histological micrograph.

The STOIC2021 COVID-19 AI challenge: Applying reusable training methodologies to private data

Challenges drive the state-of-the-art of automated medical image analysis. The quantity of public training data that they provide can limit the performance of their solutions. Public access to the training methodology for these solutions remains absent. This study implements the Type Three (T3) challenge format, which allows for training solutions on private data and guarantees reusable training methodologies. With T3, challenge organizers train a codebase provided by the participants on sequestered training data. T3 was implemented in the STOIC2021 challenge, with the goal of predicting from a computed tomography (CT) scan whether subjects had a severe COVID-19 infection, defined as intubation or death within one month. STOIC2021 consisted of a Qualification phase, where participants developed challenge solutions using 2000 publicly available CT scans, and a Final phase, where participants submitted their training methodologies with which solutions were trained on CT scans of 9724 subjects. The organizers successfully trained six of the eight Final phase submissions. The submitted codebases for training and running inference were released publicly. The winning solution obtained an area under the receiver operating characteristic curve for discerning between severe and non-severe COVID-19 of 0.815. The Final phase solutions of all finalists improved upon their Qualification phase solutions.

Deep Learning for Histopathological Assessment of Esophageal Adenocarcinoma Precursor Lesions

Histopathological assessment of esophageal biopsies is a key part in the management of patients with Barrett esophagus (BE) but prone to observer variability and reliable diagnostic methods are needed. Artificial intelligence (AI) is emerging as a powerful tool for aided diagnosis but often relies on abstract test and validation sets while real-world behavior is unknown. In this study, we developed a 2-stage AI system for histopathological assessment of BE-related dysplasia using deep learning to enhance the efficiency and accuracy of the pathology workflow. The AI system was developed and trained on 290 whole-slide images (WSIs) that were annotated at glandular and tissue levels. The system was designed to identify individual glands, grade dysplasia, and assign a WSI-level diagnosis. The proposed method was evaluated by comparing the performance of our AI system with that of a large international and heterogeneous group of 55 gastrointestinal pathologists assessing 55 digitized biopsies spanning the complete spectrum of BE-related dysplasia. The AI system correctly graded 76.4% of the WSIs, surpassing the performance of 53 out of the 55 participating pathologists. Furthermore, the receiver-operating characteristic analysis showed that the system's ability to predict the absence (nondysplastic BE) versus the presence of any dysplasia was with an area under the curve of 0.94 and a sensitivity of 0.92 at a specificity of 0.94. These findings demonstrate that this AI system has the potential to assist pathologists in assessment of BE-related dysplasia. The system's outputs could provide a reliable and consistent secondary diagnosis in challenging cases or be used for triaging low-risk nondysplastic biopsies, thereby reducing the workload of pathologists and increasing throughput.

Development and deployment of a histopathology-based deep learning algorithm for patient prescreening in a clinical trial

Accurate identification of genetic alterations in tumors, such as Fibroblast Growth Factor Receptor, is crucial for treating with targeted therapies; however, molecular testing can delay patient care due to the time and tissue required. Successful development, validation, and deployment of an AI-based, biomarker-detection algorithm could reduce screening cost and accelerate patient recruitment. Here, we develop a deep-learning algorithm using >3000 H&E-stained whole slide images from patients with advanced urothelial cancers, optimized for high sensitivity to avoid ruling out trial-eligible patients. The algorithm is validated on a dataset of 350 patients, achieving an area under the curve of 0.75, specificity of 31.8% at 88.7% sensitivity, and projected 28.7% reduction in molecular testing. We successfully deploy the system in a non-interventional study comprising 89 global study clinical sites and demonstrate its potential to prioritize/deprioritize molecular testing resources and provide substantial cost savings in the drug development and clinical settings.

Lung Cancer Diagnosis on Virtual Histologically Stained Tissue Using Weakly Supervised Learning

Lung adenocarcinoma (LUAD) is the most common primary lung cancer and accounts for 40% of all lung cancer cases. The current gold standard for lung cancer analysis is based on the pathologists' interpretation of hematoxylin and eosin (H&E)-stained tissue slices viewed under a brightfield microscope or a digital slide scanner. Computational pathology using deep learning has been proposed to detect lung cancer on histology images. However, the histological staining workflow to acquire the H&E-stained images and the subsequent cancer diagnosis procedures are labor-intensive and time-consuming with tedious sample preparation steps and repetitive manual interpretation, respectively. In this work, we propose a weakly supervised learning method for LUAD classification on label-free tissue slices with virtual histological staining. The autofluorescence images of label-free tissue with histopathological information can be converted into virtual H&E-stained images by a weakly supervised deep generative model. For the downstream LUAD classification task, we trained the attention-based multiple-instance learning model with different settings on the open-source LUAD H&E-stained whole-slide images (WSIs) dataset from the Cancer Genome Atlas (TCGA). The model was validated on the 150 H&E-stained WSIs collected from patients in Queen Mary Hospital and Prince of Wales Hospital with an average area under the curve (AUC) of 0.961. The model also achieved an average AUC of 0.973 on 58 virtual H&E-stained WSIs, comparable to the results on 58 standard H&E-stained WSIs with an average AUC of 0.977. The attention heatmaps of virtual H&E-stained WSIs and ground-truth H&E-stained WSIs can indicate tumor regions of LUAD tissue slices. In conclusion, the proposed diagnostic workflow on virtual H&E-stained WSIs of label-free tissue is a rapid, cost effective, and interpretable approach to assist clinicians in postoperative pathological examinations. The method could serve as a blueprint for other label-free imaging modalities and disease contexts.

A real-time augmented reality system integrated with artificial intelligence for skin tumor surgery: experimental study and case series

BACKGROUND

Skin tumors affect many people worldwide, and surgery is the first treatment choice. Achieving precise preoperative planning and navigation of intraoperative sampling remains a problem and is excessively reliant on the experience of surgeons, especially for Mohs surgery for malignant tumors.

MATERIALS AND METHODS

To achieve precise preoperative planning and navigation of intraoperative sampling, we developed a real-time augmented reality (AR) surgical system integrated with artificial intelligence (AI) to enhance three functions: AI-assisted tumor boundary segmentation, surgical margin design, and navigation in intraoperative tissue sampling. Non-randomized controlled trials were conducted on manikin, tumor-simulated rabbits, and human volunteers in Hunan Engineering Research Center of Skin Health and Disease Laboratory to evaluate the surgical system.

RESULTS

The results showed that the accuracy of the benign and malignant tumor segmentation was 0.9556 and 0.9548, respectively, and the average AR navigation mapping error was 0.644 mm. The proposed surgical system was applied in 106 skin tumor surgeries, including intraoperative navigation of sampling in 16 Mohs surgery cases. Surgeons who have used this system highly recognize it.

CONCLUSIONS

The surgical system highlighted the potential to achieve accurate treatment of skin tumors and to fill the gap in global research on skin tumor surgery systems.

Artificial intelligence to unlock real-world evidence in clinical oncology: A primer on recent advances

PURPOSE

Real world evidence is crucial to understanding the diffusion of new oncologic therapies, monitoring cancer outcomes, and detecting unexpected toxicities. In practice, real world evidence is challenging to collect rapidly and comprehensively, often requiring expensive and time-consuming manual case-finding and annotation of clinical text. In this Review, we summarise recent developments in the use of artificial intelligence to collect and analyze real world evidence in oncology.

METHODS

We performed a narrative review of the major current trends and recent literature in artificial intelligence applications in oncology.

RESULTS

Artificial intelligence (AI) approaches are increasingly used to efficiently phenotype patients and tumors at large scale. These tools also may provide novel biological insights and improve risk prediction through multimodal integration of radiographic, pathological, and genomic datasets. Custom language processing pipelines and large language models hold great promise for clinical prediction and phenotyping.

CONCLUSIONS

Despite rapid advances, continued progress in computation, generalizability, interpretability, and reliability as well as prospective validation are needed to integrate AI approaches into routine clinical care and real-time monitoring of novel therapies.

Clinical Application of Digital and Computational Pathology in Renal Cell Carcinoma: A Systematic Review

CONTEXT

Computational pathology is a new interdisciplinary field that combines traditional pathology with modern technologies such as digital imaging and machine learning to better understand the diagnosis, prognosis, and natural history of many diseases.

OBJECTIVE

To provide an overview of digital and computational pathology and its current and potential applications in renal cell carcinoma (RCC).

EVIDENCE ACQUISITION

A systematic review of the English-language literature was conducted using the PubMed, Web of Science, and Scopus databases in December 2022 according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines (PROSPERO ID: CRD42023389282). Risk of bias was assessed according to the Prediction Model Study Risk of Bias Assessment Tool.

EVIDENCE SYNTHESIS

In total, 20 articles were included in the review. All the studies used a retrospective design, and all digital pathology techniques were implemented retrospectively. The studies were classified according to their primary objective: detection, tumor characterization, and patient outcome. Regarding the transition to clinical practice, several studies showed promising potential. However, none presented a comprehensive assessment of clinical utility and implementation. Notably, there was substantial heterogeneity for both the strategies used for model building and the performance metrics reported.

CONCLUSIONS

This review highlights the vast potential of digital and computational pathology for the detection, classification, and assessment of oncological outcomes in RCC. Preliminary work in this field has yielded promising results. However, these models have not yet reached a stage where they can be integrated into routine clinical practice.

PATIENT SUMMARY

Computational pathology combines traditional pathology and technologies such as digital imaging and artificial intelligence to improve diagnosis of disease and identify prognostic factors and new biomarkers. The number of studies exploring its potential in kidney cancer is rapidly increasing. However, despite the surge in research activity, computational pathology is not yet ready for widespread routine use.