Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer

Gastric cancer metastasis-related NT5DC2 indicates unfavorable prognosis of patients

PURPOSE

Approximately 80 to 90% of patients with gastric cancer (GC) eventually develop into metastatic GC nowadays,because GC is difficult to be diagnosed at an early stage. GC patients with metastases typically have a poor prognosis. It is necessary to explore a potential prognostic marker in metastatic GC.

METHODS

All GC data were obtained from The Cancer Genome Atlas and Gene Expression Omnibus databases. The metastasis-related candidate gene and its role in GC were analyzed by comprehensive analysis.

RESULTS

Totally 1049 metastasis-related genes were identified in GC. Univariate Cox regression analysis screened the top 10 genes (PDHX, SLC43A1, CSAG2, NT5DC2, CSAG1, FMN1, MED1, HIVEP2, FNDC3A, and PPP1R2) that were closely correlated with prognosis of GC patients. Among which, NT5DC2 was screened as the target gene for subsequent study. The NT5DC2 expression were increased in primary GC and metastatic GC samples. Moreover, GC patients with high NT5DC2 expression exhibited shorter overall survival and post progression survival, and the NT5DC2 was metastatic GC patients' independent prognostic factor. Totally 29 pathways were activated in metastatic GC samples with high NT5DC2 expression. Four immune cells' infiltration were significantly different between NT5DC2 high and low expressed metastatic GC patients. NT5DC2 showed significantly negative correlations with 6 types of immune cells' critical marker genes and 5 types of immune cell infiltration. The 10 immune checkpoint expressions were decreased in high NTDC2 expression metastatic GC patients.

CONCLUSIONS

NT5DC2 plays a prognostic role in metastatic GC. GC patients with high NT5DC2 expression indicates unfavorable prognosis.

The use of artificial intelligence tools in cancer detection compared to the traditional diagnostic imaging methods: An overview of the systematic reviews

BACKGROUND AND PURPOSE

In comparison to conventional medical imaging diagnostic modalities, the aim of this overview article is to analyze the accuracy of the application of Artificial Intelligence (AI) techniques in the identification and diagnosis of malignant tumors in adult patients.

DATA SOURCES

The acronym PIRDs was used and a comprehensive literature search was conducted on PubMed, Cochrane, Scopus, Web of Science, LILACS, Embase, Scielo, EBSCOhost, and grey literature through Proquest, Google Scholar, and JSTOR for systematic reviews of AI as a diagnostic model and/or detection tool for any cancer type in adult patients, compared to the traditional diagnostic radiographic imaging model. There were no limits on publishing status, publication time, or language. For study selection and risk of bias evaluation, pairs of reviewers worked separately.

RESULTS

In total, 382 records were retrieved in the databases, 364 after removing duplicates, 32 satisfied the full-text reading criterion, and 09 papers were considered for qualitative synthesis. Although there was heterogeneity in terms of methodological aspects, patient differences, and techniques used, the studies found that several AI approaches are promising in terms of specificity, sensitivity, and diagnostic accuracy in the detection and diagnosis of malignant tumors. When compared to other machine learning algorithms, the Super Vector Machine method performed better in cancer detection and diagnosis. Computer-assisted detection (CAD) has shown promising in terms of aiding cancer detection, when compared to the traditional method of diagnosis.

CONCLUSIONS

The detection and diagnosis of malignant tumors with the help of AI seems to be feasible and accurate with the use of different technologies, such as CAD systems, deep and machine learning algorithms and radiomic analysis when compared with the traditional model, although these technologies are not capable of to replace the professional radiologist in the analysis of medical images. Although there are limitations regarding the generalization for all types of cancer, these AI tools might aid professionals, serving as an auxiliary and teaching tool, especially for less trained professionals. Therefore, further longitudinal studies with a longer follow-up duration are required for a better understanding of the clinical application of these artificial intelligence systems.

TRIAL REGISTRATION

Systematic review registration. Prospero registration number: CRD42022307403.

Artificial intelligence in diagnostic pathology

Digital pathology (DP) is being increasingly employed in cancer diagnostics, providing additional tools for faster, higher-quality, accurate diagnosis. The practice of diagnostic pathology has gone through a staggering transformation wherein new tools such as digital imaging, advanced artificial intelligence (AI) algorithms, and computer-aided diagnostic techniques are being used for assisting, augmenting and empowering the computational histopathology and AI-enabled diagnostics. This is paving the way for advancement in precision medicine in cancer. Automated whole slide imaging (WSI) scanners are now rendering diagnostic quality, high-resolution images of entire glass slides and combining these images with innovative digital pathology tools is making it possible to integrate imaging into all aspects of pathology reporting including anatomical, clinical, and molecular pathology. The recent approvals of WSI scanners for primary diagnosis by the FDA as well as the approval of prostate AI algorithm has paved the way for starting to incorporate this exciting technology for use in primary diagnosis. AI tools can provide a unique platform for innovations and advances in anatomical and clinical pathology workflows. In this review, we describe the milestones and landmark trials in the use of AI in clinical pathology with emphasis on future directions.

A deep learning model to triage and predict adenocarcinoma on pancreas cytology whole slide imaging

Pancreatic fine-needle aspirations are the gold-standard diagnostic procedure for the evaluation of pancreatic ductal adenocarcinoma. A suspicion for malignancy can escalate towards chemotherapy followed by a major surgery and therefore is a high-stakes task for the pathologist. In this paper, we propose a deep learning framework, MIPCL, that can serve as a helpful screening tool, predicting the presence or absence of cancer. We also reproduce two deep learning models that have found success in surgical pathology for our cytopathology study. Our MIPCL significantly improves over both models across all evaluated metrics (F1-Score: 87.97% vs 88.70% vs 91.07%; AUROC: 0.9159 vs. 0.9051 vs 0.9435). Additionally, our model is able to recover the most contributing regions on the slide for the final prediction. We also present a dataset curation strategy that increases the number of training examples from an existing dataset, thereby reducing the resource burden tied to collecting and scanning additional cases.

Artificial Intelligence in Breast Cancer Diagnosis and Personalized Medicine

Breast cancer is a significant cause of cancer-related mortality in women worldwide. Early and precise diagnosis is crucial, and clinical outcomes can be markedly enhanced. The rise of artificial intelligence (AI) has ushered in a new era, notably in image analysis, paving the way for major advancements in breast cancer diagnosis and individualized treatment regimens. In the diagnostic workflow for patients with breast cancer, the role of AI encompasses screening, diagnosis, staging, biomarker evaluation, prognostication, and therapeutic response prediction. Although its potential is immense, its complete integration into clinical practice is challenging. Particularly, these challenges include the imperatives for extensive clinical validation, model generalizability, navigating the "black-box" conundrum, and pragmatic considerations of embedding AI into everyday clinical environments. In this review, we comprehensively explored the diverse applications of AI in breast cancer care, underlining its transformative promise and existing impediments. In radiology, we specifically address AI in mammography, tomosynthesis, risk prediction models, and supplementary imaging methods, including magnetic resonance imaging and ultrasound. In pathology, our focus is on AI applications for pathologic diagnosis, evaluation of biomarkers, and predictions related to genetic alterations, treatment response, and prognosis in the context of breast cancer diagnosis and treatment. Our discussion underscores the transformative potential of AI in breast cancer management and emphasizes the importance of focused research to realize the full spectrum of benefits of AI in patient care.

Minimum resolution requirements of digital pathology images for accurate classification

Digitization of pathology has been proposed as an essential mitigation strategy for the severe staffing crisis facing most pathology departments. Despite its benefits, several barriers have prevented widespread adoption of digital workflows, including cost and pathologist reluctance due to subjective image quality concerns. In this work, we quantitatively determine the minimum image quality requirements for binary classification of histopathology images of breast tissue in terms of spatial and sampling resolution. We train an ensemble of deep learning classifier models on publicly available datasets to obtain a baseline accuracy and computationally degrade these images according to our derived theoretical model to identify the minimum resolution necessary for acceptable diagnostic accuracy. Our results show that images can be degraded significantly below the resolution of most commercial whole-slide imaging systems while maintaining reasonable accuracy, demonstrating that macroscopic features are sufficient for binary classification of stained breast tissue. A rapid low-cost imaging system capable of identifying healthy tissue not requiring human assessment could serve as a triage system for reducing caseloads and alleviating the significant strain on the current workforce.

SGCL: Spatial guided contrastive learning on whole-slide pathological images

Self-supervised representation learning (SSL) has achieved remarkable success in its application to natural images while falling behind in performance when applied to whole-slide pathological images (WSIs). This is because the inherent characteristics of WSIs in terms of gigapixel resolution and multiple objects in training patches are fundamentally different from natural images. Directly transferring the state-of-the-art (SOTA) SSL methods designed for natural images to WSIs will inevitably compromise their performance. We present a novel scheme SGCL: Spatial Guided Contrastive Learning, to fully explore the inherent properties of WSIs, leveraging the spatial proximity and multi-object priors for stable self-supervision. Beyond the self-invariance of instance discrimination, we expand and propagate the spatial proximity for the intra-invariance from the same WSI and inter-invariance from different WSIs, as well as propose the spatial-guided multi-cropping for inner-invariance within patches. To adaptively explore such spatial information without supervision, we propose a new loss function and conduct a theoretical analysis to validate it. This novel scheme of SGCL is able to achieve additional improvements over the SOTA pre-training methods on diverse downstream tasks across multiple datasets. Extensive ablation studies have been carried out and visualizations of these results have been presented to aid understanding of the proposed SGCL scheme. As open science, all codes and pre-trained models are available at https://github.com/HHHedo/SGCL.

Deep Learning-Based Stratification of Gastric Cancer Patients from Hematoxylin and Eosin-Stained Whole Slide Images by Predicting Molecular Features for Immunotherapy Response

Determining the molecular characteristics of cancer patients is crucial for optimal immunotherapy decisions. The aim of this study was to screen immunotherapy beneficiaries by predicting key molecular features from hematoxylin and eosin-stained images based on deep learning models. An independent data set from Asian gastric cancer patients was included for external validation. In addition, a segmentation model (Horizontal-Vertical Network) was used to quantify the cellular composition of tumor stroma. The model performance was evaluated by measuring the area under the curve (AUC). The tumor extraction model achieved an AUC of 0.9386 and 0.9062 in the internal and external test sets, respectively. The stratification model could predict the immunotherapy-sensitive subtypes (AUC range, 0.8685 to 0.9461), the genetic mutations (AUC range, 0.8283 to 0.9225), and the pathway activity (AUC range, 0.7568 to 0.8612) fairly accurately. In external validation, the prediction performance of Epstein-Barr virus and programmed cell death ligand 1 expression status achieved AUCs of 0.7906 and 0.6384, respectively. The segmentation model identified a relatively high proportion of inflammatory cells and connective cells in some immunotherapy-sensitive subtypes. The deep learning-based models potentially may serve as a valuable tool to screen for the beneficiaries of immunotherapy in gastric cancer patients.

Utility of Machine Learning to Detect Cytomegalovirus in Digital Hematoxylin and Eosin-Stained Slides

Rapid and accurate cytomegalovirus (CMV) identification in immunosuppressed or immunocompromised patients presenting with diarrhea is essential for therapeutic management. Due to viral latency, however, the gold standard for CMV diagnosis remains to identify viral cytopathic inclusions on routine hematoxylin and eosin (H&E)-stained tissue sections. Therefore, biopsies may be taken and "rushed" for pathology evaluation. Here, we propose the use of artificial intelligence to detect CMV inclusions on routine H&E-stained whole-slide images to aid pathologists in evaluating these cases. Fifty-eight representative H&E slides from 30 cases with CMV inclusions were identified and scanned. The resulting whole-slide images were manually annotated for CMV inclusions and tiled into 300 × 300 pixel patches. Patches containing annotations were labeled "positive," and these tiles were oversampled with image augmentation to account for class imbalance. The remaining patches were labeled "negative." Data were then divided into training, validation, and holdout sets. Multiple deep learning models were provided with training data, and their performance was analyzed. All tested models showed excellent performance. The highest performance was seen using the EfficientNetV2BO model, which had a test (holdout) accuracy of 99.93%, precision of 100.0%, recall (sensitivity) of 99.85%, and area under the curve of 0.9998. Of 518,941 images in the holdout set, there were only 346 false negatives and 2 false positives. This shows proof of concept for the use of digital tools to assist pathologists in screening "rush" biopsies for CMV infection. Given the high precision, cases screened as "positive" can be quickly confirmed by a pathologist, reducing missed CMV inclusions and improving the confidence of preliminary results. Additionally, this may reduce the need for immunohistochemistry in limited tissue samples, reducing associated costs and turnaround time.

Deep learning approach for accurate prostate cancer identification and stratification using combined immunostaining of cytokeratin, p63, and racemase

BACKGROUND

Prostate cancer (PCa) is the most frequently diagnosed cancer in men worldwide, affecting around 1.4 million individuals. Current PCa diagnosis relies on histological analysis of prostate biopsy samples, an activity that is both time-consuming and prone to observer bias. Previous studies have demonstrated that immunostaining of cytokeratin, p63, and racemase can significantly improve the sensitivity and the specificity of PCa detection compared to traditional H&E staining.

METHODS

This study introduces a novel approach that combines diagnosis-specific immunohistochemical (IHC) staining and deep learning techniques to provide reliable stratification of prostate glands. Our approach leverages a customized segmentation network, called K-PPM, that incorporates adaptive kernels and multiscale feature integration to enhance the functional information of IHC. To address the high class-imbalance problem in the dataset, we propose a weighted adaptive patch-extraction and specific-class kernel update.

RESULTS

Our system achieved noteworthy results, with a mean Dice Score Coefficient of 90.36% and a mean absolute error of 1.64 % in specific-class gland quantification on whole slides. These findings demonstrate the potential of our system as a valuable support tool for pathologists, reducing workload and decreasing diagnostic inter-observer variability.

CONCLUSIONS

Our study presents innovative approaches that have broad applicability to other digital pathology areas beyond PCa diagnosis. As a fully automated system, this model can serve as a framework for improving the histological and IHC diagnosis of other types of cancer.

Leveraging physiology and artificial intelligence to deliver advancements in health care

Artificial intelligence in health care has experienced remarkable innovation and progress in the last decade. Significant advancements can be attributed to the utilization of artificial intelligence to transform physiology data to advance health care. In this review, we explore how past work has shaped the field and defined future challenges and directions. In particular, we focus on three areas of development. First, we give an overview of artificial intelligence, with special attention to the most relevant artificial intelligence models. We then detail how physiology data have been harnessed by artificial intelligence to advance the main areas of health care: automating existing health care tasks, increasing access to care, and augmenting health care capabilities. Finally, we discuss emerging concerns surrounding the use of individual physiology data and detail an increasingly important consideration for the field, namely the challenges of deploying artificial intelligence models to achieve meaningful clinical impact.

Artificial intelligence & clinical nutrition: What the future might have in store

Artificial Intelligence (AI) is a rapidly emerging technology in healthcare that has the potential to revolutionise clinical nutrition. AI can assist in analysing complex data, interpreting medical images, and providing personalised nutrition interventions for patients. Clinical nutrition is a critical aspect of patient care, and AI can help clinicians make more informed decisions regarding patients' nutritional requirements, disease prevention, and management. AI algorithms can analyse large datasets to identify novel associations between diet and disease outcomes, enabling clinicians to make evidence-based nutritional recommendations. AI-powered devices and applications can also assist in tracking dietary intake, providing feedback, and motivating patients towards healthier food choices. However, the adoption of AI in clinical nutrition raises several ethical and regulatory concerns, such as data privacy and bias. Further research is needed to assess the clinical effectiveness and safety of AI-powered nutrition interventions. In conclusion, AI has the potential to transform clinical nutrition, but its integration into clinical practice should be carefully monitored to ensure patient safety and benefit. This article discusses the current and future applications of AI in clinical nutrition and highlights its potential benefits.

Pyramid-based self-supervised learning for histopathological image classification

Large-scale labeled datasets are crucial for the success of supervised learning in medical imaging. However, annotating histopathological images is a time-consuming and labor-intensive task that requires highly trained professionals. To address this challenge, self-supervised learning (SSL) can be utilized to pre-train models on large amounts of unsupervised data and transfer the learned representations to various downstream tasks. In this study, we propose a self-supervised Pyramid-based Local Wavelet Transformer (PLWT) model for effectively extracting rich image representations. The PLWT model extracts both local and global features to pre-train a large number of unlabeled histopathology images in a self-supervised manner. Wavelet is used to replace average pooling in the downsampling of the multi-head attention, achieving a significant reduction in information loss during the transmission of image features. Additionally, we introduce a Local Squeeze-and-Excitation (Local SE) module in the feedforward network in combination with the inverse residual to capture local image information. We evaluate PLWT's performance on three histopathological images and demonstrate the impact of pre-training. Our experiment results indicate that PLWT with self-supervised learning performs highly competitive when compared with other SSL methods, and the transferability of visual representations generated by SSL on domain-relevant histopathological images exceeds that of the supervised baseline trained on ImageNet.

CReg-KD: Model refinement via confidence regularized knowledge distillation for brain imaging

One of the core challenges of deep learning in medical image analysis is data insufficiency, especially for 3D brain imaging, which may lead to model over-fitting and poor generalization. Regularization strategies such as knowledge distillation are powerful tools to mitigate the issue by penalizing predictive distributions and introducing additional knowledge to reinforce the training process. In this paper, we revisit knowledge distillation as a regularization paradigm by penalizing attentive output distributions and intermediate representations. In particular, we propose a Confidence Regularized Knowledge Distillation (CReg-KD) framework, which adaptively transfers knowledge for distillation in light of knowledge confidence. Two strategies are advocated to regularize the global and local dependencies between teacher and student knowledge. In detail, a gated distillation mechanism is proposed to soften the transferred knowledge globally by utilizing the teacher loss as a confidence score. Moreover, the intermediate representations are attentively and locally refined with key semantic context to mimic meaningful features. To demonstrate the superiority of our proposed framework, we evaluated the framework on two brain imaging analysis tasks (i.e. Alzheimer's Disease classification and brain age estimation based on T1-weighted MRI) on the Alzheimer's Disease Neuroimaging Initiative dataset including 902 subjects and a cohort of 3655 subjects from 4 public datasets. Extensive experimental results show that CReg-KD achieves consistent improvements over the baseline teacher model and outperforms other state-of-the-art knowledge distillation approaches, manifesting that CReg-KD as a powerful medical image analysis tool in terms of both promising prediction performance and generalizability.

Predictive Modeling Using Artificial Intelligence and Machine Learning Algorithms on Electronic Health Record Data: Advantages and Challenges

The rapid adoption of electronic health record (EHR) systems in US hospitals from 2008 to 2014 produced novel data elements for analysis. Concurrent innovations in computing architecture and machine learning (ML) algorithms have made rapid consumption of health data feasible and a powerful engine for clinical innovation. In critical care research, the net convergence of these trends has resulted in an exponential increase in outcome prediction research. In the following article, we explore the history of outcome prediction in the intensive care unit (ICU), the growing use of EHR data, and the rise of artificial intelligence and ML (AI) in critical care.

A prospective observational multicentric clinical trial to evaluate microscopic examination of acid-fast bacilli in sputum by artificial intelligence-based microscopy system

Microscopy-based tuberculosis (TB) diagnosis i.e., Ziehl-Neelsen (ZN) stained smear screening still remains the primary diagnostic method in resource poor and high TB burden countries, however itrequires considerable experience and is bound to human errors. In remote areas, wherever expert microscopist is not available, timely diagnosis at initial level is not possible. Artificial intelligence (AI)-based microscopy may be a solution to this problem. A prospective observational multi-centric clinical trial to evaluate microscopic examination of acid-fast bacilli (AFB) in sputum by the AI based system was done in three hospitals in Northern India. Sputum samples from 400 clinically suspected cases of pulmonary tuberculosis were collected from three centres. Ziehl-Neelsen staining of smears was done. All the smears were observed by 3 microscopist and the AI based microscopy system. AI based microscopy was found to have a sensitivity, specificity, positive predictive value, negative predictive value and diagnostic accuracy of 89.25%, 92.15%, 75.45%, 96.94%, 91.53% respectively. AI based sputum microscopy has an acceptable degree of accuracy, PPV, NPV, specificity and sensitivity and thus may be used as a screening tool for the diagnosis of pulmonary tuberculosis.

The promise of digital healthcare technologies

Digital health technologies have been in use for many years in a wide spectrum of healthcare scenarios. This narrative review outlines the current use and the future strategies and significance of digital health technologies in modern healthcare applications. It covers the current state of the scientific field (delineating major strengths, limitations, and applications) and envisions the future impact of relevant emerging key technologies. Furthermore, we attempt to provide recommendations for innovative approaches that would accelerate and benefit the research, translation and utilization of digital health technologies.

Colorectal cancer risk stratification on histological slides based on survival curves predicted by deep learning

Studies have shown that colorectal cancer prognosis can be predicted by deep learning-based analysis of histological tissue sections of the primary tumor. So far, this has been achieved using a binary prediction. Survival curves might contain more detailed information and thus enable a more fine-grained risk prediction. Therefore, we established survival curve-based CRC survival predictors and benchmarked them against standard binary survival predictors, comparing their performance extensively on the clinical high and low risk subsets of one internal and three external cohorts. Survival curve-based risk prediction achieved a very similar risk stratification to binary risk prediction for this task. Exchanging other components of the pipeline, namely input tissue and feature extractor, had largely identical effects on model performance independently of the type of risk prediction. An ensemble of all survival curve-based models exhibited a more robust performance, as did a similar ensemble based on binary risk prediction. Patients could be further stratified within clinical risk groups. However, performance still varied across cohorts, indicating limited generalization of all investigated image analysis pipelines, whereas models using clinical data performed robustly on all cohorts.

Computerized tertiary lymphoid structures density on H&E-images is a prognostic biomarker in resectable lung adenocarcinoma

The increased amount of tertiary lymphoid structures (TLSs) is associated with a favorable prognosis in patients with lung adenocarcinoma (LUAD). However, evaluating TLSs manually is an experience-dependent and time-consuming process, which limits its clinical application. In this multi-center study, we developed an automated computational workflow for quantifying the TLS density in the tumor region of routine hematoxylin and eosin (H&E)-stained whole-slide images (WSIs). The association between the computerized TLS density and disease-free survival (DFS) was further explored in 802 patients with resectable LUAD of three cohorts. Additionally, a Cox proportional hazard regression model, incorporating clinicopathological variables and the TLS density, was established to assess its prognostic ability. The computerized TLS density was an independent prognostic biomarker in patients with resectable LUAD. The integration of the TLS density with clinicopathological variables could support individualized clinical decision-making by improving prognostic stratification.

Enhancing Kidney Transplant Care through the Integration of Chatbot

Kidney transplantation is a critical treatment option for end-stage kidney disease patients, offering improved quality of life and increased survival rates. However, the complexities of kidney transplant care necessitate continuous advancements in decision making, patient communication, and operational efficiency. This article explores the potential integration of a sophisticated chatbot, an AI-powered conversational agent, to enhance kidney transplant practice and potentially improve patient outcomes. Chatbots and generative AI have shown promising applications in various domains, including healthcare, by simulating human-like interactions and generating contextually appropriate responses. Noteworthy AI models like ChatGPT by OpenAI, BingChat by Microsoft, and Bard AI by Google exhibit significant potential in supporting evidence-based research and healthcare decision making. The integration of chatbots in kidney transplant care may offer transformative possibilities. As a clinical decision support tool, it could provide healthcare professionals with real-time access to medical literature and guidelines, potentially enabling informed decision making and improved knowledge dissemination. Additionally, the chatbot has the potential to facilitate patient education by offering personalized and understandable information, addressing queries, and providing guidance on post-transplant care. Furthermore, under clinician or transplant pharmacist supervision, it has the potential to support post-transplant care and medication management by analyzing patient data, which may lead to tailored recommendations on dosages, monitoring schedules, and potential drug interactions. However, to fully ascertain its effectiveness and safety in these roles, further studies and validation are required. Its integration with existing clinical decision support systems may enhance risk stratification and treatment planning, contributing to more informed and efficient decision making in kidney transplant care. Given the importance of ethical considerations and bias mitigation in AI integration, future studies may evaluate long-term patient outcomes, cost-effectiveness, user experience, and the generalizability of chatbot recommendations. By addressing these factors and potentially leveraging AI capabilities, the integration of chatbots in kidney transplant care holds promise for potentially improving patient outcomes, enhancing decision making, and fostering the equitable and responsible use of AI in healthcare.

Deep learning transforms colorectal cancer biomarker prediction from histopathology images

Artificial intelligence (AI) is rapidly gaining interest in medicine, including pathological assessments for personalized medicine. In this issue of Cancer Cell, Wagner et al. demonstrate superior accuracy of transformer-based deep learning in predicting biomarker status in CRC. The work has implications for increased efficiency and accuracy in clinical diagnostics guiding treatment decisions in precision oncology.

Clinical applications of graph neural networks in computational histopathology: A review

Pathological examination is the optimal approach for diagnosing cancer, and with the advancement of digital imaging technologies, it has spurred the emergence of computational histopathology. The objective of computational histopathology is to assist in clinical tasks through image processing and analysis techniques. In the early stages, the technique involved analyzing histopathology images by extracting mathematical features, but the performance of these models was unsatisfactory. With the development of artificial intelligence (AI) technologies, traditional machine learning methods were applied in this field. Although the performance of the models improved, there were issues such as poor model generalization and tedious manual feature extraction. Subsequently, the introduction of deep learning techniques effectively addressed these problems. However, models based on traditional convolutional architectures could not adequately capture the contextual information and deep biological features in histopathology images. Due to the special structure of graphs, they are highly suitable for feature extraction in tissue histopathology images and have achieved promising performance in numerous studies. In this article, we review existing graph-based methods in computational histopathology and propose a novel and more comprehensive graph construction approach. Additionally, we categorize the methods and techniques in computational histopathology according to different learning paradigms. We summarize the common clinical applications of graph-based methods in computational histopathology. Furthermore, we discuss the core concepts in this field and highlight the current challenges and future research directions.

Screening of normal endoscopic large bowel biopsies with interpretable graph learning: a retrospective study

OBJECTIVE

To develop an interpretable artificial intelligence algorithm to rule out normal large bowel endoscopic biopsies, saving pathologist resources and helping with early diagnosis.

DESIGN

A graph neural network was developed incorporating pathologist domain knowledge to classify 6591 whole-slides images (WSIs) of endoscopic large bowel biopsies from 3291 patients (approximately 54% female, 46% male) as normal or abnormal (non-neoplastic and neoplastic) using clinically driven interpretable features. One UK National Health Service (NHS) site was used for model training and internal validation. External validation was conducted on data from two other NHS sites and one Portuguese site.

RESULTS

Model training and internal validation were performed on 5054 WSIs of 2080 patients resulting in an area under the curve-receiver operating characteristic (AUC-ROC) of 0.98 (SD=0.004) and AUC-precision-recall (PR) of 0.98 (SD=0.003). The performance of the model, named Interpretable Gland-Graphs using a Neural Aggregator (IGUANA), was consistent in testing over 1537 WSIs of 1211 patients from three independent external datasets with mean AUC-ROC=0.97 (SD=0.007) and AUC-PR=0.97 (SD=0.005). At a high sensitivity threshold of 99%, the proposed model can reduce the number of normal slides to be reviewed by a pathologist by approximately 55%. IGUANA also provides an explainable output highlighting potential abnormalities in a WSI in the form of a heatmap as well as numerical values associating the model prediction with various histological features.

CONCLUSION

The model achieved consistently high accuracy showing its potential in optimising increasingly scarce pathologist resources. Explainable predictions can guide pathologists in their diagnostic decision-making and help boost their confidence in the algorithm, paving the way for its future clinical adoption.

Pathologists' first opinions on barriers and facilitators of computational pathology adoption in oncological pathology: an international study

Computational pathology (CPath) algorithms detect, segment or classify cancer in whole slide images, approaching or even exceeding the accuracy of pathologists. Challenges have to be overcome before these algorithms can be used in practice. We therefore aim to explore international perspectives on the future role of CPath in oncological pathology by focusing on opinions and first experiences regarding barriers and facilitators. We conducted an international explorative eSurvey and semi-structured interviews with pathologists utilizing an implementation framework to classify potential influencing factors. The eSurvey results showed remarkable variation in opinions regarding attitude, understandability and validation of CPath. Interview results showed that barriers focused on the quality of available evidence, while most facilitators concerned strengths of CPath. A lack of consensus was present for multiple factors, such as the determination of sufficient validation using CPath, the preferred function of CPath within the digital workflow and the timing of CPath introduction in pathology education. The diversity in opinions illustrates variety in influencing factors in CPath adoption. A next step would be to quantitatively determine important factors for adoption and initiate validation studies. Both should include clear case descriptions and be conducted among a more homogenous panel of pathologists based on sub specialization.

Survival Prediction via Hierarchical Multimodal Co-Attention Transformer: A Computational Histology-Radiology Solution

The rapid advances in deep learning-based computational pathology and radiology have demonstrated the promise of using whole slide images (WSIs) and radiology images for survival prediction in cancer patients. However, most image-based survival prediction methods are limited to using either histology or radiology alone, leaving integrated approaches across histology and radiology relatively underdeveloped. There are two main challenges in integrating WSIs and radiology images: (1) the gigapixel nature of WSIs and (2) the vast difference in spatial scales between WSIs and radiology images. To address these challenges, in this work, we propose an interpretable, weakly-supervised, multimodal learning framework, called Hierarchical Multimodal Co-Attention Transformer (HMCAT), to integrate WSIs and radiology images for survival prediction. Our approach first uses hierarchical feature extractors to capture various information including cellular features, cellular organization, and tissue phenotypes in WSIs. Then the hierarchical radiology-guided co- attention (HRCA) in HMCAT characterizes the multimodal interactions between hierarchical histology-based visual concepts and radiology features and learns hierarchical co- attention mappings for two modalities. Finally, HMCAT combines their complementary information into a multimodal risk score and discovers prognostic features from two modalities by multimodal interpretability. We apply our approach to two cancer datasets (365 WSIs with matched magnetic resonance [MR] images and 213 WSIs with matched computed tomography [CT] images). Our results demonstrate that the proposed HMCAT consistently achieves superior performance over the unimodal approaches trained on either histology or radiology data alone, as well as other state-of-the-art methods.

The Use of Deep Learning-Based Computer Diagnostic Algorithm for Detection of Lymph Node Metastases of Gastric Adenocarcinoma

Objectives. The diversifying modalities of treatment for gastric cancer raise urgent demands for the rapid and precise diagnosis of metastases in regional lymph nodes, thereby significantly impact the workload of pathologists. Meanwhile, the recent advent of whole-slide scanners and deep-learning techniques have enabled the computer-assisted analysis of histopathological images, which could help to alleviate this impact. Thus, we developed a deep learning-based diagnostic algorithm to detect lymph node metastases of gastric adenocarcinoma and evaluated its performance. Methods. We randomly selected 20 patients with gastric adenocarcinoma who underwent surgery as definitive treatment and were found to be node metastasis-positive. HEMATOXYLIN-eosin (HE) stained glass slides, including a total of 51 metastasis-positive nodes, were retrieved from the specimens of these cases. Other slides with 776 metastasis-negative nodes were also retrieved from other twenty cases with the same disease that were diagnosed as metastasis-negative by the final pathological examinations. All glass slides were digitized using a whole-slide scanner. A deep-learning algorithm to detect metastases was developed using the data in which metastasis-positive parts of the images were annotated by a well-trained pathologist, and its performance in detecting metastases was evaluated. Results. Cross-validation analysis indicated an area of 0.9994 under the receiver operating characteristic curve. Free-response receiver operating characteristic curve (FROC) analysis indicated a sensitivity of 1.00 with three false positives. Further evaluation using an independent dataset also showed similar level of accuracies. Conclusion. This deep learning-based diagnosis-aid system is a promising tool that can assist pathologists involved in gastric cancer care and reduce their workload.

Identification of lymph node metastasis in pre-operation cervical cancer patients by weakly supervised deep learning from histopathological whole-slide biopsy images

BACKGROUND

Lymph node metastasis (LNM) significantly impacts the prognosis of individuals diagnosed with cervical cancer, as it is closely linked to disease recurrence and mortality, thereby impacting therapeutic schedule choices for patients. However, accurately predicting LNM prior to treatment remains challenging. Consequently, this study seeks to utilize digital pathological features extracted from histopathological slides of primary cervical cancer patients to preoperatively predict the presence of LNM.

METHODS

A deep learning (DL) model was trained using the Vision transformer (ViT) and recurrent neural network (RNN) frameworks to predict LNM. This prediction was based on the analysis of 554 histopathological whole-slide images (WSIs) obtained from Qilu Hospital of Shandong University. To validate the model's performance, an external test was conducted using 336 WSIs from four other hospitals. Additionally, the efficiency of the DL model was evaluated using 190 cervical biopsies WSIs in a prospective set.

RESULTS

In the internal test set, our DL model achieved an area under the curve (AUC) of 0.919, with sensitivity and specificity values of 0.923 and 0.905, respectively, and an accuracy (ACC) of 0.909. The performance of the DL model remained strong in the external test set. In the prospective cohort, the AUC was 0.91, and the ACC was 0.895. Additionally, the DL model exhibited higher accuracy compared to imaging examination in the evaluation of LNM. By utilizing the transformer visualization method, we generated a heatmap that illustrates the local pathological features in primary lesions relevant to LNM.

CONCLUSION

DL-based image analysis has demonstrated efficiency in predicting LNM in early operable cervical cancer through the utilization of biopsies WSI. This approach has the potential to enhance therapeutic decision-making for patients diagnosed with cervical cancer.

Predictive value of machine learning for breast cancer recurrence: a systematic review and meta-analysis

PURPOSE

Recurrence of breast cancer leads to a high lifetime risk and a low 5 year survival rate. Researchers have used machine learning to predict the risk of recurrence in patients with breast cancer, but the predictive performance of machine learning remains controversial. Hence, this study aimed to explore the accuracy of machine learning in predicting breast cancer recurrence risk and aggregate predictive variables to provide guidance for the development of subsequent risk scoring systems.

METHODS

We searched Pubmed, EMBASE, Cochrane, and Web of Science. The risk of bias in the included studies was evaluated using prediction model risk of bias assessment tool (PROBAST). Meta-regression was adopted to explore whether there was a significant difference in the recurrence time by machine learning.

RESULTS

Thirty-four studies involving 67,560 subjects were included, among whom 8695 experienced breast cancer recurrence. The c-index of prediction models was 0.814 (95%CI 0.802-0.826) and 0.770 (95%CI 0.737-0.803) in the training and validation sets, respectively; the sensitivity and specificity were 0.69 (95% CI 0.64-0.74), 0.89 (95% CI 0.86-0.92) in the training, and 0.64 (95% CI 0.58-0.70), 0.88 (95% CI 0.82-0.92) in the validation, respectively. Age, histological grading, and lymph node status are the most commonly used variables in model construction. Attention should be paid to unhealthy lifestyles such as drinking, smoking and BMI as modeling variables. Risk prediction models based on machine learning have long-term monitoring value for breast cancer population, and subsequent studies should consider using large-sample and multi-center data to establish risk equations for verification.

CONCLUSION

Machine learning may be used as a predictive tool for breast cancer recurrence. Currently, there is a lack of effective and universally applicable machine learning models in clinical practice. We expect to incorporate multi-center studies in the future and attempt to develop tools for predicting breast cancer recurrence risk, so as to effectively identify populations at high risk of recurrence and develop personalized follow-up strategies and prognostic interventions to reduce the risk of recurrence.

Applications of Artificial Intelligence in Breast Pathology

CONTEXT.—

Increasing implementation of whole slide imaging together with digital workflow and advances in computing capacity enable the use of artificial intelligence (AI) in pathology, including breast pathology. Breast pathologists often face a significant workload, with diagnosis complexity, tedious repetitive tasks, and semiquantitative evaluation of biomarkers. Recent advances in developing AI algorithms have provided promising approaches to meet the demand in breast pathology.

OBJECTIVE.—

To provide an updated review of AI in breast pathology. We examined the success and challenges of current and potential AI applications in diagnosing and grading breast carcinomas and other pathologic changes, detecting lymph node metastasis, quantifying breast cancer biomarkers, predicting prognosis and therapy response, and predicting potential molecular changes.

DATA SOURCES.—

We obtained data and information by searching and reviewing literature on AI in breast pathology from PubMed and based our own experience.

CONCLUSIONS.—

With the increasing application in breast pathology, AI not only assists in pathology diagnosis to improve accuracy and reduce pathologists' workload, but also provides new information in predicting prognosis and therapy response.

ChampKit: A framework for rapid evaluation of deep neural networks for patch-based histopathology classification

BACKGROUND AND OBJECTIVE

Histopathology is the gold standard for diagnosis of many cancers. Recent advances in computer vision, specifically deep learning, have facilitated the analysis of histopathology images for many tasks, including the detection of immune cells and microsatellite instability. However, it remains difficult to identify optimal models and training configurations for different histopathology classification tasks due to the abundance of available architectures and the lack of systematic evaluations. Our objective in this work is to present a software tool that addresses this need and enables robust, systematic evaluation of neural network models for patch classification in histology in a light-weight, easy-to-use package for both algorithm developers and biomedical researchers.

METHODS

Here we present ChampKit (Comprehensive Histopathology Assessment of Model Predictions toolKit): an extensible, fully reproducible evaluation toolkit that is a one-stop-shop to train and evaluate deep neural networks for patch classification. ChampKit curates a broad range of public datasets. It enables training and evaluation of models supported by timm directly from the command line, without the need for users to write any code. External models are enabled through a straightforward API and minimal coding. As a result, Champkit facilitates the evaluation of existing and new models and deep learning architectures on pathology datasets, making it more accessible to the broader scientific community. To demonstrate the utility of ChampKit, we establish baseline performance for a subset of possible models that could be employed with ChampKit, focusing on several popular deep learning models, namely ResNet18, ResNet50, and R26-ViT, a hybrid vision transformer. In addition, we compare each model trained either from random weight initialization or with transfer learning from ImageNet pretrained models. For ResNet18, we also consider transfer learning from a self-supervised pretrained model.

RESULTS

The main result of this paper is the ChampKit software. Using ChampKit, we were able to systemically evaluate multiple neural networks across six datasets. We observed mixed results when evaluating the benefits of pretraining versus random intialization, with no clear benefit except in the low data regime, where transfer learning was found to be beneficial. Surprisingly, we found that transfer learning from self-supervised weights rarely improved performance, which is counter to other areas of computer vision.

CONCLUSIONS

Choosing the right model for a given digital pathology dataset is nontrivial. ChampKit provides a valuable tool to fill this gap by enabling the evaluation of hundreds of existing (or user-defined) deep learning models across a variety of pathology tasks. Source code and data for the tool are freely accessible at https://github.com/SBU-BMI/champkit.

Machine learning in computational histopathology: Challenges and opportunities

Digital histopathological images, high-resolution images of stained tissue samples, are a vital tool for clinicians to diagnose and stage cancers. The visual analysis of patient state based on these images are an important part of oncology workflow. Although pathology workflows have historically been conducted in laboratories under a microscope, the increasing digitization of histopathological images has led to their analysis on computers in the clinic. The last decade has seen the emergence of machine learning, and deep learning in particular, a powerful set of tools for the analysis of histopathological images. Machine learning models trained on large datasets of digitized histopathology slides have resulted in automated models for prediction and stratification of patient risk. In this review, we provide context for the rise of such models in computational histopathology, highlight the clinical tasks they have found success in automating, discuss the various machine learning techniques that have been applied to this domain, and underscore open problems and opportunities.

Bibliometric analysis of the global scientific production on machine learning applied to different cancer types

Cancer disease is one of the main causes of death in the world, with million annual cases in the last decades. The need to find a cure has stimulated the search for efficient treatments and diagnostic procedures. One of the most promising tools that has emerged against cancer in recent years is machine learning (ML), which has raised a huge number of scientific papers published in a relatively short period of time. The present study analyzes global scientific production on ML applied to the most relevant cancer types through various bibliometric indicators. We find that over 30,000 studies have been published so far and observe that cancers with the highest number of published studies using ML (breast, lung, and colon cancer) are those with the highest incidence, being the USA and China the main scientific producers on the subject. Interestingly, the role of China and Japan in stomach cancer is correlated with the number of cases of this cancer type in Asia (78% of the worldwide cases). Knowing the countries and institutions that most study each area can be of great help for improving international collaborations between research groups and countries. Our analysis shows that medical and computer science journals lead the number of publications on the subject and could be useful for researchers in the field. Finally, keyword co-occurrence analysis suggests that ML-cancer research trends are focused not only on the use of ML as an effective diagnostic method, but also for the improvement of radiotherapy- and chemotherapy-based treatments.

Recent progress in transformer-based medical image analysis

The transformer is primarily used in the field of natural language processing. Recently, it has been adopted and shows promise in the computer vision (CV) field. Medical image analysis (MIA), as a critical branch of CV, also greatly benefits from this state-of-the-art technique. In this review, we first recap the core component of the transformer, the attention mechanism, and the detailed structures of the transformer. After that, we depict the recent progress of the transformer in the field of MIA. We organize the applications in a sequence of different tasks, including classification, segmentation, captioning, registration, detection, enhancement, localization, and synthesis. The mainstream classification and segmentation tasks are further divided into eleven medical image modalities. A large number of experiments studied in this review illustrate that the transformer-based method outperforms existing methods through comparisons with multiple evaluation metrics. Finally, we discuss the open challenges and future opportunities in this field. This task-modality review with the latest contents, detailed information, and comprehensive comparison may greatly benefit the broad MIA community.

Cueing effect of attention among nurses with different anxiety levels: an EEG study

The attention to cueing among nurses with anxiety affects their nursing quality seriously. Nevertheless, the neural mechanism of attention under anxiety among nurses has not been revealed. In this study, we utilized the event-related potential (ERP) and functional brain networks to investigate the neural mechanism of the cueing attention differences between anxiety and non-anxiety nurse groups (AG-20 nurses; NAG-20 nurses) in the spatial cueing task. The results revealed that in the invalid cues (144 trials), longer reaction times, larger P2 amplitudes, and more linkages between the right frontal and parietal areas were found in AG compared to NAG. In the valid cues (288 trials), there were no significant behavioral and neural differences between the two groups. The AG in the invalid cues showed slower response times, larger P2 and N5 amplitudes, and denser linkages originating from the occipital cortex than those in the valid cues. The convolutional neural network was trained for discriminating between the anxiety nurses and the normal ones, with the average accuracy being 0.76. The findings provided a potential physiological biomarker to predict the anxiety group who need to give more psychological attention. Nurse leaders maybe get more information for offering solutions to retain mental health among nurses.

Kernel Attention Transformer for Histopathology Whole Slide Image Analysis and Assistant Cancer Diagnosis

Transformer has been widely used in histopathology whole slide image analysis. However, the design of token-wise self-attention and positional embedding strategy in the common Transformer limits its effectiveness and efficiency when applied to gigapixel histopathology images. In this paper, we propose a novel kernel attention Transformer (KAT) for histopathology WSI analysis and assistant cancer diagnosis. The information transmission in KAT is achieved by cross-attention between the patch features and a set of kernels related to the spatial relationship of the patches on the whole slide images. Compared to the common Transformer structure, KAT can extract the hierarchical context information of the local regions of the WSI and provide diversified diagnosis information. Meanwhile, the kernel-based cross-attention paradigm significantly reduces the computational amount. The proposed method was evaluated on three large-scale datasets and was compared with 8 state-of-the-art methods. The experimental results have demonstrated the proposed KAT is effective and efficient in the task of histopathology WSI analysis and is superior to the state-of-the-art methods.

An open competition involving thousands of competitors failed to construct useful abstract classifiers for new diagnostic test accuracy systematic reviews

There are currently no abstract classifiers, which can be used for new diagnostic test accuracy (DTA) systematic reviews to select primary DTA study abstracts from database searches. Our goal was to develop machine-learning-based abstract classifiers for new DTA systematic reviews through an open competition. We prepared a dataset of abstracts obtained through database searches from 11 reviews in different clinical areas. As the reference standard, we used the abstract lists that required manual full-text review. We randomly splitted the datasets into a train set, a public test set, and a private test set. Competition participants used the training set to develop classifiers and validated their classifiers using the public test set. The classifiers were refined based on the performance of the public test set. They could submit as many times as they wanted during the competition. Finally, we used the private test set to rank the submitted classifiers. To reduce false exclusions, we used the Fbeta measure with a beta set to seven for evaluating classifiers. After the competition, we conducted the external validation using a dataset from a cardiology DTA review. We received 13,774 submissions from 1429 teams or persons over 4 months. The top-honored classifier achieved a Fbeta score of 0.4036 and a recall of 0.2352 in the external validation. In conclusion, we were unable to develop an abstract classifier with sufficient recall for immediate application to new DTA systematic reviews. Further studies are needed to update and validate classifiers with datasets from other clinical areas.

Deep Learning-based Automatic Diagnosis of Breast Cancer on MRI Using Mask R-CNN for Detection Followed by ResNet50 for Classification

RATIONALE AND OBJECTIVES

Diagnosis of breast cancer on MRI requires, first, the identification of suspicious lesions; second, the characterization to give a diagnostic impression. We implemented Mask Reginal-Convolutional Neural Network (R-CNN) to detect abnormal lesions, followed by ResNet50 to estimate the malignancy probability.

MATERIALS AND METHODS

Two datasets were used. The first set had 176 cases, 103 cancer, and 73 benign. The second set had 84 cases, 53 cancer, and 31 benign. For detection, the pre-contrast image and the subtraction images of left and right breasts were used as inputs, so the symmetry could be considered. The detected suspicious area was characterized by ResNet50, using three DCE parametric maps as inputs. The results obtained using slice-based analyses were combined to give a lesion-based diagnosis.

RESULTS

In the first dataset, 101 of 103 cancers were detected by Mask R-CNN as suspicious, and 99 of 101 were correctly classified by ResNet50 as cancer, with a sensitivity of 99/103 = 96%. 48 of 73 benign lesions and 131 normal areas were identified as suspicious. Following classification by ResNet50, only 16 benign and 16 normal areas remained as malignant. The second dataset was used for independent testing. The sensitivity was 43/53 = 81%. Of the total of 121 identified non-cancerous lesions, only 6 of 31 benign lesions and 22 normal tissues were classified as malignant.

CONCLUSION

ResNet50 could eliminate approximately 80% of false positives detected by Mask R-CNN. Combining Mask R-CNN and ResNet50 has the potential to develop a fully-automatic computer-aided diagnostic system for breast cancer on MRI.

Three-Dimensional Histological Electrophoresis for High-Throughput Cancer Margin Detection in Multiple Types of Tumor Specimens

Accurate identification of tumor margins during cancer surgeries relies on a rapid detection technique that can perform high-throughput detection of multiple suspected tumor lesions at the same time. Unfortunately, the conventional histopathological analysis of frozen tissue sections, which is considered the gold standard, often demonstrates considerable variability, especially in many regions without adequate access to trained pathologists. Therefore, there is a clinical need for a multitumor-suitable complementary tool that can accurately and high-throughput assess tumor margins in every direction within the surgically resected tissue. We herein describe a high-throughput three-dimensional (3D) histological electrophoresis device that uses tumor-specific proteins to identify and contour tumor margins intraoperatively. Testing on seven cell-line xenograft models and human cervical cancer models (representing five types of tissues) demonstrated the high-throughput detection utility of this approach. We anticipate that the 3D histological electrophoresis device will improve the accuracy and efficiency of diagnosing a wide range of cancers.

A large public dataset of annotated clinical MRIs and metadata of patients with acute stroke

To extract meaningful and reproducible models of brain function from stroke images, for both clinical and research proposes, is a daunting task severely hindered by the great variability of lesion frequency and patterns. Large datasets are therefore imperative, as well as fully automated image post-processing tools to analyze them. The development of such tools, particularly with artificial intelligence, is highly dependent on the availability of large datasets to model training and testing. We present a public dataset of 2,888 multimodal clinical MRIs of patients with acute and early subacute stroke, with manual lesion segmentation, and metadata. The dataset provides high quality, large scale, human-supervised knowledge to feed artificial intelligence models and enable further development of tools to automate several tasks that currently rely on human labor, such as lesion segmentation, labeling, calculation of disease-relevant scores, and lesion-based studies relating function to frequency lesion maps.

Perineural invasion detection in pancreatic ductal adenocarcinoma using artificial intelligence

Perineural invasion (PNI) refers to the presence of cancer cells around or within nerves, raising the risk of residual tumor. Linked to worse prognosis in pancreatic ductal adenocarcinoma (PDAC), PNI is also being explored as a therapeutic target. The purpose of this work was to build a PNI detection algorithm to enhance accuracy and efficiency in identifying PNI in PDAC specimens. Training used 260 manually segmented nerve and tumor HD images from 6 scanned PDAC cases; Analytical performance analysis used 168 additional images; clinical analysis used 59 PDAC cases. The algorithm pinpointed key areas of tumor-nerve proximity for pathologist confirmation. Analytical performance reached sensitivity of 88% and 54%, and specificity of 78% and 85% for the detection of nerve and tumor, respectively. Incorporating tumor-nerve distance in clinical evaluation raised PNI detection from 52 to 81% of all cases. Interestingly, pathologist analysis required an average of only 24 s per case. This time-efficient tool accurately identifies PNI in PDAC, even with a small training cohort, by imitating pathologist thought processes.

Developing a weakly supervised deep learning framework for breast cancer diagnosis with HR status based on mammography images

The status of hormone receptors (HR) at the molecular level is crucial for accurate diagnosis and effective treatment of breast cancer. Meanwhile, mammography is an effective screening method for detecting breast cancer, which significantly improve survival. However, diagnosing the molecular status of breast cancer involves a pathological biopsy, which can affect the accuracy of the diagnosis. To non-invasively diagnose the hormone receptor (HR) status of breast cancer and reduced manual annotation, we proposed a weakly supervised deep learning framework BSNet which detected breast cancer with HR status and benign tumors. BSNet was trained on 2321 multi-view mammography cases from female undergoing digital mammography for the general population at Harbin Medical University Cancer Hospital in Heilongjiang Province during the period 2017-2018 and was validated on the external cohort. The average AUCs of BSNet on the test set and the external validation set were 0.89 and 0.92, respectively. BSNet demonstrated excellent performance in non-invasive breast cancer diagnosis with HR status, using multiple mammography views without pixel annotation. Furthermore, we developed a web server (http://bsnet.edbc.org) for easy use. BSNet described high-dimensional mammography of breast cancer subtypes, which helped inform early management options.

Delineating intra-tumoral heterogeneity and tumor evolution in breast cancer using precision-based approaches

The burden of breast cancer continues to increase worldwide as it remains the most diagnosed tumor in females and the second leading cause of cancer-related deaths. Breast cancer is a heterogeneous disease characterized by different subtypes which are driven by aberrations in key genes such as BRCA1 and BRCA2, and hormone receptors. However, even within each subtype, heterogeneity that is driven by underlying evolutionary mechanisms is suggested to underlie poor response to therapy, variance in disease progression, recurrence, and relapse. Intratumoral heterogeneity highlights that the evolvability of tumor cells depends on interactions with cells of the tumor microenvironment. The complexity of the tumor microenvironment is being unraveled by recent advances in screening technologies such as high throughput sequencing; however, there remain challenges that impede the practical use of these approaches, considering the underlying biology of the tumor microenvironment and the impact of selective pressures on the evolvability of tumor cells. In this review, we will highlight the advances made thus far in defining the molecular heterogeneity in breast cancer and the implications thereof in diagnosis, the design and application of targeted therapies for improved clinical outcomes. We describe the different precision-based approaches to diagnosis and treatment and their prospects. We further propose that effective cancer diagnosis and treatment are dependent on unpacking the tumor microenvironment and its role in driving intratumoral heterogeneity. Underwriting such heterogeneity are Darwinian concepts of natural selection that we suggest need to be taken into account to ensure evolutionarily informed therapeutic decisions.

Impact of H&E Stain Normalization on Deep Learning Models in Cancer Image Classification: Performance, Complexity, and Trade-Offs

Accurate classification of cancer images plays a crucial role in diagnosis and treatment planning. Deep learning (DL) models have shown promise in achieving high accuracy, but their performance can be influenced by variations in Hematoxylin and Eosin (H&E) staining techniques. In this study, we investigate the impact of H&E stain normalization on the performance of DL models in cancer image classification. We evaluate the performance of VGG19, VGG16, ResNet50, MobileNet, Xception, and InceptionV3 on a dataset of H&E-stained cancer images. Our findings reveal that while VGG16 exhibits strong performance, VGG19 and ResNet50 demonstrate limitations in this context. Notably, stain normalization techniques significantly improve the performance of less complex models such as MobileNet and Xception. These models emerge as competitive alternatives with lower computational complexity and resource requirements and high computational efficiency. The results highlight the importance of optimizing less complex models through stain normalization to achieve accurate and reliable cancer image classification. This research holds tremendous potential for advancing the development of computationally efficient cancer classification systems, ultimately benefiting cancer diagnosis and treatment.

High-accuracy morphological identification of bone marrow cells using deep learning-based Morphogo system

Accurate identification and classification of bone marrow (BM) nucleated cell morphology are crucial for the diagnosis of hematological diseases. However, the subjective and time-consuming nature of manual identification by pathologists hinders prompt diagnosis and patient treatment. To address this issue, we developed Morphogo, a convolutional neural network-based system for morphological examination. Morphogo was trained using a vast dataset of over 2.8 million BM nucleated cell images. Its performance was evaluated using 508 BM cases that were categorized into five groups based on the degree of morphological abnormalities, comprising a total of 385,207 BM nucleated cells. The results demonstrated Morphogo's ability to identify over 25 different types of BM nucleated cells, achieving a sensitivity of 80.95%, specificity of 99.48%, positive predictive value of 76.49%, negative predictive value of 99.44%, and an overall accuracy of 99.01%. In most groups, Morphogo cell analysis and Pathologists' proofreading showed high intragroup correlation coefficients for granulocytes, erythrocytes, lymphocytes, monocytes, and plasma cells. These findings further validate the practical applicability of the Morphogo system in clinical practice and emphasize its value in assisting pathologists in diagnosing blood disorders.

Non-invasive tumor microenvironment evaluation and treatment response prediction in gastric cancer using deep learning radiomics

The tumor microenvironment (TME) plays a critical role in disease progression and is a key determinant of therapeutic response in cancer patients. Here, we propose a noninvasive approach to predict the TME status from radiological images by combining radiomics and deep learning analyses. Using multi-institution cohorts of 2,686 patients with gastric cancer, we show that the radiological model accurately predicted the TME status and is an independent prognostic factor beyond clinicopathologic variables. The model further predicts the benefit from adjuvant chemotherapy for patients with localized disease. In patients treated with checkpoint blockade immunotherapy, the model predicts clinical response and further improves predictive accuracy when combined with existing biomarkers. Our approach enables noninvasive assessment of the TME, which opens the door for longitudinal monitoring and tracking response to cancer therapy. Given the routine use of radiologic imaging in oncology, our approach can be extended to many other solid tumor types.

Detection of the pathological exposure of pulp using an artificial intelligence tool: a multicentric study over periapical radiographs

BACKGROUND

Introducing artificial intelligence (AI) into the medical field proved beneficial in automating tasks and streamlining the practitioners' lives. Hence, this study was conducted to design and evaluate an AI tool called Make Sure Caries Detector and Classifier (MSc) for detecting pathological exposure of pulp on digital periapical radiographs and to compare its performance with dentists.

METHODS

This study was a diagnostic, multi-centric study, with 3461 digital periapical radiographs from three countries and seven centers. MSc was built using Yolov5-x model, and it was used for exposed and unexposed pulp detection. The dataset was split into a train, validate, and test dataset; the ratio was 8-1-1 to prevent overfitting. 345 images with 752 labels were randomly allocated to test MSc. The performance metrics used to test MSc performance included mean average precision (mAP), precision, F1 score, recall, and area under receiver operating characteristic curve (AUC). The metrics used to compare the performance with that of 10 certified dentists were: right diagnosis exposed (RDE), right diagnosis not exposed (RDNE), false diagnosis exposed (FDE), false diagnosis not exposed (FDNE), missed diagnosis (MD), and over diagnosis (OD).

RESULTS

MSc achieved a performance of more than 90% in all metrics examined: an average precision of 0.928, recall of 0.918, F1-score of 0.922, and AUC of 0.956 (P<.05). The results showed a higher mean of 1.94 for all right (correct) diagnosis parameters in MSc group, while a higher mean of 0.64 for all wrong diagnosis parameters in the dentists group (P<.05).

CONCLUSIONS

The designed MSc tool proved itself reliable in the detection and differentiating between exposed and unexposed pulp in the internally validated model. It also showed a better performance for the detection of exposed and unexposed pulp when compared to the 10 dentists' consensus.

Efficient Convolution Network to Assist Breast Cancer Diagnosis and Target Therapy

Breast cancer is the leading cause of cancer-related deaths among women worldwide, and early detection and treatment has been shown to significantly reduce fatality rates from severe illness. Moreover, determination of the human epidermal growth factor receptor-2 (HER2) gene amplification by Fluorescence in situ hybridization (FISH) and Dual in situ hybridization (DISH) is critical for the selection of appropriate breast cancer patients for HER2-targeted therapy. However, visual examination of microscopy is time-consuming, subjective and poorly reproducible due to high inter-observer variability among pathologists and cytopathologists. The lack of consistency in identifying carcinoma-like nuclei has led to divergences in the calculation of sensitivity and specificity. This manuscript introduces a highly efficient deep learning method with low computing cost. The experimental results demonstrate that the proposed framework achieves high precision and recall on three essential clinical applications, including breast cancer diagnosis and human epidermal receptor factor 2 (HER2) amplification detection on FISH and DISH slides for HER2 target therapy. Furthermore, the proposed method outperforms the majority of the benchmark methods in terms of IoU by a significant margin (p<0.001) on three essential clinical applications. Importantly, run time analysis shows that the proposed method obtains excellent segmentation results with notably reduced time for Artificial intelligence (AI) training (16.93%), AI inference (17.25%) and memory usage (18.52%), making the proposed framework feasible for practical clinical usage.

Artificial Intelligence-Driven Morphology-Based Enrichment of Malignant Cells from Body Fluid

Cell morphology is a fundamental feature used to evaluate patient specimens in pathologic analysis. However, traditional cytopathology analysis of patient effusion samples is limited by low tumor cell abundance coupled with the high background of nonmalignant cells, restricting the ability of downstream molecular and functional analyses to identify actionable therapeutic targets. We applied the Deepcell platform that combines microfluidic sorting, brightfield imaging, and real-time deep learning interpretations based on multidimensional morphology to enrich carcinoma cells from malignant effusions without cell staining or labels. Carcinoma cell enrichment was validated with whole genome sequencing and targeted mutation analysis, which showed a higher sensitivity for detection of tumor fractions and critical somatic variant mutations that were initially at low levels or undetectable in presort patient samples. Our study demonstrates the feasibility and added value of supplementing traditional morphology-based cytology with deep learning, multidimensional morphology analysis, and microfluidic sorting.

Neural Network Based Classification of Breast Cancer Histopathological Image from Intraoperative Rapid Frozen Sections

Breast cancer is the leading cause of cancer-related mortality in women worldwide. Despite the rapid developments in diagnostic techniques and medical sciences, pathologic diagnosis is still recognized as the gold standard for disease diagnose. Pathologic diagnosis is a time-consuming task performed for pathologists, needing profound professional knowledge and long-term accumulated diagnostic experience. Therefore, the development of automatic and precise histopathological image classification is essential for medical diagnosis. In this study, an improved VGG network was used to classify the breast cancer histopathological image from intraoperative rapid frozen sections. We adopt a transformed loss function by adding a penalty to cross-entropy in our training stage, which improved the accuracy on test data by 4.39%. Laplacian-4 was used for the enhancement of images, which contributes to the improvement of the accuracy. The accuracy of the proposed model on training data and test data reached 88.70% and 82.27%, respectively, which outperforms the original model by 9.39% of accuracy in test data. The process time was less than 0.25 s per image on average. Meanwhile, the heat maps of predictions were given to show the evidential regions in histopathological images, which could drive improvements in the accuracy, speed, and clinical value of pathological diagnoses. In addition to helping with the actual diagnosis, this technology may be a benefit to pathologists, surgeons, and patients. It might prove to be a helpful tool for pathologists in the future.

Clinical utility of artificial intelligence assistance in histopathologic review of lymph node metastasis for gastric adenocarcinoma

BACKGROUND

Advances in whole-slide image capture and computer image analyses using deep learning technologies have enabled the development of computer-assisted diagnostics in pathology. Herein, we built a deep learning algorithm to detect lymph node (LN) metastasis on whole-slide images of LNs retrieved from patients with gastric adenocarcinoma and evaluated its performance in clinical settings.

METHODS

We randomly selected 18 patients with gastric adenocarcinoma who underwent surgery with curative intent and were positive for LN metastasis at Chiba University Hospital. A ResNet-152-based assistance system was established to detect LN metastases and to outline regions that are highly probable for metastasis in LN images. Reference standards comprising 70 LN images from two different institutions were reviewed by six pathologists with or without algorithm assistance, and their diagnostic performances were compared between the two settings.

RESULTS

No statistically significant differences were observed between these two settings regarding sensitivity, review time, or confidence levels in classifying macrometastases, isolated tumor cells, and metastasis-negative. Meanwhile, the sensitivity for detecting micrometastases significantly improved with algorithm assistance, although the review time was significantly longer than that without assistance. Analysis of the algorithm's sensitivity in detecting metastasis in the reference standard indicated an area under the curve of 0.869, whereas that for the detection of micrometastases was 0.785.

CONCLUSIONS

A wide variety of histological types in gastric adenocarcinoma could account for these relatively low performances; however, this level of algorithm performance could suffice to help pathologists improve diagnostic accuracy.