AI-based pathology predicts origins for cancers of unknown primary

Computational pathology: A survey review and the way forward

Computational Pathology (CPath) is an interdisciplinary science that augments developments of computational approaches to analyze and model medical histopathology images. The main objective for CPath is to develop infrastructure and workflows of digital diagnostics as an assistive CAD system for clinical pathology, facilitating transformational changes in the diagnosis and treatment of cancer that are mainly address by CPath tools. With evergrowing developments in deep learning and computer vision algorithms, and the ease of the data flow from digital pathology, currently CPath is witnessing a paradigm shift. Despite the sheer volume of engineering and scientific works being introduced for cancer image analysis, there is still a considerable gap of adopting and integrating these algorithms in clinical practice. This raises a significant question regarding the direction and trends that are undertaken in CPath. In this article we provide a comprehensive review of more than 800 papers to address the challenges faced in problem design all-the-way to the application and implementation viewpoints. We have catalogued each paper into a model-card by examining the key works and challenges faced to layout the current landscape in CPath. We hope this helps the community to locate relevant works and facilitate understanding of the field's future directions. In a nutshell, we oversee the CPath developments in cycle of stages which are required to be cohesively linked together to address the challenges associated with such multidisciplinary science. We overview this cycle from different perspectives of data-centric, model-centric, and application-centric problems. We finally sketch remaining challenges and provide directions for future technical developments and clinical integration of CPath. For updated information on this survey review paper and accessing to the original model cards repository, please refer to GitHub. Updated version of this draft can also be found from arXiv.

A deep learning method for predicting the origins of cervical lymph node metastatic cancer on digital pathological images

The metastatic cancer of cervical lymph nodes presents complex shapes and poses significant challenges for doctors in determining its origin. We established a deep learning framework to predict the status of lymph nodes in patients with cervical lymphadenopathy (CLA) by hematoxylin and eosin (H&E) stained slides. This retrospective study utilized 1,036 cervical lymph node biopsy specimens at the First Affiliated Hospital of Sun Yat-Sen University (FAHSYSU). A multiple-instance learning algorithm designed for key region identification was applied, and cross-validation experiments were conducted in the dataset. Additionally, the model distinguished between primary lymphoma and metastatic cancer with high prediction accuracy. We also validated our model and other models on an external dataset. Our model showed better generalization and achieved the best results on both internal and external datasets. This algorithm offers an approach for evaluating cervical lymph node status before surgery, significantly aiding physicians in preoperative diagnosis and treatment planning.

Re-identification from histopathology images

In numerous studies, deep learning algorithms have proven their potential for the analysis of histopathology images, for example, for revealing the subtypes of tumors or the primary origin of metastases. These models require large datasets for training, which must be anonymized to prevent possible patient identity leaks. This study demonstrates that even relatively simple deep learning algorithms can re-identify patients in large histopathology datasets with substantial accuracy. In addition, we compared a comprehensive set of state-of-the-art whole slide image classifiers and feature extractors for the given task. We evaluated our algorithms on two TCIA datasets including lung squamous cell carcinoma (LSCC) and lung adenocarcinoma (LUAD). We also demonstrate the algorithm's performance on an in-house dataset of meningioma tissue. We predicted the source patient of a slide with F1 scores of up to 80.1% and 77.19% on the LSCC and LUAD datasets, respectively, and with 77.09% on our meningioma dataset. Based on our findings, we formulated a risk assessment scheme to estimate the risk to the patient's privacy prior to publication.

Towards next-generation diagnostic pathology: AI-empowered label-free multiphoton microscopy

Diagnostic pathology, historically dependent on visual scrutiny by experts, is essential for disease detection. Advances in digital pathology and developments in computer vision technology have led to the application of artificial intelligence (AI) in this field. Despite these advancements, the variability in pathologists' subjective interpretations of diagnostic criteria can lead to inconsistent outcomes. To meet the need for precision in cancer therapies, there is an increasing demand for accurate pathological diagnoses. Consequently, traditional diagnostic pathology is evolving towards "next-generation diagnostic pathology", prioritizing on the development of a multi-dimensional, intelligent diagnostic approach. Using nonlinear optical effects arising from the interaction of light with biological tissues, multiphoton microscopy (MPM) enables high-resolution label-free imaging of multiple intrinsic components across various human pathological tissues. AI-empowered MPM further improves the accuracy and efficiency of diagnosis, holding promise for providing auxiliary pathology diagnostic methods based on multiphoton diagnostic criteria. In this review, we systematically outline the applications of MPM in pathological diagnosis across various human diseases, and summarize common multiphoton diagnostic features. Moreover, we examine the significant role of AI in enhancing multiphoton pathological diagnosis, including aspects such as image preprocessing, refined differential diagnosis, and the prognostication of outcomes. We also discuss the challenges and perspectives faced by the integration of MPM and AI, encompassing equipment, datasets, analytical models, and integration into the existing clinical pathways. Finally, the review explores the synergy between AI and label-free MPM to forge novel diagnostic frameworks, aiming to accelerate the adoption and implementation of intelligent multiphoton pathology systems in clinical settings.

A pathology foundation model for cancer diagnosis and prognosis prediction

Histopathology image evaluation is indispensable for cancer diagnoses and subtype classification. Standard artificial intelligence methods for histopathology image analyses have focused on optimizing specialized models for each diagnostic task1,2. Although such methods have achieved some success, they often have limited generalizability to images generated by different digitization protocols or samples collected from different populations3. Here, to address this challenge, we devised the Clinical Histopathology Imaging Evaluation Foundation (CHIEF) model, a general-purpose weakly supervised machine learning framework to extract pathology imaging features for systematic cancer evaluation. CHIEF leverages two complementary pretraining methods to extract diverse pathology representations: unsupervised pretraining for tile-level feature identification and weakly supervised pretraining for whole-slide pattern recognition. We developed CHIEF using 60,530 whole-slide images spanning 19 anatomical sites. Through pretraining on 44 terabytes of high-resolution pathology imaging datasets, CHIEF extracted microscopic representations useful for cancer cell detection, tumour origin identification, molecular profile characterization and prognostic prediction. We successfully validated CHIEF using 19,491 whole-slide images from 32 independent slide sets collected from 24 hospitals and cohorts internationally. Overall, CHIEF outperformed the state-of-the-art deep learning methods by up to 36.1%, showing its ability to address domain shifts observed in samples from diverse populations and processed by different slide preparation methods. CHIEF provides a generalizable foundation for efficient digital pathology evaluation for patients with cancer.

Deep neural networks integrating genomics and histopathological images for predicting stages and survival time-to-event in colon cancer

MOTIVATION

There exists an unexplained diverse variation within the predefined colon cancer stages using only features from either genomics or histopathological whole slide images as prognostic factors. Unraveling this variation will bring about improved staging and treatment outcomes. Hence, motivated by the advancement of Deep Neural Network (DNN) libraries and complementary factors within some genomics datasets, we aggregate atypia patterns in histopathological images with diverse carcinogenic expression from mRNA, miRNA and DNA methylation as an integrative input source into a deep neural network for colon cancer stages classification, and samples stratification into low or high-risk survival groups.

RESULTS

The genomics-only and integrated input features return Area Under Curve-Receiver Operating Characteristic curve (AUC-ROC) of 0.97 compared with AUC-ROC of 0.78 obtained when only image features are used for the stage's classification. A further analysis of prediction accuracy using the confusion matrix shows that the integrated features have a weakly improved accuracy of 0.08% more than the accuracy obtained with genomics features. Also, the extracted features were used to split the patients into low or high-risk survival groups. Among the 2,700 fused features, 1,836 (68%) features showed statistically significant survival probability differences in aggregating samples into either low or high between the two risk survival groups. Availability and Implementation: https://github.com/Ogundipe-L/EDCNN.

Perovskite Probe-Based Machine Learning Imaging Model for Rapid Pathologic Diagnosis of Cancers

Accurately distinguishing tumor cells from normal cells is a key issue in tumor diagnosis, evaluation, and treatment. Fluorescence-based immunohistochemistry as the standard method faces the inherent challenges of the heterogeneity of tumor cells and the lack of big data analysis of probing images. Here, we have demonstrated a machine learning-driven imaging method for rapid pathological diagnosis of five types of cancers (breast, colon, liver, lung, and stomach) using a perovskite nanocrystal probe. After conducting the bioanalysis of survivin expression in five different cancers, high-efficiency perovskite nanocrystal probes modified with the survivin antibody can recognize the cancer tissue section at the single cell level. The tumor to normal (T/N) ratio is 10.3-fold higher than that of a conventional fluorescent probe, which can successfully differentiate between tumors and adjacent normal tissues within 10 min. The features of the fluorescence intensity and pathological texture morphology have been extracted and analyzed from 1000 fluorescence images by machine learning. The final integrated decision model makes the area under the receiver operating characteristic curve (area under the curve) value of machine learning classification of breast, colon, liver, lung, and stomach above 90% while predicting the tumor organ of 92% of positive patients. This method demonstrates a high T/N ratio probe in the precise diagnosis of multiple cancers, which will be good for improving the accuracy of surgical resection and reducing cancer mortality.

Image-based deep learning model using DNA methylation data predicts the origin of cancer of unknown primary

Cancer of unknown primary (CUP) is a rare type of metastatic cancer in which the origin of the tumor is unknown. Since the treatment strategy for patients with metastatic tumors depends on knowing the primary site, accurate identification of the origin site is important. Here, we developed an image-based deep-learning model that utilizes a vision transformer algorithm for predicting the origin of CUP. Using DNA methylation dataset of 8,233 primary tumors from The Cancer Genome Atlas (TCGA), we categorized 29 cancer types into 18 organ classes and extracted 2,312 differentially methylated CpG sites (DMCs) from non-squamous cancer group and 420 DMCs from squamous cell cancer group. Using these DMCs, we created organ-specific DNA methylation images and used them for model training and testing. Model performance was evaluated using 394 metastatic cancer samples from TCGA (TCGA-meta) and 995 samples (693 primary and 302 metastatic cancers) obtained from 20 independent external studies. We identified that the DNA methylation image reveals a distinct pattern based on the origin of cancer. Our model achieved an overall accuracy of 96.95 % in the TCGA-meta dataset. In the external validation datasets, our classifier achieved overall accuracies of 96.39 % and 94.37 % in primary and metastatic tumors, respectively. Especially, the overall accuracies for both primary and metastatic samples of non-squamous cell cancer were exceptionally high, with 96.79 % and 96.85 %, respectively.

The evolution of molecular management of carcinoma of unknown primary

PURPOSE OF REVIEW

There is significant need to improve diagnostic and therapeutic options for patients with cancer of unknown primary (CUP). In this review, we discuss the evolving landscape of molecular profiling in CUP.

RECENT FINDINGS

Molecular profiling is becoming accepted into the diagnostic work-up of CUP patients with tumour mutation profiling now described in international CUP guidelines. Although tissue-of-origin (ToO) molecular tests utilising gene-expression and DNA methylation have existed some time, their clinical benefit remains unclear. Novel technologies utilising whole genome sequencing and machine learning algorithms are showing promise in determining ToO, however further research is required prior to clinical application. A recent international clinical trial found patients treated with molecularly-guided therapy based on comprehensive-panel DNA sequencing had improved progression-free survival compared to chemotherapy alone, confirming utility of performing genomic profiling early in the patient journey. Small phase 2 trials have demonstrated that some CUP patients are responsive to immunotherapy, but the best way to select patients for treatment is not clear.

SUMMARY

Management of CUP requires a multifaceted approach incorporating clinical, histopathological, radiological and molecular sequencing results to assist with identifying the likely ToO and clinically actionable genomic alternations. Rapidly identifying a subset of CUP patients who are likely to benefit from site specific therapy, targeted therapy and/or immunotherapy will improve patient outcomes.

Potential application of artificial intelligence in cancer therapy

PURPOSE OF REVIEW

This review underscores the critical role and challenges associated with the widespread adoption of artificial intelligence in cancer care to enhance disease management, streamline clinical processes, optimize data retrieval of health information, and generate and synthesize evidence.

RECENT FINDINGS

Advancements in artificial intelligence models and the development of digital biomarkers and diagnostics are applicable across the cancer continuum from early detection to survivorship care. Additionally, generative artificial intelligence has promised to streamline clinical documentation and patient communications, generate structured data for clinical trial matching, automate cancer registries, and facilitate advanced clinical decision support. Widespread adoption of artificial intelligence has been slow because of concerns about data diversity and data shift, model reliability and algorithm bias, legal oversight, and high information technology and infrastructure costs.

SUMMARY

Artificial intelligence models have significant potential to transform cancer care. Efforts are underway to deploy artificial intelligence models in the cancer practice, evaluate their clinical impact, and enhance their fairness and explainability. Standardized guidelines for the ethical integration of artificial intelligence models in cancer care pathways and clinical operations are needed. Clear governance and oversight will be necessary to gain trust in artificial intelligence-assisted cancer care by clinicians, scientists, and patients.

Seeing the primary tumor because of all the trees: Cancer type prediction on low-dimensional data

The Cancer of Unknown Primary (CUP) syndrome is characterized by identifiable metastases while the primary tumor remains hidden. In recent years, various data-driven approaches have been suggested to predict the location of the primary tumor (LOP) in CUP patients promising improved diagnosis and outcome. These LOP prediction approaches use high-dimensional input data like images or genetic data. However, leveraging such data is challenging, resource-intensive and therefore a potential translational barrier. Instead of using high-dimensional data, we analyzed the LOP prediction performance of low-dimensional data from routine medical care. With our findings, we show that such low-dimensional routine clinical information suffices as input data for tree-based LOP prediction models. The best model reached a mean Accuracy of 94% and a mean Matthews correlation coefficient (MCC) score of 0.92 in 10-fold nested cross-validation (NCV) when distinguishing four types of cancer. When considering eight types of cancer, this model achieved a mean Accuracy of 85% and a mean MCC score of 0.81. This is comparable to the performance achieved by approaches using high-dimensional input data. Additionally, the distribution pattern of metastases appears to be important information in predicting the LOP.

Artificial intelligence applied to 'omics data in liver disease: towards a personalised approach for diagnosis, prognosis and treatment

Advancements in omics technologies and artificial intelligence (AI) methodologies are fuelling our progress towards personalised diagnosis, prognosis and treatment strategies in hepatology. This review provides a comprehensive overview of the current landscape of AI methods used for analysis of omics data in liver diseases. We present an overview of the prevalence of different omics levels across various liver diseases, as well as categorise the AI methodology used across the studies. Specifically, we highlight the predominance of transcriptomic and genomic profiling and the relatively sparse exploration of other levels such as the proteome and methylome, which represent untapped potential for novel insights. Publicly available database initiatives such as The Cancer Genome Atlas and The International Cancer Genome Consortium have paved the way for advancements in the diagnosis and treatment of hepatocellular carcinoma. However, the same availability of large omics datasets remains limited for other liver diseases. Furthermore, the application of sophisticated AI methods to handle the complexities of multiomics datasets requires substantial data to train and validate the models and faces challenges in achieving bias-free results with clinical utility. Strategies to address the paucity of data and capitalise on opportunities are discussed. Given the substantial global burden of chronic liver diseases, it is imperative that multicentre collaborations be established to generate large-scale omics data for early disease recognition and intervention. Exploring advanced AI methods is also necessary to maximise the potential of these datasets and improve early detection and personalised treatment strategies.

Towards Digital Quantification of Ploidy from Pan-Cancer Digital Pathology Slides using Deep Learning

Abnormal DNA ploidy, found in numerous cancers, is increasingly being recognized as a contributor in driving chromosomal instability, genome evolution, and the heterogeneity that fuels cancer cell progression. Furthermore, it has been linked with poor prognosis of cancer patients. While next-generation sequencing can be used to approximate tumor ploidy, it has a high error rate for near-euploid states, a high cost and is time consuming, motivating alternative rapid quantification methods. We introduce PloiViT, a transformer-based model for tumor ploidy quantification that outperforms traditional machine learning models, enabling rapid and cost-effective quantification directly from pathology slides. We trained PloiViT on a dataset of fifteen cancer types from The Cancer Genome Atlas and validated its performance in multiple independent cohorts. Additionally, we explored the impact of self-supervised feature extraction on performance. PloiViT, using self-supervised features, achieved the lowest prediction error in multiple independent cohorts, exhibiting better generalization capabilities. Our findings demonstrate that PloiViT predicts higher ploidy values in aggressive cancer groups and patients with specific mutations, validating PloiViT potential as complementary for ploidy assessment to next-generation sequencing data. To further promote its use, we release our models as a user-friendly inference application and a Python package for easy adoption and use.

Harnessing Deep Learning for Accurate Pathological Assessment of Brain Tumor Cell Types

Primary diffuse central nervous system large B-cell lymphoma (CNS-pDLBCL) and high-grade glioma (HGG) often present similarly, clinically and on imaging, making differentiation challenging. This similarity can complicate pathologists' diagnostic efforts, yet accurately distinguishing between these conditions is crucial for guiding treatment decisions. This study leverages a deep learning model to classify brain tumor pathology images, addressing the common issue of limited medical imaging data. Instead of training a convolutional neural network (CNN) from scratch, we employ a pre-trained network for extracting deep features, which are then used by a support vector machine (SVM) for classification. Our evaluation shows that the Resnet50 (TL + SVM) model achieves a 97.4% accuracy, based on tenfold cross-validation on the test set. These results highlight the synergy between deep learning and traditional diagnostics, potentially setting a new standard for accuracy and efficiency in the pathological diagnosis of brain tumors.

Decoding pathology: the role of computational pathology in research and diagnostics

Traditional histopathology, characterized by manual quantifications and assessments, faces challenges such as low-throughput and inter-observer variability that hinder the introduction of precision medicine in pathology diagnostics and research. The advent of digital pathology allowed the introduction of computational pathology, a discipline that leverages computational methods, especially based on deep learning (DL) techniques, to analyze histopathology specimens. A growing body of research shows impressive performances of DL-based models in pathology for a multitude of tasks, such as mutation prediction, large-scale pathomics analyses, or prognosis prediction. New approaches integrate multimodal data sources and increasingly rely on multi-purpose foundation models. This review provides an introductory overview of advancements in computational pathology and discusses their implications for the future of histopathology in research and diagnostics.

Deep learning-based diagnosis and survival prediction of patients with renal cell carcinoma from primary whole slide images

There is an urgent clinical demand to explore novel diagnostic and prognostic biomarkers for renal cell carcinoma (RCC). We proposed deep learning-based artificial intelligence strategies. The study included 1752 whole slide images from multiple centres. Based on the pixel-level of RCC segmentation, the diagnosis diagnostic model achieved an area under the receiver operating characteristic curve (AUC) of 0.977 (95% CI 0.969-0.984) in the external validation cohort. In addition, our diagnostic model exhibited excellent performance in the differential diagnosis of RCC from renal oncocytoma, which achieved an AUC of 0.951 (0.922-0.972). The graderisk for the recognition of high-grade tumour achieved AUCs of 0.840 (0.805-0.871) in the Cancer Genome Atlas (TCGA) cohort, 0.857 (0.813-0.894) in the Shanghai General Hospital (General) cohort, and 0.894 (0.842-0.933) in the Clinical Proteomic Tumor Analysis Consortium (CPTAC) cohort, for the recognition of high-grade tumour. The OSrisk for predicting 5-year survival status achieved an AUC of 0.784 (0.746-0.819) in the TCGA cohort, which was further verified in the independent general cohort and the CPTAC cohort, with AUCs of 0.774 (0.723-0.820) and 0.702 (0.632-0.765), respectively. Moreover, the competing-risk nomogram (CRN) showed its potential to be a prognostic indicator, with a hazard ratio (HR) of 5.664 (3.893-8.239, p<0.0001), outperforming other traditional clinical prognostic indicators. Kaplan-Meier survival analysis further illustrated that our CRN could significantly distinguish patients with high survival risk. Deep learning-based artificial intelligence could be a useful tool for clinicians to diagnose and predict the prognosis of RCC patients, thus improving the process of individualised treatment.

Artificial intelligence in liver cancer - new tools for research and patient management

Liver cancer has high incidence and mortality globally. Artificial intelligence (AI) has advanced rapidly, influencing cancer care. AI systems are already approved for clinical use in some tumour types (for example, colorectal cancer screening). Crucially, research demonstrates that AI can analyse histopathology, radiology and natural language in liver cancer, and can replace manual tasks and access hidden information in routinely available clinical data. However, for liver cancer, few of these applications have translated into large-scale clinical trials or clinically approved products. Here, we advocate for the incorporation of AI in all stages of liver cancer management. We present a taxonomy of AI approaches in liver cancer, highlighting areas with academic and commercial potential, and outline a policy for AI-based liver cancer management, including interdisciplinary training of researchers, clinicians and patients. The potential of AI in liver cancer is immense, but effort is required to ensure that AI can fulfil expectations.

Unsupervised mutual transformer learning for multi-gigapixel Whole Slide Image classification

The classification of gigapixel Whole Slide Images (WSIs) is an important task in the emerging area of computational pathology. There has been a surge of interest in deep learning models for WSI classification with clinical applications such as cancer detection or prediction of cellular mutations. Most supervised methods require expensive and labor-intensive manual annotations by expert pathologists. Weakly supervised Multiple Instance Learning (MIL) methods have recently demonstrated excellent performance; however, they still require large-scale slide-level labeled training datasets that require a careful inspection of each slide by an expert pathologist. In this work, we propose a fully unsupervised WSI classification algorithm based on mutual transformer learning. The instances (i.e., patches) from gigapixel WSIs are transformed into a latent space and then inverse-transformed to the original space. Using the transformation loss, pseudo labels are generated and cleaned using a transformer label cleaner. The proposed transformer-based pseudo-label generator and cleaner modules mutually train each other iteratively in an unsupervised manner. A discriminative learning mechanism is introduced to improve normal versus cancerous instance labeling. In addition to the unsupervised learning, we demonstrate the effectiveness of the proposed framework for weakly supervised learning and cancer subtype classification as downstream analysis. Extensive experiments on four publicly available datasets show better performance of the proposed algorithm compared to the existing state-of-the-art methods.

A multi-classifier system integrated by clinico-histology-genomic analysis for predicting recurrence of papillary renal cell carcinoma

Integrating genomics and histology for cancer prognosis demonstrates promise. Here, we develop a multi-classifier system integrating a lncRNA-based classifier, a deep learning whole-slide-image-based classifier, and a clinicopathological classifier to accurately predict post-surgery localized (stage I-III) papillary renal cell carcinoma (pRCC) recurrence. The multi-classifier system demonstrates significantly higher predictive accuracy for recurrence-free survival (RFS) compared to the three single classifiers alone in the training set and in both validation sets (C-index 0.831-0.858 vs. 0.642-0.777, p < 0.05). The RFS in our multi-classifier-defined high-risk stage I/II and grade 1/2 groups is significantly worse than in the low-risk stage III and grade 3/4 groups (p < 0.05). Our multi-classifier system is a practical and reliable predictor for recurrence of localized pRCC after surgery that can be used with the current staging system to more accurately predict disease course and inform strategies for individualized adjuvant therapy.

Multi-omics based artificial intelligence for cancer research

With significant advancements of next generation sequencing technologies, large amounts of multi-omics data, including genomics, epigenomics, transcriptomics, proteomics, and metabolomics, have been accumulated, offering an unprecedented opportunity to explore the heterogeneity and complexity of cancer across various molecular levels and scales. One of the promising aspects of multi-omics lies in its capacity to offer a holistic view of the biological networks and pathways underpinning cancer, facilitating a deeper understanding of its development, progression, and response to treatment. However, the exponential growth of data generated by multi-omics studies present significant analytical challenges. Processing, analyzing, integrating, and interpreting these multi-omics datasets to extract meaningful insights is an ambitious task that stands at the forefront of current cancer research. The application of artificial intelligence (AI) has emerged as a powerful solution to these challenges, demonstrating exceptional capabilities in deciphering complex patterns and extracting valuable information from large-scale, intricate omics datasets. This review delves into the synergy of AI and multi-omics, highlighting its revolutionary impact on oncology. We dissect how this confluence is reshaping the landscape of cancer research and clinical practice, particularly in the realms of early detection, diagnosis, prognosis, treatment and pathology. Additionally, we elaborate the latest AI methods for multi-omics integration to provide a comprehensive insight of the complex biological mechanisms and inherent heterogeneity of cancer. Finally, we discuss the current challenges of data harmonization, algorithm interpretability, and ethical considerations. Addressing these challenges necessitates a multidisciplinary collaboration, paving the promising way for more precise, personalized, and effective treatments for cancer patients.

Application of Photoactive Compounds in Cancer Theranostics: Review on Recent Trends from Photoactive Chemistry to Artificial Intelligence

According to the World Health Organization (WHO) and the International Agency for Research on Cancer (IARC), the number of cancer cases and deaths worldwide is predicted to nearly double by 2030, reaching 21.7 million cases and 13 million fatalities. The increase in cancer mortality is due to limitations in the diagnosis and treatment options that are currently available. The close relationship between diagnostics and medicine has made it possible for cancer patients to receive precise diagnoses and individualized care. This article discusses newly developed compounds with potential for photodynamic therapy and diagnostic applications, as well as those already in use. In addition, it discusses the use of artificial intelligence in the analysis of diagnostic images obtained using, among other things, theranostic agents.

Deceptive learning in histopathology

AIMS

Deep learning holds immense potential for histopathology, automating tasks that are simple for expert pathologists and revealing novel biology for tasks that were previously considered difficult or impossible to solve by eye alone. However, the extent to which the visual strategies learned by deep learning models in histopathological analysis are trustworthy or not has yet to be systematically analysed. Here, we systematically evaluate deep neural networks (DNNs) trained for histopathological analysis in order to understand if their learned strategies are trustworthy or deceptive.

METHODS AND RESULTS

We trained a variety of DNNs on a novel data set of 221 whole-slide images (WSIs) from lung adenocarcinoma patients, and evaluated their effectiveness at (1) molecular profiling of KRAS versus EGFR mutations, (2) determining the primary tissue of a tumour and (3) tumour detection. While DNNs achieved above-chance performance on molecular profiling, they did so by exploiting correlations between histological subtypes and mutations, and failed to generalise to a challenging test set obtained through laser capture microdissection (LCM). In contrast, DNNs learned robust and trustworthy strategies for determining the primary tissue of a tumour as well as detecting and localising tumours in tissue.

CONCLUSIONS

Our work demonstrates that DNNs hold immense promise for aiding pathologists in analysing tissue. However, they are also capable of achieving seemingly strong performance by learning deceptive strategies that leverage spurious correlations, and are ultimately unsuitable for research or clinical work. The framework we propose for model evaluation and interpretation is an important step towards developing reliable automated systems for histopathological analysis.

Prediction of DNA methylation-based tumor types from histopathology in central nervous system tumors with deep learning

Precision in the diagnosis of diverse central nervous system (CNS) tumor types is crucial for optimal treatment. DNA methylation profiles, which capture the methylation status of thousands of individual CpG sites, are state-of-the-art data-driven means to enhance diagnostic accuracy but are also time consuming and not widely available. Here, to address these limitations, we developed Deep lEarning from histoPathoLOgy and methYlation (DEPLOY), a deep learning model that classifies CNS tumors to ten major categories from histopathology. DEPLOY integrates three distinct components: the first classifies CNS tumors directly from slide images ('direct model'), the second initially generates predictions for DNA methylation beta values, which are subsequently used for tumor classification ('indirect model'), and the third classifies tumor types directly from routinely available patient demographics. First, we find that DEPLOY accurately predicts beta values from histopathology images. Second, using a ten-class model trained on an internal dataset of 1,796 patients, we predict the tumor categories in three independent external test datasets including 2,156 patients, achieving an overall accuracy of 95% and balanced accuracy of 91% on samples that are predicted with high confidence. These results showcase the potential future use of DEPLOY to assist pathologists in diagnosing CNS tumors within a clinically relevant short time frame.

[Artificial intelligence in kidney transplant pathology]

BACKGROUND

Artificial intelligence (AI) systems have showed promising results in digital pathology, including digital nephropathology and specifically also kidney transplant pathology.

AIM

Summarize the current state of research and limitations in the field of AI in kidney transplant pathology diagnostics and provide a future outlook.

MATERIALS AND METHODS

Literature search in PubMed and Web of Science using the search terms "deep learning", "transplant", and "kidney". Based on these results and studies cited in the identified literature, a selection was made of studies that have a histopathological focus and use AI to improve kidney transplant diagnostics.

RESULTS AND CONCLUSION

Many studies have already made important contributions, particularly to the automation of the quantification of some histopathological lesions in nephropathology. This likely can be extended to automatically quantify all relevant lesions for a kidney transplant, such as Banff lesions. Important limitations and challenges exist in the collection of representative data sets and the updates of Banff classification, making large-scale studies challenging. The already positive study results make future AI support in kidney transplant pathology appear likely.

Advancements in computer vision and pathology: Unraveling the potential of artificial intelligence for precision diagnosis and beyond

The integration of computer vision into pathology through slide digitalization represents a transformative leap in the field's evolution. Traditional pathology methods, while reliable, are often time-consuming and susceptible to intra- and interobserver variability. In contrast, computer vision, empowered by artificial intelligence (AI) and machine learning (ML), promises revolutionary changes, offering consistent, reproducible, and objective results with ever-increasing speed and scalability. The applications of advanced algorithms and deep learning architectures like CNNs and U-Nets augment pathologists' diagnostic capabilities, opening new frontiers in automated image analysis. As these technologies mature and integrate into digital pathology workflows, they are poised to provide deeper insights into disease processes, quantify and standardize biomarkers, enhance patient outcomes, and automate routine tasks, reducing pathologists' workload. However, this transformative force calls for cross-disciplinary collaboration between pathologists, computer scientists, and industry innovators to drive research and development. While acknowledging its potential, this chapter addresses the limitations of AI in pathology, encompassing technical, practical, and ethical considerations during development and implementation.

Deep learning-based pathological prediction of lymph node metastasis for patient with renal cell carcinoma from primary whole slide images

BACKGROUND

Metastasis renal cell carcinoma (RCC) patients have extremely high mortality rate. A predictive model for RCC micrometastasis based on pathomics could be beneficial for clinicians to make treatment decisions.

METHODS

A total of 895 formalin-fixed and paraffin-embedded whole slide images (WSIs) derived from three cohorts, including Shanghai General Hospital (SGH), Clinical Proteomic Tumor Analysis Consortium (CPTAC) and Cancer Genome Atlas (TCGA) cohorts, and another 588 frozen section WSIs from TCGA dataset were involved in the study. The deep learning-based strategy for predicting lymphatic metastasis was developed based on WSIs through clustering-constrained-attention multiple-instance learning method and verified among the three cohorts. The performance of the model was further verified in frozen-pathological sections. In addition, the model was also tested the prognosis prediction of patients with RCC in multi-source patient cohorts.

RESULTS

The AUC of the lymphatic metastasis prediction performance was 0.836, 0.865 and 0.812 in TCGA, SGH and CPTAC cohorts, respectively. The performance on frozen section WSIs was with the AUC of 0.801. Patients with high deep learning-based prediction of lymph node metastasis values showed worse prognosis.

CONCLUSIONS

In this study, we developed and verified a deep learning-based strategy for predicting lymphatic metastasis from primary RCC WSIs, which could be applied in frozen-pathological sections and act as a prognostic factor for RCC to distinguished patients with worse survival outcomes.

A multimodal generative AI copilot for human pathology

Computational pathology1,2 has witnessed considerable progress in the development of both task-specific predictive models and task-agnostic self-supervised vision encoders3,4. However, despite the explosive growth of generative artificial intelligence (AI), there have been few studies on building general-purpose multimodal AI assistants and copilots5 tailored to pathology. Here we present PathChat, a vision-language generalist AI assistant for human pathology. We built PathChat by adapting a foundational vision encoder for pathology, combining it with a pretrained large language model and fine-tuning the whole system on over 456,000 diverse visual-language instructions consisting of 999,202 question and answer turns. We compare PathChat with several multimodal vision-language AI assistants and GPT-4V, which powers the commercially available multimodal general-purpose AI assistant ChatGPT-4 (ref. 6). PathChat achieved state-of-the-art performance on multiple-choice diagnostic questions from cases with diverse tissue origins and disease models. Furthermore, using open-ended questions and human expert evaluation, we found that overall PathChat produced more accurate and pathologist-preferable responses to diverse queries related to pathology. As an interactive vision-language AI copilot that can flexibly handle both visual and natural language inputs, PathChat may potentially find impactful applications in pathology education, research and human-in-the-loop clinical decision-making.

Deep learning radiomics of multimodal ultrasound for classifying metastatic cervical lymphadenopathy into primary cancer sites: a feasibility study

PURPOSE

To investigate the feasibility of deep learning radiomics (DLR) based on multimodal ultrasound to differentiate the primary cancer sites of metastatic cervical lymphadenopathy (CLA).

MATERIALS AND METHODS

This study analyzed 280 biopsy-confirmed metastatic CLAs from 280 cancer patients, including 54 from head and neck squamous cell carcinoma (HNSCC), 58 from thyroid cancer (TC), 92 from lung cancer (LC), and 76 from gastrointestinal cancer (GIC). Before biopsy, patients underwent conventional ultrasound (CUS), ultrasound elastography (UE), and contrast-enhanced ultrasound (CEUS). Based on CUS, DLR models using CUS, CUS+UE, CUS+CEUS, and CUS+UE+CEUS data were developed and compared. The best model was integrated with key clinical indicators selected by univariate analysis to achieve the best classification performance.

RESULTS

All DLR models achieved similar performance with respect to classifying four primary tumor sites of metastatic CLA (AUC:0.708~0.755). After integrating key clinical indicators (age, sex, and neck level), the US+UE+CEUS+clinical model yielded the best performance with an overall AUC of 0.822 in the validation cohort, but there was no significance compared with the basal CUS+clinical model (P>0.05), both of which identified metastasis from HNSCC, TC, LC, and GIC with 0.869 and 0.911, 0.838 and 0.916, 0.750 and 0.610, and 0.829 and 0.769, respectively.

CONCLUSION

The ultrasound-based DLR model can be used to classify the primary cancer sites of metastatic CLA, and the CUS combined with clinical indicators is adequate to provide a high discriminatory performance. The addition of the combination of UE and CEUS data is expected to further improve performance.

A guide to artificial intelligence for cancer researchers

Artificial intelligence (AI) has been commoditized. It has evolved from a specialty resource to a readily accessible tool for cancer researchers. AI-based tools can boost research productivity in daily workflows, but can also extract hidden information from existing data, thereby enabling new scientific discoveries. Building a basic literacy in these tools is useful for every cancer researcher. Researchers with a traditional biological science focus can use AI-based tools through off-the-shelf software, whereas those who are more computationally inclined can develop their own AI-based software pipelines. In this article, we provide a practical guide for non-computational cancer researchers to understand how AI-based tools can benefit them. We convey general principles of AI for applications in image analysis, natural language processing and drug discovery. In addition, we give examples of how non-computational researchers can get started on the journey to productively use AI in their own work.

Automated curation of large-scale cancer histopathology image datasets using deep learning

BACKGROUND

Artificial intelligence (AI) has numerous applications in pathology, supporting diagnosis and prognostication in cancer. However, most AI models are trained on highly selected data, typically one tissue slide per patient. In reality, especially for large surgical resection specimens, dozens of slides can be available for each patient. Manually sorting and labelling whole-slide images (WSIs) is a very time-consuming process, hindering the direct application of AI on the collected tissue samples from large cohorts. In this study we addressed this issue by developing a deep-learning (DL)-based method for automatic curation of large pathology datasets with several slides per patient.

METHODS

We collected multiple large multicentric datasets of colorectal cancer histopathological slides from the United Kingdom (FOXTROT, N = 21,384 slides; CR07, N = 7985 slides) and Germany (DACHS, N = 3606 slides). These datasets contained multiple types of tissue slides, including bowel resection specimens, endoscopic biopsies, lymph node resections, immunohistochemistry-stained slides, and tissue microarrays. We developed, trained, and tested a deep convolutional neural network model to predict the type of slide from the slide overview (thumbnail) image. The primary statistical endpoint was the macro-averaged area under the receiver operating curve (AUROCs) for detection of the type of slide.

RESULTS

In the primary dataset (FOXTROT), with an AUROC of 0.995 [95% confidence interval [CI]: 0.994-0.996] the algorithm achieved a high classification performance and was able to accurately predict the type of slide from the thumbnail image alone. In the two external test cohorts (CR07, DACHS) AUROCs of 0.982 [95% CI: 0.979-0.985] and 0.875 [95% CI: 0.864-0.887] were observed, which indicates the generalizability of the trained model on unseen datasets. With a confidence threshold of 0.95, the model reached an accuracy of 94.6% (7331 classified cases) in CR07 and 85.1% (2752 classified cases) for the DACHS cohort.

CONCLUSION

Our findings show that using the low-resolution thumbnail image is sufficient to accurately classify the type of slide in digital pathology. This can support researchers to make the vast resource of existing pathology archives accessible to modern AI models with only minimal manual annotations.

Deep learning-based multi-modal data integration enhancing breast cancer disease-free survival prediction

Background

The prognosis of breast cancer is often unfavorable, emphasizing the need for early metastasis risk detection and accurate treatment predictions. This study aimed to develop a novel multi-modal deep learning model using preoperative data to predict disease-free survival (DFS).

Methods

We retrospectively collected pathology imaging, molecular and clinical data from The Cancer Genome Atlas and one independent institution in China. We developed a novel Deep Learning Clinical Medicine Based Pathological Gene Multi-modal (DeepClinMed-PGM) model for DFS prediction, integrating clinicopathological data with molecular insights. The patients included the training cohort (n = 741), internal validation cohort (n = 184), and external testing cohort (n = 95).

Result

Integrating multi-modal data into the DeepClinMed-PGM model significantly improved area under the receiver operating characteristic curve (AUC) values. In the training cohort, AUC values for 1-, 3-, and 5-year DFS predictions increased to 0.979, 0.957, and 0.871, while in the external testing cohort, the values reached 0.851, 0.878, and 0.938 for 1-, 2-, and 3-year DFS predictions, respectively. The DeepClinMed-PGM's robust discriminative capabilities were consistently evident across various cohorts, including the training cohort [hazard ratio (HR) 0.027, 95% confidence interval (CI) 0.0016-0.046, P < 0.0001], the internal validation cohort (HR 0.117, 95% CI 0.041-0.334, P < 0.0001), and the external cohort (HR 0.061, 95% CI 0.017-0.218, P < 0.0001). Additionally, the DeepClinMed-PGM model demonstrated C-index values of 0.925, 0.823, and 0.864 within the three cohorts, respectively.

Conclusion

This study introduces an approach to breast cancer prognosis, integrating imaging and molecular and clinical data for enhanced predictive accuracy, offering promise for personalized treatment strategies.

Deep learning to assess microsatellite instability directly from histopathological whole slide images in endometrial cancer

Molecular classification, particularly microsatellite instability-high (MSI-H), has gained attention for immunotherapy in endometrial cancer (EC). MSI-H is associated with DNA mismatch repair defects and is a crucial treatment predictor. The NCCN guidelines recommend pembrolizumab and nivolumab for advanced or recurrent MSI-H/mismatch repair deficient (dMMR) EC. However, evaluating MSI in all cases is impractical due to time and cost constraints. To overcome this challenge, we present an effective and efficient deep learning-based model designed to accurately and rapidly assess MSI status of EC using H&E-stained whole slide images. Our framework was evaluated on a comprehensive dataset of gigapixel histopathology images of 529 patients from the Cancer Genome Atlas (TCGA). The experimental results have shown that the proposed method achieved excellent performances in assessing MSI status, obtaining remarkably high results with 96%, 94%, 93% and 100% for endometrioid carcinoma G1G2, respectively, and 87%, 84%, 81% and 94% for endometrioid carcinoma G3, in terms of F-measure, accuracy, precision and sensitivity, respectively. Furthermore, the proposed deep learning framework outperforms four state-of-the-art benchmarked methods by a significant margin (p < 0.001) in terms of accuracy, precision, sensitivity and F-measure, respectively. Additionally, a run time analysis demonstrates that the proposed method achieves excellent quantitative results with high efficiency in AI inference time (1.03 seconds per slide), making the proposed framework viable for practical clinical usage. These results highlight the efficacy and efficiency of the proposed model to assess MSI status of EC directly from histopathological slides.

Prediction of Lymph Node Metastasis in T1 Colorectal Cancer Using Artificial Intelligence with Hematoxylin and Eosin-Stained Whole-Slide-Images of Endoscopic and Surgical Resection Specimens

According to the current guidelines, additional surgery is performed for endoscopically resected specimens of early colorectal cancer (CRC) with a high risk of lymph node metastasis (LNM). However, the rate of LNM is 2.1-25.0% in cases treated endoscopically followed by surgery, indicating a high rate of unnecessary surgeries. Therefore, this study aimed to develop an artificial intelligence (AI) model using H&E-stained whole slide images (WSIs) without handcrafted features employing surgically and endoscopically resected specimens to predict LNM in T1 CRC. To validate with an independent cohort, we developed a model with four versions comprising various combinations of training and test sets using H&E-stained WSIs from endoscopically (400 patients) and surgically resected specimens (881 patients): Version 1, Train and Test: surgical specimens; Version 2, Train and Test: endoscopic and surgically resected specimens; Version 3, Train: endoscopic and surgical specimens and Test: surgical specimens; Version 4, Train: endoscopic and surgical specimens and Test: endoscopic specimens. The area under the curve (AUC) of the receiver operating characteristic curve was used to determine the accuracy of the AI model for predicting LNM with a 5-fold cross-validation in the training set. Our AI model with H&E-stained WSIs and without annotations showed good performance power with the validation of an independent cohort in a single center. The AUC of our model was 0.758-0.830 in the training set and 0.781-0.824 in the test set, higher than that of previous AI studies with only WSI. Moreover, the AI model with Version 4, which showed the highest sensitivity (92.9%), reduced unnecessary additional surgery by 14.2% more than using the current guidelines (68.3% vs. 82.5%). This revealed the feasibility of using an AI model with only H&E-stained WSIs to predict LNM in T1 CRC.

Analysis of 3D pathology samples using weakly supervised AI

Human tissue, which is inherently three-dimensional (3D), is traditionally examined through standard-of-care histopathology as limited two-dimensional (2D) cross-sections that can insufficiently represent the tissue due to sampling bias. To holistically characterize histomorphology, 3D imaging modalities have been developed, but clinical translation is hampered by complex manual evaluation and lack of computational platforms to distill clinical insights from large, high-resolution datasets. We present TriPath, a deep-learning platform for processing tissue volumes and efficiently predicting clinical outcomes based on 3D morphological features. Recurrence risk-stratification models were trained on prostate cancer specimens imaged with open-top light-sheet microscopy or microcomputed tomography. By comprehensively capturing 3D morphologies, 3D volume-based prognostication achieves superior performance to traditional 2D slice-based approaches, including clinical/histopathological baselines from six certified genitourinary pathologists. Incorporating greater tissue volume improves prognostic performance and mitigates risk prediction variability from sampling bias, further emphasizing the value of capturing larger extents of heterogeneous morphology.

Artificial intelligence in digital pathology: a systematic review and meta-analysis of diagnostic test accuracy

Ensuring diagnostic performance of artificial intelligence (AI) before introduction into clinical practice is essential. Growing numbers of studies using AI for digital pathology have been reported over recent years. The aim of this work is to examine the diagnostic accuracy of AI in digital pathology images for any disease. This systematic review and meta-analysis included diagnostic accuracy studies using any type of AI applied to whole slide images (WSIs) for any disease. The reference standard was diagnosis by histopathological assessment and/or immunohistochemistry. Searches were conducted in PubMed, EMBASE and CENTRAL in June 2022. Risk of bias and concerns of applicability were assessed using the QUADAS-2 tool. Data extraction was conducted by two investigators and meta-analysis was performed using a bivariate random effects model, with additional subgroup analyses also performed. Of 2976 identified studies, 100 were included in the review and 48 in the meta-analysis. Studies were from a range of countries, including over 152,000 whole slide images (WSIs), representing many diseases. These studies reported a mean sensitivity of 96.3% (CI 94.1-97.7) and mean specificity of 93.3% (CI 90.5-95.4). There was heterogeneity in study design and 99% of studies identified for inclusion had at least one area at high or unclear risk of bias or applicability concerns. Details on selection of cases, division of model development and validation data and raw performance data were frequently ambiguous or missing. AI is reported as having high diagnostic accuracy in the reported areas but requires more rigorous evaluation of its performance.

Prediction of tumor origin in cancers of unknown primary origin with cytology-based deep learning

Cancer of unknown primary (CUP) site poses diagnostic challenges due to its elusive nature. Many cases of CUP manifest as pleural and peritoneal serous effusions. Leveraging cytological images from 57,220 cases at four tertiary hospitals, we developed a deep-learning method for tumor origin differentiation using cytological histology (TORCH) that can identify malignancy and predict tumor origin in both hydrothorax and ascites. We examined its performance on three internal (n = 12,799) and two external (n = 14,538) testing sets. In both internal and external testing sets, TORCH achieved area under the receiver operating curve values ranging from 0.953 to 0.991 for cancer diagnosis and 0.953 to 0.979 for tumor origin localization. TORCH accurately predicted primary tumor origins, with a top-1 accuracy of 82.6% and top-3 accuracy of 98.9%. Compared with results derived from pathologists, TORCH showed better prediction efficacy (1.677 versus 1.265, P < 0.001), enhancing junior pathologists' diagnostic scores significantly (1.326 versus 1.101, P < 0.001). Patients with CUP whose initial treatment protocol was concordant with TORCH-predicted origins had better overall survival than those who were administrated discordant treatment (27 versus 17 months, P = 0.006). Our study underscores the potential of TORCH as a valuable ancillary tool in clinical practice, although further validation in randomized trials is warranted.

Cross-scale multi-instance learning for pathological image diagnosis

Analyzing high resolution whole slide images (WSIs) with regard to information across multiple scales poses a significant challenge in digital pathology. Multi-instance learning (MIL) is a common solution for working with high resolution images by classifying bags of objects (i.e. sets of smaller image patches). However, such processing is typically performed at a single scale (e.g., 20× magnification) of WSIs, disregarding the vital inter-scale information that is key to diagnoses by human pathologists. In this study, we propose a novel cross-scale MIL algorithm to explicitly aggregate inter-scale relationships into a single MIL network for pathological image diagnosis. The contribution of this paper is three-fold: (1) A novel cross-scale MIL (CS-MIL) algorithm that integrates the multi-scale information and the inter-scale relationships is proposed; (2) A toy dataset with scale-specific morphological features is created and released to examine and visualize differential cross-scale attention; (3) Superior performance on both in-house and public datasets is demonstrated by our simple cross-scale MIL strategy. The official implementation is publicly available at https://github.com/hrlblab/CS-MIL.

Artificial Intelligence in Oncology: Current Landscape, Challenges, and Future Directions

Artificial intelligence (AI) in oncology is advancing beyond algorithm development to integration into clinical practice. This review describes the current state of the field, with a specific focus on clinical integration. AI applications are structured according to cancer type and clinical domain, focusing on the four most common cancers and tasks of detection, diagnosis, and treatment. These applications encompass various data modalities, including imaging, genomics, and medical records. We conclude with a summary of existing challenges, evolving solutions, and potential future directions for the field.

SIGNIFICANCE

AI is increasingly being applied to all aspects of oncology, where several applications are maturing beyond research and development to direct clinical integration. This review summarizes the current state of the field through the lens of clinical translation along the clinical care continuum. Emerging areas are also highlighted, along with common challenges, evolving solutions, and potential future directions for the field.

Artificial intelligence in digital histopathology for predicting patient prognosis and treatment efficacy in breast cancer

INTRODUCTION

Histological images contain phenotypic information predictive of patient outcomes. Due to the heavy workload of pathologists, the time-consuming nature of quantitatively assessing histological features, and human eye limitations to recognize spatial patterns, manually extracting prognostic information in routine pathological workflows remains challenging. Digital pathology has facilitated the mining and quantification of these features utilizing whole-slide image (WSI) scanners and artificial intelligence (AI) algorithms. AI algorithms to identify image-based biomarkers from the tumor microenvironment (TME) have the potential to revolutionize the field of oncology, reducing delays between diagnosis and prognosis determination, allowing for rapid stratification of patients and prescription of optimal treatment regimes, thereby improving patient outcomes.

AREAS COVERED

In this review, the authors discuss how AI algorithms and digital pathology can predict breast cancer patient prognosis and treatment outcomes using image-based biomarkers, along with the challenges of adopting this technology in clinical settings.

EXPERT OPINION

The integration of AI and digital pathology presents significant potential for analyzing the TME and its diagnostic, prognostic, and predictive value in breast cancer patients. Widespread clinical adoption of AI faces ethical, regulatory, and technical challenges, although prospective trials may offer reassurance and promote uptake, ultimately improving patient outcomes by reducing diagnosis-to-prognosis delivery delays.

Deep learning framework for comprehensive molecular and prognostic stratifications of triple-negative breast cancer

Triple-negative breast cancer (TNBC) is the most challenging breast cancer subtype. Molecular stratification and target therapy bring clinical benefit for TNBC patients, but it is difficult to implement comprehensive molecular testing in clinical practice. Here, using our multi-omics TNBC cohort (N = 425), a deep learning-based framework was devised and validated for comprehensive predictions of molecular features, subtypes and prognosis from pathological whole slide images. The framework first incorporated a neural network to decompose the tissue on WSIs, followed by a second one which was trained based on certain tissue types for predicting different targets. Multi-omics molecular features were analyzed including somatic mutations, copy number alterations, germline mutations, biological pathway activities, metabolomics features and immunotherapy biomarkers. It was shown that the molecular features with therapeutic implications can be predicted including the somatic PIK3CA mutation, germline BRCA2 mutation and PD-L1 protein expression (area under the curve [AUC]: 0.78, 0.79 and 0.74 respectively). The molecular subtypes of TNBC can be identified (AUC: 0.84, 0.85, 0.93 and 0.73 for the basal-like immune-suppressed, immunomodulatory, luminal androgen receptor, and mesenchymal-like subtypes respectively) and their distinctive morphological patterns were revealed, which provided novel insights into the heterogeneity of TNBC. A neural network integrating image features and clinical covariates stratified patients into groups with different survival outcomes (log-rank P < 0.001). Our prediction framework and neural network models were externally validated on the TNBC cases from TCGA (N = 143) and appeared robust to the changes in patient population. For potential clinical translation, we built a novel online platform, where we modularized and deployed our framework along with the validated models. It can realize real-time one-stop prediction for new cases. In summary, using only pathological WSIs, our proposed framework can enable comprehensive stratifications of TNBC patients and provide valuable information for therapeutic decision-making. It had the potential to be clinically implemented and promote the personalized management of TNBC.

Preoperative CECT-Based Multitask Model Predicts Peritoneal Recurrence and Disease-Free Survival in Advanced Ovarian Cancer: A Multicenter Study

RATIONALE AND OBJECTIVES

Peritoneal recurrence is the predominant pattern of recurrence in advanced ovarian cancer (AOC) and portends a dismal prognosis. Accurate prediction of peritoneal recurrence and disease-free survival (DFS) is crucial to identify patients who might benefit from intensive treatment. We aimed to develop a predictive model for peritoneal recurrence and prognosis in AOC.

METHODS

In this retrospective multi-institution study of 515 patients, an end-to-end multi-task convolutional neural network (MCNN) comprising a segmentation convolutional neural network (CNN) and a classification CNN was developed and tested using preoperative CT images, and MCNN-score was generated to indicate the peritoneal recurrence and DFS status in patients with AOC. We evaluated the accuracy of the model for automatic segmentation and predict prognosis.

RESULTS

The MCNN achieved promising segmentation performances with a mean Dice coefficient of 84.3% (range: 78.8%-87.0%). The MCNN was able to predict peritoneal recurrence in the training (AUC 0.87; 95% CI 0.82-0.90), internal test (0.88; 0.85-0.92), and external test set (0.82; 0.78-0.86). Similarly, MCNN demonstrated consistently high accuracy in predicting recurrence, with an AUC of 0.85; 95% CI 0.82-0.88, 0.83; 95% CI 0.80-0.86, and 0.85; 95% CI 0.83-0.88. For patients with a high MCNN-score of recurrence, it was associated with poorer DFS with P < 0.0001 and hazard ratios of 0.1964 (95% CI: 0.1439-0.2680), 0.3249 (95% CI: 0.1896-0.5565), and 0.3458 (95% CI: 0.2582-0.4632).

CONCLUSION

The MCNN approach demonstrated high performance in predicting peritoneal recurrence and DFS in patients with AOC.

Computational pathology: an evolving concept

The initial enthusiasm about computational pathology (CP) and artificial intelligence (AI) was that they will replace pathologists entirely on the way to fully automated diagnostics. It is becoming clear that currently this is not the immediate model to pursue. On top of the legal and regulatory complexities surrounding its implementation, the majority of tested machine learning (ML)-based predictive algorithms do not display the exquisite performance needed to render them unequivocal, standalone decision makers for matters with direct implications to human health. We are thus moving into a different model of "computer-assisted diagnostics", where AI is there to provide support, rather than replacing, the pathologist. Herein we focus on the practical aspects of CP, from a pathologist perspective. There is a wide range of potential applications where CP can enhance precision of pathology diagnosis, tailor prognostic and predictive information, as well as save time. There are, however, a number of potential limitations for CP that currently hinder their wider adoption in the clinical setting. We address the key necessary steps towards clinical implementation of computational pathology, discuss the significant obstacles that hinders its adoption in the clinical context and summarize some proposed solutions. We conclude that the advancement of CP in the clinic is a promising resource-intensive endeavour that requires broad and inclusive collaborations between academia, industry, and regulatory bodies.

Machine-learning-based models to predict cardiovascular risk using oculomics and clinic variables in KNHANES

BACKGROUND

Recent researches have found a strong correlation between the triglyceride-glucose (TyG) index or the atherogenic index of plasma (AIP) and cardiovascular disease (CVD) risk. However, there is a lack of research on non-invasive and rapid prediction of cardiovascular risk. We aimed to develop and validate a machine-learning model for predicting cardiovascular risk based on variables encompassing clinical questionnaires and oculomics.

METHODS

We collected data from the Korean National Health and Nutrition Examination Survey (KNHANES). The training dataset (80% from the year 2008 to 2011 KNHANES) was used for machine learning model development, with internal validation using the remaining 20%. An external validation dataset from the year 2012 assessed the model's predictive capacity for TyG-index or AIP in new cases. We included 32122 participants in the final dataset. Machine learning models used 25 algorithms were trained on oculomics measurements and clinical questionnaires to predict the range of TyG-index and AIP. The area under the receiver operating characteristic curve (AUC), accuracy, precision, recall, and F1 score were used to evaluate the performance of our machine learning models.

RESULTS

Based on large-scale cohort studies, we determined TyG-index cut-off points at 8.0, 8.75 (upper one-third values), 8.93 (upper one-fourth values), and AIP cut-offs at 0.318, 0.34. Values surpassing these thresholds indicated elevated cardiovascular risk. The best-performing algorithm revealed TyG-index cut-offs at 8.0, 8.75, and 8.93 with internal validation AUCs of 0.812, 0.873, and 0.911, respectively. External validation AUCs were 0.809, 0.863, and 0.901. For AIP at 0.34, internal and external validation achieved similar AUCs of 0.849 and 0.842. Slightly lower performance was seen for the 0.318 cut-off, with AUCs of 0.844 and 0.836. Significant gender-based variations were noted for TyG-index at 8 (male AUC=0.832, female AUC=0.790) and 8.75 (male AUC=0.874, female AUC=0.862) and AIP at 0.318 (male AUC=0.853, female AUC=0.825) and 0.34 (male AUC=0.858, female AUC=0.831). Gender similarity in AUC (male AUC=0.907 versus female AUC=0.906) was observed only when the TyG-index cut-off point equals 8.93.

CONCLUSION

We have established a simple and effective non-invasive machine learning model that has good clinical value for predicting cardiovascular risk in the general population.

A cfDNA methylation-based tissue-of-origin classifier for cancers of unknown primary

Cancers of Unknown Primary (CUP) remains a diagnostic and therapeutic challenge due to biological heterogeneity and poor responses to standard chemotherapy. Predicting tissue-of-origin (TOO) molecularly could help refine this diagnosis, with tissue acquisition barriers mitigated via liquid biopsies. However, TOO liquid biopsies are unexplored in CUP cohorts. Here we describe CUPiD, a machine learning classifier for accurate TOO predictions across 29 tumour classes using circulating cell-free DNA (cfDNA) methylation patterns. We tested CUPiD on 143 cfDNA samples from patients with 13 cancer types alongside 27 non-cancer controls, with overall sensitivity of 84.6% and TOO accuracy of 96.8%. In an additional cohort of 41 patients with CUP CUPiD predictions were made in 32/41 (78.0%) cases, with 88.5% of the predictions clinically consistent with a subsequent or suspected primary tumour diagnosis, when available (23/26 patients). Combining CUPiD with cfDNA mutation data demonstrated potential diagnosis re-classification and/or treatment change in this hard-to-treat cancer group.

The current state of digital cytology and artificial intelligence (AI): global survey results from the American Society of Cytopathology Digital Cytology Task Force

INTRODUCTION

The integration of whole slide imaging (WSI) and artificial intelligence (AI) with digital cytology has been growing gradually. Therefore, there is a need to evaluate the current state of digital cytology. This study aimed to determine the current landscape of digital cytology via a survey conducted as part of the American Society of Cytopathology (ASC) Digital Cytology White Paper Task Force.

MATERIALS AND METHODS

A survey with 43 questions pertaining to the current practices and experiences of WSI and AI in both surgical pathology and cytology was created. The survey was sent to members of the ASC, the International Academy of Cytology (IAC), and the Papanicolaou Society of Cytopathology (PSC). Responses were recorded and analyzed.

RESULTS

In total, 327 individuals participated in the survey, spanning a diverse array of practice settings, roles, and experiences around the globe. The majority of responses indicated there was routine scanning of surgical pathology slides (n = 134; 61%) with fewer respondents scanning cytology slides (n = 150; 46%). The primary challenge for surgical WSI is the need for faster scanning and cost minimization, whereas image quality is the top issue for cytology WSI. AI tools are not widely utilized, with only 16% of participants using AI for surgical pathology samples and 13% for cytology practice.

CONCLUSIONS

Utilization of digital pathology is limited in cytology laboratories as compared to surgical pathology. However, as more laboratories are willing to implement digital cytology in the near future, the establishment of practical clinical guidelines is needed.

Current status and prospects of artificial intelligence in breast cancer pathology: convolutional neural networks to prospective Vision Transformers

Breast cancer is the most prevalent cancer among women, and its diagnosis requires the accurate identification and classification of histological features for effective patient management. Artificial intelligence, particularly through deep learning, represents the next frontier in cancer diagnosis and management. Notably, the use of convolutional neural networks and emerging Vision Transformers (ViT) has been reported to automate pathologists' tasks, including tumor detection and classification, in addition to improving the efficiency of pathology services. Deep learning applications have also been extended to the prediction of protein expression, molecular subtype, mutation status, therapeutic efficacy, and outcome prediction directly from hematoxylin and eosin-stained slides, bypassing the need for immunohistochemistry or genetic testing. This review explores the current status and prospects of deep learning in breast cancer diagnosis with a focus on whole-slide image analysis. Artificial intelligence applications are increasingly applied to many tasks in breast pathology ranging from disease diagnosis to outcome prediction, thus serving as valuable tools for assisting pathologists and supporting breast cancer management.

Performance and application of the total-body PET/CT scanner: a literature review

BACKGROUND

The total-body positron emission tomography/computed tomography (PET/CT) system, with a long axial field of view, represents the state-of-the-art PET imaging technique. Recently, the total-body PET/CT system has been commercially available. The total-body PET/CT system enables high-resolution whole-body imaging, even under extreme conditions such as ultra-low dose, extremely fast imaging speed, delayed imaging more than 10 h after tracer injection, and total-body dynamic scan. The total-body PET/CT system provides a real-time picture of the tracers of all organs across the body, which not only helps to explain normal human physiological process, but also facilitates the comprehensive assessment of systemic diseases. In addition, the total-body PET/CT system may play critical roles in other medical fields, including cancer imaging, drug development and immunology.

MAIN BODY

Therefore, it is of significance to summarize the existing studies of the total-body PET/CT systems and point out its future direction. This review collected research literatures from the PubMed database since the advent of commercially available total-body PET/CT systems to the present, and was divided into the following sections: Firstly, a brief introduction to the total-body PET/CT system was presented, followed by a summary of the literature on the performance evaluation of the total-body PET/CT. Then, the research and clinical applications of the total-body PET/CT were discussed. Fourthly, deep learning studies based on total-body PET imaging was reviewed. At last, the shortcomings of existing research and future directions for the total-body PET/CT were discussed.

CONCLUSION

Due to its technical advantages, the total-body PET/CT system is bound to play a greater role in clinical practice in the future.

Applying the UTAUT2 framework to patients' attitudes toward healthcare task shifting with artificial intelligence

BACKGROUND

Increasing patient loads, healthcare inflation and ageing population have put pressure on the healthcare system. Artificial intelligence and machine learning innovations can aid in task shifting to help healthcare systems remain efficient and cost effective. To gain an understanding of patients' acceptance toward such task shifting with the aid of AI, this study adapted the Unified Theory of Acceptance and Use of Technology 2 (UTAUT2), looking at performance and effort expectancy, facilitating conditions, social influence, hedonic motivation and behavioural intention.

METHODS

This was a cross-sectional study which took place between September 2021 to June 2022 at the National Heart Centre, Singapore. One hundred patients, aged ≥ 21 years with at least one heart failure symptom (pedal oedema, New York Heart Association II-III effort limitation, orthopnoea, breathlessness), who presented to the cardiac imaging laboratory for physician-ordered clinical echocardiogram, underwent both echocardiogram by skilled sonographers and the experience of echocardiogram by a novice guided by AI technologies. They were then given a survey which looked at the above-mentioned constructs using the UTAUT2 framework.

RESULTS

Significant, direct, and positive effects of all constructs on the behavioral intention of accepting the AI-novice combination were found. Facilitating conditions, hedonic motivation and performance expectancy were the top 3 constructs. The analysis of the moderating variables, age, gender and education levels, found no impact on behavioral intention.

CONCLUSIONS

These results are important for stakeholders and changemakers such as policymakers, governments, physicians, and insurance companies, as they design adoption strategies to ensure successful patient engagement by focusing on factors affecting the facilitating conditions, hedonic motivation and performance expectancy for AI technologies used in healthcare task shifting.

Artificial intelligence in liver imaging: methods and applications

Liver disease is regarded as one of the major health threats to humans. Radiographic assessments hold promise in terms of addressing the current demands for precisely diagnosing and treating liver diseases, and artificial intelligence (AI), which excels at automatically making quantitative assessments of complex medical image characteristics, has made great strides regarding the qualitative interpretation of medical imaging by clinicians. Here, we review the current state of medical-imaging-based AI methodologies and their applications concerning the management of liver diseases. We summarize the representative AI methodologies in liver imaging with focusing on deep learning, and illustrate their promising clinical applications across the spectrum of precise liver disease detection, diagnosis and treatment. We also address the current challenges and future perspectives of AI in liver imaging, with an emphasis on feature interpretability, multimodal data integration and multicenter study. Taken together, it is revealed that AI methodologies, together with the large volume of available medical image data, might impact the future of liver disease care.