Deep learning model for the prediction of microsatellite instability in colorectal cancer: a diagnostic study

Kernel Attention Transformer for Histopathology Whole Slide Image Analysis and Assistant Cancer Diagnosis

Transformer has been widely used in histopathology whole slide image analysis. However, the design of token-wise self-attention and positional embedding strategy in the common Transformer limits its effectiveness and efficiency when applied to gigapixel histopathology images. In this paper, we propose a novel kernel attention Transformer (KAT) for histopathology WSI analysis and assistant cancer diagnosis. The information transmission in KAT is achieved by cross-attention between the patch features and a set of kernels related to the spatial relationship of the patches on the whole slide images. Compared to the common Transformer structure, KAT can extract the hierarchical context information of the local regions of the WSI and provide diversified diagnosis information. Meanwhile, the kernel-based cross-attention paradigm significantly reduces the computational amount. The proposed method was evaluated on three large-scale datasets and was compared with 8 state-of-the-art methods. The experimental results have demonstrated the proposed KAT is effective and efficient in the task of histopathology WSI analysis and is superior to the state-of-the-art methods.

Integration of Theory and Practice in Medical Morphology Curriculum in Postgraduate Training: A Flipped Classroom and Case-based Learning Exercise

OBJECTIVE

The integration of training in theory and practice across the medical education spectrum is being encouraged to increase student understanding and skills in the sciences. This study aimed to determine the deciding factors that drive students' perceived advantages in class to improve precision education and the teaching model.

METHODS

A mixed strategy of an existing flipped classroom (FC) and a case-based learning (CBL) model was conducted in a medical morphology curriculum for 575 postgraduate students. The subjective learning evaluation of the individuals (learning time, engagement, study interest and concentration, and professional integration) was collected and analyzed after FC-CBL model learning.

RESULTS

The results from the general evaluation showed promising results of the medical morphology in the FC-CBL model. Students felt more engaged by instructors in person and benefited in terms of time-saving, flexible arrangements, and professional improvement. Our study contributed to the FC-CBL model in Research Design in postgraduate training in 4 categories: 1) advancing a guideline of precision teaching according to individual characteristics; 2) revealing whether a learning background is needed for a Research Design course to guide setting up a preliminary course; 3) understanding the perceived advantages and their interfaces; and 4) barriers and/or improvement to implement the FC-CBL model in the Research Design class, such as a richer description of e-learning and hands-on practice.

CONCLUSION

Undertaking a FC-CBL combined model could be a useful addition to pedagogy for medical morphology learning in postgraduate training.

Artificial intelligence in colorectal surgery: an AI-powered systematic review

Artificial intelligence (AI) has the potential to revolutionize surgery in the coming years. Still, it is essential to clarify what the meaningful current applications are and what can be reasonably expected. This AI-powered review assessed the role of AI in colorectal surgery. A Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA)-compliant systematic search of PubMed, Embase, Scopus, Cochrane Library databases, and gray literature was conducted on all available articles on AI in colorectal surgery (from January 1 1997 to March 1 2021), aiming to define the perioperative applications of AI. Potentially eligible studies were identified using novel software powered by natural language processing (NLP) and machine learning (ML) technologies dedicated to systematic reviews. Out of 1238 articles identified, 115 were included in the final analysis. Available articles addressed the role of AI in several areas of interest. In the preoperative phase, AI can be used to define tailored treatment algorithms, support clinical decision-making, assess the risk of complications, and predict surgical outcomes and survival. Intraoperatively, AI-enhanced surgery and integration of AI in robotic platforms have been suggested. After surgery, AI can be implemented in the Enhanced Recovery after Surgery (ERAS) pathway. Additional areas of applications included the assessment of patient-reported outcomes, automated pathology assessment, and research. Available data on these aspects are limited, and AI in colorectal surgery is still in its infancy. However, the rapid evolution of technologies makes it likely that it will increasingly be incorporated into everyday practice.

Identification of crucial genes related to heart failure based on GEO database

BACKGROUND

The molecular biological mechanisms underlying heart failure (HF) remain poorly understood. Therefore, it is imperative to use innovative approaches, such as high-throughput sequencing and artificial intelligence, to investigate the pathogenesis, diagnosis, and potential treatment of HF.

METHODS

First, we initially screened Two data sets (GSE3586 and GSE5406) from the GEO database containing HF and control samples from the GEO database to establish the Train group, and selected another dataset (GSE57345) to construct the Test group for verification. Next, we identified the genes with significantly different expression levels in patients with or without HF and performed functional and pathway enrichment analyses. HF-specific genes were identified, and an artificial neural network was constructed by Random Forest. The ROC curve was used to evaluate the accuracy and reliability of the constructed model in the Train and Test groups. Finally, immune cell infiltration was analyzed to determine the role of the inflammatory response and the immunological microenvironment in the pathogenesis of HF.

RESULTS

In the Train group, 153 significant differentially expressed genes (DEGs) associated with HF were found to be abnormal, including 81 down-regulated genes and 72 up-regulated genes. GO and KEGG enrichment analyses revealed that the down-regulated genes were primarily enriched in organic anion transport, neutrophil activation, and the PI3K-Akt signaling pathway. The upregulated genes were mainly enriched in neutrophil activation and the calcium signaling. DEGs were identified using Random Forest, and finally, 16 HF-specific genes were obtained. In the ROC validation and evaluation, the area under the curve (AUC) of the Train and Test groups were 0.996 and 0.863, respectively.

CONCLUSIONS

Our research revealed the potential functions and pathways implicated in the progression of HF, and designed an RNA diagnostic model for HF tissues using machine learning and artificial neural networks. Sensitivity, specificity, and stability were confirmed by ROC curves in the two different cohorts.

Artificial Intelligence to Predict the BRAF V595E Mutation in Canine Urinary Bladder Urothelial Carcinomas

In dogs, the BRAF mutation (V595E) is common in bladder and prostate cancer and represents a specific diagnostic marker. Recent advantages in artificial intelligence (AI) offer new opportunities in the field of tumour marker detection. While AI histology studies have been conducted in humans to detect BRAF mutation in cancer, comparable studies in animals are lacking. In this study, we used commercially available AI histology software to predict BRAF mutation in whole slide images (WSI) of bladder urothelial carcinomas (UC) stained with haematoxylin and eosin (HE), based on a training (n = 81) and a validation set (n = 96). Among 96 WSI, 57 showed identical PCR and AI-based BRAF predictions, resulting in a sensitivity of 58% and a specificity of 63%. The sensitivity increased substantially to 89% when excluding small or poor-quality tissue sections. Test reliability depended on tumour differentiation (p < 0.01), presence of inflammation (p < 0.01), slide quality (p < 0.02) and sample size (p < 0.02). Based on a small subset of cases with available adjacent non-neoplastic urothelium, AI was able to distinguish malignant from benign epithelium. This is the first study to demonstrate the use of AI histology to predict BRAF mutation status in canine UC. Despite certain limitations, the results highlight the potential of AI in predicting molecular alterations in routine tissue sections.

Recent advances of pathomics in colorectal cancer diagnosis and prognosis

Colorectal cancer (CRC) is one of the most common malignancies, with the third highest incidence and the second highest mortality in the world. To improve the therapeutic outcome, the risk stratification and prognosis predictions would help guide clinical treatment decisions. Achieving these goals have been facilitated by the fast development of artificial intelligence (AI) -based algorithms using radiological and pathological data, in combination with genomic information. Among them, features extracted from pathological images, termed pathomics, are able to reflect sub-visual characteristics linking to better stratification and prediction of therapeutic responses. In this paper, we review recent advances in pathological image-based algorithms in CRC, focusing on diagnosis of benign and malignant lesions, micro-satellite instability, as well as prediction of neoadjuvant chemoradiotherapy and the prognosis of CRC patients.

Global development of artificial intelligence in cancer field: a bibliometric analysis range from 1983 to 2022

Background

Artificial intelligence (AI) is widely applied in cancer field nowadays. The aim of this study is to explore the hotspots and trends of AI in cancer research.

Methods

The retrieval term includes four topic words ("tumor," "cancer," "carcinoma," and "artificial intelligence"), which were searched in the database of Web of Science from January 1983 to December 2022. Then, we documented and processed all data, including the country, continent, Journal Impact Factor, and so on using the bibliometric software.

Results

A total of 6,920 papers were collected and analyzed. We presented the annual publications and citations, most productive countries/regions, most influential scholars, the collaborations of journals and institutions, and research focus and hotspots in AI-based cancer research.

Conclusion

This study systematically summarizes the current research overview of AI in cancer research so as to lay the foundation for future research.

RNA-to-image multi-cancer synthesis using cascaded diffusion models

Data scarcity presents a significant obstacle in the field of biomedicine, where acquiring diverse and sufficient datasets can be costly and challenging. Synthetic data generation offers a potential solution to this problem by expanding dataset sizes, thereby enabling the training of more robust and generalizable machine learning models. Although previous studies have explored synthetic data generation for cancer diagnosis, they have predominantly focused on single modality settings, such as whole-slide image tiles or RNA-Seq data. To bridge this gap, we propose a novel approach, RNA-Cascaded-Diffusion-Model or RNA-CDM, for performing RNA-to-image synthesis in a multi-cancer context, drawing inspiration from successful text-to-image synthesis models used in natural images. In our approach, we employ a variational auto-encoder to reduce the dimensionality of a patient's gene expression profile, effectively distinguishing between different types of cancer. Subsequently, we employ a cascaded diffusion model to synthesize realistic whole-slide image tiles using the latent representation derived from the patient's RNA-Seq data. Our results demonstrate that the generated tiles accurately preserve the distribution of cell types observed in real-world data, with state-of-the-art cell identification models successfully detecting important cell types in the synthetic samples. Furthermore, we illustrate that the synthetic tiles maintain the cell fraction observed in bulk RNA-Seq data and that modifications in gene expression affect the composition of cell types in the synthetic tiles. Next, we utilize the synthetic data generated by RNA-CDM to pretrain machine learning models and observe improved performance compared to training from scratch. Our study emphasizes the potential usefulness of synthetic data in developing machine learning models in sarce-data settings, while also highlighting the possibility of imputing missing data modalities by leveraging the available information. In conclusion, our proposed RNA-CDM approach for synthetic data generation in biomedicine, particularly in the context of cancer diagnosis, offers a novel and promising solution to address data scarcity. By generating synthetic data that aligns with real-world distributions and leveraging it to pretrain machine learning models, we contribute to the development of robust clinical decision support systems and potential advancements in precision medicine.

Deep learning-based pathology signature could reveal lymph node status and act as a novel prognostic marker across multiple cancer types

BACKGROUND

Identifying lymph node metastasis (LNM) relies mainly on indirect radiology. Current studies omitted the quantified associations with traits beyond cancer types, failing to provide generalisation performance across various tumour types.

METHODS

4400 whole slide images across 11 cancer types were collected for training, cross-verification, and external validation of the pan-cancer lymph node metastasis (PC-LNM) model. We proposed an attention-based weakly supervised neural network based on self-supervised cancer-invariant features for the prediction task.

RESULTS

PC-LNM achieved a test area under the curve (AUC) of 0.732 (95% confidence interval: 0.717-0.746, P < 0.0001) in fivefold cross-validation of multiple cancer types, which also demonstrated good generalisation in the external validation cohort with AUC of 0.699 (95% confidence interval: 0.658-0.737, P < 0.0001). The interpretability results derived from PC-LNM revealed that the regions with the highest attention scores identified by the model generally correspond to tumours with poorly differentiated morphologies. PC-LNM achieved superior performance over previously reported methods and could also act as an independent prognostic factor for patients across multiple tumour types.

DISCUSSION

We presented an automated pan-cancer model for predicting the LNM status from primary tumour histology, which could act as a novel prognostic marker across multiple cancer types.

An aggregation of aggregation methods in computational pathology

Image analysis and machine learning algorithms operating on multi-gigapixel whole-slide images (WSIs) often process a large number of tiles (sub-images) and require aggregating predictions from the tiles in order to predict WSI-level labels. In this paper, we present a review of existing literature on various types of aggregation methods with a view to help guide future research in the area of computational pathology (CPath). We propose a general CPath workflow with three pathways that consider multiple levels and types of data and the nature of computation to analyse WSIs for predictive modelling. We categorize aggregation methods according to the context and representation of the data, features of computational modules and CPath use cases. We compare and contrast different methods based on the principle of multiple instance learning, perhaps the most commonly used aggregation method, covering a wide range of CPath literature. To provide a fair comparison, we consider a specific WSI-level prediction task and compare various aggregation methods for that task. Finally, we conclude with a list of objectives and desirable attributes of aggregation methods in general, pros and cons of the various approaches, some recommendations and possible future directions.

A multi-class classification algorithm based on hematoxylin-eosin staining for neoadjuvant therapy in rectal cancer: a retrospective study

Neoadjuvant therapy (NAT) is a major treatment option for locally advanced rectal cancer. With recent advancement of machine/deep learning algorithms, predicting the treatment response of NAT has become possible using radiological and/or pathological images. However, programs reported thus far are limited to binary classifications, and they can only distinguish the pathological complete response (pCR). In the clinical setting, the pathological NAT responses are classified as four classes: (TRG0-3), with 0 as pCR, 1 as moderate response, 2 as minimal response and 3 as poor response. Therefore, the actual clinical need for risk stratification remains unmet. By using ResNet (Residual Neural Network), we developed a multi-class classifier based on Hematoxylin-Eosin (HE) images to divide the response to three groups (TRG0, TRG1/2, and TRG3). Overall, the model achieved the AUC 0.97 at 40× magnification and AUC 0.89 at 10× magnification. For TRG0, the model under 40× magnification achieved a precision of 0.67, a sensitivity of 0.67, and a specificity of 0.95. For TRG1/2, a precision of 0.92, a sensitivity of 0.86, and a specificity of 0.89 were achieved. For TRG3, the model obtained a precision of 0.71, a sensitivity of 0.83, and a specificity of 0.88. To find the relationship between the treatment response and pathological images, we constructed a visual heat map of tiles using Class Activation Mapping (CAM). Notably, we found that tumor nuclei and tumor-infiltrating lymphocytes appeared to be potential features of the algorithm. Taken together, this multi-class classifier represents the first of its kind to predict different NAT responses in rectal cancer.

Overcoming the challenges to implementation of artificial intelligence in pathology

Pathologists worldwide are facing remarkable challenges with increasing workloads and lack of time to provide consistently high-quality patient care. The application of artificial intelligence (AI) to digital whole-slide images has the potential of democratizing the access to expert pathology and affordable biomarkers by supporting pathologists in the provision of timely and accurate diagnosis as well as supporting oncologists by directly extracting prognostic and predictive biomarkers from tissue slides. The long-awaited adoption of AI in pathology, however, has not materialized, and the transformation of pathology is happening at a much slower pace than that observed in other fields (eg, radiology). Here, we provide a critical summary of the developments in digital and computational pathology in the last 10 years, outline key hurdles and ways to overcome them, and provide a perspective for AI-supported precision oncology in the future.

Modeling the survival of colorectal cancer patients based on colonoscopic features in a feature ensemble vision transformer

The prognosis of patients with colorectal cancer (CRC) mostly relies on the classic tumor node metastasis (TNM) staging classification. A more accurate and convenient prediction model would provide a better prognosis and assist in treatment. From May 2014 to December 2017, patients who underwent an operation for CRC were enrolled. The proposed feature ensemble vision transformer (FEViT) used ensemble classifiers to benefit the combinations of relevant colonoscopy features from the pretrained vision transformer and clinical features, including sex, age, family history of CRC, and tumor location, to establish the prognostic model. A total of 1729 colonoscopy images were enrolled in the current retrospective study. For the prediction of patient survival, FEViT achieved an accuracy of 94 % with an area under the receiver operating characteristic curve of 0.93, which was better than the TNM staging classification (90 %, 0.83) in the experiment. FEViT reduced the limited receptive field and gradient disappearance in the conventional convolutional neural network and was a relatively effective and efficient procedure. The promising accuracy of FEViT in modeling survival makes the prognosis of CRC patients more predictable and practical.

Deep learning-based methods for classification of microsatellite instability in endometrial cancer from HE-stained pathological images

BACKGROUND

Microsatellite instability (MSI) is one of the essential tumor biomarkers for cancer treatment and prognosis. The presence of more significant PD-L1 expression on the surface of tumor cells in endometrial cancer with MSI suggests that MSI may be a promising biomarker for anti-PD-1/PD-L1 immunotherapy. However, the conventional testing methods are labor-intensive and expensive for patients.

METHODS

Inspired by classifiers for MSI based on fast and low-cost deep-learning methods in previous investigations, a new architecture for MSI classification based on an attention module is proposed to extract features from pathological images. Especially, slide-level microsatellite status will be obtained by the bag of words method to aggregate probabilities predicted by the proposed model. The H&E-stained whole slide images (WSIs) from The Cancer Genome Atlas endometrial cohort are collected as the dataset. The performances of the proposed model were primarily evaluated by the area under the receiver-operating characteristic curve, accuracy, sensitivity, and F1-Score.

RESULTS

On the randomly divided test dataset, the proposed model achieved an accuracy of 0.80, a sensitivity of 0.857, a F1-Score of 0.826, and an AUROC of 0.799. We then visualize the results of the microsatellite status classification to capture more specific morphological features, helping pathologists better understand how deep learning performs the classification.

CONCLUSIONS

This study implements the prediction of microsatellite status in endometrial cancer cases using deep-learning methods directly from H&E-stained WSIs. The proposed architecture can help the model capture more valuable features for classification. In contrast to current laboratory testing methods, the proposed model creates a more convenient screening tool for rapid automated testing for patients. This method can potentially be a clinical method for detecting the microsatellite status of endometrial cancer.

Prediction of Microsatellite Instability in Colorectal Cancer Using a Machine Learning Model Based on PET/CT Radiomics

PURPOSE

We investigated the feasibility of preoperative ¹⁸F-fluorodeoxyglucose (FDG) positron emission tomography (PET)/computed tomography (CT) radiomics with machine learning to predict microsatellite instability (MSI) status in colorectal cancer (CRC) patients.

MATERIALS AND METHODS

Altogether, 233 patients with CRC who underwent preoperative FDG PET/CT were enrolled and divided into training (n=139) and test (n=94) sets. A PET-based radiomics signature (rad_score) was established to predict the MSI status in patients with CRC. The predictive ability of the rad_score was evaluated using the area under the receiver operating characteristic curve (AUROC) in the test set. A logistic regression model was used to determine whether the rad_score was an independent predictor of MSI status in CRC. The predictive performance of rad_score was compared with conventional PET parameters.

RESULTS

The incidence of MSI-high was 15 (10.8%) and 10 (10.6%) in the training and test sets, respectively. The rad_score was constructed based on the two radiomic features and showed similar AUROC values for predicting MSI status in the training and test sets (0.815 and 0.867, respectively; p=0.490). Logistic regression analysis revealed that the rad_score was an independent predictor of MSI status in the training set. The rad_score performed better than metabolic tumor volume when assessed using the AUROC (0.867 vs. 0.794, p=0.015).

CONCLUSION

Our predictive model incorporating PET radiomic features successfully identified the MSI status of CRC, and it also showed better performance than the conventional PET image parameters.

Artificial intelligence-assisted selection and efficacy prediction of antineoplastic strategies for precision cancer therapy

The rapid development of artificial intelligence (AI) technologies in the context of the vast amount of collectable data obtained from high-throughput sequencing has led to an unprecedented understanding of cancer and accelerated the advent of a new era of clinical oncology with a tone of precision treatment and personalized medicine. However, the gains achieved by a variety of AI models in clinical oncology practice are far from what one would expect, and in particular, there are still many uncertainties in the selection of clinical treatment options that pose significant challenges to the application of AI in clinical oncology. In this review, we summarize emerging approaches, relevant datasets and open-source software of AI and show how to integrate them to address problems from clinical oncology and cancer research. We focus on the principles and procedures for identifying different antitumor strategies with the assistance of AI, including targeted cancer therapy, conventional cancer therapy, and cancer immunotherapy. In addition, we also highlight the current challenges and directions of AI in clinical oncology translation. Overall, we hope this article will provide researchers and clinicians with a deeper understanding of the role and implications of AI in precision cancer therapy, and help AI move more quickly into accepted cancer guidelines.

Contribution of whole slide imaging-based deep learning in the assessment of intraoperative and postoperative sections in neuropathology

The pathological diagnosis of intracranial germinoma (IG), oligodendroglioma, and low-grade astrocytoma on intraoperative frozen section (IFS) and hematoxylin-eosin (HE)-staining section directly determines patients' treatment options, but it is a difficult task for pathologists. We aimed to investigate whether whole-slide imaging (WSI)-based deep learning can contribute new precision to the diagnosis of IG, oligodendroglioma, and low-grade astrocytoma. Two types of WSIs (500 IFSs and 832 HE-staining sections) were collected from 379 patients at multiple medical centers. Patients at Center 1 were split into the training, testing, and internal validation sets (3:1:1), while the other centers were the external validation sets. First, we subdivided WSIs into small tiles and selected tissue tiles using a tissue tile selection model. Then a tile-level classification model was established, and the majority voting method was used to determine the final diagnoses. Color jitter was applied to the tiles so that the deep learning (DL) models could adapt to the variations in the staining. Last, we investigated the effectiveness of model assistance. The internal validation accuracies of the IFS and HE models were 93.9% and 95.3%, respectively. The external validation accuracies of the IFS and HE models were 82.0% and 76.9%, respectively. Furthermore, the IFS and HE models can predict Ki-67 positive cell areas with R² of 0.81 and 0.86, respectively. With model assistance, the IFS and HE diagnosis accuracy of pathologists improved from 54.6%-69.7% and 53.5%-83.7% to 87.9%-93.9% and 86.0%-90.7%, respectively. Both the IFS model and the HE model can differentiate the three tumors, predict the expression of Ki-67, and improve the diagnostic accuracy of pathologists. The use of our model can assist clinicians in providing patients with optimal and timely treatment options.

Histopathological bladder cancer gene mutation prediction with hierarchical deep multiple-instance learning

Gene mutation detection is usually carried out by molecular biological methods, which is expensive and has a long-time cycle. In contrast, pathological images are ubiquitous. If clinically significant gene mutations can be predicted only through pathological images, it will greatly promote the widespread use of gene mutation detection in clinical practice. However, current gene mutation prediction methods based on pathological images are ineffective because of the inability to identify mutated regions in gigapixel Whole Slide Image (WSI). To address this challenge, hereby we propose a carefully designed framework for WSI-based gene mutation prediction, which consists of three parts. (i) The first part of cancerous area segmentation, based on supervised learning, quickly filters out a large number of non-mutated regions; (ii) the second part of cancerous patch clustering, based on the representations derived from contrastive learning, ensures the comprehensiveness of patch selection; and (iii) the third part of mutation classification, based on the proposed hierarchical deep multi-instance learning method (HDMIL), ensures that sufficient patches are considered and inaccurate selections are ignored. In addition, benefiting from a two-stage attention mechanism in HDMIL, the patches that are highly correlated with gene mutations can be identified. This interpretability can help a pathologist to analyze the correlation between gene mutation and histopathological morphology. Experimental results demonstrate that the proposed gene mutation prediction framework significantly outperforms the state-of-the-art methods. In the TCGA bladder cancer dataset, five clinically relevant gene mutations are well predicted.

Histopathology images predict multi-omics aberrations and prognoses in colorectal cancer patients

Histopathologic assessment is indispensable for diagnosing colorectal cancer (CRC). However, manual evaluation of the diseased tissues under the microscope cannot reliably inform patient prognosis or genomic variations crucial for treatment selections. To address these challenges, we develop the Multi-omics Multi-cohort Assessment (MOMA) platform, an explainable machine learning approach, to systematically identify and interpret the relationship between patients' histologic patterns, multi-omics, and clinical profiles in three large patient cohorts (n = 1888). MOMA successfully predicts the overall survival, disease-free survival (log-rank test P-value<0.05), and copy number alterations of CRC patients. In addition, our approaches identify interpretable pathology patterns predictive of gene expression profiles, microsatellite instability status, and clinically actionable genetic alterations. We show that MOMA models are generalizable to multiple patient populations with different demographic compositions and pathology images collected from distinctive digitization methods. Our machine learning approaches provide clinically actionable predictions that could inform treatments for colorectal cancer patients.

A review and comparative study of cancer detection using machine learning: SBERT and SimCSE application

BACKGROUND

Using visual, biological, and electronic health records data as the sole input source, pretrained convolutional neural networks and conventional machine learning methods have been heavily employed for the identification of various malignancies. Initially, a series of preprocessing steps and image segmentation steps are performed to extract region of interest features from noisy features. Then, the extracted features are applied to several machine learning and deep learning methods for the detection of cancer.

METHODS

In this work, a review of all the methods that have been applied to develop machine learning algorithms that detect cancer is provided. With more than 100 types of cancer, this study only examines research on the four most common and prevalent cancers worldwide: lung, breast, prostate, and colorectal cancer. Next, by using state-of-the-art sentence transformers namely: SBERT (2019) and the unsupervised SimCSE (2021), this study proposes a new methodology for detecting cancer. This method requires raw DNA sequences of matched tumor/normal pair as the only input. The learnt DNA representations retrieved from SBERT and SimCSE will then be sent to machine learning algorithms (XGBoost, Random Forest, LightGBM, and CNNs) for classification. As far as we are aware, SBERT and SimCSE transformers have not been applied to represent DNA sequences in cancer detection settings.

RESULTS

The XGBoost model, which had the highest overall accuracy of 73 ± 0.13 % using SBERT embeddings and 75 ± 0.12 % using SimCSE embeddings, was the best performing classifier. In light of these findings, it can be concluded that incorporating sentence representations from SimCSE's sentence transformer only marginally improved the performance of machine learning models.

CT-based deep learning model for the prediction of DNA mismatch repair deficient colorectal cancer: a diagnostic study

BACKGROUND

Stratification of DNA mismatch repair (MMR) status in patients with colorectal cancer (CRC) enables individual clinical treatment decision making. The present study aimed to develop and validate a deep learning (DL) model based on the pre-treatment CT images for predicting MMR status in CRC.

METHODS

1812 eligible participants (training cohort: n = 1124; internal validation cohort: n = 482; external validation cohort: n = 206) with CRC were enrolled from two institutions. All pretherapeutic CT images from three dimensions were trained by the ResNet101, then integrated by Gaussian process regression (GPR) to develop a full-automatic DL model for MMR status prediction. The predictive performance of the DL model was evaluated using the area under the receiver operating characteristic curve (AUC) and then tested in the internal and external validation cohorts. Additionally, the participants from institution 1 were sub-grouped by various clinical factors for subgroup analysis, then the predictive performance of the DL model for identifying MMR status between participants in different groups were compared.

RESULTS

The full-automatic DL model was established in the training cohort to stratify the MMR status, which presented promising discriminative ability with the AUCs of 0.986 (95% CI 0.971-1.000) in the internal validation cohort and 0.915 (95% CI 0.870-0.960) in the external validation cohort. In addition, the subgroup analysis based on the thickness of CT images, clinical T and N stages, gender, the longest diameter, and the location of tumors revealed that the DL model showed similar satisfying prediction performance.

CONCLUSIONS

The DL model may potentially serve as a noninvasive tool to facilitate the pre-treatment individualized prediction of MMR status in patients with CRC, which could promote the personalized clinical-making decision.

Generalizable biomarker prediction from cancer pathology slides with self-supervised deep learning: A retrospective multi-centric study

Deep learning (DL) can predict microsatellite instability (MSI) from routine histopathology slides of colorectal cancer (CRC). However, it is unclear whether DL can also predict other biomarkers with high performance and whether DL predictions generalize to external patient populations. Here, we acquire CRC tissue samples from two large multi-centric studies. We systematically compare six different state-of-the-art DL architectures to predict biomarkers from pathology slides, including MSI and mutations in BRAF, KRAS, NRAS, and PIK3CA. Using a large external validation cohort to provide a realistic evaluation setting, we show that models using self-supervised, attention-based multiple-instance learning consistently outperform previous approaches while offering explainable visualizations of the indicative regions and morphologies. While the prediction of MSI and BRAF mutations reaches a clinical-grade performance, mutation prediction of PIK3CA, KRAS, and NRAS was clinically insufficient.

Artificial intelligence in cancer immunotherapy: Applications in neoantigen recognition, antibody design and immunotherapy response prediction

Cancer immunotherapy is a method of controlling and eliminating tumors by reactivating the body's cancer-immunity cycle and restoring its antitumor immune response. The increased availability of data, combined with advancements in high-performance computing and innovative artificial intelligence (AI) technology, has resulted in a rise in the use of AI in oncology research. State-of-the-art AI models for functional classification and prediction in immunotherapy research are increasingly used to support laboratory-based experiments. This review offers a glimpse of the current AI applications in immunotherapy, including neoantigen recognition, antibody design, and prediction of immunotherapy response. Advancing in this direction will result in more robust predictive models for developing better targets, drugs, and treatments, and these advancements will eventually make their way into the clinical setting, pushing AI forward in the field of precision oncology.

Unassisted Clinicians Versus Deep Learning-Assisted Clinicians in Image-Based Cancer Diagnostics: Systematic Review With Meta-analysis

BACKGROUND

A number of publications have demonstrated that deep learning (DL) algorithms matched or outperformed clinicians in image-based cancer diagnostics, but these algorithms are frequently considered as opponents rather than partners. Despite the clinicians-in-the-loop DL approach having great potential, no study has systematically quantified the diagnostic accuracy of clinicians with and without the assistance of DL in image-based cancer identification.

OBJECTIVE

We systematically quantified the diagnostic accuracy of clinicians with and without the assistance of DL in image-based cancer identification.

METHODS

PubMed, Embase, IEEEXplore, and the Cochrane Library were searched for studies published between January 1, 2012, and December 7, 2021. Any type of study design was permitted that focused on comparing unassisted clinicians and DL-assisted clinicians in cancer identification using medical imaging. Studies using medical waveform-data graphics material and those investigating image segmentation rather than classification were excluded. Studies providing binary diagnostic accuracy data and contingency tables were included for further meta-analysis. Two subgroups were defined and analyzed, including cancer type and imaging modality.

RESULTS

In total, 9796 studies were identified, of which 48 were deemed eligible for systematic review. Twenty-five of these studies made comparisons between unassisted clinicians and DL-assisted clinicians and provided sufficient data for statistical synthesis. We found a pooled sensitivity of 83% (95% CI 80%-86%) for unassisted clinicians and 88% (95% CI 86%-90%) for DL-assisted clinicians. Pooled specificity was 86% (95% CI 83%-88%) for unassisted clinicians and 88% (95% CI 85%-90%) for DL-assisted clinicians. The pooled sensitivity and specificity values for DL-assisted clinicians were higher than for unassisted clinicians, at ratios of 1.07 (95% CI 1.05-1.09) and 1.03 (95% CI 1.02-1.05), respectively. Similar diagnostic performance by DL-assisted clinicians was also observed across the predefined subgroups.

CONCLUSIONS

The diagnostic performance of DL-assisted clinicians appears better than unassisted clinicians in image-based cancer identification. However, caution should be exercised, because the evidence provided in the reviewed studies does not cover all the minutiae involved in real-world clinical practice. Combining qualitative insights from clinical practice with data-science approaches may improve DL-assisted practice, although further research is required.

TRIAL REGISTRATION

PROSPERO CRD42021281372; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=281372.

Fast and label-free automated detection of microsatellite status in early colon cancer using artificial intelligence integrated infrared imaging

PURPOSE

Microsatellite instability (MSI) due to mismatch repair (MMR) defects accounts for 15-20% of colon cancers (CC). MSI testing is currently standard of care in CC with immunohistochemistry of the four MMR proteins representing the gold standard. Instead, label-free quantum cascade laser (QCL) based infrared (IR) imaging combined with artificial intelligence (AI) may classify MSI/microsatellite stability (MSS) in unstained tissue sections user-independently and tissue preserving.

METHODS

Paraffin-embedded unstained tissue sections of early CC from patients participating in the multicentre AIO ColoPredict Plus (CPP) 2.0 registry were analysed after dividing into three groups (training, test, and validation). IR images of tissue sections using QCL-IR microscopes were classified by AI (convolutional neural networks [CNN]) using a two-step approach. The first CNN (modified U-Net) detected areas of cancer while the second CNN (VGG-Net) classified MSI/MSS. End-points were area under receiver operating characteristic (AUROC) and area under precision recall curve (AUPRC).

RESULTS

The cancer detection in the first step was based on 629 patients (train n = 273, test n = 138, and validation n = 218). Resulting classification AUROC was 1.0 for the validation dataset. The second step classifying MSI/MSS was performed on 547 patients (train n = 331, test n = 69, and validation n = 147) reaching AUROC and AUPRC of 0.9 and 0.74, respectively, for the validation cohort.

CONCLUSION

Our novel label-free digital pathology approach accurately and rapidly classifies MSI vs. MSS. The tissue sections analysed were not processed leaving the sample unmodified for subsequent analyses. Our approach demonstrates an AI-based decision support tool potentially driving improved patient stratification and precision oncology in the future.

Colorectal cancer lymph node metastasis prediction with weakly supervised transformer-based multi-instance learning

Lymph node metastasis examined by the resected lymph nodes is considered one of the most important prognostic factors for colorectal cancer (CRC). However, it requires careful and comprehensive inspection by expert pathologists. To relieve the pathologists' burden and speed up the diagnostic process, in this paper, we develop a deep learning system with the binary positive/negative labels of the lymph nodes to solve the CRC lymph node classification task. The multi-instance learning (MIL) framework is adopted in our method to handle the whole slide images (WSIs) of gigapixels in size at once and get rid of the labor-intensive and time-consuming detailed annotations. First, a transformer-based MIL model, DT-DSMIL, is proposed in this paper based on the deformable transformer backbone and the dual-stream MIL (DSMIL) framework. The local-level image features are extracted and aggregated with the deformable transformer, and the global-level image features are obtained with the DSMIL aggregator. The final classification decision is made based on both the local and the global-level features. After the effectiveness of our proposed DT-DSMIL model is demonstrated by comparing its performance with its predecessors, a diagnostic system is developed to detect, crop, and finally identify the single lymph nodes within the slides based on the DT-DSMIL and the Faster R-CNN model. The developed diagnostic model is trained and tested on a clinically collected CRC lymph node metastasis dataset composed of 843 slides (864 metastasis lymph nodes and 1415 non-metastatic lymph nodes), achieving the accuracy of 95.3% and the area under the receiver operating characteristic curve (AUC) of 0.9762 (95% confidence interval [CI]: 0.9607-0.9891) for the single lymph node classification. As for the lymph nodes with micro-metastasis and macro-metastasis, our diagnostic system achieves the AUC of 0.9816 (95% CI: 0.9659-0.9935) and 0.9902 (95% CI: 0.9787-0.9983), respectively. Moreover, the system shows reliable diagnostic region localizing performance: the model can always identify the most likely metastases, no matter the model's predictions or manual labels, showing great potential in avoiding false negatives and discovering incorrectly labeled slides in actual clinical use.

Predicting colorectal cancer microsatellite instability with a self-attention-enabled convolutional neural network

This study develops a method combining a convolutional neural network model, INSIGHT, with a self-attention model, WiseMSI, to predict microsatellite instability (MSI) based on the tiles in colorectal cancer patients from a multicenter Chinese cohort. After INSIGHT differentiates tumor tiles from normal tissue tiles in a whole slide image, features of tumor tiles are extracted with a ResNet model pre-trained on ImageNet. Attention-based pooling is adopted to aggregate tile-level features into slide-level representation. INSIGHT has an area under the curve (AUC) of 0.985 for tumor patch classification. The Spearman correlation coefficient of tumor cell fraction given by expert pathologist and INSIGHT is 0.7909. WiseMSI achieves a specificity of 94.7% (95% confidence interval [CI] 93.7%-95.7%), a sensitivity of 84.7% (95% CI 82.6%-86.9%), and an AUC of 0.954 (95% CI 0.948-0.960). Comparative analysis shows that this method has better performance than the other five classic deep learning methods.

Predicting gene mutation status via artificial intelligence technologies based on multimodal integration (MMI) to advance precision oncology

Personalized treatment strategies for cancer frequently rely on the detection of genetic alterations which are determined by molecular biology assays. Historically, these processes typically required single-gene sequencing, next-generation sequencing, or visual inspection of histopathology slides by experienced pathologists in a clinical context. In the past decade, advances in artificial intelligence (AI) technologies have demonstrated remarkable potential in assisting physicians with accurate diagnosis of oncology image-recognition tasks. Meanwhile, AI techniques make it possible to integrate multimodal data such as radiology, histology, and genomics, providing critical guidance for the stratification of patients in the context of precision therapy. Given that the mutation detection is unaffordable and time-consuming for a considerable number of patients, predicting gene mutations based on routine clinical radiological scans or whole-slide images of tissue with AI-based methods has become a hot issue in actual clinical practice. In this review, we synthesized the general framework of multimodal integration (MMI) for molecular intelligent diagnostics beyond standard techniques. Then we summarized the emerging applications of AI in the prediction of mutational and molecular profiles of common cancers (lung, brain, breast, and other tumor types) pertaining to radiology and histology imaging. Furthermore, we concluded that there truly exist multiple challenges of AI techniques in the way of its real-world application in the medical field, including data curation, feature fusion, model interpretability, and practice regulations. Despite these challenges, we still prospect the clinical implementation of AI as a highly potential decision-support tool to aid oncologists in future cancer treatment management.

A bibliometric and visual analysis of publications on artificial intelligence in colorectal cancer (2002-2022)

Background

Colorectal cancer (CRC) has the third-highest incidence and second-highest mortality rate of all cancers worldwide. Early diagnosis and screening of CRC have been the focus of research in this field. With the continuous development of artificial intelligence (AI) technology, AI has advantages in many aspects of CRC, such as adenoma screening, genetic testing, and prediction of tumor metastasis.

Objective

This study uses bibliometrics to analyze research in AI in CRC, summarize the field's history and current status of research, and predict future research directions.

Method

We searched the SCIE database for all literature on CRC and AI. The documents span the period 2002-2022. we used bibliometrics to analyze the data of these papers, such as authors, countries, institutions, and references. Co-authorship, co-citation, and co-occurrence analysis were the main methods of analysis. Citespace, VOSviewer, and SCImago Graphica were used to visualize the results.

Result

This study selected 1,531 articles on AI in CRC. China has published a maximum number of 580 such articles in this field. The U.S. had the most quality publications, boasting an average citation per article of 46.13. Mori Y and Ding K were the two authors with the highest number of articles. Scientific Reports, Cancers, and Frontiers in Oncology are this field's most widely published journals. Institutions from China occupy the top 9 positions among the most published institutions. We found that research on AI in this field mainly focuses on colonoscopy-assisted diagnosis, imaging histology, and pathology examination.

Conclusion

AI in CRC is currently in the development stage with good prospects. AI is currently widely used in colonoscopy, imageomics, and pathology. However, the scope of AI applications is still limited, and there is a lack of inter-institutional collaboration. The pervasiveness of AI technology is the main direction of future housing development in this field.

The Applications of Artificial Intelligence in Digestive System Neoplasms: A Review

Importance

Digestive system neoplasms (DSNs) are the leading cause of cancer-related mortality with a 5-year survival rate of less than 20%. Subjective evaluation of medical images including endoscopic images, whole slide images, computed tomography images, and magnetic resonance images plays a vital role in the clinical practice of DSNs, but with limited performance and increased workload of radiologists or pathologists. The application of artificial intelligence (AI) in medical image analysis holds promise to augment the visual interpretation of medical images, which could not only automate the complicated evaluation process but also convert medical images into quantitative imaging features that associated with tumor heterogeneity.

Highlights

We briefly introduce the methodology of AI for medical image analysis and then review its clinical applications including clinical auxiliary diagnosis, assessment of treatment response, and prognosis prediction on 4 typical DSNs including esophageal cancer, gastric cancer, colorectal cancer, and hepatocellular carcinoma.

Conclusion

AI technology has great potential in supporting the clinical diagnosis and treatment decision-making of DSNs. Several technical issues should be overcome before its application into clinical practice of DSNs.

Tumour-infiltrating lymphocytes: from prognosis to treatment selection

Tumour-infiltrating lymphocytes (TILs) are considered crucial in anti-tumour immunity. Accordingly, the presence of TILs contains prognostic and predictive value. In 2011, we performed a systematic review and meta-analysis on the prognostic value of TILs across cancer types. Since then, the advent of immune checkpoint blockade (ICB) has renewed interest in the analysis of TILs. In this review, we first describe how our understanding of the prognostic value of TIL has changed over the last decade. New insights on novel TIL subsets are discussed and give a broader view on the prognostic effect of TILs in cancer. Apart from prognostic value, evidence on the predictive significance of TILs in the immune therapy era are discussed, as well as new techniques, such as machine learning that strive to incorporate these predictive capacities within clinical trials.

Interpretable deep learning model to predict the molecular classification of endometrial cancer from haematoxylin and eosin-stained whole-slide images: a combined analysis of the PORTEC randomised trials and clinical cohorts

BACKGROUND

Endometrial cancer can be molecularly classified into POLE^mut, mismatch repair deficient (MMRd), p53 abnormal (p53abn), and no specific molecular profile (NSMP) subgroups. We aimed to develop an interpretable deep learning pipeline for whole-slide-image-based prediction of the four molecular classes in endometrial cancer (im4MEC), to identify morpho-molecular correlates, and to refine prognostication.

METHODS

This combined analysis included diagnostic haematoxylin and eosin-stained slides and molecular and clinicopathological data from 2028 patients with intermediate-to-high-risk endometrial cancer from the PORTEC-1 (n=466), PORTEC-2 (n=375), and PORTEC-3 (n=393) randomised trials and the TransPORTEC pilot study (n=110), the Medisch Spectrum Twente cohort (n=242), a case series of patients with POLE^mut endometrial cancer in the Leiden Endometrial Cancer Repository (n=47), and The Cancer Genome Atlas-Uterine Corpus Endometrial Carcinoma cohort (n=395). PORTEC-3 was held out as an independent test set and a four-fold cross validation was performed. Performance was measured with the macro and class-wise area under the receiver operating characteristic curve (AUROC). Whole-slide images were segmented into tiles of 360 μm resized to 224 × 224 pixels. im4MEC was trained to learn tile-level morphological features with self-supervised learning and to molecularly classify whole-slide images with an attention mechanism. The top 20 tiles with the highest attention scores were reviewed to identify morpho-molecular correlates. Predictions of a nuclear classification deep learning model serve to derive interpretable morphological features. We analysed 5-year recurrence-free survival and explored prognostic refinement by molecular class using the Kaplan-Meier method.

FINDINGS

im4MEC attained macro-average AUROCs of 0·874 (95% CI 0·856-0·893) on four-fold cross-validation and 0·876 on the independent test set. The class-wise AUROCs were 0·849 for POLE^mut (n=51), 0·844 for MMRd (n=134), 0·883 for NSMP (n=120), and 0·928 for p53abn (n=88). POLE^mut and MMRd tiles had a high density of lymphocytes, p53abn tiles had strong nuclear atypia, and the morphology of POLE^mut and MMRd endometrial cancer overlapped. im4MEC highlighted a low tumour-to-stroma ratio as a potentially novel characteristic feature of the NSMP class. 5-year recurrence-free survival was significantly different between im4MEC predicted molecular classes in PORTEC-3 (log-rank p<0·0001). The ten patients with aggressive p53abn endometrial cancer that was predicted as MMRd showed inflammatory morphology and appeared to have a better prognosis than patients with correctly predicted p53abn endometrial cancer (p=0·30). The four patients with NSMP endometrial cancer that was predicted as p53abn showed higher nuclear atypia and appeared to have a worse prognosis than patients with correctly predicted NSMP (p=0·13). Patients with MMRd endometrial cancer predicted as POLE^mut had an excellent prognosis, as do those with true POLE^mut endometrial cancer.

INTERPRETATION

We present the first interpretable deep learning model, im4MEC, for haematoxylin and eosin-based prediction of molecular endometrial cancer classification. im4MEC robustly identified morpho-molecular correlates and could enable further prognostic refinement of patients with endometrial cancer.

FUNDING

The Hanarth Foundation, the Promedica Foundation, and the Swiss Federal Institutes of Technology.

Prognostic Significance of Tumor-Infiltrating Lymphocytes Determined Using LinkNet on Colorectal Cancer Pathology Images

PURPOSE

Tumor-infiltrating lymphocytes (TILs) have a significant prognostic value in cancers. However, very few automated, deep learning-based TIL scoring algorithms have been developed for colorectal cancer (CRC).

MATERIALS AND METHODS

We developed an automated, multiscale LinkNet workflow for quantifying TILs at the cellular level in CRC tumors using H&E-stained images from the Lizard data set with annotations of lymphocytes. The predictive performance of the automatic TIL scores (TILsLink) for disease progression and overall survival (OS) was evaluated using two international data sets, including 554 patients with CRC from The Cancer Genome Atlas (TCGA) and 1,130 patients with CRC from Molecular and Cellular Oncology (MCO).

RESULTS

The LinkNet model provided outstanding precision (0.9508), recall (0.9185), and overall F1 score (0.9347). Clear continuous TIL-hazard relationships were observed between TILsLink and the risk of disease progression or death in both TCGA and MCO cohorts. Both univariate and multivariate Cox regression analyses for the TCGA data demonstrated that patients with high TIL abundance had a significant (approximately 75%) reduction in risk for disease progression. In both the MCO and TCGA cohorts, the TIL-high group was significantly associated with improved OS in univariate analysis (30% and 54% reduction in risk, respectively). The favorable effects of high TIL levels were consistently observed in different subgroups (classified according to known risk factors).

CONCLUSION

The proposed deep-learning workflow for automatic TIL quantification on the basis of LinkNet can be a useful tool for CRC. TILsLink is likely an independent risk factor for disease progression and carries predictive information of disease progression beyond the current clinical risk factors and biomarkers. The prognostic significance of TILsLink for OS is also evident.

Using deep learning to predict survival outcome in non-surgical cervical cancer patients based on pathological images

PURPOSE

We analyzed clinical features and the representative HE-stained pathologic images to predict 5-year overall survival via the deep-learning approach in cervical cancer patients in order to assist oncologists in designing the optimal treatment strategies.

METHODS

The research retrospectively collected 238 non-surgical cervical cancer patients treated with radiochemotherapy from 2014 to 2017. These patients were randomly divided into the training set (n = 165) and test set (n = 73). Then, we extract deep features after segmenting the HE-stained image into patches of size 224 × 224. A Lasso-Cox model was constructed with clinical data to predict 5-year OS. C-index evaluated this model performance with 95% CI, calibration curve, and ROC.

RESULTS

Based on multivariate analysis, 2 of 11 clinical characteristics (C-index 0.68) and 2 of 2048 pathomic features (C-index 0.74) and clinical-pathomic model (C-index 0.83) of nomograms predict 5-year survival in the training set, respectively. In test set, compared with the pathomic and clinical characteristics used alone, the clinical-pathomic model had an AUC of 0.750 (95% CI 0.540-0.959), the clinical predictor model had an AUC of 0.729 (95% CI 0.551-0.909), and the pathomic model AUC was 0.703 (95% CI 0.487-0.919). Based on appropriate nomogram scores, we divided patients into high-risk and low-risk groups, and Kaplan-Meier survival probability curves for both groups showed statistical differences.

CONCLUSION

We built a clinical-pathomic model to predict 5-year OS in non-surgical cervical cancer patients, which may be a promising method to improve the precision of personalized therapy.

Facts and Hopes on the Use of Artificial Intelligence for Predictive Immunotherapy Biomarkers in Cancer

Immunotherapy by immune checkpoint inhibitors has become a standard treatment strategy for many types of solid tumors. However, the majority of patients with cancer will not respond, and predicting response to this therapy is still a challenge. Artificial intelligence (AI) methods can extract meaningful information from complex data, such as image data. In clinical routine, radiology or histopathology images are ubiquitously available. AI has been used to predict the response to immunotherapy from radiology or histopathology images, either directly or indirectly via surrogate markers. While none of these methods are currently used in clinical routine, academic and commercial developments are pointing toward potential clinical adoption in the near future. Here, we summarize the state of the art in AI-based image biomarkers for immunotherapy response based on radiology and histopathology images. We point out limitations, caveats, and pitfalls, including biases, generalizability, and explainability, which are relevant for researchers and health care providers alike, and outline key clinical use cases of this new class of predictive biomarkers.

Dietary fiber as a wide pillar of colorectal cancer prevention and adjuvant therapy

Colorectal cancer is the third most incident and second most lethal type of cancer worldwide. Lifestyle and dietary patterns are the key factors for higher disease development risk. The dietary fiber intake from fruits and vegetables, mainly formed by food hydrocolloids, can help to lower the incidence of this type of neoplasia. Different food polysaccharides have applications in anti-tumoral therapy, such as coadjuvant to mainstream drugs, carriage-like properties, or direct influence on tumoral cells. Some classes include inulin, β-glucans, pectins, fucoidans, alginates, mucilages, and gums. Therefore, it is fundamental to discuss colorectal cancer mechanisms and the roles played by different polysaccharides in intestinal health. Genetic, environmental, and immunological modulation of mutated pathways regarding colorectal cancer has been explored before. Microbial diversity, byproduct formation (primarily short-chain fatty acids), inflammatory profile control, and tumoral mutated pathways regulation are thoroughly explored mechanisms by which dietary fiber sources influence a healthy gut ambiance.

Survey of Explainable AI Techniques in Healthcare

Artificial intelligence (AI) with deep learning models has been widely applied in numerous domains, including medical imaging and healthcare tasks. In the medical field, any judgment or decision is fraught with risk. A doctor will carefully judge whether a patient is sick before forming a reasonable explanation based on the patient's symptoms and/or an examination. Therefore, to be a viable and accepted tool, AI needs to mimic human judgment and interpretation skills. Specifically, explainable AI (XAI) aims to explain the information behind the black-box model of deep learning that reveals how the decisions are made. This paper provides a survey of the most recent XAI techniques used in healthcare and related medical imaging applications. We summarize and categorize the XAI types, and highlight the algorithms used to increase interpretability in medical imaging topics. In addition, we focus on the challenging XAI problems in medical applications and provide guidelines to develop better interpretations of deep learning models using XAI concepts in medical image and text analysis. Furthermore, this survey provides future directions to guide developers and researchers for future prospective investigations on clinical topics, particularly on applications with medical imaging.

Development and Validation of a Machine Learning Model for Detection and Classification of Tertiary Lymphoid Structures in Gastrointestinal Cancers

IMPORTANCE

Tertiary lymphoid structures (TLSs) are associated with a favorable prognosis and improved response to cancer immunotherapy. The current approach for evaluation of TLSs is limited by interobserver variability and high complexity and cost of specialized imaging techniques.

OBJECTIVE

To develop a machine learning model for automated and quantitative evaluation of TLSs based on routine histopathology images.

DESIGN, SETTING, AND PARTICIPANTS

In this multicenter, international diagnostic/prognostic study, an interpretable machine learning model was developed and validated for automated detection, enumeration, and classification of TLSs in hematoxylin-eosin-stained images. A quantitative scoring system for TLSs was proposed, and its association with survival was investigated in patients with 1 of 6 types of gastrointestinal cancers. Data analysis was performed between June 2021 and March 2022.

MAIN OUTCOMES AND MEASURES

The diagnostic accuracy for classification of TLSs into 3 maturation states and the association of TLS score with survival were investigated.

RESULTS

A total of 1924 patients with gastrointestinal cancer from 7 independent cohorts (median [IQR] age ranging from 57 [49-64] years to 68 [58-77] years; proportion by sex ranging from 214 of 409 patients who were male [52.3%] to 134 of 155 patients who were male [86.5%]). The machine learning model achieved high accuracies for detecting and classifying TLSs into 3 states (TLS1: 97.7%; 95% CI, 96.4%-99.0%; TLS2: 96.3%; 95% CI, 94.6%-98.0%; TLS3: 95.7%; 95% CI, 93.9%-97.5%). TLSs were detected in 62 of 155 esophageal cancers (40.0%) and up to 267 of 353 gastric cancers (75.6%). Across 6 cancer types, patients were stratified into 3 risk groups (higher and lower TLS score and no TLS) and survival outcomes compared between groups: higher vs lower TLS score (hazard ratio [HR]; 0.27; 95% CI, 0.18-0.41; P < .001) and lower TLS score vs no TLSs (HR, 0.65; 95% CI, 0.56-0.76; P < .001). TLS score remained an independent prognostic factor associated with survival after adjusting for clinicopathologic variables and tumor-infiltrating lymphocytes (eg, for colon cancer: HR, 0.11; 95% CI, 0.02-0.47; P = .003).

CONCLUSIONS AND RELEVANCE

In this study, an interpretable machine learning model was developed that may allow automated and accurate detection of TLSs on routine tissue slide. This model is complementary to the cancer staging system for risk stratification in gastrointestinal cancers.

Artificial intelligence in cancer research and precision medicine: Applications, limitations and priorities to drive transformation in the delivery of equitable and unbiased care

Artificial intelligence (AI) has experienced explosive growth in oncology and related specialties in recent years. The improved expertise in data capture, the increased capacity for data aggregation and analytic power, along with decreasing costs of genome sequencing and related biologic "omics", set the foundation and need for novel tools that can meaningfully process these data from multiple sources and of varying types. These advances provide value across biomedical discovery, diagnosis, prognosis, treatment, and prevention, in a multimodal fashion. However, while big data and AI tools have already revolutionized many fields, medicine has partially lagged due to its complexity and multi-dimensionality, leading to technical challenges in developing and validating solutions that generalize to diverse populations. Indeed, inner biases and miseducation of algorithms, in view of their implementation in daily clinical practice, are increasingly relevant concerns; critically, it is possible for AI to mirror the unconscious biases of the humans who generated these algorithms. Therefore, to avoid worsening existing health disparities, it is critical to employ a thoughtful, transparent, and inclusive approach that involves addressing bias in algorithm design and implementation along the cancer care continuum. In this review, a broad landscape of major applications of AI in cancer care is provided, with a focus on cancer research and precision medicine. Major challenges posed by the implementation of AI in the clinical setting will be discussed. Potentially feasible solutions for mitigating bias are provided, in the light of promoting cancer health equity.

Artificial intelligence-based multi-omics analysis fuels cancer precision medicine

With biotechnological advancements, innovative omics technologies are constantly emerging that have enabled researchers to access multi-layer information from the genome, epigenome, transcriptome, proteome, metabolome, and more. A wealth of omics technologies, including bulk and single-cell omics approaches, have empowered to characterize different molecular layers at unprecedented scale and resolution, providing a holistic view of tumor behavior. Multi-omics analysis allows systematic interrogation of various molecular information at each biological layer while posing tricky challenges regarding how to extract valuable insights from the exponentially increasing amount of multi-omics data. Therefore, efficient algorithms are needed to reduce the dimensionality of the data while simultaneously dissecting the mysteries behind the complex biological processes of cancer. Artificial intelligence has demonstrated the ability to analyze complementary multi-modal data streams within the oncology realm. The coincident development of multi-omics technologies and artificial intelligence algorithms has fuelled the development of cancer precision medicine. Here, we present state-of-the-art omics technologies and outline a roadmap of multi-omics integration analysis using an artificial intelligence strategy. The advances made using artificial intelligence-based multi-omics approaches are described, especially concerning early cancer screening, diagnosis, response assessment, and prognosis prediction. Finally, we discuss the challenges faced in multi-omics analysis, along with tentative future trends in this field. With the increasing application of artificial intelligence in multi-omics analysis, we anticipate a shifting paradigm in precision medicine becoming driven by artificial intelligence-based multi-omics technologies.

Systematic review of machine learning-based radiomics approach for predicting microsatellite instability status in colorectal cancer

This study aimed to systematically summarize the performance of the machine learning-based radiomics models in the prediction of microsatellite instability (MSI) in patients with colorectal cancer (CRC). It was conducted according to the preferred reporting items for a systematic review and meta-analysis of diagnostic test accuracy studies (PRISMA-DTA) guideline and was registered at the PROSPERO website with an identifier CRD42022295787. Systematic literature searching was conducted in databases of PubMed, Embase, Web of Science, and Cochrane Library up to November 10, 2022. Research which applied radiomics analysis on preoperative CT/MRI/PET-CT images for predicting the MSI status in CRC patients with no history of anti-tumor therapies was eligible. The radiomics quality score (RQS) and Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2) were applied to evaluate the research quality (full score 100%). Twelve studies with 4,320 patients were included. All studies were retrospective, and only four had an external validation cohort. The median incidence of MSI was 19% (range 8-34%). The area under the receiver operator curve of the models ranged from 0.78 to 0.96 (median 0.83) in the external validation cohort. The median sensitivity was 0.76 (range 0.32-1.00), and the median specificity was 0.87 (range 0.69-1.00). The median RQS score was 38% (range 14-50%), and half of the studies showed high risk in patient selection as evaluated by QUADAS-2. In conclusion, while radiomics based on pretreatment imaging modalities had a high performance in the prediction of MSI status in CRC, so far it does not appear to be ready for clinical use due to insufficient methodological quality.

Role of AI and digital pathology for colorectal immuno-oncology

Immunotherapy deals with therapeutic interventions to arrest the progression of tumours using the immune system. These include checkpoint inhibitors, T-cell manipulation, cytokines, oncolytic viruses and tumour vaccines. In this paper, we present a survey of the latest developments on immunotherapy in colorectal cancer (CRC) and the role of artificial intelligence (AI) in this context. Among these, microsatellite instability (MSI) is perhaps the most popular IO biomarker globally. We first discuss the MSI status of tumours, its implications for patient management, and its relationship to immune response. In recent years, several aspiring studies have used AI to predict the MSI status of patients from digital whole-slide images (WSIs) of routine diagnostic slides. We present a survey of AI literature on the prediction of MSI and tumour mutation burden from digitised WSIs of haematoxylin and eosin-stained diagnostic slides. We discuss AI approaches in detail and elaborate their contributions, limitations and key takeaways to drive future research. We further expand this survey to other IO-related biomarkers like immune cell infiltrates and alternate data modalities like immunohistochemistry and gene expression. Finally, we underline possible future directions in immunotherapy for CRC and promise of AI to accelerate this exploration for patient benefits.

Application of machine learning in the prediction of deficient mismatch repair in patients with colorectal cancer based on routine preoperative characterization

Simple summary

Detecting deficient mismatch repair (dMMR) in patients with colorectal cancer is essential for clinical decision-making, including evaluation of prognosis, guidance of adjuvant chemotherapy and immunotherapy, and primary screening for Lynch syndrome. However, outside of tertiary care centers, existing detection methods are not widely disseminated and highly depend on the experienced pathologist. Therefore, it is of great clinical significance to develop a broadly accessible and low-cost tool for dMMR prediction, particularly prior to surgery. In this study, we developed a convenient and reliable model for predicting dMMR status in CRC patients on routine preoperative characterization utilizing multiple machine learning algorithms. This model will work as an automated screening tool for identifying patients suitable for mismatch repair testing and consequently for improving the detection rate of dMMR, while reducing unnecessary labor and cost in patients with proficient mismatch repair.

Background

Deficient mismatch repair (dMMR) indicates a sustained anti-tumor immune response and has a favorable prognosis in patients with colorectal cancer (CRC). Although all CRC patients are recommended to undergo dMMR testing after surgery, current diagnostic approaches are not available for all country hospitals and patients. Therefore, efficient and low-cost predictive models for dMMR, especially for preoperative evaluations, are warranted.

Methods

A large scale of 5596 CRC patients who underwent surgical resection and mismatch repair testing were enrolled and randomly divided into training and validation cohorts. The clinical features exploited for predicting dMMR comprised the demographic characteristics, preoperative laboratory data, and tumor burden information. Machine learning (ML) methods involving eight basic algorithms, ensemble learning methods, and fusion algorithms were adopted with 10-fold cross-validation, and their performance was evaluated based on the area under the receiver operating characteristic curve (AUC) and calibration curves. The clinical net benefits were assessed using a decision curve analysis (DCA), and a nomogram was developed to facilitate model clinical practicality.

Results

All models achieved an AUC of nearly 0.80 in the validation cohort, with the stacking model exhibiting the best performance (AUC = 0.832). Logistical DCA revealed that the stacking model yielded more clinical net benefits than the conventional regression models. In the subgroup analysis, the stacking model also predicted dMMR regardless of the clinical stage. The nomogram showed a favorable consistence with the actual outcome in the calibration curve.

Conclusion

With the aid of ML algorithms, we developed a novel and robust model for predicting dMMR in CRC patients with satisfactory discriminative performance and designed a user-friendly and convenient nomogram.

Machine learning–based multiparametric magnetic resonance imaging radiomics model for distinguishing central neurocytoma from glioma of lateral ventricle

OBJECTIVES

To develop a machine learning-based radiomics model based on multiparametric magnetic resonance imaging (MRI) for preoperative discrimination between central neurocytomas (CNs) and gliomas of lateral ventricles.

METHODS

A total of 132 patients from two medical centers were enrolled in this retrospective study. Patients from the first medical center were divided into a training cohort (n = 74) and an internal validation cohort (n = 30). Patients from the second medical center were used as the external validation cohort (n = 28). Features were extracted from contrast-enhanced T1-weighted and T2-weighted images. A support vector machine was used for radiomics model investigation. Performance was evaluated using the sensitivity, specificity, and the area under the receiver operating characteristic curve (AUC). The model's performance was also compared with those of three radiologists.

RESULTS

The radiomics model achieved an AUC of 0.986 in the training cohort, 0.933 in the internal validation cohort, and 0.903 in the external validation cohort. In the three cohorts, the AUC values were 0.657, 0.786, and 0.708 for radiologist 1; 0.838, 0.799, and 0.790 for radiologist 2; and 0.827, 0.871, and 0.862 for radiologist 3. When assisted by the radiomics model, two radiologists improved their performance in the training cohort (p < 0.05) but not in the internal or external validation cohorts.

CONCLUSIONS

The machine learning radiomics model based on multiparametric MRI showed better performance for distinguishing CNs from lateral ventricular gliomas than did experienced radiologists, and it showed the potential to improve radiologist performance.

KEY POINTS

• The machine learning radiomics model shows excellent performance in distinguishing CNs from gliomas. • The radiomics model outweighs two experienced radiologists (area under the receiver operating characteristic curve, 0.90 vs 0.79 and 0.86, respectively). • The radiomics model has the potential to enhance radiologist performance.

Using Whole Slide Gray Value Map to Predict HER2 Expression and FISH Status in Breast Cancer

Accurate detection of HER2 expression through immunohistochemistry (IHC) is of great clinical significance in the treatment of breast cancer. However, manual interpretation of HER2 is challenging, due to the interobserver variability among pathologists. We sought to explore a deep learning method to predict HER2 expression level and gene status based on a Whole Slide Image (WSI) of the HER2 IHC section. When applied to 228 invasive breast carcinoma of no special type (IBC-NST) DAB-stained slides, our GrayMap+ convolutional neural network (CNN) model accurately classified HER2 IHC level with mean accuracy 0.952 ± 0.029 and predicted HER2 FISH status with mean accuracy 0.921 ± 0.029. Our result also demonstrated strong consistency in HER2 expression score between our system and experienced pathologists (intraclass correlation coefficient (ICC) = 0.903, Cohen's κ = 0.875). The discordant cases were found to be largely caused by high intra-tumor staining heterogeneity in the HER2 IHC group and low copy number in the HER2 FISH group.

Artificial Intelligence-Based Prediction of Recurrence after Curative Resection for Colorectal Cancer from Digital Pathological Images

BACKGROUND

To develop an artificial intelligence-based model to predict recurrence after curative resection for stage I-III colorectal cancer from digitized pathological slides.

PATIENTS AND METHODS

In this retrospective study, 471 consecutive patients who underwent curative resection for stage I-III colorectal cancer at our institution from 2004 to 2015 were enrolled, and 512 randomly selected tiles from digitally scanned images of hematoxylin and eosin-stained tumor tissue sections were used to train a convolutional neural network. Five-fold cross-validation was used to validate the model. The association between recurrence and the model's output scores were analyzed in the test cohorts.

RESULTS

The area under the receiver operating characteristic curve of the cross-validation was 0.7245 [95% confidence interval (CI) 0.6707-0.7783; P < 0.0001]. The score successfully classified patients into those with better and worse recurrence free survival (P < 0.0001). Multivariate analysis revealed that a high score was significantly associated with worse recurrence free survival [odds ratio (OR) 1.857; 95% CI 1.248-2.805; P = 0.0021], which was independent from other predictive factors: male sex (P = 0.0238), rectal cancer (P = 0.0396), preoperative abnormal carcinoembryonic antigen (CEA) level (P = 0.0216), pathological T3/T4 stage (P = 0.0162), and pathological positive lymph node metastasis (P < 0.0001).

CONCLUSIONS

The artificial intelligence-based prediction model discriminated patients with a high risk of recurrence. This approach could help decision-makers consider the benefits of adjuvant chemotherapy.

Quantitative Pathologic Analysis of Digitized Images of Colorectal Carcinoma Improves Prediction of Recurrence-Free Survival

BACKGROUND & AIMS

To examine whether quantitative pathologic analysis of digitized hematoxylin and eosin slides of colorectal carcinoma (CRC) correlates with clinicopathologic features, molecular alterations, and prognosis.

METHODS

A quantitative segmentation algorithm (QuantCRC) was applied to 6468 digitized hematoxylin and eosin slides of CRCs. Fifteen parameters were recorded from each image and tested for associations with clinicopathologic features and molecular alterations. A prognostic model was developed to predict recurrence-free survival using data from the internal cohort (n = 1928) and validated on an internal test (n = 483) and external cohort (n = 938).

RESULTS

There were significant differences in QuantCRC according to stage, histologic subtype, grade, venous/lymphatic/perineural invasion, tumor budding, CD8 immunohistochemistry, mismatch repair status, KRAS mutation, BRAF mutation, and CpG methylation. A prognostic model incorporating stage, mismatch repair, and QuantCRC resulted in a Harrell's concordance (c)-index of 0.714 (95% confidence interval [CI], 0.702-0.724) in the internal test and 0.744 (95% CI, 0.741-0.754) in the external cohort. Removing QuantCRC from the model reduced the c-index to 0.679 (95% CI, 0.673-0.694) in the external cohort. Prognostic risk groups were identified, which provided a hazard ratio of 2.24 (95% CI, 1.33-3.87, P = .004) for low vs high-risk stage III CRCs and 2.36 (95% CI, 1.07-5.20, P = .03) for low vs high-risk stage II CRCs, in the external cohort after adjusting for established risk factors. The predicted median 36-month recurrence rate for high-risk stage III CRCs was 32.7% vs 13.4% for low-risk stage III and 15.8% for high-risk stage II vs 5.4% for low-risk stage II CRCs.

CONCLUSIONS

QuantCRC provides a powerful adjunct to routine pathologic reporting of CRC. A prognostic model using QuantCRC improves prediction of recurrence-free survival.