Deep learning model for the prediction of microsatellite instability in colorectal cancer: a diagnostic study

Quantitative Pathologic Analysis of Digitized Images of Colorectal Carcinoma Improves Prediction of Recurrence-Free Survival

BACKGROUND & AIMS

To examine whether quantitative pathologic analysis of digitized hematoxylin and eosin slides of colorectal carcinoma (CRC) correlates with clinicopathologic features, molecular alterations, and prognosis.

METHODS

A quantitative segmentation algorithm (QuantCRC) was applied to 6468 digitized hematoxylin and eosin slides of CRCs. Fifteen parameters were recorded from each image and tested for associations with clinicopathologic features and molecular alterations. A prognostic model was developed to predict recurrence-free survival using data from the internal cohort (n = 1928) and validated on an internal test (n = 483) and external cohort (n = 938).

RESULTS

There were significant differences in QuantCRC according to stage, histologic subtype, grade, venous/lymphatic/perineural invasion, tumor budding, CD8 immunohistochemistry, mismatch repair status, KRAS mutation, BRAF mutation, and CpG methylation. A prognostic model incorporating stage, mismatch repair, and QuantCRC resulted in a Harrell's concordance (c)-index of 0.714 (95% confidence interval [CI], 0.702-0.724) in the internal test and 0.744 (95% CI, 0.741-0.754) in the external cohort. Removing QuantCRC from the model reduced the c-index to 0.679 (95% CI, 0.673-0.694) in the external cohort. Prognostic risk groups were identified, which provided a hazard ratio of 2.24 (95% CI, 1.33-3.87, P = .004) for low vs high-risk stage III CRCs and 2.36 (95% CI, 1.07-5.20, P = .03) for low vs high-risk stage II CRCs, in the external cohort after adjusting for established risk factors. The predicted median 36-month recurrence rate for high-risk stage III CRCs was 32.7% vs 13.4% for low-risk stage III and 15.8% for high-risk stage II vs 5.4% for low-risk stage II CRCs.

CONCLUSIONS

QuantCRC provides a powerful adjunct to routine pathologic reporting of CRC. A prognostic model using QuantCRC improves prediction of recurrence-free survival.

Colorectal Cancer Survival Prediction Using Deep Distribution Based Multiple-Instance Learning

Most deep-learning algorithms that use Hematoxylin- and Eosin-stained whole slide images (WSIs) to predict cancer survival incorporate image patches either with the highest scores or a combination of both the highest and lowest scores. In this study, we hypothesize that incorporating wholistic patch information can predict colorectal cancer (CRC) cancer survival more accurately. As such, we developed a distribution-based multiple-instance survival learning algorithm (DeepDisMISL) to validate this hypothesis on two large international CRC WSIs datasets called MCO CRC and TCGA COAD-READ. Our results suggest that combining patches that are scored based on percentile distributions together with the patches that are scored as highest and lowest drastically improves the performance of CRC survival prediction. Including multiple neighborhood instances around each selected distribution location (e.g., percentiles) could further improve the prediction. DeepDisMISL demonstrated superior predictive ability compared to other recently published, state-of-the-art algorithms. Furthermore, DeepDisMISL is interpretable and can assist clinicians in understanding the relationship between cancer morphological phenotypes and a patient's cancer survival risk.

Development of a method for digital assessment of tumor regression grade in patients with rectal cancer following neoadjuvant therapy

Neoadjuvant chemo-radiotherapy (nCRT) followed by surgical resection is the standard treatment strategy in patients with locally advanced rectal cancer (RC). The pathological effect of nCRT is assessed by determining the tumor regression grade (TRG) of the resected tumor. Various methods exist for assessing TRG and all are performed manually by the pathologist with an accompanying risk of interobserver variation. Automated digital image analysis could be a more objective and reproducible approach to evaluate TRG. This study aimed at developing a digital method to assess TRG in RC following nCRT, and correlate the results to the currently used Mandard method. A deep learning-based semi-automatic Epithelium-Tumor area Percentage (ETP) algorithm enabling quantification of tumor regression by determining the percentage of residual tumor epithelium out of the total tumor area was developed. The ETP was quantified in 50 cases treated with nCRT and 25 cases with no prior nCRT served as controls. Median ETP was 39.25% in untreated compared with 6.64% in patients who received nCRT (P < .001). The ETP of the resected tumors treated with nCRT increased along with increasing Mandard grade (P < .001). As new treatment strategies in RC are emerging, performing an accurate and reproducible evaluation of TRG is important in the assessment of treatment response and prognosis. TRG is often used as an outcome point in clinical trials. The ETP algorithm is capable of performing a precise and objective value of tumor regression.

Deep learning model to predict Epstein-Barr virus associated gastric cancer in histology

The detection of Epstein-Barr virus (EBV) in gastric cancer patients is crucial for clinical decision making, as it is related with specific treatment responses and prognoses. Despite its importance, the limited medical resources preclude universal EBV testing. Herein, we propose a deep learning-based EBV prediction method from H&E-stained whole-slide images (WSI). Our model was developed using 319 H&E stained WSI (26 EBV positive; TCGA dataset) from the Cancer Genome Atlas, and 108 WSI (8 EBV positive; ISH dataset) from an independent institution. Our deep learning model, EBVNet consists of two sequential components: a tumor classifier and an EBV classifier. We visualized the learned representation by the classifiers using UMAP. We externally validated the model using 60 additional WSI (7 being EBV positive; HGH dataset). We compared the model's performance with those of four pathologists. EBVNet achieved an AUPRC of 0.65, whereas the four pathologists yielded a mean AUPRC of 0.41. Moreover, EBVNet achieved an negative predictive value, sensitivity, specificity, precision, and F1-score of 0.98, 0.86, 0.92, 0.60, and 0.71, respectively. Our proposed model is expected to contribute to prescreen patients for confirmatory testing, potentially to save test-related cost and labor.

Molecular Pathology of Gastric Cancer

Gastric cancer (GC) is one of the most common lethal malignant neoplasms worldwide, with limited treatment options for both locally advanced and/or metastatic conditions, resulting in a dismal prognosis. Although the widely used morphological classifications may be helpful for endoscopic or surgical treatment choices, they are still insufficient to guide precise and/or personalized therapy for individual patients. Recent advances in genomic technology and high-throughput analysis may improve the understanding of molecular pathways associated with GC pathogenesis and aid in the classification of GC at the molecular level. Advances in next-generation sequencing have enabled the identification of several genetic alterations through single experiments. Thus, understanding the driver alterations involved in gastric carcinogenesis has become increasingly important because it can aid in the discovery of potential biomarkers and therapeutic targets. In this article, we review the molecular classifications of GC, focusing on The Cancer Genome Atlas (TCGA) classification. We further describe the currently available biomarker-targeted therapies and potential biomarker-guided therapies. This review will help clinicians by providing an inclusive understanding of the molecular pathology of GC and may assist in selecting the best treatment approaches for patients with GC.

Deep learning-based prediction of molecular cancer biomarkers from tissue slides: A new tool for precision oncology

Molecular tests are necessary to stratify cancer patients for targeted therapy. However, high cost and technical barriers limit the application of these tests, hindering optimal treatment. Recently, deep learning (DL) has been applied to predict molecular test results from digitized images of tissue slides. Furthermore, treatment response and prognosis can be predicted from tissue slides using DL. In this review, we summarized DL-based studies regarding the prediction of genetic mutation, microsatellite instability, tumor mutational burden, molecular subtypes, gene expression, treatment response, and prognosis directly from hematoxylin- and eosin-stained tissue slides. Although performance needs to be improved, these studies clearly demonstrated the feasibility of DL-based prediction of key molecular features in cancer tissues. With the accumulation of data and technical advances, the performance of the DL system could be improved in the near future. Therefore, we expect that DL could provide cost- and time-effective alternative tools for patient stratification in the era of precision oncology.

PPsNet: An improved deep learning model for microsatellite instability high prediction in colorectal cancer from whole slide images

BACKGROUND AND OBJECTIVE

Recent studies have shown that colorectal cancer (CRC) patients with microsatellite instability high (MSI-H) are more likely to benefit from immunotherapy. However, current MSI testing methods are not available for all patients due to the lack of available equipment and trained personnel, as well as the high cost of the assay. Here, we developed an improved deep learning model to predict MSI-H in CRC from whole slide images (WSIs).

METHODS

We established the MSI-H prediction model based on two stages: tumor detection and MSI classification. Previous works applied fine-tuning strategy directly for tumor detection, but ignoring the challenge of vanishing gradient due to the large number of convolutional layers. We added auxiliary classifiers to intermediate layers of pre-trained models to help propagate gradients back through in an effective manner. To predict MSI status, we constructed a pair-wise learning model with a synergic network, named parameter partial sharing network (PPsNet), where partial parameters are shared among two deep convolutional neural networks (DCNNs). The proposed PPsNet contained fewer parameters and reduced the problem of intra-class variation and inter-class similarity. We validated the proposed model on a holdout test set and two external test sets.

RESULTS

144 H&E-stained WSIs from 144 CRC patients (81 cases with MSI-H and 63 cases with MSI-L/MSS) were collected retrospectively from three hospitals. The experimental results indicate that deep supervision based fine-tuning almost outperforms training from scratch and utilizing fine-tuning directly. The proposed PPsNet always achieves better accuracy and area under the receiver operating characteristic curve (AUC) than other solutions with four different neural network architectures on validation. The proposed method finally achieves obvious improvements than other state-of-the-art methods on the validation dataset with an accuracy of 87.28% and AUC of 94.29%.

CONCLUSIONS

The proposed method can obviously increase model performance and our model yields better performance than other methods. Additionally, this work also demonstrates the feasibility of MSI-H prediction using digital pathology images based on deep learning in the Asian population. It is hoped that this model could serve as an auxiliary tool to identify CRC patients with MSI-H more time-saving and efficiently.

Multi-modality artificial intelligence in digital pathology

In common medical procedures, the time-consuming and expensive nature of obtaining test results plagues doctors and patients. Digital pathology research allows using computational technologies to manage data, presenting an opportunity to improve the efficiency of diagnosis and treatment. Artificial intelligence (AI) has a great advantage in the data analytics phase. Extensive research has shown that AI algorithms can produce more up-to-date and standardized conclusions for whole slide images. In conjunction with the development of high-throughput sequencing technologies, algorithms can integrate and analyze data from multiple modalities to explore the correspondence between morphological features and gene expression. This review investigates using the most popular image data, hematoxylin-eosin stained tissue slide images, to find a strategic solution for the imbalance of healthcare resources. The article focuses on the role that the development of deep learning technology has in assisting doctors' work and discusses the opportunities and challenges of AI.

DEMoS: a deep learning-based ensemble approach for predicting the molecular subtypes of gastric adenocarcinomas from histopathological images

MOTIVATION

The molecular subtyping of gastric cancer (adenocarcinoma) into four main subtypes based on integrated multiomics profiles, as proposed by The Cancer Genome Atlas (TCGA) initiative, represents an effective strategy for patient stratification. However, this approach requires the use of multiple technological platforms, and is quite expensive and time-consuming to perform. A computational approach that uses histopathological image data to infer molecular subtypes could be a practical, cost- and time-efficient complementary tool for prognostic and clinical management purposes.

RESULTS

Here, we propose a deep learning ensemble approach (called DEMoS) capable of predicting the four recognized molecular subtypes of gastric cancer directly from histopathological images. DEMoS achieved tile-level area under the receiver-operating characteristic curve (AUROC) values of 0.785, 0.668, 0.762 and 0.811 for the prediction of these four subtypes of gastric cancer [i.e. (i) Epstein-Barr (EBV)-infected, (ii) microsatellite instability (MSI), (iii) genomically stable (GS) and (iv) chromosomally unstable tumors (CIN)] using an independent test dataset, respectively. At the patient-level, it achieved AUROC values of 0.897, 0.764, 0.890 and 0.898, respectively. Thus, these four subtypes are well-predicted by DEMoS. Benchmarking experiments further suggest that DEMoS is able to achieve an improved classification performance for image-based subtyping and prevent model overfitting. This study highlights the feasibility of using a deep learning ensemble-based method to rapidly and reliably subtype gastric cancer (adenocarcinoma) solely using features from histopathological images.

AVAILABILITY AND IMPLEMENTATION

All whole slide images used in this study was collected from the TCGA database. This study builds upon our previously published HEAL framework, with related documentation and tutorials available at http://heal.erc.monash.edu.au. The source code and related models are freely accessible at https://github.com/Docurdt/DEMoS.git.

SUPPLEMENTARY INFORMATION

Supplementary data are available at Bioinformatics online.

The value of magnetic resonance imaging-based tumor shape features for assessing microsatellite instability status in endometrial cancer

BACKGROUND

Microsatellite instability (MSI) status can be used for the classification and risk stratification of endometrial cancer (EC). This study aimed to investigate whether magnetic resonance imaging (MRI)-based tumor shape features can help assess MSI status in EC before surgery.

METHODS

The medical records of 88 EC patients with MSI status were retrospectively reviewed. Quantitative and subjective shape features based on MRI were used to assess MSI status. Variables were compared using the Student's t-test, χ² test, or Wilcoxon rank-sum test where appropriate. Univariate and multivariate analyses were performed by the logistic regression model. The area under the curve (AUC) was used to estimate the discrimination performance of variables.

RESULTS

There were 23 patients with MSI, and 65 patients with microsatellite stability (MSS) in this study. Eccentricity and shape type showed significant differences between MSI and MSS (P=0.039 and P=0.033, respectively). The AUC values of eccentricity, shape type, and the combination of 2 features for assessing MSI were 0.662 [95% confidence interval (CI): 0.554-0.770], 0.627 (95% CI: 0.512-0.743), and 0.727 (95% CI: 0.613-0.842), respectively. Considering the International Federation of Gynecology and Obstetrics (FIGO) staging, eccentricity maintained a significant difference in stages I-II (P=0.039), while there was no statistical difference in stages III-IV (P=0.601).

CONCLUSIONS

It is possible that MRI-based tumor shape features, including eccentricity and shape type, could be promising markers for assessing MSI status. The features may aid in the preliminary screening of EC patients with MSI.

Deep Neural Network for the Prediction of KRAS Genotype in Rectal Cancer

BACKGROUND

KRAS mutation can alter the treatment plan after resection of colorectal cancer. Despite its importance, the KRAS status of several patients remains unchecked because of the high cost and limited resources. This study developed a deep neural network (DNN) to predict the KRAS genotype using hematoxylin and eosin (H&E)-stained histopathological images.

STUDY DESIGN

Three DNNs were created (KRAS_Mob, KRAS_Shuff, and KRAS_Ince) using the structural backbone of the MobileNet, ShuffleNet, and Inception networks, respectively. The Cancer Genome Atlas was screened to extract 49,684 image tiles that were used for deep learning and internal validation. An independent cohort of 43,032 image tiles was used for external validation. The performance was compared with humans, and a virtual cost-saving analysis was done.

RESULTS

The KRAS_Mob network (area under the receiver operating curve [AUC] 0.8, 95% CI 0.71 to 0.89) was the best-performing model for predicting the KRAS genotype, followed by the KRAS_Shuff (AUC 0.73, 95% CI 0.62 to 0.84) and KRAS_Ince (AUC 0.71, 95% CI 0.6 to 0.82) networks. Combing the KRAS_Mob and KRAS_Shuff networks as a double prediction approach showed improved performance. KRAS_Mob network accuracy surpassed that of two independent pathologists (AUC 0.79 [95% CI 0.64 to 0.93], 0.51 [95% CI 0.34 to 0.69], and 0.51 (95% CI 0.34 to 0.69]; p < 0.001 for all comparisons).

CONCLUSION

The DNN has the potential to predict the KRAS genotype directly from H&E-stained histopathological slide images. As an algorithmic screening method to prioritize patients for laboratory confirmation, such a model might possibly reduce the number of patients screened, resulting in significant test-related time and economic savings.

A clinical decision support system optimising adjuvant chemotherapy for colorectal cancers by integrating deep learning and pathological staging markers: a development and validation study

BACKGROUND

The DoMore-v1-CRC marker was recently developed using deep learning and conventional haematoxylin and eosin-stained tissue sections, and was observed to outperform established molecular and morphological markers of patient outcome after primary colorectal cancer resection. The aim of the present study was to develop a clinical decision support system based on DoMore-v1-CRC and pathological staging markers to facilitate individualised selection of adjuvant treatment.

METHODS

We estimated cancer-specific survival in subgroups formed by pathological tumour stage (pT<4 or pT4), pathological nodal stage (pN0, pN1, or pN2), number of lymph nodes sampled (≤12 or >12) if not pN2, and DoMore-v1-CRC classification (good, uncertain, or poor prognosis) in 997 patients with stage II or III colorectal cancer considered to have no residual tumour (R0) from two community-based cohorts in Norway and the UK, and used these data to define three risk groups. An external cohort of 1075 patients with stage II or III R0 colorectal cancer from the QUASAR 2 trial was used for validation; these patients were treated with single-agent capecitabine. The proposed risk stratification system was evaluated using Cox regression analysis. We similarly evaluated a risk stratification system intended to reflect current guidelines and clinical practice. The primary outcome was cancer-specific survival.

FINDINGS

The new risk stratification system provided a hazard ratio of 10·71 (95% CI 6·39-17·93; p<0·0001) for high-risk versus low-risk patients and 3·06 (1·73-5·42; p=0·0001) for intermediate versus low risk in the primary analysis of the validation cohort. Estimated 3-year cancer-specific survival was 97·2% (95% CI 95·1-98·4; n=445 [41%]) for the low-risk group, 94·8% (91·7-96·7; n=339 [32%]) for the intermediate-risk group, and 77·6% (72·1-82·1; n=291 [27%]) for the high-risk group. The guideline-based risk grouping was observed to be less prognostic and informative (the low-risk group comprised only 142 [13%] of the 1075 patients).

INTERPRETATION

Integrating DoMore-v1-CRC and pathological staging markers provided a clinical decision support system that risk stratifies more accurately than its constituent elements, and identifies substantially more patients with stage II and III colorectal cancer with similarly good prognosis as the low-risk group in current guidelines. Avoiding adjuvant chemotherapy in these patients might be safe, and could reduce morbidity, mortality, and treatment costs.

FUNDING

The Research Council of Norway.

Artificial intelligence in histopathology: enhancing cancer research and clinical oncology

Artificial intelligence (AI) methods have multiplied our capabilities to extract quantitative information from digital histopathology images. AI is expected to reduce workload for human experts, improve the objectivity and consistency of pathology reports, and have a clinical impact by extracting hidden information from routinely available data. Here, we describe how AI can be used to predict cancer outcome, treatment response, genetic alterations and gene expression from digitized histopathology slides. We summarize the underlying technologies and emerging approaches, noting limitations, including the need for data sharing and standards. Finally, we discuss the broader implications of AI in cancer research and oncology.

The evolving role of morphology in endometrial cancer diagnostics: From histopathology and molecular testing towards integrative data analysis by deep learning

Endometrial cancer (EC) diagnostics is evolving into a system in which molecular aspects are increasingly important. The traditional histological subtype-driven classification has shifted to a molecular-based classification that stratifies EC into DNA polymerase epsilon mutated (POLEmut), mismatch repair deficient (MMRd), and p53 abnormal (p53abn), and the remaining EC as no specific molecular profile (NSMP). The molecular EC classification has been implemented in the World Health Organization 2020 classification and the 2021 European treatment guidelines, as it serves as a better basis for patient management. As a result, the integration of the molecular class with histopathological variables has become a critical focus of recent EC research. Pathologists have observed and described several morphological characteristics in association with specific genomic alterations, but these appear insufficient to accurately classify patients according to molecular subgroups. This requires pathologists to rely on molecular ancillary tests in routine workup. In this new era, it has become increasingly challenging to assign clinically relevant weights to histological and molecular features on an individual patient basis. Deep learning (DL) technology opens new options for the integrative analysis of multi-modal image and molecular datasets with clinical outcomes. Proof-of-concept studies in other cancers showed promising accuracy in predicting molecular alterations from H&E-stained tumor slide images. This suggests that some morphological characteristics that are associated with molecular alterations could be identified in EC, too, expanding the current understanding of the molecular-driven EC classification. Here in this review, we report the morphological characteristics of the molecular EC classification currently identified in the literature. Given the new challenges in EC diagnostics, this review discusses, therefore, the potential supportive role that DL could have, by providing an outlook on all relevant studies using DL on histopathology images in various cancer types with a focus on EC. Finally, we touch upon how DL might shape the management of future EC patients.

Derivation of prognostic contextual histopathological features from whole-slide images of tumours via graph deep learning

Methods of computational pathology applied to the analysis of whole-slide images (WSIs) do not typically consider histopathological features from the tumour microenvironment. Here, we show that a graph deep neural network that considers such contextual features in gigapixel-sized WSIs in a semi-supervised manner can provide interpretable prognostic biomarkers. We designed a neural-network model that leverages attention techniques to learn features of the heterogeneous tumour microenvironment from memory-efficient representations of aggregates of highly correlated image patches. We trained the model with WSIs of kidney, breast, lung and uterine cancers and validated it by predicting the prognosis of 3,950 patients with these four different types of cancer. We also show that the model provides interpretable contextual features of clear cell renal cell carcinoma that allowed for the risk-based retrospective stratification of 1,333 patients. Deep graph neural networks that derive contextual histopathological features from WSIs may aid diagnostic and prognostic tasks.

Predicting Mismatch-Repair Status in Rectal Cancer Using Multiparametric MRI-Based Radiomics Models: A Preliminary Study

Detecting mismatch-repair (MMR) status is crucial for personalized treatment strategies and prognosis in rectal cancer (RC). A preoperative, noninvasive, and cost-efficient predictive tool for MMR is critically needed. Therefore, this study developed and validated machine learning radiomics models for predicting MMR status in patients directly on preoperative MRI scans. Pathologically confirmed RC cases administered surgical resection in two distinct hospitals were examined in this retrospective trial. Totally, 78 and 33 cases were included in the training and test sets, respectively. Then, 65 cases were enrolled as an external validation set. Radiomics features were obtained from preoperative rectal MR images comprising T2-weighted imaging (T2WI), diffusion-weighted imaging (DWI), contrast-enhanced T1-weighted imaging (T1WI), and combined multisequences. Four optimal features related to MMR status were selected by the least absolute shrinkage and selection operator (LASSO) method. Support vector machine (SVM) learning was adopted to establish four predictive models, i.e., Model_T2WI, Model_DWI, Model_CE-T1WI, and Model_combination, whose diagnostic performances were determined and compared by receiver operating characteristic (ROC) curves and decision curve analysis (DCA). Model_combination had better diagnostic performance compared with the other models in all datasets (all p < 0.05). The usefulness of the proposed model was confirmed by DCA. Therefore, the present pilot study showed the radiomics model combining multiple sequences derived from preoperative MRI is effective in predicting MMR status in RC cases.

Contrastive learning-based computational histopathology predict differential expression of cancer driver genes

MOTIVATION

Digital pathological analysis is run as the main examination used for cancer diagnosis. Recently, deep learning-driven feature extraction from pathology images is able to detect genetic variations and tumor environment, but few studies focus on differential gene expression in tumor cells.

RESULTS

In this paper, we propose a self-supervised contrastive learning framework, HistCode, to infer differential gene expression from whole slide images (WSIs). We leveraged contrastive learning on large-scale unannotated WSIs to derive slide-level histopathological features in latent space, and then transfer it to tumor diagnosis and prediction of differentially expressed cancer driver genes. Our experiments showed that our method outperformed other state-of-the-art models in tumor diagnosis tasks, and also effectively predicted differential gene expression. Interestingly, we found the genes with higher fold change can be more precisely predicted. To intuitively illustrate the ability to extract informative features from pathological images, we spatially visualized the WSIs colored by the attention scores of image tiles. We found that the tumor and necrosis areas were highly consistent with the annotations of experienced pathologists. Moreover, the spatial heatmap generated by lymphocyte-specific gene expression patterns was also consistent with the manually labeled WSIs.

Computational Analysis of Pathological Image Enables Interpretable Prediction for Microsatellite Instability

Background

Microsatellite instability (MSI) is associated with several tumor types and has become increasingly vital in guiding patient treatment decisions; however, reasonably distinguishing MSI from its counterpart is challenging in clinical practice.

Methods

In this study, interpretable pathological image analysis strategies are established to help medical experts to identify MSI. The strategies only require ubiquitous hematoxylin and eosin–stained whole-slide images and perform well in the three cohorts collected from The Cancer Genome Atlas. Equipped with machine learning and image processing technique, intelligent models are established to diagnose MSI based on pathological images, providing the rationale of the decision in both image level and pathological feature level.

Findings

The strategies achieve two levels of interpretability. First, the image-level interpretability is achieved by generating localization heat maps of important regions based on deep learning. Second, the feature-level interpretability is attained through feature importance and pathological feature interaction analysis. Interestingly, from both the image-level and feature-level interpretability, color and texture characteristics, as well as their interaction, are shown to be mostly contributed to the MSI prediction.

Interpretation

The developed transparent machine learning pipeline is able to detect MSI efficiently and provide comprehensive clinical insights to pathologists. The comprehensible heat maps and features in the intelligent pipeline reflect extra- and intra-cellular acid–base balance shift in MSI tumor.

A Radiomics Nomogram Integrated With Clinic-Radiological Features for Preoperative Prediction of DNA Mismatch Repair Deficiency in Gastric Adenocarcinoma

Purpose

To develop and validate a radiomics nomogram integrated with clinic-radiological features for preoperative prediction of DNA mismatch repair deficiency (dMMR) in gastric adenocarcinoma.

Materials and Methods

From March 2014 to August 2020, 161 patients with pathologically confirmed gastric adenocarcinoma were included from two centers (center 1 as the training and internal testing sets, n = 101; center 2 as the external testing sets, n = 60). All patients underwent preoperative contrast-enhanced computerized tomography (CT) examination. Radiomics features were extracted from portal-venous phase CT images. Max-relevance and min-redundancy (mRMR) and least absolute shrinkage and selection operator (LASSO) methods were used to select features, and then radiomics signature was constructed using logistic regression analysis. A radiomics nomogram was built incorporating the radiomics signature and independent clinical predictors. The model performance was assessed using receiver operating characteristic (ROC) curve analysis, calibration curve, and decision curve analysis (DCA).

Results

The radiomics signature, which was constructed using two selected features, was significantly associated with dMMR gastric adenocarcinoma in the training and internal testing sets (P < 0.05). The radiomics signature model showed a moderate discrimination ability with an area under the ROC curve (AUC) of 0.81 in the training set, which was confirmed with an AUC of 0.78 in the internal testing set. The radiomics nomogram consisting of the radiomics signature and clinical factors (age, sex, and location) showed excellent discrimination in the training, internal testing, and external testing sets with AUCs of 0.93, 0.82, and 0.83, respectively. Further, calibration curves and DCA analysis demonstrated good fit and clinical utility of the radiomics nomogram.

Conclusions

The radiomics nomogram combining radiomics signature and clinical characteristics (age, sex, and location) may be used to individually predict dMMR of gastric adenocarcinoma.

Clinical actionability of triaging DNA mismatch repair deficient colorectal cancer from biopsy samples using deep learning

Background

We aimed to develop a deep learning (DL) model to predict DNA mismatch repair (MMR) status in colorectal cancers (CRC) based on hematoxylin and eosin-stained whole-slide images (WSIs) and assess its clinical applicability.

Methods

The DL model was developed and validated through three-fold cross validation using 441 WSIs from the Cancer Genome Atlas (TCGA) and externally validated using 78 WSIs from the Pathology AI Platform (PAIP), and 355 WSIs from surgical specimens and 341 WSIs from biopsy specimens of the Sun Yet-sun University Cancer Center (SYSUCC). Domain adaption and multiple instance learning (MIL) techniques were adopted for model development. The performance of the models was evaluated using the area under the receiver operating characteristic curve (AUROC). A dual-threshold strategy was also built from the surgical cohorts and validated in the biopsy cohort. Sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), F1-score, and the percentage of patients avoiding IHC testing were evaluated.

Findings

The MIL model achieved an AUROC of 0·8888±0·0357 in the TCGA-validation cohort, 0·8806±0·0232 in the PAIP cohort, 0·8457±0·0233 in the SYSUCC-surgical cohort, and 0·7679±0·0342 in the SYSUCC-biopsy cohort. A dual-threshold triage strategy was used to rule-in and rule-out dMMR patients with remaining uncertain patients recommended for further IHC testing, which kept sensitivity higher than 90% and specificity higher than 95% on deficient MMR patient triage from both the surgical and biopsy specimens, result in more than half of patients avoiding IHC based MMR testing.

Interpretation

A DL-based method that could directly predict CRC MMR status from WSIs was successfully developed, and a dual-threshold triage strategy was established to minimize the number of patients for further IHC testing.

Funding

The study was funded by the National Natural Science Foundation of China (82073159, 81871971 and 81700576), the Natural Science Foundation of Guangdong Province (No. 2021A1515011792 and No.2022A1515012403) and Medical Scientific Research Foundation of Guangdong Province of China (No. A2020392).

Spatial analysis of tumor-infiltrating lymphocytes in histological sections using deep learning techniques predicts survival in colorectal carcinoma

This study aimed to explore the prognostic impact of spatial distribution of tumor‐infiltrating lymphocytes (TILs) quantified by deep learning (DL) approaches based on digitalized whole‐slide images stained with hematoxylin and eosin in patients with colorectal cancer (CRC). The prognostic impact of spatial distributions of TILs in patients with CRC was explored in the Yonsei cohort (n = 180) and validated in The Cancer Genome Atlas (TCGA) cohort (n = 268). Two experienced pathologists manually measured TILs at the most invasive margin (IM) as 0–3 by the Klintrup–Mäkinen (KM) grading method and this was compared to DL approaches. Inter‐rater agreement for TILs was measured using Cohen's kappa coefficient. On multivariate analysis of spatial TIL features derived by DL approaches and clinicopathological variables including tumor stage, microsatellite instability, and KRAS mutation, TIL densities within 200 μm of the IM (f_im200) remained the most significant prognostic factor for progression‐free survival (PFS) (hazard ratio [HR] 0.004 [95% confidence interval, CI, 0.0001–0.15], p = 0.0028) in the Yonsei cohort. On multivariate analysis using the TCGA dataset, f_im200 retained prognostic significance for PFS (HR 0.031 [95% CI 0.001–0.645], p = 0.024). Inter‐rater agreement of manual KM grading was insignificant in the Yonsei (κ = 0.109) and the TCGA (κ = 0.121) cohorts. The survival analysis based on KM grading showed statistically significant different PFS in the TCGA cohort, but not the Yonsei cohort. Automatic quantification of TILs at the IM based on DL approaches shows prognostic utility to predict PFS, and could provide robust and reproducible TIL density measurement in patients with CRC.

Immune Response in Colorectal Carcinoma: A Review of Its Significance as a Predictive and Prognostic Biomarker

Colorectal carcinoma is a leading cause of cancer-related death worldwide. There is significant prognostic heterogeneity in stage II and III tumours, necessitating the development of new biomarkers to better identify patients at risk of disease progression. Recently, the tumour immune environment, particularly the type and quantity of T lymphocytes, has been shown to be a useful biomarker in predicting prognosis for patients with colorectal carcinoma. In this review, the significance of the immune response in colorectal carcinoma, including its influence on prognosis and response to therapy, will be detailed.

Prediction and Risk Stratification of Cardiovascular Disease in Diabetic Kidney Disease Patients

Background

Diabetic kidney disease (DKD) patients are facing an extremely high risk of cardiovascular disease (CVD), which is a major cause of death for DKD patients. We aimed to build a deep learning model to predict CVD risk among DKD patients and perform risk stratifying, which could help them perform early intervention and improve personal health management.

Methods

A retrospective cohort study was conducted to assess the risk of the occurrence of composite cardiovascular disease, which includes coronary heart disease, cerebrovascular diseases, congestive heart failure, and peripheral artery disease, in DKD patients. A least absolute shrinkage and selection operator (LASSO) regression was used to perform the variable selection. A deep learning-based survival model called DeepSurv, based on a feed-forward neural network was developed to predict CVD risk among DKD patients. We compared the model performance with the conventional Cox proportional hazards (CPH) model and the Random survival forest (RSF) model using the concordance index (C-index), the area under the curve (AUC), and integrated Brier scores (IBS).

Results

We recruited 890 patients diagnosed with DKD in this retrospective study. During a median follow-up of 10.4 months, there are 289 patients who sustained a subsequent CVD. Seven variables, including age, high density lipoprotein (HDL), hemoglobin (Hb), systolic blood pressure (SBP), smoking status, 24 h urinary protein excretion, and total cholesterol (TC), chosen by LASSO regression were used to develop the predictive model. The DeepSurv model showed the best performance, achieved a C-index of 0.767(95% confidence intervals [CI]: 0.717–0.817), AUC of 0.780(95%CI: 0.721–0.839), and IBS of 0.067 in the validation set. Then we used the cut-off value determined by ROC (receiver operating characteristic) curve to divide the patients into different risk groups. Moreover, the DeepSurv model was also applied to develop an online calculation tool for patients to conduct risk monitoring.

Conclusion

A deep-learning-based predictive model using seven clinical variables can effectively predict CVD risk among DKD patients and perform risk stratification. An online calculator allows its easy implementation.

Characteristics and clinical outcomes of patients with advanced gastric or gastroesophageal cancer treated in and out of randomized clinical trials of first-line immune checkpoint inhibitors

BACKGROUND

Although several randomized trials (RCTs) showed survival benefits of immune checkpoint inhibitor (ICI) plus first-line chemotherapy for advanced gastric or gastroesophageal cancer (AGC), these trials could enroll patients who fulfilled the strict eligibility criteria or waited for certain screening period for central assessment of PD-L1 status.

METHODS

We retrospectively compared characteristics and clinical outcomes of the patients with AGC who received first-line chemotherapy in control arm of RCTs with ICIs (control group) or clinical practice (practice group) at our institution from February 2016 to April 2019.

RESULTS

The control group had a better baseline Eastern Cooperative Oncology Group performance status (PS0, 81.2% vs. 51.4%, p < 0.001) and a longer interval from first visit to first-line chemotherapy initiation (19 days vs. 9 days, p < 0.001) than the practice group. Median overall survival (OS) was 20.3 months in control group and 15.7 months in practice group, with a trend of longer OS in control group than that in practice group (hazard ratio, 0.71; p = 0.062). More patients in control group were treated with subsequent chemotherapy including ICIs.

CONCLUSION

Patients with AGC in RCTs of ICIs had a better PS or a higher chance to receive subsequent chemotherapy, resulting in a better prognosis than those treated in clinical practice. This information should be considered when interpreting RCT results and applying new treatments into clinical practice.

Usability of deep learning and H&E images predict disease outcome-emerging tool to optimize clinical trials

Understanding factors that impact prognosis for cancer patients have high clinical relevance for treatment decisions and monitoring of the disease outcome. Advances in artificial intelligence (AI) and digital pathology offer an exciting opportunity to capitalize on the use of whole slide images (WSIs) of hematoxylin and eosin (H&E) stained tumor tissue for objective prognosis and prediction of response to targeted therapies. AI models often require hand-delineated annotations for effective training which may not be readily available for larger data sets. In this study, we investigated whether AI models can be trained without region-level annotations and solely on patient-level survival data. We present a weakly supervised survival convolutional neural network (WSS-CNN) approach equipped with a visual attention mechanism for predicting overall survival. The inclusion of visual attention provides insights into regions of the tumor microenvironment with the pathological interpretation which may improve our understanding of the disease pathomechanism. We performed this analysis on two independent, multi-center patient data sets of lung (which is publicly available data) and bladder urothelial carcinoma. We perform univariable and multivariable analysis and show that WSS-CNN features are prognostic of overall survival in both tumor indications. The presented results highlight the significance of computational pathology algorithms for predicting prognosis using H&E stained images alone and underpin the use of computational methods to improve the efficiency of clinical trial studies.

A Hemagglutinin Stem Vaccine Designed Rationally by AlphaFold2 Confers Broad Protection against Influenza B Infection

Two lineages of influenza B viruses (IBV) co-circulating in human beings have been posing a significant public health burden worldwide. A substantial number of broadly neutralizing antibodies (bnAbs) have been identified targeting conserved epitopes on hemagglutinin (HA) stem domain, posing great interest for universal influenza vaccine development. Various strategies to design immunogens that selectively present these conserved epitopes are being explored. However, it has been a challenge to retain native conformation of the HA stem region, especially for soluble expression in prokaryotic systems. Here, using a structure prediction tool AlphaFold2, we rationally designed a stable stem antigen “B60-Stem-8071”, an HA stem vaccine derived from B/Brisbane/60/2006 grafted with a CR8071 epitope as a linker. The B60-Stem-8071 exhibited better solubility and more stable expression in the E. coli system compared to the naïve HA stem antigen. Immunization with B60-Stem-8071 in mice generated cross-reactive antibodies and protected mice broadly against lethal challenge with Yamagata and Victoria lineages of influenza B virus. Notably, soluble expression of B60-stem-8071 in the E. coli system showed the potential to produce the influenza B vaccine in a low-cost way. This study represents a proof of concept for the rational design of HA stem antigen based on structure prediction and analysis.

Diagnostic study on clinical feasibility of an AI-based diagnostic system as a second reader on mobile CT images: a preliminary result

Background

Artificial intelligence (AI) has breathed new life into the lung nodules detection and diagnosis. However, whether the output information from AI will translate into benefits for clinical workflow or patient outcomes in a real-world setting remains unknown. This study was to demonstrate the feasibility of an AI-based diagnostic system deployed as a second reader in imaging interpretation for patients screened for pulmonary abnormalities in a clinical setting.

Methods

The study included patients from a lung cancer screening program conducted in Sichuan Province, China using a mobile computed tomography (CT) scanner which traveled to medium-size cities between July 10^th, 2020 and September 10^th, 2020. Cases that were suspected to have malignant nodules by junior radiologists, senior radiologists or AI were labeled a high risk (HR) tag as HR-junior, HR-senior and HR-AI, respectively, and included into final analysis. The diagnosis efficacy of the AI was evaluated by calculating negative predictive value and positive predictive value when referring to the senior readers’ final results as the gold standard. Besides, characteristics of the lesions were compared among cases with different HR labels.

Results

In total, 251/3,872 patients (6.48%, male/female: 91/160, median age, 66 years) with HR lung nodules were included. The AI algorithm achieved a negative predictive value of 88.2% [95% confidence interval (CI): 62.2–98.0%] and a positive predictive value of 55.6% (95% CI: 49.0–62.0%). The diagnostic duration was significantly reduced when AI was used as a second reader (223±145.6 vs. 270±143.17 s, P<0.001). The information yielded by AI affected the radiologist’s decision-making in 35/145 cases. Lesions of HR cases had a higher volume [309.9 (214.9–732.5) vs. 141.3 (79.3–380.8) mm³, P<0.001], lower average CT number [−511.0 (−576.5 to −100.5) vs. −191.5 (−487.3 to 22.5), P=0.010], and pure ground glass opacity rather than solid.

Conclusions

The AI algorithm had high negative predictive value but low positive predictive value in diagnosing HR lung lesions in a clinical setting. Deploying AI as a second reader could help avoid missed diagnoses, reduce diagnostic duration, and strengthen diagnostic confidence for radiologists.

Interpretable tumor differentiation grade and microsatellite instability recognition in gastric cancer using deep learning

Gastric cancer possesses great histological and molecular diversity, which creates obstacles for rapid and efficient diagnoses. Classic diagnoses either depend on the pathologist's judgment, which relies heavily on subjective experience, or time-consuming molecular assays for subtype diagnosis. Here, we present a deep learning (DL) system to achieve interpretable tumor differentiation grade and microsatellite instability (MSI) recognition in gastric cancer directly using hematoxylin-eosin (HE) staining whole-slide images (WSIs). WSIs from 467 patients were divided into three cohorts: the training cohort with 348 annotated WSIs, the testing cohort with 88 annotated WSIs, and the integration testing cohort with 31 original WSIs without tumor contour annotation. First, the DL models comprehensibly achieved tumor differentiation recognition with an F1 values of 0.8615 and 0.8977 for poorly differentiated adenocarcinoma (PDA) and well-differentiated adenocarcinoma (WDA) classes. Its ability to extract pathological features about the glandular structure formation, which is the key to distinguishing between PDA and WDA, increased the interpretability of the DL models. Second, the DL models achieved MSI status recognition with a patient-level accuracy of 86.36% directly from HE-stained WSIs in the testing cohort. Finally, the integrated end-to-end system achieved patient-level MSI recognition from original HE staining WSIs with an accuracy of 83.87% in the integration testing cohort with no tumor contour annotation. The proposed system, therefore, demonstrated high accuracy and interpretability, which can potentially promote the implementation of artificial intelligence healthcare.

Turning cold tumors hot: from molecular mechanisms to clinical applications

Immune checkpoint blockade (ICB) therapies have achieved clinical benefit, but most 'immune-cold' solid tumors are not responsive. The diversity of immune evasion mechanisms remains a key obstacle in turning nonresponsive 'cold' tumors into responsive 'hot' ones. Therefore, exploring the mechanisms of such transitions and tumor immunotyping can provide significant insights into designing effective therapeutic strategies against cancer. Here, we focus on the latest advances regarding local and systemic regulatory mechanisms of immune responses in cold and hot tumors. We also highlight the necessity for tumor immunotyping through the assessment of multiple immunological variables using various diagnostic techniques and biomarkers. Finally, we discuss the challenges and potential clinical applications of immunophenotyping to turn cold tumors hot, which may further guide combined immunotherapies.

Recent Applications of Artificial Intelligence from Histopathologic Image-Based Prediction of Microsatellite Instability in Solid Cancers: A Systematic Review

Cancers with high microsatellite instability (MSI-H) have a better prognosis and respond well to immunotherapy. However, MSI is not tested in all cancers because of the additional costs and time of diagnosis. Therefore, artificial intelligence (AI)-based models have been recently developed to evaluate MSI from whole slide images (WSIs). Here, we aimed to assess the current state of AI application to predict MSI based on WSIs analysis in MSI-related cancers and suggest a better study design for future studies. Studies were searched in online databases and screened by reference type, and only the full texts of eligible studies were reviewed. The included 14 studies were published between 2018 and 2021, and most of the publications were from developed countries. The commonly used dataset is The Cancer Genome Atlas dataset. Colorectal cancer (CRC) was the most common type of cancer studied, followed by endometrial, gastric, and ovarian cancers. The AI models have shown the potential to predict MSI with the highest AUC of 0.93 in the case of CRC. The relatively limited scale of datasets and lack of external validation were the limitations of most studies. Future studies with larger datasets are required to implicate AI models in routine diagnostic practice for MSI prediction.

A deep learning model and human-machine fusion for prediction of EBV-associated gastric cancer from histopathology

Epstein–Barr virus-associated gastric cancer (EBVaGC) shows a robust response to immune checkpoint inhibitors. Therefore, a cost-efficient and accessible tool is needed for discriminating EBV status in patients with gastric cancer. Here we introduce a deep convolutional neural network called EBVNet and its fusion with pathologists for predicting EBVaGC from histopathology. The EBVNet yields an averaged area under the receiver operating curve (AUROC) of 0.969 from the internal cross validation, an AUROC of 0.941 on an external dataset from multiple institutes and an AUROC of 0.895 on The Cancer Genome Atlas dataset. The human-machine fusion significantly improves the diagnostic performance of both the EBVNet and the pathologist. This finding suggests that our EBVNet could provide an innovative approach for the identification of EBVaGC and may help effectively select patients with gastric cancer for immunotherapy.

Epstein–Barr virus-associated gastric cancer shows a robust response to immune checkpoint inhibitors. Here the authors introduce a deep convolutional neural network and its fusion with pathologists for predicting it from histopathology.

Modern Machine Learning Practices in Colorectal Surgery: A Scoping Review

Objective: The use of machine learning (ML) has revolutionized every domain of medicine. Surgeons are now using ML models for disease detection and outcome prediction with high precision. ML-guided colorectal surgeries are more efficient than conventional surgical procedures. The primary aim of this paper is to provide an overview of the latest research on “ML in colorectal surgery”, with its viable applications. Methods: PubMed, Google Scholar, Medline, and Cochrane library were searched. Results: After screening, 27 articles out of 172 were eventually included. Among all of the reviewed articles, those found to fit the criteria for inclusion had exclusively focused on ML in colorectal surgery, with justified applications. We identified existing applications of ML in colorectal surgery. Additionally, we discuss the benefits, risks, and safety issues. Conclusions: A better, more sustainable, and more efficient method, with useful applications, for ML in surgery is possible if we and data scientists work together to address the drawbacks of the current approach. Potential problems related to patients’ perspectives also need to be resolved. The development of accurate technologies alone will not solve the problem of perceived unreliability from the patients’ end. Confidence can only be developed within society if more research with precise results is carried out.

Artificial intelligence for detection of microsatellite instability in colorectal cancer-a multicentric analysis of a pre-screening tool for clinical application

BACKGROUND

Microsatellite instability (MSI)/mismatch repair deficiency (dMMR) is a key genetic feature which should be tested in every patient with colorectal cancer (CRC) according to medical guidelines. Artificial intelligence (AI) methods can detect MSI/dMMR directly in routine pathology slides, but the test performance has not been systematically investigated with predefined test thresholds.

METHOD

We trained and validated AI-based MSI/dMMR detectors and evaluated predefined performance metrics using nine patient cohorts of 8343 patients across different countries and ethnicities.

RESULTS

Classifiers achieved clinical-grade performance, yielding an area under the receiver operating curve (AUROC) of up to 0.96 without using any manual annotations. Subsequently, we show that the AI system can be applied as a rule-out test: by using cohort-specific thresholds, on average 52.73% of tumors in each surgical cohort [total number of MSI/dMMR = 1020, microsatellite stable (MSS)/ proficient mismatch repair (pMMR) = 7323 patients] could be identified as MSS/pMMR with a fixed sensitivity at 95%. In an additional cohort of N = 1530 (MSI/dMMR = 211, MSS/pMMR = 1319) endoscopy biopsy samples, the system achieved an AUROC of 0.89, and the cohort-specific threshold ruled out 44.12% of tumors with a fixed sensitivity at 95%. As a more robust alternative to cohort-specific thresholds, we showed that with a fixed threshold of 0.25 for all the cohorts, we can rule-out 25.51% in surgical specimens and 6.10% in biopsies.

INTERPRETATION

When applied in a clinical setting, this means that the AI system can rule out MSI/dMMR in a quarter (with global thresholds) or half of all CRC patients (with local fine-tuning), thereby reducing cost and turnaround time for molecular profiling.

Deep Learning on Histopathological Images for Colorectal Cancer Diagnosis: A Systematic Review

Colorectal cancer (CRC) is the second most common cancer in women and the third most common in men, with an increasing incidence. Pathology diagnosis complemented with prognostic and predictive biomarker information is the first step for personalized treatment. The increased diagnostic load in the pathology laboratory, combined with the reported intra- and inter-variability in the assessment of biomarkers, has prompted the quest for reliable machine-based methods to be incorporated into the routine practice. Recently, Artificial Intelligence (AI) has made significant progress in the medical field, showing potential for clinical applications. Herein, we aim to systematically review the current research on AI in CRC image analysis. In histopathology, algorithms based on Deep Learning (DL) have the potential to assist in diagnosis, predict clinically relevant molecular phenotypes and microsatellite instability, identify histological features related to prognosis and correlated to metastasis, and assess the specific components of the tumor microenvironment.

Artificial intelligence to identify genetic alterations in conventional histopathology

Precision oncology relies on the identification of targetable molecular alterations in tumor tissues. In many tumor types, a limited set of molecular tests is currently part of standard diagnostic workflows. However, universal testing for all targetable alterations, especially rare ones, is limited by the cost and availability of molecular assays. From 2017 to 2021, multiple studies have shown that artificial intelligence (AI) methods can predict the probability of specific genetic alterations directly from conventional hematoxylin and eosin (H&E) tissue slides. Although these methods are currently less accurate than gold-standard testing (e.g. immunohistochemistry, polymerase chain reaction or next-generation sequencing), they could be used as pre-screening tools to reduce the workload of genetic analyses. In this systematic literature review, we summarize the state of the art in predicting molecular alterations from H&E using AI. We found that AI methods perform reasonably well across multiple tumor types, although few algorithms have been broadly validated. In addition, we found that genetic alterations in FGFR, IDH, PIK3CA, BRAF, TP53 and DNA repair pathways are predictable from H&E in multiple tumor types, while many other genetic alterations have rarely been investigated or were only poorly predictable. Finally, we discuss the next steps for the implementation of AI-based surrogate tests in diagnostic workflows. This article is protected by copyright. All rights reserved.

Fully automatic segmentation of abdominal aortic thrombus in pre-operative CTA images using deep convolutional neural networks

BACKGROUND

Endovascular aortic aneurysm repair (EVAR) is currently established as the first-line treatment for anatomically suitable abdominal aortic aneurysm (AAA).

OBJECTIVE

To establish a deep convolutional neural networks (DCNN) model for fully automatic segmentation intraluminal thrombosis (ILT) of abdominal aortic aneurysm (AAA) in pre-operative computed tomography angiography (CTA) images.

METHODS

We retrospectively reviewed 340 patients of AAA with ILT at our single center. The software ITKSNAP was used to draw AAA and ILT region of interests (ROIs), respectively. Image preprocessing and DCNN model build using MATLAB. Randomly divided, 80% of patients was classified as training set, 20% of patients was classified as test set. Accuracy, intersection over union (IOU), Boundary F1 (BF) Score were used to evaluate the predictive effect of the model.

RESULTS

By training in 34760-35652 CTA images (n= 204) and validation in 6968-7860 CTA images (n=68), the DCNN model achieved encouraging predictive performance in test set (n= 68, 6898 slices): Global accuracy 0.9988 ± 5.7735E-05, mean accuracy 0.9546 ± 0.0054, ILT IOU 0.8650 ± 0.0033, aortic lumen IOU 0.8595 ± 0.0085, ILT weighted IOU 0.9976 ± 0.0001, mean IOU 0.9078 ± 0.0029, mean BF Score 0.9829 ± 0.0011. Our DCNN model achieved a mean IOU of more than 90.78% for segmentation of ILT and aortic lumen. It provides a mean relative volume difference between automatic segmentation and ground truth (P> 0.05).

CONCLUSION

An end-to-end DCNN model could be used as an efficient and adjunctive tool for fully automatic segmentation of abdominal aortic thrombus in pre-operative CTA image.

A whole-slide image (WSI)-based immunohistochemical feature prediction system improves the subtyping of lung cancer

BACKGROUND

Clinically, accurate pathological diagnosis is often challenged by insufficient tissue amounts and the unaffordability of additional immunohistochemical or genetic tests; thus, there is an urgent need for a universal approach to improve the subtyping of lung cancer without the above limitations. Here we aimed to develop a deep learning system to predict the immunohistochemistry (IHC) phenotype directly from whole-slide images (WSIs) to improve the subtyping of lung cancer from surgical resection and biopsy specimens.

METHODS

A total of 1914 patients with lung cancer from three independent hospitals in China were enrolled for WSI-based immunohistochemical feature prediction system (WIFPS) development and validation.

RESULTS

The WIFPS could directly predict the IHC status of nine subtype-specific biomarkers, including CK7, TTF-1, Napsin A, CK5/6, P63, P40, CD56, Synaptophysin, and Chromogranin A, achieving average areas under the curve (AUCs) of 0.912, 0.906, and 0.888 and overall diagnostic accuracies of 0.925, 0.941, and 0.887 in the validation datasets of total, external surgical resection specimens and biopsy specimens, respectively. The histological subtyping performance of the WIFPS remained comparable with that of general pathologists (GPs), with Cohen's kappa values ranging from 0.7646 to 0.8282. Furthermore, the WIFPS could be trained to not only predict the IHC status of anaplastic lymphoma kinase (ALK), programmed death-1 (PD-1), and programmed death ligand 1 (PD-L1), but also predict EGFR and KRAS mutation status, with AUCs from 0.525 to 0.917, as detected in separate populations.

CONCLUSIONS

In this study, the WIFPS showed its proficiency as a useful complement to traditional histologic subtyping for integrated immunohistochemical spectrum prediction as well as potential in the detection of gene mutations.

Artificial Intelligence for Predicting Microsatellite Instability Based on Tumor Histomorphology: A Systematic Review

Microsatellite instability (MSI)/defective DNA mismatch repair (dMMR) is receiving more attention as a biomarker for eligibility for immune checkpoint inhibitors in advanced diseases. However, due to high costs and resource limitations, MSI/dMMR testing is not widely performed. Some attempts are in progress to predict MSI/dMMR status through histomorphological features on H&E slides using artificial intelligence (AI) technology. In this study, the potential predictive role of this new methodology was reviewed through a systematic review. Studies up to September 2021 were searched through PubMed and Embase database searches. The design and results of each study were summarized, and the risk of bias for each study was evaluated. For colorectal cancer, AI-based systems showed excellent performance with the highest standard of 0.972; for gastric and endometrial cancers they showed a relatively low but satisfactory performance, with the highest standard of 0.81 and 0.82, respectively. However, analyzing the risk of bias, most studies were evaluated at high-risk. AI-based systems showed a high potential in predicting the MSI/dMMR status of different cancer types, and particularly of colorectal cancers. Therefore, a confirmation test should be required only for the results that are positive in the AI test.

Non-Coding RNAs in Colorectal Cancer: Their Functions and Mechanisms

Colorectal cancer (CRC) is a common malignancy with high mortality. However, the molecular mechanisms underlying CRC remain unclear. Controversies over the exact functions of non-coding RNAs (ncRNAs) in the progression of CRC have been prevailing for multiple years. Recently, accumulating evidence has demonstrated the regulatory roles of ncRNAs in various human cancers, including CRC. The intracellular signaling pathways by which ncRNAs act on tumor cells have been explored, and in CRC, various studies have identified numerous dysregulated ncRNAs that serve as oncogenes or tumor suppressors in the process of tumorigenesis through diverse mechanisms. In this review, we have summarized the functions and mechanisms of ncRNAs (mainly lncRNAs, miRNAs, and circRNAs) in the tumorigenesis of CRC. We also discuss the potential applications of ncRNAs as diagnostic and prognostic tools, as well as therapeutic targets in CRC. This review details strategies that trigger the recognition of CRC-related ncRNAs, as well as the methodologies and challenges of studying these molecules, and the forthcoming clinical applications of these findings.

Deep Learning With Radiomics for Disease Diagnosis and Treatment: Challenges and Potential

The high-throughput extraction of quantitative imaging features from medical images for the purpose of radiomic analysis, i.e., radiomics in a broad sense, is a rapidly developing and emerging research field that has been attracting increasing interest, particularly in multimodality and multi-omics studies. In this context, the quantitative analysis of multidimensional data plays an essential role in assessing the spatio-temporal characteristics of different tissues and organs and their microenvironment. Herein, recent developments in this method, including manually defined features, data acquisition and preprocessing, lesion segmentation, feature extraction, feature selection and dimension reduction, statistical analysis, and model construction, are reviewed. In addition, deep learning-based techniques for automatic segmentation and radiomic analysis are being analyzed to address limitations such as rigorous workflow, manual/semi-automatic lesion annotation, and inadequate feature criteria, and multicenter validation. Furthermore, a summary of the current state-of-the-art applications of this technology in disease diagnosis, treatment response, and prognosis prediction from the perspective of radiology images, multimodality images, histopathology images, and three-dimensional dose distribution data, particularly in oncology, is presented. The potential and value of radiomics in diagnostic and therapeutic strategies are also further analyzed, and for the first time, the advances and challenges associated with dosiomics in radiotherapy are summarized, highlighting the latest progress in radiomics. Finally, a robust framework for radiomic analysis is presented and challenges and recommendations for future development are discussed, including but not limited to the factors that affect model stability (medical big data and multitype data and expert knowledge in medical), limitations of data-driven processes (reproducibility and interpretability of studies, different treatment alternatives for various institutions, and prospective researches and clinical trials), and thoughts on future directions (the capability to achieve clinical applications and open platform for radiomics analysis).

Does artificial intelligence have a role in the IVF clinic?

The success of IVF has remained stagnant for a decade. The focus of a great deal of research is to improve on the current ~30% success rate of IVF. Artificial intelligence (AI), or machines that mimic human intelligence, has been gaining traction for its potential to improve outcomes in medicine, such as cancer diagnosis from medical images. In this commentary, we discuss whether AI has the potential to improve fertility outcomes in the IVF clinic. Based on existing research, we examine the potential of adopting AI within multiple facets of an IVF cycle, including egg/sperm and embryo selection, as well as formulation of an IVF treatment regimen. We discuss both the potential benefits and concerns of the patient and clinician in adopting AI in the clinic. We outline hurdles that need to be overcome prior to implementation. We conclude that AI has an important future in improving IVF success.

Direct identification of ALK and ROS1 fusions in non-small cell lung cancer from hematoxylin and eosin-stained slides using deep learning algorithms

Anaplastic lymphoma kinase (ALK) and ROS oncogene 1 (ROS1) gene fusions are well-established key players in non-small cell lung cancer (NSCLC). Although their frequency is relatively low, their detection is important for patient care and guides therapeutic decisions. The accepted methods used for their detection are immunohistochemistry (IHC) and fluorescence in situ hybridization (FISH) assay, as well as DNA and RNA-based sequencing methodologies. These assays are expensive, time-consuming, and require technical expertise and specialized equipment as well as biological specimens that are not always available. Here we present an alternative detection method using a computer vision deep learning approach. An advanced convolutional neural network (CNN) was used to generate classifier models to detect ALK and ROS1-fusions directly from scanned hematoxylin and eosin (H&E) whole slide images prepared from NSCLC tumors of patients. A two-step training approach was applied, with an initial unsupervised training step performed on a pan-cancer sample cohort followed by a semi-supervised fine-tuning step, which supported the development of a classifier with performances equal to those accepted for diagnostic tests. Validation of the ALK/ROS1 classifier on a cohort of 72 lung cancer cases who underwent ALK and ROS1-fusion testing at the pathology department at Sheba Medical Center displayed sensitivities of 100% for both genes (six ALK-positive and two ROS1-positive cases) and specificities of 100% and 98.6% respectively for ALK and ROS1, with only one false-positive result for ROS1-alteration. These results demonstrate the potential advantages that machine learning solutions may have in the molecular pathology domain, by allowing fast, standardized, accurate, and robust biomarker detection overcoming many limitations encountered when using current techniques. The integration of such novel solutions into the routine pathology workflow can support and improve the current clinical pipeline.

Development and validation of a weakly supervised deep learning framework to predict the status of molecular pathways and key mutations in colorectal cancer from routine histology images: a retrospective study

BACKGROUND

Determining the status of molecular pathways and key mutations in colorectal cancer is crucial for optimal therapeutic decision making. We therefore aimed to develop a novel deep learning pipeline to predict the status of key molecular pathways and mutations from whole-slide images of haematoxylin and eosin-stained colorectal cancer slides as an alternative to current tests.

METHODS

In this retrospective study, we used 502 diagnostic slides of primary colorectal tumours from 499 patients in The Cancer Genome Atlas colon and rectal cancer (TCGA-CRC-DX) cohort and developed a weakly supervised deep learning framework involving three separate convolutional neural network models. Whole-slide images were divided into equally sized tiles and model 1 (ResNet18) extracted tumour tiles from non-tumour tiles. These tumour tiles were inputted into model 2 (adapted ResNet34), trained by iterative draw and rank sampling to calculate a prediction score for each tile that represented the likelihood of a tile belonging to the molecular labels of high mutation density (vs low mutation density), microsatellite instability (vs microsatellite stability), chromosomal instability (vs genomic stability), CpG island methylator phenotype (CIMP)-high (vs CIMP-low), BRAFmut (vs BRAFWT), TP53mut (vs TP53WT), and KRASWT (vs KRASmut). These scores were used to identify the top-ranked titles from each slide, and model 3 (HoVer-Net) segmented and classified the different types of cell nuclei in these tiles. We calculated the area under the convex hull of the receiver operating characteristic curve (AUROC) as a model performance measure and compared our results with those of previously published methods.

FINDINGS

Our iterative draw and rank sampling method yielded mean AUROCs for the prediction of hypermutation (0·81 [SD 0·03] vs 0·71), microsatellite instability (0·86 [0·04] vs 0·74), chromosomal instability (0·83 [0·02] vs 0·73), BRAFmut (0·79 [0·01] vs 0·66), and TP53mut (0·73 [0·02] vs 0·64) in the TCGA-CRC-DX cohort that were higher than those from previously published methods, and an AUROC for KRASmut that was similar to previously reported methods (0·60 [SD 0·04] vs 0·60). Mean AUROC for predicting CIMP-high status was 0·79 (SD 0·05). We found high proportions of tumour-infiltrating lymphocytes and necrotic tumour cells to be associated with microsatellite instability, and high proportions of tumour-infiltrating lymphocytes and a low proportion of necrotic tumour cells to be associated with hypermutation.

INTERPRETATION

After large-scale validation, our proposed algorithm for predicting clinically important mutations and molecular pathways, such as microsatellite instability, in colorectal cancer could be used to stratify patients for targeted therapies with potentially lower costs and quicker turnaround times than sequencing-based or immunohistochemistry-based approaches.

FUNDING

The UK Medical Research Council.