Predicting Rectal Cancer Response to Neoadjuvant Chemoradiotherapy Using Deep Learning of Diffusion Kurtosis MRI

Transfer Learning Strategy Based on Unsupervised Learning and Ensemble Learning for Breast Cancer Molecular Subtype Prediction Using Dynamic Contrast-Enhanced MRI

BACKGROUND

Imaging-driven deep learning strategies focus on training from scratch and transfer learning. However, the performance of training from scratch is often impeded by the lack of large-scale labeled training data. Additionally, owing to the differences between source and target domains, analyzing medical image tasks satisfactorily via transfer learning based on ImageNet is difficult.

PURPOSE

To investigate two transfer learning algorithms for breast cancer molecular subtype prediction (luminal and non-luminal) based on unsupervised pre-training and ensemble learning: M_EL and B_EL, using malignant and benign datasets as the source domain, respectively.

STUDY TYPE

Retrospective.

POPULATION

Eight hundred and thirty-three female patients with histologically confirmed breast lesions (567 benign and 266 malignant cases) were selected. In the 5-fold cross-validation, the malignant cohort was randomly divided into 5 subsets to form a training set (80%) and a validation set (20%).

FIELD STRENGTH/SEQUENCE

3.0 T, dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) using T1-weighted high-resolution isotropic volume examination.

ASSESSMENT

First, three datasets acquired at different times post-contrast were preprocessed as unlabeled source domains. Second, three baseline networks corresponding to the different MRI post-contrast phases were built, optimized by a combination of mutual information maximization between high- and low-level representations and prior distribution constraints. Next, the pre-trained networks were fine-tuned on the labeled target domain. Finally, prediction results were integrated using weighted voting-based ensemble learning.

STATISTICAL TESTS

Mean accuracy, precision, specificity, and area under receiver operating characteristic curve (AUC) were obtained with 5-fold cross-validation. P < 0.05 was considered to be statistically significant.

RESULTS

Compared with a convolutional long short-term memory network, pre-trained VGG-16, VGG-19, and DenseNet-121 from ImageNet, M_EL and B_EL exhibited significantly more optimized prediction performance (specificity: 90.5% and 89.9%; accuracy: 82.6% and 81.1%; precision: 91.2% and 90.9%; AUC: 0.836 and 0.823, respectively).

DATA CONCLUSION

Transfer learning based on unsupervised pre-training may facilitate automatic prediction of breast cancer molecular subtypes.

LEVEL OF EVIDENCE

3 TECHNICAL EFFICACY: Stage 2.

Automatic segmentation of rectal tumor on diffusion-weighted images by deep learning with U-Net

Purpose

Manual delineation of a rectal tumor on a volumetric image is time‐consuming and subjective. Deep learning has been used to segment rectal tumors automatically on T2‐weighted images, but automatic segmentation on diffusion‐weighted imaging is challenged by noise, artifact, and low resolution. In this study, a volumetric U‐shaped neural network (U‐Net) is proposed to automatically segment rectal tumors on diffusion‐weighted images.

Methods

Three hundred patients of locally advanced rectal cancer were enrolled in this study and divided into a training group, a validation group, and a test group. The region of rectal tumor was delineated on the diffusion‐weighted images by experienced radiologists as the ground truth. A U‐Net was designed with a volumetric input of the diffusion‐weighted images and an output of segmentation with the same size. A semi‐automatic segmentation method was used for comparison by manually choosing a threshold of gray level and automatically selecting the largest connected region. Dice similarity coefficient (DSC) was calculated to evaluate the methods.

Results

On the test group, deep learning method (DSC = 0.675 ± 0.144, median DSC is 0.702, maximum DSC is 0.893, and minimum DSC is 0.297) showed higher segmentation accuracy than the semi‐automatic method (DSC = 0.614 ± 0.225, median DSC is 0.685, maximum DSC is 0.869, and minimum DSC is 0.047). Paired t‐test shows significant difference (T = 2.160, p = 0.035) in DSC between the deep learning method and the semi‐automatic method in the test group.

Conclusion

Volumetric U‐Net can automatically segment rectal tumor region on DWI images of locally advanced rectal cancer.

Predicting Treatment Response of Neoadjuvant Chemoradiotherapy in Locally Advanced Rectal Cancer Using Amide Proton Transfer MRI Combined With Diffusion-Weighted Imaging

Objective

To evaluate amide proton weighted (APTw) MRI combined with diffusion-weighted imaging (DWI) in predicting neoadjuvant chemoradiotherapy (NCRT) response in patients with locally advanced rectal cancer (LARC).

Methods

53 patients with LARC were enrolled in this retrospective study. MR examination including APTw MRI and DWI was performed before and after NCRT. APTw SI, ADC value, tumor size, CEA level before and after NCRT were assessed. The difference of the above parameters between before and after NCRT was calculated. The tumor regression grading (TRG) was assessed by American Joint Committee on Cancer’s Cancer Staging Manual AJCC 8th score. The Shapiro-Wilk test, paired t-test and Wilcoxon Signed Ranks test, two-sample t-test, Mann-Whitney U test and multivariate analysis were used for statistical analysis.

Results

Of the 53 patients, 19 had good responses (TRG 0-1), 34 had poor responses (TRG 2-3). After NCRT, all the rectal tumors demonstrated decreased APT values, increased ADC values, reduced tumor volumes and CEA levels (all p < 0.001). Good responders demonstrated higher pre-APT values, higher Δ APT values, lower pre- ADC values and higher Δ tumor volumes than poor responders. Pre-APT combined with pre-ADC achieved the best diagnostic performance, with AUC of 0.895 (sensitivity of 85.29%, specificity of 89.47%, p < 0.001) in predicting good response to NCRT.

Conclusion

The combination of APTw and DWI may serve as a noninvasive biomarker for evaluating and identifying response to NCRT in LARC patients.

Image-based deep learning model for predicting pathological response in rectal cancer using post-chemoradiotherapy magnetic resonance imaging

INTRODUCTION

To develop an image-based deep learning model for predicting pathological response in rectal cancer using post-chemoradiotherapy magnetic resonance (MR) imaging.

MATERIALS AND METHODS

A total of 466 patients with locally advanced rectal cancer who received preoperative chemoradiotherapy followed by surgical resection were collected from single center, among whom 113 (24.3%) were allocated to the holdout testing set. Complete response (pCR) was defined as Dworak tumor regression grade (TRG) 4, while good response (GR) was defined as TRG 3 or 4. Based on post-chemoradiotherapy T2-weighted axial MR images, two deep learning models were developed to predict pCR and GR, respectively. The prediction performance of the deep learning models was evaluated in the testing set and was compared to that of a senior radiologist and a radiation oncologist.

RESULTS

The deep learning model showed an area under the receiver operating characteristic curve, sensitivity, specificity, positive predictive value, negative predictive value, and accuracy of 0.76, 0.30, 0.96, 0.67, 0.87, and 85.0% for predicting pCR and 0.72, 0.54, 0.81, 0.60, 0.77, and 71.7% for predicting GR, respectively. The deep learning model had a superior predictive performance than the observers. Fair agreement between the ground truth and the model was shown for pCR prediction (kappa = 0.34) and GR prediction (kappa = 0.36).

CONCLUSIONS

The post-chemoradiotherapy T2-weighted axial MR image-based deep learning model showed acceptable performance in predicting pCR or GR in patients with rectal cancer, compared with human observers.

Deep Learning on Enhanced CT Images Can Predict the Muscular Invasiveness of Bladder Cancer

Background

Clinical treatment decision making of bladder cancer (BCa) relies on the absence or presence of muscle invasion and tumor staging. Deep learning (DL) is a novel technique in image analysis, but its potential for evaluating the muscular invasiveness of bladder cancer remains unclear. The purpose of this study was to develop and validate a DL model based on computed tomography (CT) images for prediction of muscle-invasive status of BCa.

Methods

A total of 441 BCa patients were retrospectively enrolled from two centers and were divided into development (n=183), tuning (n=110), internal validation (n=73) and external validation (n=75) cohorts. The model was built based on nephrographic phase images of preoperative CT urography. Receiver operating characteristic (ROC) curves were performed and the area under the ROC curve (AUC) for discrimination between muscle-invasive BCa and non-muscle-invasive BCa was calculated. The performance of the model was evaluated and compared with that of the subjective assessment by two radiologists.

Results

The DL model exhibited relatively good performance in all cohorts [AUC: 0.861 in the internal validation cohort, 0.791 in the external validation cohort] and outperformed the two radiologists. The model yielded a sensitivity of 0.733, a specificity of 0.810 in the internal validation cohort and a sensitivity of 0.710 and a specificity of 0.773 in the external validation cohort.

Conclusion

The proposed DL model based on CT images exhibited relatively good prediction ability of muscle-invasive status of BCa preoperatively, which may improve individual treatment of BCa.

MRI Evaluation of Complete Response of Locally Advanced Rectal Cancer After Neoadjuvant Therapy: Current Status and Future Trends

Complete tumor response can be achieved in a certain proportion of patients with locally advanced rectal cancer, who achieve maximal response to neoadjuvant therapy (NAT). For these patients, a watch-and-wait (WW) or nonsurgical strategy has been proposed and is becoming widely practiced in order to avoid unnecessary surgical complications. Therefore, a non-invasive, reliable diagnostic tool for accurately evaluating complete tumor response is needed. Magnetic resonance imaging (MRI) plays a crucial role in both primary staging and restaging tumor response to NAT in rectal cancer without relying on resected specimen. In recent years, numerous efforts have been made to research the value of MRI in predicting and evaluating complete response in rectal cancer. Current MRI evaluation is mainly based on morphological and functional images. Morphologic MRI yields high soft tissue resolution, multiplanar images, and provides detailed depictions of rectal cancer and its surrounding structures. Functional MRI may help to distinguish residual tumor from fibrosis, therefore improving the diagnostic performance of morphologic MRI in identifying complete tumor response. Both morphologic and functional MRI have several promising parameters that may help accurately evaluate and/or predict complete response of rectal cancer. However, these parameters still have limitations and the results remain inconsistent. Recent development of new techniques, such as textural analysis, radiomics analysis and deep learning, demonstrate great potential based on MRI-derived parameters. This article aimed to review and help better understand the strengths, limitations, and future trends of these MRI-derived methods in evaluating complete response in rectal cancer.

Predicting Treatment Response to Neoadjuvant Chemoradiotherapy in Rectal Mucinous Adenocarcinoma Using an MRI-Based Radiomics Nomogram

Objective

To build and validate an MRI-based radiomics nomogram to predict the therapeutic response to neoadjuvant chemoradiotherapy (nCRT) in rectal mucinous adenocarcinoma (RMAC).

Methods

Totally, 92 individuals with pathologically confirmed RMAC administered surgical resection upon nCRT in two different centers were assessed retrospectively (training set, n = 52, validation set, n = 40). Rectal MRI was performed pre-nCRT. Radiomics parameters were obtained from high-resolution T2-weighted images and selected to construct a radiomics signature. Then, radiomics nomogram construction integrated patient variables and the radiomics signature. The resulting radiomics nomogram was utilized to assess the tumor regression grade (TRG). Diagnostic performance was determined by generating receiver operating characteristic (ROC) curves and decision curve analysis (DCA).

Results

Six optimal features related to TRG were obtained to construct a radiomics signature. The nomogram combining the radiomics signature with age and mucin deposit outperformed the radiomics signature alone in the training (AUC, 0.950 vs 0.843, p < 0.05) and validation (AUC, 0.868 vs 0.719, p < 0.05) cohorts. DCA demonstrated a clinical utility for the radiomics nomogram model.

Conclusions

The established quantitative MRI-based radiomics nomogram is effective in predicting treatment response to neoadjuvant therapy in patients with RMAC.

Magnetic resonance imaging-based artificial intelligence model in rectal cancer

Rectal magnetic resonance imaging (MRI) is the preferred method for the diagnosis of rectal cancer as recommended by the guidelines. Rectal MRI can accurately evaluate the tumor location, tumor stage, invasion depth, extramural vascular invasion, and circumferential resection margin. We summarize the progress of research on the use of artificial intelligence (AI) in rectal cancer in recent years. AI, represented by machine learning, is being increasingly used in the medical field. The application of AI models based on high-resolution MRI in rectal cancer has been increasingly reported. In addition to staging the diagnosis and localizing radiotherapy, an increasing number of studies have reported that AI models based on high-resolution MRI can be used to predict the response to chemotherapy and prognosis of patients.

Predicting treatment response from longitudinal images using multi-task deep learning

Radiographic imaging is routinely used to evaluate treatment response in solid tumors. Current imaging response metrics do not reliably predict the underlying biological response. Here, we present a multi-task deep learning approach that allows simultaneous tumor segmentation and response prediction. We design two Siamese subnetworks that are joined at multiple layers, which enables integration of multi-scale feature representations and in-depth comparison of pre-treatment and post-treatment images. The network is trained using 2568 magnetic resonance imaging scans of 321 rectal cancer patients for predicting pathologic complete response after neoadjuvant chemoradiotherapy. In multi-institution validation, the imaging-based model achieves AUC of 0.95 (95% confidence interval: 0.91–0.98) and 0.92 (0.87–0.96) in two independent cohorts of 160 and 141 patients, respectively. When combined with blood-based tumor markers, the integrated model further improves prediction accuracy with AUC 0.97 (0.93–0.99). Our approach to capturing dynamic information in longitudinal images may be broadly used for screening, treatment response evaluation, disease monitoring, and surveillance.

Radiographic imaging is routinely used to evaluate treatment response in solid tumors. Here, the authors present a multi-task deep learning approach that allows simultaneous tumor segmentation and response prediction from longitudinal images in a multi-center study on rectal cancer.

Evaluating treatment response to neoadjuvant chemoradiotherapy in rectal cancer using various MRI-based radiomics models

Background

To validate and compare various MRI-based radiomics models to evaluate treatment response to neoadjuvant chemoradiotherapy (nCRT) of rectal cancer.

Methods

A total of 80 patients with locally advanced rectal cancer (LARC) who underwent surgical resection after nCRT were enrolled retrospectively. Rectal MR images were scanned pre- and post-nCRT. The radiomics features were extracted from T2-weighted images, then reduced separately by least absolute shrinkage and selection operator (LASSO) and principal component analysis (PCA). Four classifiers of Logistic Regression, Random Forest (RF), Decision Tree and K-nearest neighbor (KNN) models were constructed to assess the tumor regression grade (TRG) and pathologic complete response (pCR), respectively. The diagnostic performances of models were determined with leave-one-out cross-validation by generating receiver operating characteristic curves and decision curve analysis.

Results

Three features related to the TRG and 11 features related to the pCR were obtained by LASSO. Top five principal components representing a cumulative contribution of 80% to overall features were selected by PCA. For TRG, the area under the curve (AUC) of RF model was 0.943 for LASSO and 0.930 for PCA, higher than other models (P < 0.05 for both). As for pCR, the AUCs of KNN for LASSO and PCA were 0.945 and 0.712, higher than other models (P < 0.05 for both). The DCA showed that LASSO algorithm was clinically superior to PCA.

Conclusion

MRI-based radiomics models demonstrated good performance for evaluating the treatment response of LARC after nCRT and LASSO algorithm yielded more clinical benefit.

Precision medicine for adjuvant chemotherapy of resected colorectal cancer

Colorectal cancer (CRC) is the most common cancer and the second leading cause of cancer death in Japan. Surgical resection is the only curative option for localized disease. However, undetectable micrometastases remaining after curative surgery may cause disease recurrence. Adjuvant chemotherapy aims to eradicate these micrometastases to improve the cure rate. Unfortunately, few reliable prognostic and predictive markers are available that identify patients at high risk for CRC during early-stage disease. However, promising biomarkers may become available in the near future. Such biomarkers provide information for stratifying a patient's risk and for selecting the optimal treatment. Here, we provide an overview of current relevant prognostic and predictive biomarkers applicable to adjuvant treatment of early-stage CRC and focus on the future of this field.

A Deep Learning Model to Predict the Response to Neoadjuvant Chemoradiotherapy by the Pretreatment Apparent Diffusion Coefficient Images of Locally Advanced Rectal Cancer

Background and Purpose

Pretreatment prediction of the response to neoadjuvant chemoradiotherapy (NCRT) helps to determine the subsequent plans for the patients with locally advanced rectal cancer (LARC). If the good responders (GR) and non-good responders (non-GR) can be accurately predicted, they can choose to intensify the neoadjuvant chemoradiotherapy to decrease the risk of tumor progression during NCRT and increase the chance of organ preservation. Compared with radiomics methods, deep learning (DL) may adaptively extract features from the images without the need of feature definition. However, DL suffers from limited training samples and signal discrepancy among different scanners. This study aims to construct a DL model to predict GRs by training apparent diffusion coefficient (ADC) images from different scanners.

Methods

The study retrospectively recruited 700 participants, chronologically divided into a training group (n = 500) and a test group (n = 200). Deep convolutional neural networks were constructed to classify GRs and non-GRs. The networks were designed with a max-pooling layer parallelized by a center-cropping layer to extract features from both the macro and micro scale. ADC images and T2-weighted images were collected at 1.5 Tesla and 3.0 Tesla. The networks were trained by the image patches delineated by radiologists in ADC images and T2-weighted images, respectively. Pathological results were used as the ground truth. The deep learning models were evaluated on the test group and compared with the prediction by mean ADC value.

Results

Area under curve (AUC) of receiver operating characteristic (ROC) is 0.851 (95% CI: 0.789–0.914) for DL model with ADC images (DL_ADC), significantly larger (P = 0.018, Z = 2.367) than that of mean ADC with AUC = 0.723 (95% CI: 0.637–0.809). The sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) of DL_ADC model are 94.3%, 68.3%, 87.4% and 83.7%, respectively. The DL model with T2-weighted images (DL_T2) produces an AUC of 0.721 (95% CI: 0.640–0.802), significantly (P = 0.000, Z = 3.554) lower than that of DL_ADC model.

Conclusion

Deep learning model reveals the potential of pretreatment apparent diffusion coefficient images for the prediction of good responders to neoadjuvant chemoradiotherapy.