Deep learning radiomics-based prediction of distant metastasis in patients with locally advanced rectal cancer after neoadjuvant chemoradiotherapy: A multicentre study

BACKGROUND

Accurate predictions of distant metastasis (DM) in locally advanced rectal cancer (LARC) patients receiving neoadjuvant chemoradiotherapy (nCRT) are helpful in developing appropriate treatment plans. This study aimed to perform DM prediction through deep learning radiomics.

METHODS

We retrospectively sampled 235 patients receiving nCRT with the minimum 36 months' postoperative follow-up from three hospitals. Through transfer learning, a deep learning radiomic signature (DLRS) based on multiparametric magnetic resonance imaging (MRI) was constructed. A nomogram was established integrating deep MRI information and clinicopathologic factors for better prediction. Harrell's concordance index (C-index) and time-dependent receiver operating characteristic (ROC) were used as performance metrics. Furthermore, the risk of DM in patients with different response to nCRT was evaluated with the nomogram.

FINDINGS

DLRS performed well in DM prediction, with a C-index of 0·747 and an area under curve (AUC) at three years of 0·894 in the validation cohort. The performance of nomogram was better, with a C-index of 0·775. In addition, the nomogram could stratify patients with different responses to nCRT into high- and low-risk groups of DM (P < 0·05).

INTERPRETATION

MRI-based deep learning radiomics had potential in predicting the DM of LARC patients receiving nCRT and could help evaluate the risk of DM in patients who have different responses to nCRT.

FUNDING

The funding bodies that contributed to this study are listed in the Acknowledgements section.

Deep learning radiomics of ultrasonography: Identifying the risk of axillary non-sentinel lymph node involvement in primary breast cancer

Background

Completion axillary lymph node dissection is overtreatment for patients with sentinel lymph node (SLN) metastasis in whom the metastatic risk of residual non-SLN (NSLN) is low. However, the National Comprehensive Cancer Network panel posits that none of the previous studies has successfully identified such subset patients. Here, we develop a multicentre deep learning radiomics of ultrasonography model (DLRU) to predict the risk of SLN and NSLN metastasis.

Methods

In total, 937 eligible breast cancer patients with ultrasound images were enrolled from two hospitals as the training set (n = 542) and independent test set (n = 395) respectively. Using the images, we developed and validated a prediction model combined with deep learning radiomics and axillary ultrasound to sequentially identify the metastatic risk of SLN and NSLN, thereby, classifying patients to relevant axillary management groups.

Findings

In the test set, the DLRU yields the best performance in identifying patients with metastatic disease in SLNs (sensitivity=98.4%, 95% CI 96.6–100) and NSLNs (sensitivity=98.4%, 95% CI 95.6–99.9). The DLRU also accurately stratifies patients without metastasis in SLN or NSLN into the corresponding low-risk (LR)-SLN and high-risk (HR)-SLN&LR-NSLN category with the negative predictive value of 97% (95% CI 94.2–100) and 91.7% (95% CI 88.8–97.9), respectively. Moreover, compared with the current clinical management, DLRU appropriately assigned 51% (39.6%/77.4%) of overtreated patients in the entire study cohort into the LR group, perhaps avoiding overtreatment.

Interpretation

The performance of the DLRU indicates that it may offer a simple preoperative tool to promote personalized axillary management of breast cancer.

Funding

The National Nature Science Foundation of China; The National Outstanding Youth Science Fund Project of National Natural Science Foundation of China; The Scientific research project of Heilongjiang Health Committee; The Postgraduate Research &Practice Innovation Program of Harbin Medical University.

Semi-supervised GAN-based Radiomics Model for Data Augmentation in Breast Ultrasound Mass Classification

BACKGROUND AND OBJECTIVE

The capability of deep learning radiomics (DLR) to extract high-level medical imaging features has promoted the use of computer-aided diagnosis of breast mass detected on ultrasound. Recently, generative adversarial network (GAN) has aided in tackling a general issue in DLR, i.e., obtaining a sufficient number of medical images. However, GAN methods require a pair of input and labeled images, which require an exhaustive human annotation process that is very time-consuming. The aim of this paper is to develop a radiomics model based on a semi-supervised GAN method to perform data augmentation in breast ultrasound images.

METHODS

A total of 1447 ultrasound images, including 767 benign masses and 680 malignant masses were acquired from a tertiary hospital. A semi-supervised GAN model was developed to augment the breast ultrasound images. The synthesized images were subsequently used to classify breast masses using a convolutional neural network (CNN). The model was validated using a 5-fold cross-validation method.

RESULTS

The proposed GAN architecture generated high-quality breast ultrasound images, verified by two experienced radiologists. The improved performance of semi-supervised learning increased the quality of the synthetic data produced in comparison to the baseline method. We achieved more accurate breast mass classification results (accuracy 90.41%, sensitivity 87.94%, specificity 85.86%) with our synthetic data augmentation compared to other state-of-the-art methods.

CONCLUSION

The proposed radiomics model has demonstrated a promising potential to synthesize and classify breast masses on ultrasound in a semi-supervised manner.

Use of deep learning-based radiomics to differentiate Parkinson's disease patients from normal controls: a study based on [18F]FDG PET imaging

OBJECTIVES

We proposed a novel deep learning-based radiomics (DLR) model to diagnose Parkinson's disease (PD) based on [¹⁸F]fluorodeoxyglucose (FDG) PET images.

METHODS

In this two-center study, 255 normal controls (NCs) and 103 PD patients were enrolled from Huashan Hospital, China; 26 NCs and 22 PD patients were enrolled as a separate test group from Wuxi 904 Hospital, China. The proposed DLR model consisted of a convolutional neural network-based feature encoder and a support vector machine (SVM) model-based classifier. The DLR model was trained and validated in the Huashan cohort and tested in the Wuxi cohort, and accuracy, sensitivity, specificity and receiver operator characteristic (ROC) curve graphs were used to describe the model's performance. Comparative experiments were performed based on four other models including the scale model, radiomics model, standard uptake value ratio (SUVR) model and DLR model.

RESULTS

The DLR model demonstrated superiority in differentiating PD patients and NCs in comparison to other models, with an accuracy of 95.17% [90.35%, 98.13%] (95% confidence intervals, CI) in the Huashan cohort. Moreover, the DLR model also demonstrated greater performance in diagnosing PD early than routine methods, with an accuracy of 85.58% [78.60%, 91.57%] in the Huashan cohort.

CONCLUSIONS

We developed a DLR model based on [¹⁸F]FDG PET images that showed good performance in the noninvasive, individualized prediction of PD and was superior to traditional handcrafted methods. This model has the potential to guide and facilitate clinical diagnosis and contribute to the development of precision treatment.

KEY POINTS

The DLR method on [¹⁸F]FDG PET images helps clinicians to diagnose PD and PD subgroups from normal controls. A prospective two-center study showed that the DLR method provides greater diagnostic accuracy.

Deep learning radiomics for prediction of axillary lymph node metastasis in patients with clinical stage T1-2 breast cancer

Background

This study investigates whether deep learning radiomics of conventional ultrasound images can predict preoperative axillary lymph node (ALN) status in patients with clinical stages T1-2 breast cancer (BC).

Methods

This study retrospectively analyzed the preoperative ultrasound data of 892 patients with BC, who were classified into training (n=535), validation (n=178), and test (n=179) cohorts. Linear combinations of the selected features were weighted by their coefficients to obtain the predicted score. Then, deep learning radiomic features were extracted from the ultrasound images to evaluate the ALN status. Receiver-operating characteristic curves were drawn, followed by the calculation of the area under the curve (AUC) to assess the accuracy of the prediction model in predicting axillary lymph node metastasis (ALNM) in the three cohorts.

Results

Deep learning radiomics combined with radiomics and clinical parameters was the optimal diagnostic predictor of the ALN status in the absence and presence of ALNM, with the AUC of 0.920 (95% confidence interval: 0.872 and 0.968, respectively). Additionally, this combination could also differentiate low-load ALNM [N + (1-2)] from heavy-load ALNM with ≥3 positive nodes [N + (≥3)] in the test cohort, with the AUC of 0.819 (95% confidence interval: 0.568 and 1.00, respectively).

Conclusions

Conclusively, deep learning radiomics of ultrasound images is a non-invasive approach to predicting preoperative ALNM in BC.

Development of a deep learning radiomics model combining lumbar CT, multi-sequence MRI, and clinical data to predict high-risk cage subsidence after lumbar fusion: a retrospective multicenter study

BACKGROUND

To develop and validate a model that integrates clinical data, deep learning radiomics, and radiomic features to predict high-risk patients for cage subsidence (CS) after lumbar fusion.

METHODS

This study analyzed preoperative CT and MRI data from 305 patients undergoing lumbar fusion surgery from three centers. Using a deep learning model based on 3D vision transformations, the data were divided the dataset into training (n = 214), validation (n = 61), and test (n = 30) groups. Feature selection was performed using LASSO regression, followed by the development of a logistic regression model. The predictive ability of the model was assessed using various machine learning algorithms, and a combined clinical model was also established.

RESULTS

Ultimately, 11 traditional radiomic features, 5 deep learning radiomic features, and 1 clinical feature were selected. The combined model demonstrated strong predictive performance, with area under the curve (AUC) values of 0.941, 0.832, and 0.935 for the training, validation, and test groups, respectively. Notably, our model outperformed predictions made by two experienced surgeons.

CONCLUSIONS

This study developed a robust predictive model that integrates clinical features and imaging data to identify high-risk patients for CS following lumbar fusion. This model has the potential to improve clinical decision-making and reduce the need for revision surgeries, easing the burden on healthcare systems.

Deep learning radiomics on shear wave elastography and b-mode ultrasound videos of diaphragm for weaning outcome prediction

PURPOSE

We proposed an automatic method based on deep learning radiomics (DLR) on shear wave elastography (SWE) and B-mode ultrasound videos of diaphragm for two classification tasks, one for differentiation between the control and patient groups, and the other for weaning outcome prediction.

MATERIALS AND METHODS

We included a total of 581 SWE and B-mode ultrasound videos, of which 466 were from the control group of 179 normal subjects, and 115 were from the patient group of 35 mechanically ventilated subjects in the intensive care unit (ICU). Among the patient group, 17 subjects successfully weaned and 18 failed. The deep neural network of U-Net was utilized to automatically segment diaphragm regions in dual-modal videos of SWE and B-mode. High-throughput radiomics features were then extracted, the statistical test and least absolute shrinkage and selection operator (LASSO) were applied for feature dimension reduction. The optimal classification models for the two tasks were established using the support vector machine (SVM).

RESULTS

The automatic segmentation model achieved Dice score of 87.89 %. A total of 4524 radiomics features were extracted, 10 and 20 important features were left after feature dimension reduction for constructing the two classification models. The best areas under receiver operating characteristic curves of the two models reached 84.01 % and 94.37 %, respectively.

CONCLUSIONS

Our proposed DLR methods are innovative for automatic segmentation of diaphragm regions in SWE and B-mode videos and deep mining of high-throughput radiomics features from dual-modal images. The approaches have been proved to be effective for prediction of weaning outcomes.

Ultrasound-based deep learning radiomics nomogram for differentiating mass mastitis from invasive breast cancer

BACKGROUND

The purpose of this study is to develop and validate the potential value of the deep learning radiomics nomogram (DLRN) based on ultrasound to differentiate mass mastitis (MM) and invasive breast cancer (IBC).

METHODS

50 cases of MM and 180 cases of IBC with ultrasound Breast Imaging Reporting and Data System 4 category were recruited (training cohort, n = 161, validation cohort, n = 69). Based on PyRadiomics and ResNet50 extractors, radiomics and deep learning features were extracted, respectively. Based on supervised machine learning methods such as logistic regression, random forest, and support vector machine, as well as unsupervised machine learning methods using K-means clustering analysis, the differences in features between MM and IBC were analyzed to develop DLRN. The performance of DLRN had been evaluated by receiver operating characteristic curve, calibration, and clinical practicality.

RESULTS

Supervised machine learning results showed that compared with radiomics models, especially random forest models, deep learning models were better at recognizing MM and IBC. The area under the curve (AUC) of the validation cohort was 0.84, the accuracy was 0.83, the sensitivity was 0.73, and the specificity was 0.83. Compared to radiomics or deep learning models, DLRN even further improved discrimination ability (AUC of 0.90 and 0.90, accuracy of 0.83 and 0.88 for training and validation cohorts), which had better clinical benefits and good calibratability. In addition, the information heterogeneity of deep learning features in MM and IBC was validated again through unsupervised machine learning clustering analysis, indicating that MM had a unique features phenotype.

CONCLUSION

The DLRN developed based on radiomics and deep learning features of ultrasound images has potential clinical value in effectively distinguishing between MM and IBC. DLRN breaks through visual limitations and quantifies more image information related to MM based on computers, further utilizing machine learning to effectively utilize this information for clinical decision-making. As DLRN becomes an autonomous screening system, it will improve the recognition rate of MM in grassroots hospitals and reduce the possibility of incorrect treatment and overtreatment.

Gd-EOB-DTPA-enhanced MRI radiomics and deep learning models to predict microvascular invasion in hepatocellular carcinoma: a multicenter study

BACKGROUND

Microvascular invasion (MVI) is an important risk factor for early postoperative recurrence of hepatocellular carcinoma (HCC). Based on gadolinium-ethoxybenzyl-diethylenetriamine pentaacetic acid (Gd-EOB-DTPA)-enhanced magnetic resonance imaging (MRI) images, we developed a novel radiomics model. It combined bi-regional features and two machine learning algorithms. The aim of this study was to validate its potential value for preoperative prediction of MVI.

METHODS

This retrospective study included 304 HCC patients (training cohort, 216 patients; testing cohort, 88 patients) from three hospitals. Intratumoral and peritumoral volumes of interest were delineated in arterial phase, portal venous phase, and hepatobiliary phase images. Conventional radiomics (CR) and deep learning radiomics (DLR) features were extracted based on FeAture Explorer software and the 3D ResNet-18 extractor, respectively. Clinical variables were selected using univariate and multivariate analyses. Clinical, CR, DLR, CR-DLR, and clinical-radiomics (Clin-R) models were built using support vector machines. The predictive capacity of the models was assessed by the area under the receiver operating characteristic curve (AUC), accuracy, sensitivity, and specificity.

RESULTS

The bi-regional CR-DLR model showed more gains and gave better predictive performance than the single-regional models or single-machine learning models. Its AUC, accuracy, sensitivity, and specificity were 0.844, 76.9%, 87.8%, and 69.1% in the training cohort and 0.740, 73.9%, 50%, and 84.5% in the testing cohort. Alpha-fetoprotein (odds ratio was 0.32) and maximum tumor diameter (odds ratio was 1.270) were independent predictors. The AUCs of the clinical model and the Clin-R model were 0.655 and 0.672, respectively. There was no significant difference in the AUCs between all the models (P > 0.005).

CONCLUSION

Based on Gd-EOB-DTPA-enhanced MRI images, we focused on developing a radiomics model that combines bi-regional features and two machine learning algorithms (CR and DLR). The application of the new model will provide a more accurate and non-invasive diagnostic solution for medical imaging. It will provide valuable information for clinical personalized treatment, thereby improving patient prognosis.

CLINICAL TRIAL NUMBER

Not applicable.

Multiscale deep learning radiomics for predicting recurrence-free survival in pancreatic cancer: A multicenter study

PURPOSE

This multicenter study aimed to develop and validate a multiscale deep learning radiomics nomogram for predicting recurrence-free survival (RFS) in patients with pancreatic ductal adenocarcinoma (PDAC).

MATERIALS AND METHODS

A total of 469 PDAC patients from four hospitals were divided into training and test sets. Handcrafted radiomics and deep learning (DL) features were extracted from optimal regions of interest, encompassing both intratumoral and peritumoral areas. Univariate Cox regression, LASSO regression, and multivariate Cox regression selected features for three image signatures (intratumoral, peritumoral radiomics, and DL). A multiscale nomogram was constructed and validated against the 8th AJCC staging system.

RESULTS

The 4 mm peritumoral VOI yielded the best radiomics prediction, while a 15 mm expansion was optimal for deep learning. The multiscale nomogram demonstrated a C-index of 0.82 (95 % CI: 0.78-0.85) in the training set and 0.70 (95 % CI: 0.64-0.76) in the external test 1 (high-volume hospital), with the external test 2 (low-volume hospital) showing a C-index of 0.78 (95 % CI: 0.65-0.91). These outperformed the AJCC system's C-index (0.54-0.57). The area under the curve (AUC) for recurrence prediction at 0.5, 1, and 2 years was 0.89, 0.94, and 0.89 in the training set, with AUC values ranging from 0.75 to 0.97 in the external validation sets, consistently surpassing the AJCC system across all sets. Kaplan-Meier analysis confirmed significant differences in prognosis between high- and low-risk groups (P < 0.01 across all cohorts).

CONCLUSION

The multiscale nomogram effectively stratifies recurrence risk in PDAC patients, enhancing presurgical clinical decision-making and potentially improving patient outcomes.