Multi-center evaluation of artificial intelligent imaging and clinical models for predicting neoadjuvant chemotherapy response in breast cancer

Deep learning analysis of serial digital breast tomosynthesis images in a prospective cohort of breast cancer patients who received neoadjuvant chemotherapy

PURPOSE

Different imaging tools, including digital breast tomosynthesis (DBT), are frequently used for evaluating tumor response during neoadjuvant chemotherapy (NACT). This study aimed to explore whether using artificial intelligence (AI) for serial DBT acquisitions during NACT for breast cancer can predict pathological complete response (pCR) after completion of NACT.

METHODS

A total of 149 women (mean age 53 years, pCR rate 22 %) with breast cancer treated with NACT at Skane University Hospital, Sweden, between 2014 and 2019, were prospectively included in this observational cohort study (ClinicalTrials.gov: NCT02306096). DBT images from both the cancer and contralateral healthy breasts acquired at three time points: pre-NACT, after two cycles of NACT, and after the completion of six cycles of NACT (pre-surgery) were analyzed. The deep learning AI system used to predict pCR consisted of a backbone 3D ResNet and an attention and prediction module. The GradCAM method was used to obtain insights into the model decision basis through a quantitative analysis of the importance maps on the validation set. Moreover, specific model choices were motivated by ablation studies.

RESULTS

The AI model reached an AUC of 0.83 (95% CI: 0.63-1.00) (test set). The spatial correlation of importance maps for input volumes from the same patient but at different time points was high, possibly indicating that the model focuses on the same areas during decision-making.

CONCLUSIONS

We demonstrate a high discriminative performance of our algorithm for predicting pCR/non-pCR. Availability of larger datasets would permit more comprehensive training of the models and more rigorous evaluation of their prediction performance for future patients.

Machine learning and new insights for breast cancer diagnosis

Breast cancer (BC) is the most prominent form of cancer among females all over the world. The current methods of BC detection include X-ray mammography, ultrasound, computed tomography, magnetic resonance imaging, positron emission tomography and breast thermographic techniques. More recently, machine learning (ML) tools have been increasingly employed in diagnostic medicine for its high efficiency in detection and intervention. The subsequent imaging features and mathematical analyses can then be used to generate ML models, which stratify, differentiate and detect benign and malignant breast lesions. Given its marked advantages, radiomics is a frequently used tool in recent research and clinics. Artificial neural networks and deep learning (DL) are novel forms of ML that evaluate data using computer simulation of the human brain. DL directly processes unstructured information, such as images, sounds and language, and performs precise clinical image stratification, medical record analyses and tumour diagnosis. Herein, this review thoroughly summarizes prior investigations on the application of medical images for the detection and intervention of BC using radiomics, namely DL and ML. The aim was to provide guidance to scientists regarding the use of artificial intelligence and ML in research and the clinic.

Predictive value of 18F-FDG PET/CT-based radiomics model for neoadjuvant chemotherapy efficacy in breast cancer: a multi-scanner/center study with external validation

PURPOSE

To develop and validate the predictive value of an 18F-fluorodeoxyglucose positron emission tomography/computed tomography (18F-FDG PET/CT) model for breast cancer neoadjuvant chemotherapy (NAC) efficacy based on the tumor-to-liver ratio (TLR) radiomic features and multiple data pre-processing methods.

METHODS

One hundred and ninety-three breast cancer patients from multiple centers were retrospectively included in this study. According to the endpoint of NAC, we divided the patients into pathological complete remission (pCR) and non-pCR groups. All patients underwent 18F-FDG PET/CT imaging before NAC treatment, and CT and PET images volume of interest (VOI) segmentation by manual segmentation and semi-automated absolute threshold segmentation, respectively. Then, feature extraction of VOI was performed with the pyradiomics package. A total of 630 models were created based on the source of radiomic features, the elimination of the batch effect approach, and the discretization method. The differences in data pre-processing approaches were compared and analyzed to identify the best-performing model, which was further tested by the permutation test.

RESULTS

A variety of data pre-processing methods contributed in varying degrees to the improvement of model effects. Among them, TLR radiomic features and Combat and Limma methods that eliminate batch effects could enhance the model prediction overall, and data discretization could be used as a potential method that can further optimize the model. A total of seven excellent models were selected and then based on the AUC of each model in the four test sets and their standard deviations, we selected the optimal model. The optimal model predicted AUC between 0.7 and 0.77 for the four test groups, with p-values less than 0.05 for the permutation test.

CONCLUSION

It is necessary to enhance the predictive effect of the model by eliminating confounding factors through data pre-processing. The model developed in this way is effective in predicting the efficacy of NAC for breast cancer.

Artificial intelligence-assisted selection and efficacy prediction of antineoplastic strategies for precision cancer therapy

The rapid development of artificial intelligence (AI) technologies in the context of the vast amount of collectable data obtained from high-throughput sequencing has led to an unprecedented understanding of cancer and accelerated the advent of a new era of clinical oncology with a tone of precision treatment and personalized medicine. However, the gains achieved by a variety of AI models in clinical oncology practice are far from what one would expect, and in particular, there are still many uncertainties in the selection of clinical treatment options that pose significant challenges to the application of AI in clinical oncology. In this review, we summarize emerging approaches, relevant datasets and open-source software of AI and show how to integrate them to address problems from clinical oncology and cancer research. We focus on the principles and procedures for identifying different antitumor strategies with the assistance of AI, including targeted cancer therapy, conventional cancer therapy, and cancer immunotherapy. In addition, we also highlight the current challenges and directions of AI in clinical oncology translation. Overall, we hope this article will provide researchers and clinicians with a deeper understanding of the role and implications of AI in precision cancer therapy, and help AI move more quickly into accepted cancer guidelines.

Early prediction of treatment response to neoadjuvant chemotherapy based on longitudinal ultrasound images of HER2-positive breast cancer patients by Siamese multi-task network: A multicentre, retrospective cohort study

BACKGROUND

Early prediction of treatment response to neoadjuvant chemotherapy (NACT) in patients with human epidermal growth factor receptor 2 (HER2)-positive breast cancer can facilitate timely adjustment of treatment regimens. We aimed to develop and validate a Siamese multi-task network (SMTN) for predicting pathological complete response (pCR) based on longitudinal ultrasound images at the early stage of NACT.

METHODS

In this multicentre, retrospective cohort study, a total of 393 patients with biopsy-proven HER2-positive breast cancer were retrospectively enrolled from three hospitals in china between December 16, 2013 and March 05, 2021, and allocated into a training cohort and two external validation cohorts. Patients receiving full cycles of NACT and with surgical pathological results available were eligible for inclusion. The key exclusion criteria were missing ultrasound images and/or clinicopathological characteristics. The proposed SMTN consists of two subnetworks that could be joined at multiple layers, which allowed for the integration of multi-scale features and extraction of dynamic information from longitudinal ultrasound images before and after the first /second cycles of NACT. We constructed the clinical model as a baseline using multivariable logistic regression analysis. Then the performance of SMTN was evaluated and compared with the clinical model.

FINDINGS

The training cohort, comprising 215 patients, were selected from Yunnan Cancer Hospital. The two independent external validation cohorts, comprising 95 and 83 patients, were selected from Guangdong Provincial People's Hospital, and Shanxi Cancer Hospital, respectively. The SMTN yielded an area under the receiver operating characteristic curve (AUC) values of 0.986 (95% CI: 0.977-0.995), 0.902 (95%CI: 0.856-0.948), and 0.957 (95%CI: 0.924-0.990) in the training cohort and two external validation cohorts, respectively, which were significantly higher than that those of the clinical model (AUC: 0.524-0.588, P _all < 0.05). The AUCs values of the SMTN within the anti-HER2 therapy subgroups were 0.833-0.972 in the two external validation cohorts. Moreover, 272 of 279 (97.5%) non-pCR patients (159 of 160 (99.4%), 53 of 54 (98.1%), and 60 of 65 (92.3%) in the training and two external validation cohorts, respectively) were successfully identified by the SMTN, suggesting that they could benefit from regime adjustment at the early-stage of NACT.

INTERPRETATION

The SMTN was able to predict pCR in the early-stage of NACT for HER2-positive breast cancer patients, which could guide clinicians in adjusting treatment regimes.

FUNDING

Key-Area Research and Development Program of Guangdong Province (No.2021B0101420006); National Natural Science Foundation of China (No.82071892, 82171920); Guangdong Provincial Key Laboratory of Artificial Intelligence in Medical Image Analysis and Application (No.2022B1212010011); the National Science Foundation for Young Scientists of China (No.82102019, 82001986); Project Funded by China Postdoctoral Science Foundation (No.2020M682643); the Outstanding Youth Science Foundation of Yunnan Basic Research Project (202101AW070001); Scientific research fund project of Department of Education of Yunnan Province(2022J0249). Science and technology Projects in Guangzhou (202201020001;202201010513); High-level Hospital Construction Project (DFJH201805, DFJHBF202105).

Ultrasound-based radiomics analysis for differentiating benign and malignant breast lesions: From static images to CEUS video analysis

Background

Continuous contrast-enhanced ultrasound (CEUS) video is a challenging direction for radiomics research. We aimed to evaluate machine learning (ML) approaches with radiomics combined with the XGBoost model and a convolutional neural network (CNN) for discriminating between benign and malignant lesions in CEUS videos with a duration of more than 1 min.

Methods

We gathered breast CEUS videos of 109 benign and 81 malignant tumors from two centers. Radiomics combined with the XGBoost model and a CNN was used to classify the breast lesions on the CEUS videos. The lesions were manually segmented by one radiologist. Radiomics combined with the XGBoost model was conducted with a variety of data sampling methods. The CNN used pretrained 3D residual network (ResNet) models with 18, 34, 50, and 101 layers. The machine interpretations were compared with prospective interpretations by two radiologists. Breast biopsies or pathological examinations were used as the reference standard. Areas under the receiver operating curves (AUCs) were used to compare the diagnostic performance of the models.

Results

The CNN model achieved the best AUC of 0.84 on the test cohort with the 3D-ResNet-50 model. The radiomics model obtained AUCs between 0.65 and 0.75. Radiologists 1 and 2 had AUCs of 0.75 and 0.70, respectively.

Conclusions

The 3D-ResNet-50 model was superior to the radiomics combined with the XGBoost model in classifying enhanced lesions as benign or malignant on CEUS videos. The CNN model was superior to the radiologists, and the radiomics model performance was close to the performance of the radiologists.

Delta-Radiomics Based on Dynamic Contrast-Enhanced MRI Predicts Pathologic Complete Response in Breast Cancer Patients Treated with Neoadjuvant Chemotherapy

Simple Summary

Neoadjuvant chemotherapy (NAC) followed with surgery is the standard strategy in the treatment of locally advanced breast cancer, but the individual efficacy varies. Early and accurate prediction of complete responders determines the NAC regimens and prognosis. Breast MRI has been recommended to monitor NAC response before, during, and after treatment. Radiomics has been heralded as a breakthrough in medicine and regarded to have changed the landscape of biomedical research in oncology. Delta-radiomics characterizing the change in feature values by applying radiomics to multiple time points, is a promising strategy for predicting response after NAC. In our study, the delta-radiomics model built with the change of radiomic features before and after one cycle NAC could effectively predict pathological complete response (pCR) in breast cancer. The model provides strong support for clinical decision-making at the earliest stage and helps patients benefit the most from NAC.

Abstract

Objective: To investigate the value of delta-radiomics after the first cycle of neoadjuvant chemotherapy (NAC) using dynamic contrast-enhanced (DCE) MRI for early prediction of pathological complete response (pCR) in patients with breast cancer. Methods: From September 2018 to May 2021, a total of 140 consecutive patients (training, n = 98: validation, n = 42), newly diagnosed with breast cancer who received NAC before surgery, were prospectively enrolled. All patients underwent DCE-MRI at pre-NAC (pre-) and after the first cycle (1st-) of NAC. Radiomic features were extracted from the postcontrast early, peak, and delay phases. Delta-radiomics features were computed in each contrast phases. Least absolute shrinkage and selection operator (LASSO) and a logistic regression model were used to select features and build models. The model performance was assessed by receiver operating characteristic (ROC) analysis and compared by DeLong test. Results: The delta-radiomics model based on the early phases of DCE-MRI showed a highest AUC (0.917/0.842 for training/validation cohort) compared with that using the peak and delay phases images. The delta-radiomics model outperformed the pre-radiomics model (AUC = 0.759/0.617, p = 0.011/0.047 for training/validation cohort) in early phase. Based on the optimal model, longitudinal fusion radiomic models achieved an AUC of 0.871/0.869 in training/validation cohort. Clinical-radiomics model generated good calibration and discrimination capacity with AUC 0.934 (95%CI: 0.882, 0.986)/0.864 (95%CI: 0.746, 0.982) for training and validation cohort. Delta-radiomics based on early contrast phases of DCE-MRI combined clinicopathology information could predict pCR after one cycle of NAC in patients with breast cancer.