Prior to Initiation of Chemotherapy, Can We Predict Breast Tumor Response? Deep Learning Convolutional Neural Networks Approach Using a Breast MRI Tumor Dataset

Deep Learning in Breast Cancer Imaging: A Decade of Progress and Future Directions

Breast cancer has reached the highest incidence rate worldwide among all malignancies since 2020. Breast imaging plays a significant role in early diagnosis and intervention to improve the outcome of breast cancer patients. In the past decade, deep learning has shown remarkable progress in breast cancer imaging analysis, holding great promise in interpreting the rich information and complex context of breast imaging modalities. Considering the rapid improvement in deep learning technology and the increasing severity of breast cancer, it is critical to summarize past progress and identify future challenges to be addressed. This paper provides an extensive review of deep learning-based breast cancer imaging research, covering studies on mammograms, ultrasound, magnetic resonance imaging, and digital pathology images over the past decade. The major deep learning methods and applications on imaging-based screening, diagnosis, treatment response prediction, and prognosis are elaborated and discussed. Drawn from the findings of this survey, we present a comprehensive discussion of the challenges and potential avenues for future research in deep learning-based breast cancer imaging.

Role of artificial intelligence in digital pathology for gynecological cancers

The diagnosis of cancer is typically based on histopathological sections or biopsies on glass slides. Artificial intelligence (AI) approaches have greatly enhanced our ability to extract quantitative information from digital histopathology images as a rapid growth in oncology data. Gynecological cancers are major diseases affecting women's health worldwide. They are characterized by high mortality and poor prognosis, underscoring the critical importance of early detection, treatment, and identification of prognostic factors. This review highlights the various clinical applications of AI in gynecological cancers using digitized histopathology slides. Particularly, deep learning models have shown promise in accurately diagnosing, classifying histopathological subtypes, and predicting treatment response and prognosis. Furthermore, the integration with transcriptomics, proteomics, and other multi-omics techniques can provide valuable insights into the molecular features of diseases. Despite the considerable potential of AI, substantial challenges remain. Further improvements in data acquisition and model optimization are required, and the exploration of broader clinical applications, such as the biomarker discovery, need to be explored.

AI Applications to Breast MRI: Today and Tomorrow

In breast imaging, there is an unrelenting increase in the demand for breast imaging services, partly explained by continuous expanding imaging indications in breast diagnosis and treatment. As the human workforce providing these services is not growing at the same rate, the implementation of artificial intelligence (AI) in breast imaging has gained significant momentum to maximize workflow efficiency and increase productivity while concurrently improving diagnostic accuracy and patient outcomes. Thus far, the implementation of AI in breast imaging is at the most advanced stage with mammography and digital breast tomosynthesis techniques, followed by ultrasound, whereas the implementation of AI in breast magnetic resonance imaging (MRI) is not moving along as rapidly due to the complexity of MRI examinations and fewer available dataset. Nevertheless, there is persisting interest in AI-enhanced breast MRI applications, even as the use of and indications of breast MRI continue to expand. This review presents an overview of the basic concepts of AI imaging analysis and subsequently reviews the use cases for AI-enhanced MRI interpretation, that is, breast MRI triaging and lesion detection, lesion classification, prediction of treatment response, risk assessment, and image quality. Finally, it provides an outlook on the barriers and facilitators for the adoption of AI in breast MRI. LEVEL OF EVIDENCE: 5 TECHNICAL EFFICACY: Stage 6.

Advancing personalized oncology: a systematic review on the integration of artificial intelligence in monitoring neoadjuvant treatment for breast cancer patients

PURPOSE

Despite suffering from the same disease, each patient exhibits a distinct microbiological profile and variable reactivity to prescribed treatments. Most doctors typically use a standardized treatment approach for all patients suffering from a specific disease. Consequently, the challenge lies in the effectiveness of this standardized treatment and in adapting it to each individual patient. Personalized medicine is an emerging field in which doctors use diagnostic tests to identify the most effective medical treatments for each patient. Prognosis, disease monitoring, and treatment planning rely on manual, error-prone methods. Artificial intelligence (AI) uses predictive techniques capable of automating prognostic and monitoring processes, thus reducing the error rate associated with conventional methods.

METHODS

This paper conducts an analysis of current literature, encompassing the period from January 2015 to 2023, based on Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA).

RESULTS

In assessing 25 pertinent studies concerning predicting neoadjuvant treatment (NAT) response in breast cancer (BC) patients, the studies explored various imaging modalities (Magnetic Resonance Imaging, Ultrasound, etc.), evaluating results based on accuracy, sensitivity, and area under the curve. Additionally, the technologies employed, such as machine learning (ML), deep learning (DL), statistics, and hybrid models, were scrutinized. The presentation of datasets used for predicting complete pathological response (PCR) was also considered.

CONCLUSION

This paper seeks to unveil crucial insights into the application of AI techniques in personalized oncology, particularly in the monitoring and prediction of responses to NAT for BC patients. Finally, the authors suggest avenues for future research into AI-based monitoring systems.

Deep-learning based discrimination of pathologic complete response using MRI in HER2-positive and triple-negative breast cancer

Distinguishing between pathologic complete response and residual cancer after neoadjuvant chemotherapy (NAC) is crucial for treatment decisions, but the current imaging methods face challenges. To address this, we developed deep-learning models using post-NAC dynamic contrast-enhanced MRI and clinical data. A total of 852 women with human epidermal growth factor receptor 2 (HER2)-positive or triple-negative breast cancer were randomly divided into a training set (n = 724) and a validation set (n = 128). A 3D convolutional neural network model was trained on the training set and validated independently. The main models were developed using cropped MRI images, but models using uncropped whole images were also explored. The delayed-phase model demonstrated superior performance compared to the early-phase model (area under the receiver operating characteristic curve [AUC] = 0.74 vs. 0.69, P = 0.013) and the combined model integrating multiple dynamic phases and clinical data (AUC = 0.74 vs. 0.70, P = 0.022). Deep-learning models using uncropped whole images exhibited inferior performance, with AUCs ranging from 0.45 to 0.54. Further refinement and external validation are necessary for enhanced accuracy.

Use of ultrasound and MRI to stage the axilla for breast cancer before and after neoadjuvant chemotherapy prior to targeted sentinel lymphadenectomy

PURPOSE

Prior data from this Center demonstrated that for patients who had biopsy-proven axillary metastases, were ycN0 after neoadjuvant chemotherapy (NAC), and had a wire-directed (targeted) sentinel lymphadenectomy (WD-SLND), 60% were node negative. The hypothesis of this study was that results of axillary imaging either before or after NAC would be predictive of final pathologic status after WD-SLND.

METHODS

For patients treated with NAC between 2015 and 2023, ultrasound and MRI images of the axilla were retrospectively reviewed by radiologists specializing in breast imaging, who were blinded to the surgical and pathology results.

RESULTS

Of 113 patients who fit the clinical criteria, 66 (58%) were ypN0 at WD-SLND and 34 (30%) had a pathologic complete response to NAC. There was no correlation between the number of abnormal lymph nodes on pre-NAC ultrasound or MRI imaging and the final pathologic status of the lymph nodes. The positive predictive value (PPV) of abnormal post-NAC axillary imaging was 48% for ultrasound and 53% for MRI. The negative predictive value (NPV) for normal post-NAC axillary imaging was 67% for ultrasound and 68% for MRI.

CONCLUSION

The results of axillary imaging were not adequate to identify lymph nodes after NAC that were persistently pathologically node positive or those which had become pathologically node negative.

Explainable 3D CNN based on baseline breast DCE-MRI to give an early prediction of pathological complete response to neoadjuvant chemotherapy

BACKGROUND

So far, baseline Dynamic Contrast-Enhanced Magnetic Resonance Imaging (DCE-MRI) has played a key role for the application of sophisticated artificial intelligence-based models using Convolutional Neural Networks (CNNs) to extract quantitative imaging information as earlier indicators of pathological Complete Response (pCR) achievement in breast cancer patients treated with neoadjuvant chemotherapy (NAC). However, these models did not exploit the DCE-MRI exams in their full geometry as 3D volume but analysed only few individual slices independently, thus neglecting the depth information.

METHOD

This study aimed to develop an explainable 3D CNN, which fulfilled the task of pCR prediction before the beginning of NAC, by leveraging the 3D information of post-contrast baseline breast DCE-MRI exams. Specifically, for each patient, the network took in input a 3D sequence containing the tumor region, which was previously automatically identified along the DCE-MRI exam. A visual explanation of the decision-making process of the network was also provided.

RESULTS

To the best of our knowledge, our proposal is competitive than other models in the field, which made use of imaging data alone, reaching a median AUC value of 81.8%, 95%CI [75.3%; 88.3%], a median accuracy value of 78.7%, 95%CI [74.8%; 82.5%], a median sensitivity value of 69.8%, 95%CI [59.6%; 79.9%] and a median specificity value of 83.3%, 95%CI [82.6%; 84.0%], respectively. The median and CIs were computed according to a 10-fold cross-validation scheme for 5 rounds.

CONCLUSION

Finally, this proposal holds high potential to support clinicians on non-invasively early pursuing or changing patient-centric NAC pathways.

Artificial Intelligence-Enhanced Breast MRI: Applications in Breast Cancer Primary Treatment Response Assessment and Prediction

ABSTRACT

Primary systemic therapy (PST) is the treatment of choice in patients with locally advanced breast cancer and is nowadays also often used in patients with early-stage breast cancer. Although imaging remains pivotal to assess response to PST accurately, the use of imaging to predict response to PST has the potential to not only better prognostication but also allow the de-escalation or omission of potentially toxic treatment with undesirable adverse effects, the accelerated implementation of new targeted therapies, and the mitigation of surgical delays in selected patients. In response to the limited ability of radiologists to predict response to PST via qualitative, subjective assessments of tumors on magnetic resonance imaging (MRI), artificial intelligence-enhanced MRI with classical machine learning, and in more recent times, deep learning, have been used with promising results to predict response, both before the start of PST and in the early stages of treatment. This review provides an overview of the current applications of artificial intelligence to MRI in assessing and predicting response to PST, and discusses the challenges and limitations of their clinical implementation.

Deep Learning-Based Segmentation of Locally Advanced Breast Cancer on MRI in Relation to Residual Cancer Burden: A Multi-Institutional Cohort Study

BACKGROUND

While several methods have been proposed for automated assessment of breast-cancer response to neoadjuvant chemotherapy on breast MRI, limited information is available about their performance across multiple institutions.

PURPOSE

To assess the value and robustness of deep learning-derived volumes of locally advanced breast cancer (LABC) on MRI to infer the presence of residual disease after neoadjuvant chemotherapy.

STUDY TYPE

Retrospective.

SUBJECTS

Training cohort: 102 consecutive female patients with LABC scheduled for neoadjuvant chemotherapy (NAC) from a single institution (age: 25-73 years). Independent testing cohort: 55 consecutive female patients with LABC from four institutions (age: 25-72 years).

FIELD STRENGTH/SEQUENCE

Training cohort: single vendor 1.5 T or 3.0 T. Testing cohort: multivendor 3.0 T. Gradient echo dynamic contrast-enhanced sequences.

ASSESSMENT

A convolutional neural network (nnU-Net) was trained to segment LABC. Based on resulting tumor volumes, an extremely randomized tree model was trained to assess residual cancer burden (RCB)-0/I vs. RCB-II/III. An independent model was developed using functional tumor volume (FTV). Models were tested on an independent testing cohort and response assessment performance and robustness across multiple institutions were assessed.

STATISTICAL TESTS

The receiver operating characteristic (ROC) was used to calculate the area under the ROC curve (AUC). DeLong's method was used to compare AUCs. Correlations were calculated using Pearson's method. P values <0.05 were considered significant.

RESULTS

Automated segmentation resulted in a median (interquartile range [IQR]) Dice score of 0.87 (0.62-0.93), with similar volumetric measurements (R = 0.95, P < 0.05). Automated volumetric measurements were significantly correlated with FTV (R = 0.80). Tumor volume-derived from deep learning of DCE-MRI was associated with RCB, yielding an AUC of 0.76 to discriminate between RCB-0/I and RCB-II/III, performing similar to the FTV-based model (AUC = 0.77, P = 0.66). Performance was comparable across institutions (IQR AUC: 0.71-0.84).

DATA CONCLUSION

Deep learning-based segmentation estimates changes in tumor load on DCE-MRI that are associated with RCB after NAC and is robust against variations between institutions.

EVIDENCE LEVEL

2.

TECHNICAL EFFICACY

Stage 4.

Deep Learning for Fully Automatic Tumor Segmentation on Serially Acquired Dynamic Contrast-Enhanced MRI Images of Triple-Negative Breast Cancer

Accurate tumor segmentation is required for quantitative image analyses, which are increasingly used for evaluation of tumors. We developed a fully automated and high-performance segmentation model of triple-negative breast cancer using a self-configurable deep learning framework and a large set of dynamic contrast-enhanced MRI images acquired serially over the patients' treatment course. Among all models, the top-performing one that was trained with the images across different time points of a treatment course yielded a Dice similarity coefficient of 93% and a sensitivity of 96% on baseline images. The top-performing model also produced accurate tumor size measurements, which is valuable for practical clinical applications.

Deep Learning Model Based on Dual-Modal Ultrasound and Molecular Data for Predicting Response to Neoadjuvant Chemotherapy in Breast Cancer

RATIONALE AND OBJECTIVES

To carry out radiomics analysis/deep convolutional neural network (CNN) based on B-mode ultrasound (BUS) and shear wave elastography (SWE) to predict response to neoadjuvant chemotherapy (NAC) in breast cancer patients.

MATERIALS AND METHODS

In this prospective study, 255 breast cancer patients who received NAC between September 2016 and December 2021 were included. Radiomics models were designed using a support vector machine classifier based on US images obtained before treatment, including BUS and SWE. And CNN models also were developed using ResNet architecture. The final predictive model was developed by combining the dual-modal US and independently associated clinicopathologic characteristics. The predictive performances of the models were assessed with five-fold cross-validation.

RESULTS

Pretreatment SWE performed better than BUS in predicting the response to NAC for breast cancer for both the CNN and radiomics models (P < 0.001). The predictive results of the CNN models were significantly better than the radiomics models, with AUCs of 0.72 versus 0.69 for BUS and 0.80 versus 0.77 for SWE, respectively (P = 0.003). The CNN model based on the dual-modal US and molecular data exhibited outstanding performance in predicting NAC response, with an accuracy of 83.60% ± 2.63%, a sensitivity of 87.76% ± 6.44%, and a specificity of 77.45% ± 4.38%.

CONCLUSION

The pretreatment CNN model based on the dual-modal US and molecular data achieved excellent performance for predicting the response to chemotherapy in breast cancer. Therefore, this model has the potential to serve as a non-invasive objective biomarker to predict NAC response and aid clinicians with individual treatments.

Multimodal Spatiotemporal Deep Learning Framework to Predict Response of Breast Cancer to Neoadjuvant Systemic Therapy

Current approaches to breast cancer therapy include neoadjuvant systemic therapy (NST). The efficacy of NST is measured by pathologic complete response (pCR). A patient who attains pCR has significantly enhanced disease-free survival progress. The accurate prediction of pCR in response to a given treatment regimen could increase the likelihood of achieving pCR and prevent toxicities caused by treatments that are not effective. Th early prediction of response to NST can increase the likelihood of survival and help with decisions regarding breast-conserving surgery. An automated NST prediction framework that is able to precisely predict which patient undergoing NST will achieve a pathological complete response (pCR) at an early stage of treatment is needed. Here, we propose an end-to-end efficient multimodal spatiotemporal deep learning framework (deep-NST) framework to predict the outcome of NST prior or at an early stage of treatment. The deep-NST model incorporates imaging data captured at different timestamps of NST regimens, a tumor's molecular data, and a patient's demographic data. The efficacy of the proposed work is validated on the publicly available ISPY-1 dataset, in terms of accuracy, area under the curve (AUC), and computational complexity. In addition, seven ablation experiments were carried out to evaluate the impact of each design module in the proposed work. The experimental results show that the proposed framework performs significantly better than other recent methods.

Predicting pathological complete response to neoadjuvant systemic therapy for triple-negative breast cancers using deep learning on multiparametric MRIs

We trained and validated a deep learning model that can predict the treatment response to neoadjuvant systemic therapy (NAST) for patients with triple negative breast cancer (TNBC). Dynamic contrast enhanced (DCE) MRI and diffusion-weighted imaging (DWI) of the pre-treatment (baseline) and after four cycles (C4) of doxorubicin/cyclophosphamide treatment were used as inputs to the model for prediction of pathologic complete response (pCR). Based on the standard pCR definition that includes disease status in either breast or axilla, the model achieved areas under the receiver operating characteristic curves (AUCs) of 0.96 ± 0.05, 0.78 ± 0.09, 0.88 ± 0.02, and 0.76 ± 0.03, for the training, validation, testing, and prospective testing groups, respectively. For the pCR status of breast only, the retrained model achieved prediction AUCs of 0.97 ± 0.04, 0.82 ± 0.10, 0.86 ± 0.03, and 0.83 ± 0.02, for the training, validation, testing, and prospective testing groups, respectively. Thus, the developed deep learning model is highly promising for predicting the treatment response to NAST of TNBC.Clinical Relevance- Deep learning based on serial and multiparametric MRIs can potentially distinguish TNBC patients with pCR from non-pCR at the early stage of neoadjuvant systemic therapy, potentially enabling more personalized treatment of TNBC patients.

Predicting pathological complete response based on weakly and semi-supervised joint learning from breast cancer MRI

Neoadjuvant chemotherapy (NAC) is the standard treatment for breast cancer patients. Patients achieving complete pathological response (pCR) after NAC usually have a good prognosis. However, automatic pCR prediction has been a challenging problem due to lacking well annotations in 3D MRI. Thus far, unifying different annotation information to predict the tumor's early response to NAC has not been systematically addressed. This paper proposes a weakly and semi-supervised joint learning method that integrates attentional features from multi-parametric MRI with radiomic features for predicting pCR to NAC in breast cancer patients. The attention-based multi-instance learning (MIL) is first developed to generate informative MRI bag-level features and mine key instances. The mean-teacher framework is then employed to segment tumor regions in a semi-supervised setting for extracting radiomic features. We perform experiments on 442 patients' data and show that our method achieves an AUC value of 0.85 in pCR prediction, which is superior to comparative methods. It is also shown that learning from multi-parametric MRI outperforms that of single-parameter MRI in pCR prediction.

A convolutional neural network with self-attention for fully automated metabolic tumor volume delineation of head and neck cancer in [Formula: see text]F]FDG PET/CT

PURPOSE

PET-derived metabolic tumor volume (MTV) and total lesion glycolysis of the primary tumor are known to be prognostic of clinical outcome in head and neck cancer (HNC). Including evaluation of lymph node metastases can further increase the prognostic value of PET but accurate manual delineation and classification of all lesions is time-consuming and prone to interobserver variability. Our goal, therefore, was development and evaluation of an automated tool for MTV delineation/classification of primary tumor and lymph node metastases in PET/CT investigations of HNC patients.

METHODS

Automated lesion delineation was performed with a residual 3D U-Net convolutional neural network (CNN) incorporating a multi-head self-attention block. 698 [Formula: see text]F]FDG PET/CT scans from 3 different sites and 5 public databases were used for network training and testing. An external dataset of 181 [Formula: see text]F]FDG PET/CT scans from 2 additional sites was employed to assess the generalizability of the network. In these data, primary tumor and lymph node (LN) metastases were interactively delineated and labeled by two experienced physicians. Performance of the trained network models was assessed by 5-fold cross-validation in the main dataset and by pooling results from the 5 developed models in the external dataset. The Dice similarity coefficient (DSC) for individual delineation tasks and the primary tumor/metastasis classification accuracy were used as evaluation metrics. Additionally, a survival analysis using univariate Cox regression was performed comparing achieved group separation for manual and automated delineation, respectively.

RESULTS

In the cross-validation experiment, delineation of all malignant lesions with the trained U-Net models achieves DSC of 0.885, 0.805, and 0.870 for primary tumor, LN metastases, and the union of both, respectively. In external testing, the DSC reaches 0.850, 0.724, and 0.823 for primary tumor, LN metastases, and the union of both, respectively. The voxel classification accuracy was 98.0% and 97.9% in cross-validation and external data, respectively. Univariate Cox analysis in the cross-validation and the external testing reveals that manually and automatically derived total MTVs are both highly prognostic with respect to overall survival, yielding essentially identical hazard ratios (HR) ([Formula: see text]; [Formula: see text] vs. [Formula: see text]; [Formula: see text] in cross-validation and [Formula: see text]; [Formula: see text] vs. [Formula: see text]; [Formula: see text] in external testing).

CONCLUSION

To the best of our knowledge, this work presents the first CNN model for successful MTV delineation and lesion classification in HNC. In the vast majority of patients, the network performs satisfactory delineation and classification of primary tumor and lymph node metastases and only rarely requires more than minimal manual correction. It is thus able to massively facilitate study data evaluation in large patient groups and also does have clear potential for supervised clinical application.

A Focus on the Synergy of Radiomics and RNA Sequencing in Breast Cancer

Radiological imaging is currently employed as the most effective technique for screening, diagnosis, and follow up of patients with breast cancer (BC), the most common type of tumor in women worldwide. However, the introduction of the omics sciences such as metabolomics, proteomics, and molecular genomics, have optimized the therapeutic path for patients and implementing novel information parallel to the mutational asset targetable by specific clinical treatments. Parallel to the "omics" clusters, radiological imaging has been gradually employed to generate a specific omics cluster termed "radiomics". Radiomics is a novel advanced approach to imaging, extracting quantitative, and ideally, reproducible data from radiological images using sophisticated mathematical analysis, including disease-specific patterns, that could not be detected by the human eye. Along with radiomics, radiogenomics, defined as the integration of "radiology" and "genomics", is an emerging field exploring the relationship between specific features extracted from radiological images and genetic or molecular traits of a particular disease to construct adequate predictive models. Accordingly, radiological characteristics of the tissue are supposed to mimic a defined genotype and phenotype and to better explore the heterogeneity and the dynamic evolution of the tumor over the time. Despite such improvements, we are still far from achieving approved and standardized protocols in clinical practice. Nevertheless, what can we learn by this emerging multidisciplinary clinical approach? This minireview provides a focused overview on the significance of radiomics integrated by RNA sequencing in BC. We will also discuss advances and future challenges of such radiomics-based approach.

Clinical applications of deep learning in breast MRI

Deep learning (DL) is one of the most powerful data-driven machine-learning techniques in artificial intelligence (AI). It can automatically learn from raw data without manual feature selection. DL models have led to remarkable advances in data extraction and analysis for medical imaging. Magnetic resonance imaging (MRI) has proven useful in delineating the characteristics and extent of breast lesions and tumors. This review summarizes the current state-of-the-art applications of DL models in breast MRI. Many recent DL models were examined in this field, along with several advanced learning approaches and methods for data normalization and breast and lesion segmentation. For clinical applications, DL-based breast MRI models were proven useful in five aspects: diagnosis of breast cancer, classification of molecular types, classification of histopathological types, prediction of neoadjuvant chemotherapy response, and prediction of lymph node metastasis. For subsequent studies, further improvement in data acquisition and preprocessing is necessary, additional DL techniques in breast MRI should be investigated, and wider clinical applications need to be explored.

Prognostic Factors for Triple-Negative Breast Cancer with Residual Disease after Neoadjuvant Chemotherapy

Valid factors to evaluate the prognosis of triple-negative breast cancer (TNBC) with residual disease after neoadjuvant chemotherapy (NAC) are still lacking. We performed this study to explore prognostic factors focusing on genetic alterations and clinicopathology features in non- pathologic complete response (pCR) TNBC patients. Patients initially diagnosed with early-stage TNBC, treated with NAC, and who had residual disease after primary tumor surgery at the China National Cancer Center during 2016 and 2020 were enrolled. Genomic analyses were performed by targeted sequencing for each tumor sample. Univariable and multivariable analyses were conducted to screen prognostic factors for the survival of patients. Fifty-seven patients were included in our study. Genomic analyses showed that TP53 (41/57, 72%), PIK3CA (12/57, 21%), and MET (7/57, 12%), and PTEN (7/57, 12%) alternations commonly occurred. The clinical TNM (cTNM) stage and PIK3CA status were independent prognostic factors of disease-free survival (DFS) (p < 0.001, p = 0.03). A prognostic stratification indicated that patients with clinical stages I &II possessed the best DFS, followed by those with clinical stage III & wild-type PIK3CA. In contrast, patients with clinical stage III & the PIK3CA mutation had the worst DFS. In TNBC patients with residual disease after NAC, prognostic stratification for DFS was observed by combining the cTNM stage and PIK3CA status.

Prediction of pathologic complete response to neoadjuvant systemic therapy in triple negative breast cancer using deep learning on multiparametric MRI

Triple-negative breast cancer (TNBC) is an aggressive subtype of breast cancer. Neoadjuvant systemic therapy (NAST) followed by surgery are currently standard of care for TNBC with 50-60% of patients achieving pathologic complete response (pCR). We investigated ability of deep learning (DL) on dynamic contrast enhanced (DCE) MRI and diffusion weighted imaging acquired early during NAST to predict TNBC patients' pCR status in the breast. During the development phase using the images of 130 TNBC patients, the DL model achieved areas under the receiver operating characteristic curves (AUCs) of 0.97 ± 0.04 and 0.82 ± 0.10 for the training and the validation, respectively. The model achieved an AUC of 0.86 ± 0.03 when evaluated in the independent testing group of 32 patients. In an additional prospective blinded testing group of 48 patients, the model achieved an AUC of 0.83 ± 0.02. These results demonstrated that DL based on multiparametric MRI can potentially differentiate TNBC patients with pCR or non-pCR in the breast early during NAST.

Deep learning prediction of pathological complete response, residual cancer burden, and progression-free survival in breast cancer patients

The goal of this study was to employ novel deep-learning convolutional-neural-network (CNN) to predict pathological complete response (PCR), residual cancer burden (RCB), and progression-free survival (PFS) in breast cancer patients treated with neoadjuvant chemotherapy using longitudinal multiparametric MRI, demographics, and molecular subtypes as inputs. In the I-SPY-1 TRIAL, 155 patients with stage 2 or 3 breast cancer with breast tumors underwent neoadjuvant chemotherapy met the inclusion/exclusion criteria. The inputs were dynamic-contrast-enhanced (DCE) MRI, and T2- weighted MRI as three-dimensional whole-images without the tumor segmentation, as well as molecular subtypes and demographics. The outcomes were PCR, RCB, and PFS. Three ("Integrated", "Stack" and "Concatenation") CNN were evaluated using receiver-operating characteristics and mean absolute errors. The Integrated approach outperformed the "Stack" or "Concatenation" CNN. Inclusion of both MRI and non-MRI data outperformed either alone. The combined pre- and post-neoadjuvant chemotherapy data outperformed either alone. Using the best model and data combination, PCR prediction yielded an accuracy of 0.81±0.03 and AUC of 0.83±0.03; RCB prediction yielded an accuracy of 0.80±0.02 and Cohen's κ of 0.73±0.03; PFS prediction yielded a mean absolute error of 24.6±0.7 months (survival ranged from 6.6 to 127.5 months). Deep learning using longitudinal multiparametric MRI, demographics, and molecular subtypes accurately predicts PCR, RCB, and PFS in breast cancer patients. This approach may prove useful for treatment selection, planning, execution, and mid-treatment adjustment.

Deep Learning Based Methods for Breast Cancer Diagnosis: A Systematic Review and Future Direction

Breast cancer is one of the precarious conditions that affect women, and a substantive cure has not yet been discovered for it. With the advent of Artificial intelligence (AI), recently, deep learning techniques have been used effectively in breast cancer detection, facilitating early diagnosis and therefore increasing the chances of patients' survival. Compared to classical machine learning techniques, deep learning requires less human intervention for similar feature extraction. This study presents a systematic literature review on the deep learning-based methods for breast cancer detection that can guide practitioners and researchers in understanding the challenges and new trends in the field. Particularly, different deep learning-based methods for breast cancer detection are investigated, focusing on the genomics and histopathological imaging data. The study specifically adopts the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA), which offer a detailed analysis and synthesis of the published articles. Several studies were searched and gathered, and after the eligibility screening and quality evaluation, 98 articles were identified. The results of the review indicated that the Convolutional Neural Network (CNN) is the most accurate and extensively used model for breast cancer detection, and the accuracy metrics are the most popular method used for performance evaluation. Moreover, datasets utilized for breast cancer detection and the evaluation metrics are also studied. Finally, the challenges and future research direction in breast cancer detection based on deep learning models are also investigated to help researchers and practitioners acquire in-depth knowledge of and insight into the area.

Integration of Clinical and CT-Based Radiomic Features for Pretreatment Prediction of Pathologic Complete Response to Neoadjuvant Systemic Therapy in Breast Cancer

The purpose of the present study was to examine the potential of a machine learning model with integrated clinical and CT-based radiomics features in predicting pathologic complete response (pCR) to neoadjuvant systemic therapy (NST) in breast cancer. Contrast-enhanced CT was performed in 329 patients with breast tumors (n = 331) before NST. Pyradiomics was used for feature extraction, and 107 features of seven classes were extracted. Feature selection was performed on the basis of the intraclass correlation coefficient (ICC), and six ICC thresholds (0.7−0.95) were examined to identify the feature set resulting in optimal model performance. Clinical factors, such as age, clinical stage, cancer cell type, and cell surface receptors, were used for prediction. We tried six machine learning algorithms, and clinical, radiomics, and clinical−radiomics models were trained for each algorithm. Radiomics and clinical−radiomics models with gray level co-occurrence matrix (GLCM) features only were also built for comparison. The linear support vector machine (SVM) regression model trained with radiomics features of ICC ≥0.85 in combination with clinical factors performed the best (AUC = 0.87). The performance of the clinical and radiomics linear SVM models showed statistically significant difference after correction for multiple comparisons (AUC = 0.69 vs. 0.78; p < 0.001). The AUC of the radiomics model trained with GLCM features was significantly lower than that of the radiomics model trained with all seven classes of radiomics features (AUC = 0.85 vs. 0.87; p = 0.011). Integration of clinical and CT-based radiomics features was helpful in the pretreatment prediction of pCR to NST in breast cancer.

Artificial Intellgence in the Era of Precision Oncological Imaging

Rapid-paced development and adaptability of artificial intelligence algorithms have secured their almost ubiquitous presence in the field of oncological imaging. Artificial intelligence models have been created for a variety of tasks, including risk stratification, automated detection, and segmentation of lesions, characterization, grading and staging, prediction of prognosis, and treatment response. Soon, artificial intelligence could become an essential part of every step of oncological workup and patient management. Integration of neural networks and deep learning into radiological artificial intelligence algorithms allow for extrapolating imaging features otherwise inaccessible to human operators and pave the way to truly personalized management of oncological patients.Although a significant proportion of currently available artificial intelligence solutions belong to basic and translational cancer imaging research, their progressive transfer to clinical routine is imminent, contributing to the development of a personalized approach in oncology. We thereby review the main applications of artificial intelligence in oncological imaging, describe the example of their successful integration into research and clinical practice, and highlight the challenges and future perspectives that will shape the field of oncological radiology.

Deep Learning Prediction of Pathologic Complete Response in Breast Cancer Using MRI and Other Clinical Data: A Systematic Review

Breast cancer patients who have pathological complete response (pCR) to neoadjuvant chemotherapy (NAC) are more likely to have better clinical outcomes. The ability to predict which patient will respond to NAC early in the treatment course is important because it could help to minimize unnecessary toxic NAC and to modify regimens mid-treatment to achieve better efficacy. Machine learning (ML) is increasingly being used in radiology and medicine because it can identify relationships amongst complex data elements to inform outcomes without the need to specify such relationships a priori. One of the most popular deep learning methods that applies to medical images is the Convolutional Neural Networks (CNN). In contrast to supervised ML, deep learning CNN can operate on the whole images without requiring radiologists to manually contour the tumor on images. Although there have been many review papers on supervised ML prediction of pCR, review papers on deep learning prediction of pCR are sparse. Deep learning CNN could also incorporate multiple image types, clinical data such as demographics and molecular subtypes, as well as data from multiple treatment time points to predict pCR. The goal of this study is to perform a systematic review of deep learning methods that use whole-breast MRI images without annotation or tumor segmentation to predict pCR in breast cancer.

The Role of Deep Learning in Advancing Breast Cancer Detection Using Different Imaging Modalities: A Systematic Review

Breast cancer is among the most common and fatal diseases for women, and no permanent treatment has been discovered. Thus, early detection is a crucial step to control and cure breast cancer that can save the lives of millions of women. For example, in 2020, more than 65% of breast cancer patients were diagnosed in an early stage of cancer, from which all survived. Although early detection is the most effective approach for cancer treatment, breast cancer screening conducted by radiologists is very expensive and time-consuming. More importantly, conventional methods of analyzing breast cancer images suffer from high false-detection rates. Different breast cancer imaging modalities are used to extract and analyze the key features affecting the diagnosis and treatment of breast cancer. These imaging modalities can be divided into subgroups such as mammograms, ultrasound, magnetic resonance imaging, histopathological images, or any combination of them. Radiologists or pathologists analyze images produced by these methods manually, which leads to an increase in the risk of wrong decisions for cancer detection. Thus, the utilization of new automatic methods to analyze all kinds of breast screening images to assist radiologists to interpret images is required. Recently, artificial intelligence (AI) has been widely utilized to automatically improve the early detection and treatment of different types of cancer, specifically breast cancer, thereby enhancing the survival chance of patients. Advances in AI algorithms, such as deep learning, and the availability of datasets obtained from various imaging modalities have opened an opportunity to surpass the limitations of current breast cancer analysis methods. In this article, we first review breast cancer imaging modalities, and their strengths and limitations. Then, we explore and summarize the most recent studies that employed AI in breast cancer detection using various breast imaging modalities. In addition, we report available datasets on the breast-cancer imaging modalities which are important in developing AI-based algorithms and training deep learning models. In conclusion, this review paper tries to provide a comprehensive resource to help researchers working in breast cancer imaging analysis.

Radiomic and Volumetric Measurements as Clinical Trial Endpoints—A Comprehensive Review

Simple Summary

The extraction of quantitative data from standard-of-care imaging modalities offers opportunities to improve the relevance and salience of imaging biomarkers used in drug development. This review aims to identify the challenges and opportunities for discovering new imaging-based biomarkers based on radiomic and volumetric assessment in the single-site solid tumor sites: breast cancer, rectal cancer, lung cancer and glioblastoma. Developing approaches to harmonize three essential areas: segmentation, validation and data sharing may expedite regulatory approval and adoption of novel cancer imaging biomarkers.

Abstract

Clinical trials for oncology drug development have long relied on surrogate outcome biomarkers that assess changes in tumor burden to accelerate drug registration (i.e., Response Evaluation Criteria in Solid Tumors version 1.1 (RECIST v1.1) criteria). Drug-induced reduction in tumor size represents an imperfect surrogate marker for drug activity and yet a radiologically determined objective response rate is a widely used endpoint for Phase 2 trials. With the addition of therapies targeting complex biological systems such as immune system and DNA damage repair pathways, incorporation of integrative response and outcome biomarkers may add more predictive value. We performed a review of the relevant literature in four representative tumor types (breast cancer, rectal cancer, lung cancer and glioblastoma) to assess the preparedness of volumetric and radiomics metrics as clinical trial endpoints. We identified three key areas—segmentation, validation and data sharing strategies—where concerted efforts are required to enable progress of volumetric- and radiomics-based clinical trial endpoints for wider clinical implementation.

Identification of women with high grade histopathology results after conisation by artificial neural networks

BACKGROUND

The aim of the study was to evaluate if artificial neural networks can predict high-grade histopathology results after conisation from risk factors and their combinations in patients undergoing conisation because of pathological changes on uterine cervix.

PATIENTS AND METHODS

We analysed 1475 patients who had conisation surgery at the University Clinic for Gynaecology and Obstetrics of University Clinical Centre Maribor from 1993-2005. The database in different datasets was arranged to deal with unbalance data and enhance classification performance. Weka open-source software was used for analysis with artificial neural networks. Last Papanicolaou smear (PAP) and risk factors for development of cervical dysplasia and carcinoma were used as input and high-grade dysplasia Yes/No as output result. 10-fold cross validation was used for defining training and holdout set for analysis.

RESULTS

Bas eline classification and multiple runs of artificial neural network on various risk factors settings were performed. We achieved 84.19% correct classifications, area under the curve 0.87, kappa 0.64, F-measure 0.884 and Matthews correlation coefficient (MCC) 0.640 in model, where baseline prediction was 69.79%.

CONCLUSIONS

With artificial neural networks we were able to identify more patients who developed high-grade squamous intraepithelial lesion on final histopathology result of conisation as with baseline prediction. But, characteristics of 1475 patients who had conisation in years 1993-2005 at the University Clinical Centre Maribor did not allow reliable prediction with artificial neural networks for every-day clinical practice.

Development and validation of a deep learning model for breast lesion segmentation and characterization in multiparametric MRI

Importance

The utilization of artificial intelligence for the differentiation of benign and malignant breast lesions in multiparametric MRI (mpMRI) assists radiologists to improve diagnostic performance.

Objectives

To develop an automated deep learning model for breast lesion segmentation and characterization and to evaluate the characterization performance of AI models and radiologists.

Materials and methods

For lesion segmentation, 2,823 patients were used for the training, validation, and testing of the VNet-based segmentation models, and the average Dice similarity coefficient (DSC) between the manual segmentation by radiologists and the mask generated by VNet was calculated. For lesion characterization, 3,303 female patients with 3,607 pathologically confirmed lesions (2,213 malignant and 1,394 benign lesions) were used for the three ResNet-based characterization models (two single-input and one multi-input models). Histopathology was used as the diagnostic criterion standard to assess the characterization performance of the AI models and the BI-RADS categorized by the radiologists, in terms of sensitivity, specificity, accuracy, and the area under the receiver operating characteristic curve (AUC). An additional 123 patients with 136 lesions (81 malignant and 55 benign lesions) from another institution were available for external testing.

Results

Of the 5,811 patients included in the study, the mean age was 46.14 (range 11–89) years. In the segmentation task, a DSC of 0.860 was obtained between the VNet-generated mask and manual segmentation by radiologists. In the characterization task, the AUCs of the multi-input and the other two single-input models were 0.927, 0.821, and 0.795, respectively. Compared to the single-input DWI or DCE model, the multi-input DCE and DWI model obtained a significant increase in sensitivity, specificity, and accuracy (0.831 vs. 0.772/0.776, 0.874 vs. 0.630/0.709, 0.846 vs. 0.721/0.752). Furthermore, the specificity of the multi-input model was higher than that of the radiologists, whether using BI-RADS category 3 or 4 as a cutoff point (0.874 vs. 0.404/0.841), and the accuracy was intermediate between the two assessment methods (0.846 vs. 0.773/0.882). For the external testing, the performance of the three models remained robust with AUCs of 0.812, 0.831, and 0.885, respectively.

Conclusions

Combining DCE with DWI was superior to applying a single sequence for breast lesion characterization. The deep learning computer-aided diagnosis (CADx) model we developed significantly improved specificity and achieved comparable accuracy to the radiologists with promise for clinical application to provide preliminary diagnoses.

Deep Learning Mechanism for Predicting the Axillary Lymph Node Metastasis in Patients with Primary Breast Cancer

The second largest cause of mortality worldwide is breast cancer, and it mostly occurs in women. Early diagnosis has improved further treatments and reduced the level of mortality. A unique deep learning algorithm is presented for predicting breast cancer in its early stages. This method utilizes numerous layers to retrieve significantly greater amounts of information from the source inputs. It could perform automatic quantitative evaluation of complicated image properties in the medical field and give greater precision and reliability during the diagnosis. The dataset of axillary lymph nodes from the breast cancer patients was collected from Erasmus Medical Center. A total of 1050 images were studied from the 850 patients during the years 2018 to 2021. For the independent test, data samples were collected for 100 images from 95 patients at national cancer institute. The existence of axillary lymph nodes was confirmed by pathologic examination. The feed forward, radial basis function, and Kohonen self-organizing are the artificial neural networks (ANNs) which are used to train 84% of the Erasmus Medical Center dataset and test the remaining 16% of the independent dataset. The proposed model performance was determined in terms of accuracy (Ac), sensitivity (Sn), specificity (Sf), and the outcome of the receiver operating curve (Roc), which was compared to the other four radiologists' mechanism. The result of the study shows that the proposed mechanism achieves 95% sensitivity, 96% specificity, and 98% accuracy, which is higher than the radiologists' models (90% sensitivity, 92% specificity, and 94% accuracy). Deep learning algorithms could accurately predict the clinical negativity of axillary lymph node metastases by utilizing images of initial breast cancer patients. This method provides an earlier diagnostic technique for axillary lymph node metastases in patients with medically negative changes in axillary lymph nodes.

Evaluation of cancer outcome assessment using MRI: A review of deep-learning methods

Accurate evaluation of tumor response to treatment is critical to allow personalized treatment regimens according to the predicted response and to support clinical trials investigating new therapeutic agents by providing them with an accurate response indicator. Recent advances in medical imaging, computer hardware, and machine-learning algorithms have resulted in the increased use of these tools in the field of medicine as a whole and specifically in cancer imaging for detection and characterization of malignant lesions, prognosis, and assessment of treatment response. Among the currently available imaging techniques, magnetic resonance imaging (MRI) plays an important role in the evaluation of treatment assessment of many cancers, given its superior soft-tissue contrast and its ability to allow multiplanar imaging and functional evaluation. In recent years, deep learning (DL) has become an active area of research, paving the way for computer-assisted clinical and radiological decision support. DL can uncover associations between imaging features that cannot be visually identified by the naked eye and pertinent clinical outcomes. The aim of this review is to highlight the use of DL in the evaluation of tumor response assessed on MRI. In this review, we will first provide an overview of common DL architectures used in medical imaging research in general. Then, we will review the studies to date that have applied DL to magnetic resonance imaging for the task of treatment response assessment. Finally, we will discuss the challenges and opportunities of using DL within the clinical workflow.

A Survey on Deep Learning for Precision Oncology

Precision oncology, which ensures optimized cancer treatment tailored to the unique biology of a patient’s disease, has rapidly developed and is of great clinical importance. Deep learning has become the main method for precision oncology. This paper summarizes the recent deep-learning approaches relevant to precision oncology and reviews over 150 articles within the last six years. First, we survey the deep-learning approaches categorized by various precision oncology tasks, including the estimation of dose distribution for treatment planning, survival analysis and risk estimation after treatment, prediction of treatment response, and patient selection for treatment planning. Secondly, we provide an overview of the studies per anatomical area, including the brain, bladder, breast, bone, cervix, esophagus, gastric, head and neck, kidneys, liver, lung, pancreas, pelvis, prostate, and rectum. Finally, we highlight the challenges and discuss potential solutions for future research directions.

Robustness Evaluation of a Deep Learning Model on Sagittal and Axial Breast DCE-MRIs to Predict Pathological Complete Response to Neoadjuvant Chemotherapy

To date, some artificial intelligence (AI) methods have exploited Dynamic Contrast-Enhanced Magnetic Resonance Imaging (DCE-MRI) to identify finer tumor properties as potential earlier indicators of pathological Complete Response (pCR) in breast cancer patients undergoing neoadjuvant chemotherapy (NAC). However, they work either for sagittal or axial MRI protocols. More flexible AI tools, to be used easily in clinical practice across various institutions in accordance with its own imaging acquisition protocol, are required. Here, we addressed this topic by developing an AI method based on deep learning in giving an early prediction of pCR at various DCE-MRI protocols (axial and sagittal). Sagittal DCE-MRIs refer to 151 patients (42 pCR; 109 non-pCR) from the public I-SPY1 TRIAL database (DB); axial DCE-MRIs are related to 74 patients (22 pCR; 52 non-pCR) from a private DB provided by Istituto Tumori “Giovanni Paolo II” in Bari (Italy). By merging the features extracted from baseline MRIs with some pre-treatment clinical variables, accuracies of 84.4% and 77.3% and AUC values of 80.3% and 78.0% were achieved on the independent tests related to the public DB and the private DB, respectively. Overall, the presented method has shown to be robust regardless of the specific MRI protocol.

Deep learning in breast imaging

Millions of breast imaging exams are performed each year in an effort to reduce the morbidity and mortality of breast cancer. Breast imaging exams are performed for cancer screening, diagnostic work-up of suspicious findings, evaluating extent of disease in recently diagnosed breast cancer patients, and determining treatment response. Yet, the interpretation of breast imaging can be subjective, tedious, time-consuming, and prone to human error. Retrospective and small reader studies suggest that deep learning (DL) has great potential to perform medical imaging tasks at or above human-level performance, and may be used to automate aspects of the breast cancer screening process, improve cancer detection rates, decrease unnecessary callbacks and biopsies, optimize patient risk assessment, and open up new possibilities for disease prognostication. Prospective trials are urgently needed to validate these proposed tools, paving the way for real-world clinical use. New regulatory frameworks must also be developed to address the unique ethical, medicolegal, and quality control issues that DL algorithms present. In this article, we review the basics of DL, describe recent DL breast imaging applications including cancer detection and risk prediction, and discuss the challenges and future directions of artificial intelligence-based systems in the field of breast cancer.

Updates in Artificial Intelligence for Breast Imaging

Artificial intelligence (AI) for breast imaging has rapidly moved from the experimental to implementation phase. As of this writing, Food and Drug Administration (FDA)-approved mammographic applications are available for triage, lesion detection and classification, and breast density assessment. For sonography and MRI, FDA-approved applications are available for lesion classification. Numerous other interpretive and noninterpretive AI applications are in the development phase. This article reviews AI applications for mammography, sonography, and MRI that are currently available for clinical use. In addition, clinical implementation and the future of AI for breast imaging are discussed.

Tomographic Ultrasound Imaging in the Diagnosis of Breast Tumors under the Guidance of Deep Learning Algorithms

This study was aimed to discuss the feasibility of distinguishing benign and malignant breast tumors under the tomographic ultrasound imaging (TUI) of deep learning algorithm. The deep learning algorithm was used to segment the images, and 120 patients with breast tumor were included in this study, all of whom underwent routine ultrasound examinations. Subsequently, TUI was used to assist in guiding the positioning, and the light scattering tomography system was used to further measure the lesions. A deep learning model was established to process the imaging results, and the pathological test results were undertaken as the gold standard for the efficiency of different imaging methods to diagnose the breast tumors. The results showed that, among 120 patients with breast tumor, 56 were benign lesions and 64 were malignant lesions. The average total amount of hemoglobin (HBT) of malignant lesions was significantly higher than that of benign lesions (P < 0.05). The sensitivity, specificity, accuracy, positive predictive value, and negative predictive value of TUI in the diagnosis of breast cancer were 90.4%, 75.6%, 81.4%, 84.7%, and 80.6%, respectively. The sensitivity, specificity, accuracy, positive predictive value, and negative predictive value of ultrasound in the diagnosis of breast cancer were 81.7%, 64.9%, 70.5%, 75.9%, and 80.6%, respectively. In addition, for suspected breast malignant lesions, the combined application of ultrasound and tomography can increase the diagnostic specificity to 82.1% and the accuracy to 83.8%. Based on the above results, it was concluded that TUI combined with ultrasound had a significant effect on benign and malignant diagnosis of breast cancer and can significantly improve the specificity and accuracy of diagnosis. It also reflected that deep learning technology had a good auxiliary role in the examination of diseases and was worth the promotion of clinical application.

Deep learning of quantitative ultrasound multi-parametric images at pre-treatment to predict breast cancer response to chemotherapy

In this study, a novel deep learning-based methodology was investigated to predict breast cancer response to neo-adjuvant chemotherapy (NAC) using the quantitative ultrasound (QUS) multi-parametric imaging at pre-treatment. QUS multi-parametric images of breast tumors were generated using the data acquired from 181 patients diagnosed with locally advanced breast cancer and planned for NAC followed by surgery. The ground truth response to NAC was identified for each patient after the surgery using the standard clinical and pathological criteria. Two deep convolutional neural network (DCNN) architectures including the residual network and residual attention network (RAN) were explored for extracting optimal feature maps from the parametric images, with a fully connected network for response prediction. In different experiments, the features maps were derived from the tumor core only, as well as the core and its margin. Evaluation results on an independent test set demonstrate that the developed model with the RAN architecture to extract feature maps from the expanded parametric images of the tumor core and margin had the best performance in response prediction with an accuracy of 88% and an area under the receiver operating characteristic curve of 0.86. Ten-year survival analyses indicate statistically significant differences between the survival of the responders and non-responders identified based on the model prediction at pre-treatment and the standard criteria at post-treatment. The results of this study demonstrate the promising capability of DCNNs with attention mechanisms in predicting breast cancer response to NAC prior to the start of treatment using QUS multi-parametric images.

Predicting cancer outcomes with radiomics and artificial intelligence in radiology

The successful use of artificial intelligence (AI) for diagnostic purposes has prompted the application of AI-based cancer imaging analysis to address other, more complex, clinical needs. In this Perspective, we discuss the next generation of challenges in clinical decision-making that AI tools can solve using radiology images, such as prognostication of outcome across multiple cancers, prediction of response to various treatment modalities, discrimination of benign treatment confounders from true progression, identification of unusual response patterns and prediction of the mutational and molecular profile of tumours. We describe the evolution of and opportunities for AI in oncology imaging, focusing on hand-crafted radiomic approaches and deep learning-derived representations, with examples of their application for decision support. We also address the challenges faced on the path to clinical adoption, including data curation and annotation, interpretability, and regulatory and reimbursement issues. We hope to demystify AI in radiology for clinicians by helping them to understand its limitations and challenges, as well as the opportunities it provides as a decision-support tool in cancer management.

Chemotherapy response prediction with diffuser elapser network

In solid tumors, elevated fluid pressure and inadequate blood perfusion resulting from unbalanced angiogenesis are the prominent reasons for the ineffective drug delivery inside tumors. To normalize the heterogeneous and tortuous tumor vessel structure, antiangiogenic treatment is an effective approach. Additionally, the combined therapy of antiangiogenic agents and chemotherapy drugs has shown promising effects on enhanced drug delivery. However, the need to find the appropriate scheduling and dosages of the combination therapy is one of the main problems in anticancer therapy. Our study aims to generate a realistic response to the treatment schedule, making it possible for future works to use these patient-specific responses to decide on the optimal starting time and dosages of cytotoxic drug treatment. Our dataset is based on our previous in-silico model with a framework for the tumor microenvironment, consisting of a tumor layer, vasculature network, interstitial fluid pressure, and drug diffusion maps. In this regard, the chemotherapy response prediction problem is discussed in the study, putting forth a proof of concept for deep learning models to capture the tumor growth and drug response behaviors simultaneously. The proposed model utilizes multiple convolutional neural network submodels to predict future tumor microenvironment maps considering the effects of ongoing treatment. Since the model has the task of predicting future tumor microenvironment maps, we use two image quality evaluation metrics, which are structural similarity and peak signal-to-noise ratio, to evaluate model performance. We track tumor cell density values of ground truth and predicted tumor microenvironments. The model predicts tumor microenvironment maps seven days ahead with the average structural similarity score of 0.973 and the average peak signal ratio of 35.41 in the test set. It also predicts tumor cell density at the end day of 7 with the mean absolute percentage error of [Formula: see text].

Assessing PD-L1 Expression Status Using Radiomic Features from Contrast-Enhanced Breast MRI in Breast Cancer Patients: Initial Results

Simple Summary

To our knowledge, this is the first study assessing radiomics coupled with machine learning from MRI-derived features to predict PD-L1 expression status in biopsy-proven triple negative breast cancers and comparing the performance of this approach with the performance of qualitative assessment by two radiologists. This pilot study shows that radiomics analysis coupled with machine learning of DCE-MRI is a promising approach to derive prognostic and predictive information and to select patients who could benefit from anti-PD-1/PD-L1 treatment. This technique could also be used to monitor PD-L1 expression, as it can vary over time and between different regions of the tumor, thus avoiding repeated biopsies.

Abstract

The purpose of this retrospective study was to assess whether radiomics analysis coupled with machine learning (ML) based on standard-of-care dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) can predict PD-L1 expression status in patients with triple negative breast cancer, and to compare the performance of this approach with radiologist review. Patients with biopsy-proven triple negative breast cancer who underwent pre-treatment breast MRI and whose PD-L1 status was available were included. Following 3D tumor segmentation and extraction of radiomic features, radiomic features with significant differences between PD-L1+ and PD-L1− patients were determined, and a final predictive model to predict PD-L1 status was developed using a coarse decision tree and five-fold cross-validation. Separately, all lesions were qualitatively assessed by two radiologists independently according to the BI-RADS lexicon. Of 62 women (mean age 47, range 31–81), 27 had PD-L1− tumors and 35 had PD-L1+ tumors. The final radiomics model to predict PD-L1 status utilized three MRI parameters, i.e., variance (FO), run length variance (RLM), and large zone low grey level emphasis (LZLGLE), for a sensitivity of 90.7%, specificity of 85.1%, and diagnostic accuracy of 88.2%. There were no significant associations between qualitative assessed DCE-MRI imaging features and PD-L1 status. Thus, radiomics analysis coupled with ML based on standard-of-care DCE-MRI is a promising approach to derive prognostic and predictive information and to select patients who could benefit from anti-PD-1/PD-L1 treatment.

Computer-aided classification of MRI for pathological complete response to neoadjuvant chemotherapy in breast cancer

Background: To determine suitable optimal classifiers and examine the general applicability of computer-aided classification to compare the differences between a computer-aided system and radiologists in predicting pathological complete response (pCR) from patients with breast cancer receiving neoadjuvant chemotherapy. Methods: We analyzed a total of 455 masses and used the U-Net network and ResNet to execute MRI segmentation and pCR classification. The diagnostic performance of radiologists, the computer-aided system and a combination of radiologists and computer-aided system were compared using receiver operating characteristic curve analysis. Results: The combination of radiologists and computer-aided system had the best performance for predicting pCR with an area under the curve (AUC) value of 0.899, significantly higher than that of radiologists alone (AUC: 0.700) and computer-aided system alone (AUC: 0.835). Conclusion: An automated classification system is feasible to predict the pCR to neoadjuvant chemotherapy in patients with breast cancer and can complement MRI.

Artificial intelligence for assisting cancer diagnosis and treatment in the era of precision medicine

Over the past decade, artificial intelligence (AI) has contributed substantially to the resolution of various medical problems, including cancer. Deep learning (DL), a subfield of AI, is characterized by its ability to perform automated feature extraction and has great power in the assimilation and evaluation of large amounts of complicated data. On the basis of a large quantity of medical data and novel computational technologies, AI, especially DL, has been applied in various aspects of oncology research and has the potential to enhance cancer diagnosis and treatment. These applications range from early cancer detection, diagnosis, classification and grading, molecular characterization of tumors, prediction of patient outcomes and treatment responses, personalized treatment, automatic radiotherapy workflows, novel anti-cancer drug discovery, and clinical trials. In this review, we introduced the general principle of AI, summarized major areas of its application for cancer diagnosis and treatment, and discussed its future directions and remaining challenges. As the adoption of AI in clinical use is increasing, we anticipate the arrival of AI-powered cancer care.

Radiomics in breast MRI: current progress toward clinical application in the era of artificial intelligence

Breast magnetic resonance imaging (MRI) is the most sensitive imaging modality for breast cancer diagnosis and is widely used clinically. Dynamic contrast-enhanced MRI is the basis for breast MRI, but ultrafast images, T2-weighted images, and diffusion-weighted images are also taken to improve the characteristics of the lesion. Such multiparametric MRI with numerous morphological and functional data poses new challenges to radiologists, and thus, new tools for reliable, reproducible, and high-volume quantitative assessments are warranted. In this context, radiomics, which is an emerging field of research involving the conversion of digital medical images into mineable data for clinical decision-making and outcome prediction, has been gaining ground in oncology. Recent development in artificial intelligence has promoted radiomics studies in various fields including breast cancer treatment and numerous studies have been conducted. However, radiomics has shown a translational gap in clinical practice, and many issues remain to be solved. In this review, we will outline the steps of radiomics workflow and investigate clinical application of radiomics focusing on breast MRI based on published literature, as well as current discussion about limitations and challenges in radiomics.

Transfer Learning Strategy Based on Unsupervised Learning and Ensemble Learning for Breast Cancer Molecular Subtype Prediction Using Dynamic Contrast-Enhanced MRI

BACKGROUND

Imaging-driven deep learning strategies focus on training from scratch and transfer learning. However, the performance of training from scratch is often impeded by the lack of large-scale labeled training data. Additionally, owing to the differences between source and target domains, analyzing medical image tasks satisfactorily via transfer learning based on ImageNet is difficult.

PURPOSE

To investigate two transfer learning algorithms for breast cancer molecular subtype prediction (luminal and non-luminal) based on unsupervised pre-training and ensemble learning: M_EL and B_EL, using malignant and benign datasets as the source domain, respectively.

STUDY TYPE

Retrospective.

POPULATION

Eight hundred and thirty-three female patients with histologically confirmed breast lesions (567 benign and 266 malignant cases) were selected. In the 5-fold cross-validation, the malignant cohort was randomly divided into 5 subsets to form a training set (80%) and a validation set (20%).

FIELD STRENGTH/SEQUENCE

3.0 T, dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) using T1-weighted high-resolution isotropic volume examination.

ASSESSMENT

First, three datasets acquired at different times post-contrast were preprocessed as unlabeled source domains. Second, three baseline networks corresponding to the different MRI post-contrast phases were built, optimized by a combination of mutual information maximization between high- and low-level representations and prior distribution constraints. Next, the pre-trained networks were fine-tuned on the labeled target domain. Finally, prediction results were integrated using weighted voting-based ensemble learning.

STATISTICAL TESTS

Mean accuracy, precision, specificity, and area under receiver operating characteristic curve (AUC) were obtained with 5-fold cross-validation. P < 0.05 was considered to be statistically significant.

RESULTS

Compared with a convolutional long short-term memory network, pre-trained VGG-16, VGG-19, and DenseNet-121 from ImageNet, M_EL and B_EL exhibited significantly more optimized prediction performance (specificity: 90.5% and 89.9%; accuracy: 82.6% and 81.1%; precision: 91.2% and 90.9%; AUC: 0.836 and 0.823, respectively).

DATA CONCLUSION

Transfer learning based on unsupervised pre-training may facilitate automatic prediction of breast cancer molecular subtypes.

LEVEL OF EVIDENCE

3 TECHNICAL EFFICACY: Stage 2.

Analysis of mammograms using artificial intelligence to predict response to neoadjuvant chemotherapy in breast cancer patients: proof of concept

Objectives

In this proof of concept study, a deep learning–based method for automatic analysis of digital mammograms (DM) as a tool to aid in assessment of neoadjuvant chemotherapy (NACT) treatment response in breast cancer (BC) was examined.

Methods

Baseline DM from 453 patients receiving NACT between 2005 and 2019 were included in the study cohort. A deep learning system, using the aforementioned baseline DM, was developed to predict pathological complete response (pCR) in the surgical specimen after completion of NACT. Two image patches, one extracted around the detected tumour and the other from the corresponding position in the reference image, were fed into a classification network. For training and validation, 1485 images obtained from 400 patients were used, and the model was ultimately applied to a test set consisting of 53 patients.

Results

A total of 95 patients (21%) achieved pCR. The median patient age was 52.5 years (interquartile range 43.7–62.1), and 255 (56%) were premenopausal. The artificial intelligence (AI) model predicted the pCR as represented by the area under the curve of 0.71 (95% confidence interval 0.53–0.90; p = 0.035). The sensitivity was 46% at a fixed specificity of 90%.

Conclusions

Our study describes an AI platform using baseline DM to predict BC patients’ responses to NACT. The initial AI performance indicated the potential to aid in clinical decision-making. In order to continue exploring the clinical utility of AI in predicting responses to NACT for BC, further research, including refining the methodology and a larger sample size, is warranted.

Key Points

• We aimed to answer the following question: Prior to initiation of neoadjuvant chemotherapy, can artificial intelligence (AI) applied to digital mammograms (DM) predict breast tumour response?

• DMs contain information that AI can make use of for predicting pathological complete (pCR) response after neoadjuvant chemotherapy for breast cancer.

• By developing an AI system designed to focus on relevant parts of the DM, fully automatic pCR prediction can be done well enough to potentially aid in clinical decision-making.