Deep learning in breast radiology: current progress and future directions

Predicting Pathological Characteristics of HER2-Positive Breast Cancer from Ultrasound Images: a Deep Ensemble Approach

The objective is to evaluate the feasibility of utilizing ultrasound images in identifying critical prognostic biomarkers for HER2-positive breast cancer (HER2 + BC). This study enrolled 512 female patients diagnosed with HER2-positive breast cancer through pathological validation at our institution from January 2016 to December 2021. Five distinct deep convolutional neural networks (DCNNs) and a deep ensemble (DE) approach were trained to classify axillary lymph node involvement (ALNM), lymphovascular invasion (LVI), and histological grade (HG). The efficacy of the models was evaluated based on accuracy, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), receiver operating characteristic (ROC) curves, areas under the ROC curve (AUCs), and heat maps. DeLong test was applied to compare differences in AUC among different models. The deep ensemble approach, as the most effective model, demonstrated AUCs and accuracy of 0.869 (95% CI: 0.802-0.936) and 69.7% in LVI, 0.973 (95% CI: 0.949-0.998) and 73.8% in HG, thus providing superior classification performance in the context of imbalanced data (p < 0.05 by the DeLong test). On ALNM, AUC and accuracy were 0.780 (95% CI: 0.688-0.873) and 77.5%, which were comparable to other single models. The pretreatment US-based DE model could hold promise as a clinical guidance for predicting pathological characteristics of patients with HER2-positive breast cancer, thereby providing benefit of facilitating timely adjustments in treatment strategies.

Automatic sequence identification in multicentric prostate multiparametric MRI datasets for clinical machine-learning

OBJECTIVES

To present an accurate machine-learning (ML) method and knowledge-based heuristics for automatic sequence-type identification in multi-centric multiparametric MRI (mpMRI) datasets for prostate cancer (PCa) ML.

METHODS

Retrospective prostate mpMRI studies were classified into 5 series types-T2-weighted (T2W), diffusion-weighted images (DWI), apparent diffusion coefficients (ADC), dynamic contrast-enhanced (DCE) and other series types (others). Metadata was processed for all series and two models were trained (XGBoost after custom categorical tokenization and CatBoost with raw categorical data) using 5-fold cross-validation (CV) with different data fractions for learning curve analyses. For validation, two test sets-hold-out test set and temporal split-were used. A leave-one-group-out (LOGO) CV analysis was performed with centres as groups to understand the effect of dataset-specific data.

RESULTS

4045 studies (31,053 series) and 1004 studies (7891 series) from 11 centres were used to train and test series identification models, respectively. Test F1-scores were consistently above 0.95 (CatBoost) and 0.97 (XGBoost). Learning curves demonstrate learning saturation, while temporal validation shows model remain capable of correctly identifying all T2W/DWI/ADC triplets. However, optimal performance requires centre-specific data-controlling for model and used feature sets when comparing CV with LOGOCV, F1-score dropped for T2W, DCE and others (-0.146, -0.181 and -0.179, respectively), with larger performance decreases for CatBoost (-0.265). Finally, we delineate heuristics to assist researchers in series classification for PCa mpMRI datasets.

CONCLUSIONS

Automatic series-type identification is feasible and can enable automated data curation. However, dataset-specific data should be included to achieve optimal performance.

CRITICAL RELEVANCE STATEMENT

Organising large collections of data is time-consuming but necessary to train clinical machine-learning models. To address this, we outline and validate an automatic series identification method that can facilitate this process. Finally, we outline a set of metadata-based heuristics that can be used to further automate series-type identification.

KEY POINTS

Multi-centric prostate MRI studies were used for sequence annotation model training. Automatic sequence annotation requires few instances and generalises temporally. Sequence annotation, necessary for clinical AI model training, can be performed automatically.

For the busy clinical-imaging professional in an AI world: Gaining intuition about deep learning without math

Medical diagnostics comprise recognizing patterns in images, tissue slides, and symptoms. Deep learning algorithms (DLs) are well suited to such tasks, but they are black boxes in various ways. To explain DL Computer-Aided Diagnostic (CAD) results and their accuracy to patients, to manage or drive the direction of future medical DLs, to make better decisions with CAD, etc., clinical professionals may benefit from hands-on, under-the-hood lessons about medical DL. For those who already have some high-level knowledge about DL, the next step is to gain a more-fundamental understanding of DLs, which may help illuminate inside the boxes. The objectives of this Continuing Medical Education (CME) article include:Better understanding can come from relatable medical analogies and personally experiencing quick simulations to observe deep learning in action, akin to the way clinicians are trained to perform other tasks. We developed readily-implementable demonstrations and simulation exercises. We framed the exercises using analogies to breast cancer, malignancy and cancer stage as example diagnostic applications. The simulations revealed a nuanced relationship between DL output accuracy and the quantity and nature of the data. The simulation results provided lessons-learned and implications for the clinical world. Although we focused on DLs for diagnosis, they are similar to DLs for treatment (e.g. radiotherapy) so that treatment providers may also benefit from this tutorial.

Mammography classification with multi-view deep learning techniques: Investigating graph and transformer-based architectures

The potential and promise of deep learning systems to provide an independent assessment and relieve radiologists' burden in screening mammography have been recognized in several studies. However, the low cancer prevalence, the need to process high-resolution images, and the need to combine information from multiple views and scales still pose technical challenges. Multi-view architectures that combine information from the four mammographic views to produce an exam-level classification score are a promising approach to the automated processing of screening mammography. However, training such architectures from exam-level labels, without relying on pixel-level supervision, requires very large datasets and may result in suboptimal accuracy. Emerging architectures such as Visual Transformers (ViT) and graph-based architectures can potentially integrate ipsi-lateral and contra-lateral breast views better than traditional convolutional neural networks, thanks to their stronger ability of modeling long-range dependencies. In this paper, we extensively evaluate novel transformer-based and graph-based architectures against state-of-the-art multi-view convolutional neural networks, trained in a weakly-supervised setting on a middle-scale dataset, both in terms of performance and interpretability. Extensive experiments on the CSAW dataset suggest that, while transformer-based architecture outperform other architectures, different inductive biases lead to complementary strengths and weaknesses, as each architecture is sensitive to different signs and mammographic features. Hence, an ensemble of different architectures should be preferred over a winner-takes-all approach to achieve more accurate and robust results. Overall, the findings highlight the potential of a wide range of multi-view architectures for breast cancer classification, even in datasets of relatively modest size, although the detection of small lesions remains challenging without pixel-wise supervision or ad-hoc networks.

Ultrasound-based radiomics nomogram for predicting axillary lymph node metastasis in invasive breast cancer

OBJECTIVE

To develop a nomogram for predicting axillary lymph node metastasis (ALNM) in patients with invasive breast cancer.

METHODS

We included 307 patients with clinicopathologically confirmed invasive breast cancer. The cohort was divided into a training group (n=215) and a validation group (n=92). Ultrasound images were used to extract radiomics features. The least absolute shrinkage and selection operator (LASSO) algorithm helped select pertinent features, from which Radiomics Scores (Radscores) were calculated using the LASSO regression equation. We developed three logistic regression models based on Radscores and 2D image features, and assessed the models' performance in the validation group. A nomogram was created from the best-performing model.

RESULTS

In the training set, the area under the curve (AUC) for the Radscore model, 2D feature model, and combined model were 0.76, 0.85, and 0.88, respectively. In the validation set, the AUCs were 0.71, 0.78, and 0.83, respectively. The combined model demonstrated good calibration and promising clinical utility.

CONCLUSION

Our ultrasound-based radiomics nomogram can accurately and non-invasively predict ALNM in breast cancer, suggesting potential clinical applications to optimize surgical and medical strategies.

Application of deep learning on mammographies to discriminate between low and high-risk DCIS for patient participation in active surveillance trials

BACKGROUND

Ductal Carcinoma In Situ (DCIS) can progress to invasive breast cancer, but most DCIS lesions never will. Therefore, four clinical trials (COMET, LORIS, LORETTA, AND LORD) test whether active surveillance for women with low-risk Ductal carcinoma In Situ is safe (E. S. Hwang et al., BMJ Open, 9: e026797, 2019, A. Francis et al., Eur J Cancer. 51: 2296-2303, 2015, Chizuko Kanbayashi et al. The international collaboration of active surveillance trials for low-risk DCIS (LORIS, LORD, COMET, LORETTA),  L. E. Elshof et al., Eur J Cancer, 51, 1497-510, 2015). Low-risk is defined as grade I or II DCIS. Because DCIS grade is a major eligibility criteria in these trials, it would be very helpful to assess DCIS grade on mammography, informed by grade assessed on DCIS histopathology in pre-surgery biopsies, since surgery will not be performed on a significant number of patients participating in these trials.

OBJECTIVE

To assess the performance and clinical utility of a convolutional neural network (CNN) in discriminating high-risk (grade III) DCIS and/or Invasive Breast Cancer (IBC) from low-risk (grade I/II) DCIS based on mammographic features. We explored whether the CNN could be used as a decision support tool, from excluding high-risk patients for active surveillance.

METHODS

In this single centre retrospective study, 464 patients diagnosed with DCIS based on pre-surgery biopsy between 2000 and 2014 were included. The collection of mammography images was partitioned on a patient-level into two subsets, one for training containing 80% of cases (371 cases, 681 images) and 20% (93 cases, 173 images) for testing. A deep learning model based on the U-Net CNN was trained and validated on 681 two-dimensional mammograms. Classification performance was assessed with the Area Under the Curve (AUC) receiver operating characteristic and predictive values on the test set for predicting high risk DCIS-and high-risk DCIS and/ or IBC from low-risk DCIS.

RESULTS

When classifying DCIS as high-risk, the deep learning network achieved a Positive Predictive Value (PPV) of 0.40, Negative Predictive Value (NPV) of 0.91 and an AUC of 0.72 on the test dataset. For distinguishing high-risk and/or upstaged DCIS (occult invasive breast cancer) from low-risk DCIS a PPV of 0.80, a NPV of 0.84 and an AUC of 0.76 were achieved.

CONCLUSION

For both scenarios (DCIS grade I/II vs. III, DCIS grade I/II vs. III and/or IBC) AUCs were high, 0.72 and 0.76, respectively, concluding that our convolutional neural network can discriminate low-grade from high-grade DCIS.

Detailed Image Data Quality and Cleaning Practices for Artificial Intelligence Tools for Breast Cancer

Standardizing image-data preparation practices to improve accuracy/consistency of AI diagnostic tools.

Barriers and facilitators of artificial intelligence conception and implementation for breast imaging diagnosis in clinical practice: a scoping review

OBJECTIVE

Although artificial intelligence (AI) has demonstrated promise in enhancing breast cancer diagnosis, the implementation of AI algorithms in clinical practice encounters various barriers. This scoping review aims to identify these barriers and facilitators to highlight key considerations for developing and implementing AI solutions in breast cancer imaging.

METHOD

A literature search was conducted from 2012 to 2022 in six databases (PubMed, Web of Science, CINHAL, Embase, IEEE, and ArXiv). The articles were included if some barriers and/or facilitators in the conception or implementation of AI in breast clinical imaging were described. We excluded research only focusing on performance, or with data not acquired in a clinical radiology setup and not involving real patients.

RESULTS

A total of 107 articles were included. We identified six major barriers related to data (B1), black box and trust (B2), algorithms and conception (B3), evaluation and validation (B4), legal, ethical, and economic issues (B5), and education (B6), and five major facilitators covering data (F1), clinical impact (F2), algorithms and conception (F3), evaluation and validation (F4), and education (F5).

CONCLUSION

This scoping review highlighted the need to carefully design, deploy, and evaluate AI solutions in clinical practice, involving all stakeholders to yield improvement in healthcare.

CLINICAL RELEVANCE STATEMENT

The identification of barriers and facilitators with suggested solutions can guide and inform future research, and stakeholders to improve the design and implementation of AI for breast cancer detection in clinical practice.

KEY POINTS

• Six major identified barriers were related to data; black-box and trust; algorithms and conception; evaluation and validation; legal, ethical, and economic issues; and education. • Five major identified facilitators were related to data, clinical impact, algorithms and conception, evaluation and validation, and education. • Coordinated implication of all stakeholders is required to improve breast cancer diagnosis with AI.

Artificial Intelligence-Enhanced Breast MRI: Applications in Breast Cancer Primary Treatment Response Assessment and Prediction

ABSTRACT

Primary systemic therapy (PST) is the treatment of choice in patients with locally advanced breast cancer and is nowadays also often used in patients with early-stage breast cancer. Although imaging remains pivotal to assess response to PST accurately, the use of imaging to predict response to PST has the potential to not only better prognostication but also allow the de-escalation or omission of potentially toxic treatment with undesirable adverse effects, the accelerated implementation of new targeted therapies, and the mitigation of surgical delays in selected patients. In response to the limited ability of radiologists to predict response to PST via qualitative, subjective assessments of tumors on magnetic resonance imaging (MRI), artificial intelligence-enhanced MRI with classical machine learning, and in more recent times, deep learning, have been used with promising results to predict response, both before the start of PST and in the early stages of treatment. This review provides an overview of the current applications of artificial intelligence to MRI in assessing and predicting response to PST, and discusses the challenges and limitations of their clinical implementation.

Preliminary exploration of deep learning-assisted recognition of superior labrum anterior and posterior lesions in shoulder MR arthrography

PURPOSE

MR arthrography (MRA) is the most accurate method for preoperatively diagnosing superior labrum anterior-posterior (SLAP) lesions, but diagnostic results can vary considerably due to factors such as experience. In this study, deep learning was used to facilitate the preliminary identification of SLAP lesions and compared with radiologists of different seniority.

METHODS

MRA data from 636 patients were retrospectively collected, and all patients were classified as having/not having SLAP lesions according to shoulder arthroscopy. The SLAP-Net model was built and tested on 514 patients (dataset 1) and independently tested on data from two other MRI devices (122 patients, dataset 2). Manual diagnosis was performed by three radiologists with different seniority levels and compared with SLAP-Net outputs. Model performance was evaluated by the receiver operating characteristic (ROC) curve, area under the ROC curve (AUC), etc. McNemar's test was used to compare performance among models and between radiologists' models. The intraclass correlation coefficient (ICC) was used to assess the radiologists' reliability. p < 0.05 was considered statistically significant.

RESULTS

SLAP-Net had AUC = 0.98 and accuracy = 0.96 for classification in dataset 1 and AUC = 0.92 and accuracy = 0.85 in dataset 2. In dataset 1, SLAP-Net had diagnostic performance similar to that of senior radiologists (p = 0.055) but higher than that of early- and mid-career radiologists (p = 0.025 and 0.011). In dataset 2, SLAP-Net had similar diagnostic performance to radiologists of all three seniority levels (p = 0.468, 0.289, and 0.495, respectively).

CONCLUSIONS

Deep learning can be used to identify SLAP lesions upon initial MR arthrography examination. SLAP-Net performs comparably to senior radiologists.

Norwegian radiologists' expectations of artificial intelligence in mammographic screening - A cross-sectional survey

PURPOSE

To explore Norwegian breast radiologists' expectations of adding artificial intelligence (AI) in the interpretation procedure of screening mammograms.

METHODS

All breast radiologists involved in interpretation of screening mammograms in BreastScreen Norway during 2021 and 2022 (n = 98) were invited to take part in this anonymous cross-sectional survey about use of AI in mammographic screening. The questionnaire included background information of the respondents, their expectations, considerations of biases, and ethical and social implications of implementing AI in screen reading. Data was collected digitally and analyzed using descriptive statistics.

RESULTS

The response rate was 61% (60/98), and 67% (40/60) of the respondents were women. Sixty percent (36/60) reported ≥10 years' experience in screen reading, while 82% (49/60) reported no or limited experience with AI in health care. Eighty-two percent of the respondents were positive to explore AI in the interpretation procedure in mammographic screening. When used as decision support, 68% (41/60) expected AI to increase the radiologists' sensitivity for cancer detection. As potential challenges, 55% (33/60) reported lack of trust in the AI system and 45% (27/60) reported discrepancy between radiologists and AI systems as possible challenges. The risk of automation bias was considered high among 47% (28/60). Reduced time spent reading mammograms was rated as a potential benefit by 70% (42/60).

CONCLUSION

The radiologists reported positive expectations of AI in the interpretation procedure of screening mammograms. Efforts to minimize the risk of automation bias and increase trust in the AI systems are important before and during future implementation of the tool.

Combining Molecular and Radiomic Features for Risk Assessment in Breast Cancer

Breast cancer risk is highly variable within the population and current research is leading the shift toward personalized medicine. By accurately assessing an individual woman's risk, we can reduce the risk of over/undertreatment by preventing unnecessary procedures or by elevating screening procedures. Breast density measured from conventional mammography has been established as one of the most dominant risk factors for breast cancer; however, it is currently limited by its ability to characterize more complex breast parenchymal patterns that have been shown to provide additional information to strengthen cancer risk models. Molecular factors ranging from high penetrance, or high likelihood that a mutation will show signs and symptoms of the disease, to combinations of gene mutations with low penetrance have shown promise for augmenting risk assessment. Although imaging biomarkers and molecular biomarkers have both individually demonstrated improved performance in risk assessment, few studies have evaluated them together. This review aims to highlight the current state of the art in breast cancer risk assessment using imaging and genetic biomarkers.

Predicting pathological complete response to neoadjuvant systemic therapy for triple-negative breast cancers using deep learning on multiparametric MRIs

We trained and validated a deep learning model that can predict the treatment response to neoadjuvant systemic therapy (NAST) for patients with triple negative breast cancer (TNBC). Dynamic contrast enhanced (DCE) MRI and diffusion-weighted imaging (DWI) of the pre-treatment (baseline) and after four cycles (C4) of doxorubicin/cyclophosphamide treatment were used as inputs to the model for prediction of pathologic complete response (pCR). Based on the standard pCR definition that includes disease status in either breast or axilla, the model achieved areas under the receiver operating characteristic curves (AUCs) of 0.96 ± 0.05, 0.78 ± 0.09, 0.88 ± 0.02, and 0.76 ± 0.03, for the training, validation, testing, and prospective testing groups, respectively. For the pCR status of breast only, the retrained model achieved prediction AUCs of 0.97 ± 0.04, 0.82 ± 0.10, 0.86 ± 0.03, and 0.83 ± 0.02, for the training, validation, testing, and prospective testing groups, respectively. Thus, the developed deep learning model is highly promising for predicting the treatment response to NAST of TNBC.Clinical Relevance- Deep learning based on serial and multiparametric MRIs can potentially distinguish TNBC patients with pCR from non-pCR at the early stage of neoadjuvant systemic therapy, potentially enabling more personalized treatment of TNBC patients.

Deep learning-based classification of breast lesions using dynamic ultrasound video

PURPOSE

We intended to develop a deep-learning-based classification model based on breast ultrasound dynamic video, then evaluate its diagnostic performance in comparison with the classic model based on ultrasound static image and that of different radiologists.

METHOD

We collected 1000 breast lesions from 888 patients from May 2020 to December 2021. Each lesion contained two static images and two dynamic videos. We divided these lesions randomly into training, validation, and test sets by the ratio of 7:2:1. Two deep learning (DL) models, namely DL-video and DL-image, were developed based on 3D Resnet-50 and 2D Resnet-50 using 2000 dynamic videos and 2000 static images, respectively. Lesions in the test set were evaluated to compare the diagnostic performance of two models and six radiologists with different seniority.

RESULTS

The area under the curve of the DL-video model was significantly higher than those of the DL-image model (0.969 vs. 0.925, P = 0.0172) and six radiologists (0.969 vs. 0.779-0.912, P < 0.05). All radiologists performed better when evaluating the dynamic videos compared to the static images. Furthermore, radiologists performed better with increased seniority both in reading images and videos.

CONCLUSIONS

The DL-video model can discern more detailed spatial and temporal information for accurate classification of breast lesions than the conventional DL-image model and radiologists, and its clinical application can further improve the diagnosis of breast cancer.

Prediction of pathologic complete response to neoadjuvant systemic therapy in triple negative breast cancer using deep learning on multiparametric MRI

Triple-negative breast cancer (TNBC) is an aggressive subtype of breast cancer. Neoadjuvant systemic therapy (NAST) followed by surgery are currently standard of care for TNBC with 50-60% of patients achieving pathologic complete response (pCR). We investigated ability of deep learning (DL) on dynamic contrast enhanced (DCE) MRI and diffusion weighted imaging acquired early during NAST to predict TNBC patients' pCR status in the breast. During the development phase using the images of 130 TNBC patients, the DL model achieved areas under the receiver operating characteristic curves (AUCs) of 0.97 ± 0.04 and 0.82 ± 0.10 for the training and the validation, respectively. The model achieved an AUC of 0.86 ± 0.03 when evaluated in the independent testing group of 32 patients. In an additional prospective blinded testing group of 48 patients, the model achieved an AUC of 0.83 ± 0.02. These results demonstrated that DL based on multiparametric MRI can potentially differentiate TNBC patients with pCR or non-pCR in the breast early during NAST.

Beyond high hopes: A scoping review of the 2019-2021 scientific discourse on machine learning in medical imaging

Machine learning has become a key driver of the digital health revolution. That comes with a fair share of high hopes and hype. We conducted a scoping review on machine learning in medical imaging, providing a comprehensive outlook of the field's potential, limitations, and future directions. Most reported strengths and promises included: improved (a) analytic power, (b) efficiency (c) decision making, and (d) equity. Most reported challenges included: (a) structural barriers and imaging heterogeneity, (b) scarcity of well-annotated, representative and interconnected imaging datasets (c) validity and performance limitations, including bias and equity issues, and (d) the still missing clinical integration. The boundaries between strengths and challenges, with cross-cutting ethical and regulatory implications, remain blurred. The literature emphasizes explainability and trustworthiness, with a largely missing discussion about the specific technical and regulatory challenges surrounding these concepts. Future trends are expected to shift towards multi-source models, combining imaging with an array of other data, in a more open access, and explainable manner.

Artificial Intelligence (AI) in Breast Imaging: A Scientometric Umbrella Review

Artificial intelligence (AI), a rousing advancement disrupting a wide spectrum of applications with remarkable betterment, has continued to gain momentum over the past decades. Within breast imaging, AI, especially machine learning and deep learning, honed with unlimited cross-data/case referencing, has found great utility encompassing four facets: screening and detection, diagnosis, disease monitoring, and data management as a whole. Over the years, breast cancer has been the apex of the cancer cumulative risk ranking for women across the six continents, existing in variegated forms and offering a complicated context in medical decisions. Realizing the ever-increasing demand for quality healthcare, contemporary AI has been envisioned to make great strides in clinical data management and perception, with the capability to detect indeterminate significance, predict prognostication, and correlate available data into a meaningful clinical endpoint. Here, the authors captured the review works over the past decades, focusing on AI in breast imaging, and systematized the included works into one usable document, which is termed an umbrella review. The present study aims to provide a panoramic view of how AI is poised to enhance breast imaging procedures. Evidence-based scientometric analysis was performed in accordance with the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) guideline, resulting in 71 included review works. This study aims to synthesize, collate, and correlate the included review works, thereby identifying the patterns, trends, quality, and types of the included works, captured by the structured search strategy. The present study is intended to serve as a "one-stop center" synthesis and provide a holistic bird's eye view to readers, ranging from newcomers to existing researchers and relevant stakeholders, on the topic of interest.

Ultrasonography and clinicopathological features of breast cancer in predicting axillary lymph node metastases

Background

Early identification of axillary lymph node metastasis (ALNM) in breast cancer (BC) is still a clinical difficulty. There is still no good method to replace sentinel lymph node biopsy (SLNB). The purpose of our study was to develop and validate a nomogram to predict the probability of ALNM preoperatively based on ultrasonography (US) and clinicopathological features of primary tumors.

Methods

From September 2019 to April 2022, the preoperative US) and clinicopathological data of 1076 T1-T2 BC patients underwent surgical treatment were collected. Patients were divided into a training set (875 patients from September 2019 to October 2021) and a validation set (201 patients from November 2021 to April 2022). Patients were divided into positive and negative axillary lymph node (ALN) group according pathology of axillary surgery. Compared the US and clinicopathological features between the two groups. The risk factors for ALNM were determined using multivariate logistic regression analysis, and a nomogram was constructed. AUC and calibration were used to assess its performance.

Results

By univariate and multivariate logistic regression analysis, age (p = 0.009), histologic grades (p = 0.000), molecular subtypes (p = 0.000), tumor location (p = 0.000), maximum diameter (p = 0.000), spiculated margin (p = 0.000) and distance from the skin (p = 0.000) were independent risk factors of ALNM. Then a nomogram was developed. The model was good discriminating with an AUC of 0.705 and 0.745 for the training and validation set, respectively. And the calibration curves demonstrated high agreement. However, in further predicting a heavy nodal disease burden (> 2 nodes), none of the variables were significant.

Conclusion

This nomogram based on the US and clinicopathological data can predict the presence of ALNM good in T1-T2 BC patients. But it cannot effectively predict a heavy nodal disease burden (> 2 nodes).

External Validation of an Ensemble Model for Automated Mammography Interpretation by Artificial Intelligence

Importance

With a shortfall in fellowship-trained breast radiologists, mammography screening programs are looking toward artificial intelligence (AI) to increase efficiency and diagnostic accuracy. External validation studies provide an initial assessment of how promising AI algorithms perform in different practice settings.

Objective

To externally validate an ensemble deep-learning model using data from a high-volume, distributed screening program of an academic health system with a diverse patient population.

Design, Setting, and Participants

In this diagnostic study, an ensemble learning method, which reweights outputs of the 11 highest-performing individual AI models from the Digital Mammography Dialogue on Reverse Engineering Assessment and Methods (DREAM) Mammography Challenge, was used to predict the cancer status of an individual using a standard set of screening mammography images. This study was conducted using retrospective patient data collected between 2010 and 2020 from women aged 40 years and older who underwent a routine breast screening examination and participated in the Athena Breast Health Network at the University of California, Los Angeles (UCLA).

Main Outcomes and Measures

Performance of the challenge ensemble method (CEM) and the CEM combined with radiologist assessment (CEM+R) were compared with diagnosed ductal carcinoma in situ and invasive cancers within a year of the screening examination using performance metrics, such as sensitivity, specificity, and area under the receiver operating characteristic curve (AUROC).

Results

Evaluated on 37 317 examinations from 26 817 women (mean [SD] age, 58.4 [11.5] years), individual model AUROC estimates ranged from 0.77 (95% CI, 0.75-0.79) to 0.83 (95% CI, 0.81-0.85). The CEM model achieved an AUROC of 0.85 (95% CI, 0.84-0.87) in the UCLA cohort, lower than the performance achieved in the Kaiser Permanente Washington (AUROC, 0.90) and Karolinska Institute (AUROC, 0.92) cohorts. The CEM+R model achieved a sensitivity (0.813 [95% CI, 0.781-0.843] vs 0.826 [95% CI, 0.795-0.856]; P = .20) and specificity (0.925 [95% CI, 0.916-0.934] vs 0.930 [95% CI, 0.929-0.932]; P = .18) similar to the radiologist performance. The CEM+R model had significantly lower sensitivity (0.596 [95% CI, 0.466-0.717] vs 0.850 [95% CI, 0.766-0.923]; P < .001) and specificity (0.803 [95% CI, 0.734-0.861] vs 0.945 [95% CI, 0.936-0.954]; P < .001) than the radiologist in women with a prior history of breast cancer and Hispanic women (0.894 [95% CI, 0.873-0.910] vs 0.926 [95% CI, 0.919-0.933]; P = .004).

Conclusions and Relevance

This study found that the high performance of an ensemble deep-learning model for automated screening mammography interpretation did not generalize to a more diverse screening cohort, suggesting that the model experienced underspecification. This study suggests the need for model transparency and fine-tuning of AI models for specific target populations prior to their clinical adoption.

Development and validation of a deep learning model for breast lesion segmentation and characterization in multiparametric MRI

Importance

The utilization of artificial intelligence for the differentiation of benign and malignant breast lesions in multiparametric MRI (mpMRI) assists radiologists to improve diagnostic performance.

Objectives

To develop an automated deep learning model for breast lesion segmentation and characterization and to evaluate the characterization performance of AI models and radiologists.

Materials and methods

For lesion segmentation, 2,823 patients were used for the training, validation, and testing of the VNet-based segmentation models, and the average Dice similarity coefficient (DSC) between the manual segmentation by radiologists and the mask generated by VNet was calculated. For lesion characterization, 3,303 female patients with 3,607 pathologically confirmed lesions (2,213 malignant and 1,394 benign lesions) were used for the three ResNet-based characterization models (two single-input and one multi-input models). Histopathology was used as the diagnostic criterion standard to assess the characterization performance of the AI models and the BI-RADS categorized by the radiologists, in terms of sensitivity, specificity, accuracy, and the area under the receiver operating characteristic curve (AUC). An additional 123 patients with 136 lesions (81 malignant and 55 benign lesions) from another institution were available for external testing.

Results

Of the 5,811 patients included in the study, the mean age was 46.14 (range 11–89) years. In the segmentation task, a DSC of 0.860 was obtained between the VNet-generated mask and manual segmentation by radiologists. In the characterization task, the AUCs of the multi-input and the other two single-input models were 0.927, 0.821, and 0.795, respectively. Compared to the single-input DWI or DCE model, the multi-input DCE and DWI model obtained a significant increase in sensitivity, specificity, and accuracy (0.831 vs. 0.772/0.776, 0.874 vs. 0.630/0.709, 0.846 vs. 0.721/0.752). Furthermore, the specificity of the multi-input model was higher than that of the radiologists, whether using BI-RADS category 3 or 4 as a cutoff point (0.874 vs. 0.404/0.841), and the accuracy was intermediate between the two assessment methods (0.846 vs. 0.773/0.882). For the external testing, the performance of the three models remained robust with AUCs of 0.812, 0.831, and 0.885, respectively.

Conclusions

Combining DCE with DWI was superior to applying a single sequence for breast lesion characterization. The deep learning computer-aided diagnosis (CADx) model we developed significantly improved specificity and achieved comparable accuracy to the radiologists with promise for clinical application to provide preliminary diagnoses.

Texture Analysis of Enhanced MRI and Pathological Slides Predicts EGFR Mutation Status in Breast Cancer

Objective

Image texture information was extracted from enhanced magnetic resonance imaging (MRI) and pathological hematoxylin and eosin- (HE-) stained images of female breast cancer patients. We established models individually, and then, we combine the two kinds of data to establish model. Through this method, we verified whether sufficient information could be obtained from enhanced MRI and pathological slides to assist in the determination of epidermal growth factor receptor (EGFR) mutation status in patients.

Methods

We obtained enhanced MRI data from patients with breast cancer before treatment and selected diffusion-weighted imaging (DWI), T1 fast-spin echo (T1 FSE), and T2 fast-spin echo (T2 FSE) as the data sources for extracting texture information. Imaging physicians manually outlined the 3D regions of interest (ROIs) and extracted texture features according to the gray level cooccurrence matrix (GLCM) of the images. For the HE staining images of the patients, we adopted a specific normalization algorithm to simulate the images dyed with only hematoxylin or eosin and extracted textures. We extracted texture features to predict the expression of EGFR. After evaluating the predictive power of each model, the models from the two data sources were combined for remodeling.

Results

For enhanced MRI data, the modeling of texture information of T1 FSE had a good predictive effect for EGFR mutation status. For pathological images, eosin-stained images can achieve a better prediction effect. We selected these two classifiers as the weak classifiers of the final model and obtained good results (training group: AUC, 0.983; 95% CI, 0.95-1.00; accuracy, 0.962; specificity, 0.936; and sensitivity, 0.979; test group: AUC, 0.983; 95% CI, 0.94-1.00; accuracy, 0.943; specificity, 1.00; and sensitivity, 0.905).

Conclusion

The EGFR mutation status of patients with breast cancer can be well predicted based on enhanced MRI data and pathological data. This helps hospitals that do not test the EGFR mutation status of patients with breast cancer. The technology gives clinicians more information about breast cancer, which helps them make accurate diagnoses and select suitable treatments.

Multi-Task Fusion for Improving Mammography Screening Data Classification

Machine learning and deep learning methods have become essential for computer-assisted prediction in medicine, with a growing number of applications also in the field of mammography. Typically these algorithms are trained for a specific task, e.g., the classification of lesions or the prediction of a mammogram's pathology status. To obtain a comprehensive view of a patient, models which were all trained for the same task(s) are subsequently ensembled or combined. In this work, we propose a pipeline approach, where we first train a set of individual, task-specific models and subsequently investigate the fusion thereof, which is in contrast to the standard model ensembling strategy. We fuse model predictions and high-level features from deep learning models with hybrid patient models to build stronger predictors on patient level. To this end, we propose a multi-branch deep learning model which efficiently fuses features across different tasks and mammograms to obtain a comprehensive patient-level prediction. We train and evaluate our full pipeline on public mammography data, i.e., DDSM and its curated version CBIS-DDSM, and report an AUC score of 0.962 for predicting the presence of any lesion and 0.791 for predicting the presence of malignant lesions on patient level. Overall, our fusion approaches improve AUC scores significantly by up to 0.04 compared to standard model ensembling. Moreover, by providing not only global patient-level predictions but also task-specific model results that are related to radiological features, our pipeline aims to closely support the reading workflow of radiologists.

An artificial intelligence system using maximum intensity projection MR images facilitates classification of non-mass enhancement breast lesions

OBJECTIVES

To build an artificial intelligence (AI) system to classify benign and malignant non-mass enhancement (NME) lesions using maximum intensity projection (MIP) of early post-contrast subtracted breast MR images.

METHODS

This retrospective study collected 965 pure NME lesions (539 benign and 426 malignant) confirmed by histopathology or follow-up in 903 women. The 754 NME lesions acquired by one MR scanner were randomly split into the training set, validation set, and test set A (482/121/151 lesions). The 211 NME lesions acquired by another MR scanner were used as test set B. The AI system was developed using ResNet-50 with the axial and sagittal MIP images. One senior and one junior radiologist reviewed the MIP images of each case independently and rated its Breast Imaging Reporting and Data System category. The performance of the AI system and the radiologists was evaluated using the area under the receiver operating characteristic curve (AUC).

RESULTS

The AI system yielded AUCs of 0.859 and 0.816 in the test sets A and B, respectively. The AI system achieved comparable performance as the senior radiologist (p = 0.558, p = 0.041) and outperformed the junior radiologist (p < 0.001, p = 0.009) in both test sets A and B. After AI assistance, the AUC of the junior radiologist increased from 0.740 to 0.862 in test set A (p < 0.001) and from 0.732 to 0.843 in test set B (p < 0.001).

CONCLUSION

Our MIP-based AI system yielded good applicability in classifying NME lesions in breast MRI and can assist the junior radiologist achieve better performance.

KEY POINTS

• Our MIP-based AI system yielded good applicability in the dataset both from the same and a different MR scanner in predicting malignant NME lesions. • The AI system achieved comparable diagnostic performance with the senior radiologist and outperformed the junior radiologist. • This AI system can assist the junior radiologist achieve better performance in the classification of NME lesions in MRI.

Leveraging Multi-Task Learning to Cope With Poor and Missing Labels of Mammograms

In breast cancer screening, binary classification of mammograms is a common task aiming to determine whether a case is malignant or benign. A Computer-Aided Diagnosis (CADx) system based on a trainable classifier requires clean data and labels coming from a confirmed diagnosis. Unfortunately, such labels are not easy to obtain in clinical practice, since the histopathological reports of biopsy may not be available alongside mammograms, while normal cases may not have an explicit follow-up confirmation. Such ambiguities result either in reducing the number of samples eligible for training or in a label uncertainty that may decrease the performances. In this work, we maximize the number of samples for training relying on multi-task learning. We design a deep-neural-network-based classifier yielding multiple outputs in one forward pass. The predicted classes include binary malignancy, cancer probability estimation, breast density, and image laterality. Since few samples have all classes available and confirmed, we propose to introduce the uncertainty related to the classes as a per-sample weight during training. Such weighting prevents updating the network's parameters when training on uncertain or missing labels. We evaluate our approach on the public INBreast and private datasets, showing statistically significant improvements compared to baseline and independent state-of-the-art approaches. Moreover, we use mammograms from Susan G. Komen Tissue Bank for fine-tuning, further demonstrating the ability to improve the performances in our multi-task learning setup from raw clinical data. We achieved the binary classification performance of AUC = 80.46 on our private dataset and AUC = 85.23 on the INBreast dataset.

Influence of the Computer-Aided Decision Support System Design on Ultrasound-Based Breast Cancer Classification

Automation of medical data analysis is an important topic in modern cancer diagnostics, aiming at robust and reproducible workflows. Therefore, we used a dataset of breast US images (252 malignant and 253 benign cases) to realize and compare different strategies for CAD support in lesion detection and classification. Eight different datasets (including pre-processed and spatially augmented images) were prepared, and machine learning algorithms (i.e., Viola-Jones; YOLOv3) were trained for lesion detection. The radiomics signature (RS) was derived from detection boxes and compared with RS derived from manually obtained segments. Finally, the classification model was established and evaluated concerning accuracy, sensitivity, specificity, and area under the Receiver Operating Characteristic curve. After training on a dataset including logarithmic derivatives of US images, we found that YOLOv3 obtains better results in breast lesion detection (IoU: 0.544 ± 0.081; LE: 0.171 ± 0.009) than the Viola-Jones framework (IoU: 0.399 ± 0.054; LE: 0.096 ± 0.016). Interestingly, our findings show that the classification model trained with RS derived from detection boxes and the model based on the RS derived from a gold standard manual segmentation are comparable (p-value = 0.071). Thus, deriving radiomics signatures from the detection box is a promising technique for building a breast lesion classification model, and may reduce the need for the lesion segmentation step in the future design of CAD systems.

Radiomics and deep learning approach to the differential diagnosis of parotid gland tumors

PURPOSE OF REVIEW

Advances in computer technology and growing expectations from computer-aided systems have led to the evolution of artificial intelligence into subsets, such as deep learning and radiomics, and the use of these systems is revolutionizing modern radiological diagnosis. In this review, artificial intelligence applications developed with radiomics and deep learning methods in the differential diagnosis of parotid gland tumors (PGTs) will be overviewed.

RECENT FINDINGS

The development of artificial intelligence models has opened new scenarios owing to the possibility of assessing features of medical images that usually are not evaluated by physicians. Radiomics and deep learning models come to the forefront in computer-aided diagnosis of medical images, even though their applications in the differential diagnosis of PGTs have been limited because of the scarcity of data sets related to these rare neoplasms. Nevertheless, recent studies have shown that artificial intelligence tools can classify common PGTs with reasonable accuracy.

SUMMARY

All studies aimed at the differential diagnosis of benign vs. malignant PGTs or the identification of the commonest PGT subtypes were identified, and five studies were found that focused on deep learning-based differential diagnosis of PGTs. Data sets were created in three of these studies with MRI and in two with computed tomography (CT). Additional seven studies were related to radiomics. Of these, four were on MRI-based radiomics, two on CT-based radiomics, and one compared MRI and CT-based radiomics in the same patients.