Deep Learning-Based Radiomics of B-Mode Ultrasonography and Shear-Wave Elastography: Improved Performance in Breast Mass Classification

Comparative analysis of convolutional neural networks and vision transformers in identifying benign and malignant breast lesions

Various deep learning models have been developed and employed for medical image classification. This study conducted comprehensive experiments on 12 models, aiming to establish reliable benchmarks for research on breast dynamic contrast-enhanced magnetic resonance imaging image classification. Twelve deep learning models were systematically compared by analyzing variations in 4 key hyperparameters: optimizer (Op), learning rate, batch size (BS), and data augmentation. The evaluation criteria encompassed a comprehensive set of metrics including accuracy (Ac), loss value, precision, recall rate, F1-score, and area under the receiver operating characteristic curve. Furthermore, the training times and model parameter counts were assessed for holistic performance comparison. Adjustments in the BS within Adam Op had a minimal impact on Ac in the convolutional neural network models. However, altering the Op and learning rate while maintaining the same BS significantly affected the Ac. The ResNet152 network model exhibited the lowest Ac. Both the recall rate and area under the receiver operating characteristic curve for the ResNet152 and Vision transformer-base (ViT) models were inferior compared to the others. Data augmentation unexpectedly reduced the Ac of ResNet50, ResNet152, VGG16, VGG19, and ViT models. The VGG16 model boasted the shortest training duration, whereas the ViT model, before data augmentation, had the longest training time and smallest model weight. The ResNet152 and ViT models were not well suited for image classification tasks involving small breast dynamic contrast-enhanced magnetic resonance imaging datasets. Although data augmentation is typically beneficial, its application should be approached cautiously. These findings provide important insights to inform and refine future research in this domain.

Variational mode directed deep learning framework for breast lesion classification using ultrasound imaging

Breast cancer is the most prevalent cancer and the second cause of cancer related death among women in the United States. Accurate and early detection of breast cancer can reduce the number of mortalities. Recent works explore deep learning techniques with ultrasound for detecting malignant breast lesions. However, the lack of explanatory features, need for segmentation, and high computational complexity limit their applicability in this detection. Therefore, we propose a novel ultrasound-based breast lesion classification framework that utilizes two-dimensional variational mode decomposition (2D-VMD) which provides self-explanatory features for guiding a convolutional neural network (CNN) with mixed pooling and attention mechanisms. The visual inspection of these features demonstrates their explainability in terms of discriminative lesion-specific boundary and texture in the decomposed modes of benign and malignant images, which further guide the deep learning network for enhanced classification. The proposed framework can classify the lesions with accuracies of 98% and 93% in two public breast ultrasound datasets and 89% in an in-house dataset without having to segment the lesions unlike existing techniques, along with an optimal trade-off between the sensitivity and specificity. 2D-VMD improves the areas under the receiver operating characteristics and precision-recall curves by 5% and 10% respectively. The proposed method achieves relative improvement of 14.47%(8.42%) (mean (SD)) in accuracy over state-of-the-art methods for one public dataset, and 5.75%(4.52%) for another public dataset with comparable performance to two existing methods. Further, it is computationally efficient with a reduction of [Formula: see text] in floating point operations as compared to existing methods.

Attention-based Fusion Network for Breast Cancer Segmentation and Classification Using Multi-modal Ultrasound Images

OBJECTIVE

Breast cancer is one of the most commonly occurring cancers in women. Thus, early detection and treatment of cancer lead to a better outcome for the patient. Ultrasound (US) imaging plays a crucial role in the early detection of breast cancer, providing a cost-effective, convenient, and safe diagnostic approach. To date, much research has been conducted to facilitate reliable and effective early diagnosis of breast cancer through US image analysis. Recently, with the introduction of machine learning technologies such as deep learning (DL), automated lesion segmentation and classification, the identification of malignant masses in US breasts has progressed, and computer-aided diagnosis (CAD) technology is being applied in clinics effectively. Herein, we propose a novel deep learning-based "segmentation + classification" model based on B- and SE-mode images.

METHODS

For the segmentation task, we propose a Multi-Modal Fusion U-Net (MMF-U-Net), which segments lesions by mixing B- and SE-mode information through fusion blocks. After segmenting, the lesion area from the B- and SE-mode images is cropped using a predicted segmentation mask. The encoder part of the pre-trained MMF-U-Net model is then used on the cropped B- and SE-mode breast US images to classify benign and malignant lesions.

RESULTS

The experimental results using the proposed method showed good segmentation and classification scores. The dice score, intersection over union (IoU), precision, and recall are 78.23%, 68.60%, 82.21%, and 80.58%, respectively, using the proposed MMF-U-Net on real-world clinical data. The classification accuracy is 98.46%.

CONCLUSION

Our results show that the proposed method will effectively segment the breast lesion area and can reliably classify the benign from malignant lesions.

BUSClean: Open-source software for breast ultrasound image pre-processing and knowledge extraction for medical AI

Development of artificial intelligence (AI) for medical imaging demands curation and cleaning of large-scale clinical datasets comprising hundreds of thousands of images. Some modalities, such as mammography, contain highly standardized imaging. In contrast, breast ultrasound imaging (BUS) can contain many irregularities not indicated by scan metadata, such as enhanced scan modes, sonographer annotations, or additional views. We present an open-source software solution for automatically processing clinical BUS datasets. The algorithm performs BUS scan filtering (flagging of invalid and non-B-mode scans), cleaning (dual-view scan detection, scan area cropping, and caliper detection), and knowledge extraction (BI-RADS Labeling and Measurement fields) from sonographer annotations. Its modular design enables users to adapt it to new settings. Experiments on an internal testing dataset of 430 clinical BUS images achieve >95% sensitivity and >98% specificity in detecting every type of text annotation, >98% sensitivity and specificity in detecting scans with blood flow highlighting, alternative scan modes, or invalid scans. A case study on a completely external, public dataset of BUS scans found that BUSClean identified text annotations and scans with blood flow highlighting with 88.6% and 90.9% sensitivity and 98.3% and 99.9% specificity, respectively. Adaptation of the lesion caliper detection method to account for a type of caliper specific to the case study demonstrates the intended use of BUSClean in new data distributions and improved performance in lesion caliper detection from 43.3% and 93.3% out-of-the-box to 92.1% and 92.3% sensitivity and specificity, respectively. Source code, example notebooks, and sample data are available at https://github.com/hawaii-ai/bus-cleaning.

A convolutional attention model for predicting response to chemo-immunotherapy from ultrasound elastography in mouse tumor models

BACKGROUND

In the era of personalized cancer treatment, understanding the intrinsic heterogeneity of tumors is crucial. Despite some patients responding favorably to a particular treatment, others may not benefit, leading to the varied efficacy observed in standard therapies. This study focuses on the prediction of tumor response to chemo-immunotherapy, exploring the potential of tumor mechanics and medical imaging as predictive biomarkers. We have extensively studied "desmoplastic" tumors, characterized by a dense and very stiff stroma, which presents a substantial challenge for treatment. The increased stiffness of such tumors can be restored through pharmacological intervention with mechanotherapeutics.

METHODS

We developed a deep learning methodology based on shear wave elastography (SWE) images, which involved a convolutional neural network (CNN) model enhanced with attention modules. The model was developed and evaluated as a predictive biomarker in the setting of detecting responsive, stable, and non-responsive tumors to chemotherapy, immunotherapy, or the combination, following mechanotherapeutics administration. A dataset of 1365 SWE images was obtained from 630 tumors from our previous experiments and used to train and successfully evaluate our methodology. SWE in combination with deep learning models, has demonstrated promising results in disease diagnosis and tumor classification but their potential for predicting tumor response prior to therapy is not yet fully realized.

RESULTS

We present strong evidence that integrating SWE-derived biomarkers with automatic tumor segmentation algorithms enables accurate tumor detection and prediction of therapeutic outcomes.

CONCLUSIONS

This approach can enhance personalized cancer treatment by providing non-invasive, reliable predictions of therapeutic outcomes.

A hybrid features fusion-based framework for classification of breast micronodules using ultrasonography

BACKGROUND

Breast cancer is one of the leading diseases worldwide. According to estimates by the National Breast Cancer Foundation, over 42,000 women are expected to die from this disease in 2024.

OBJECTIVE

The prognosis of breast cancer depends on the early detection of breast micronodules and the ability to distinguish benign from malignant lesions. Ultrasonography is a crucial radiological imaging technique for diagnosing the illness because it allows for biopsy and lesion characterization. The user's level of experience and knowledge is vital since ultrasonographic diagnosis relies on the practitioner's expertise. Furthermore, computer-aided technologies significantly contribute by potentially reducing the workload of radiologists and enhancing their expertise, especially when combined with a large patient volume in a hospital setting.

METHOD

This work describes the development of a hybrid CNN system for diagnosing benign and malignant breast cancer lesions. The models InceptionV3 and MobileNetV2 serve as the foundation for the hybrid framework. Features from these models are extracted and concatenated individually, resulting in a larger feature set. Finally, various classifiers are applied for the classification task.

RESULTS

The model achieved the best results using the softmax classifier, with an accuracy of over 95%.

CONCLUSION

Computer-aided diagnosis greatly assists radiologists and reduces their workload. Therefore, this research can serve as a foundation for other researchers to build clinical solutions.

Multiview deep learning networks based on automated breast volume scanner images for identifying breast cancer in BI-RADS 4

Objectives

To develop and validate a deep learning (DL) based automatic segmentation and classification system to classify benign and malignant BI-RADS 4 lesions imaged with ABVS.

Methods

From May to December 2020, patients with BI-RADS 4 lesions from Centre 1 and Centre 2 were retrospectively enrolled and divided into a training set (Centre 1) and an independent test set (Centre 2). All included patients underwent an ABVS examination within one week before the biopsy. A two-stage DL framework consisting of an automatic segmentation module and an automatic classification module was developed. The preprocessed ABVS images were input into the segmentation module for BI-RADS 4 lesion segmentation. The classification model was constructed to extract features and output the probability of malignancy. The diagnostic performances among different ABVS views (axial, sagittal, coronal, and multi-view) and DL architectures (Inception-v3, ResNet 50, and MobileNet) were compared.

Results

A total of 251 BI-RADS 4 lesions from 216 patients were included (178 in the training set and 73 in the independent test set). The average Dice coefficient, precision, and recall of the segmentation module in the test set were 0.817 ± 0.142, 0.903 ± 0.183, and 0.886 ± 0.187, respectively. The DL model based on multiview ABVS images and Inception-v3 achieved the best performance, with an AUC, sensitivity, specificity, PPV, and NPV of 0.949 (95% CI: 0.945-0.953), 82.14%, 95.56%, 92.00%, and 89.58%, respectively, in the test set.

Conclusions

The developed multiview DL model enables automatic segmentation and classification of BI-RADS 4 lesions in ABVS images.

Ultrasound-based radiomics for the differential diagnosis of breast masses: A systematic review and meta-analysis

OBJECTIVES

Ultrasound-based radiomics has demonstrated excellent diagnostic performance in differentiating benign and malignant breast masses. Given a few clinical studies on their diagnostic role, we conducted a meta-analysis of the potential effects of ultrasound-based radiomics for the differential diagnosis of breast masses, aiming to provide evidence-based medical basis for clinical research.

MATERIALS AND METHODS

We searched Embase, Web of Science, Cochrane Library, and PubMed databases from inception through to February 2023. The methodological quality assessment of the included studies was performed according to Quality Assessment of Diagnostic Accuracy Studies checklist. A diagnostic test accuracy systematic review and meta-analysis was performed in accordance with PRISMA guidelines. Sensitivity, specificity, and area under curve delineating benign and malignant lesions were recorded. We also used sensitivity analysis and subgroup analysis to explore potential sources of heterogeneity. Deeks' funnel plots was used to examine the publication bias.

RESULTS

A total of 11 studies were included in this meta-analysis. For the diagnosis of malignant breast masses worldwide, the overall mean rates of sensitivity and specificity of ultrasound-based radiomics were 0.90 (95% confidence interval [CI], 0.83-0.95) and 0.89 (95% CI, 0.82-0.94), respectively. The summary diagnostic odds ratio was 76 (95% CI, 26-219), and the area under the curve for the summary receiver operating characteristic curve was 0.95 (95% CI, 0.93-0.97).

CONCLUSION

Ultrasound-based radiomics has the potential to improve diagnostic accuracy to discriminate between benign and malignant breast masses, and could reduce unnecessary biopsies.

Associated factors leading to misdiagnosis of a combined diagnostic model of different types of strain imaging and conventional ultrasound in evaluation of breast lesions: Selection strategy for using different types of strain imaging in evaluation of breast lesions

OBJECTIVE

To evaluate the effectiveness of a decision tree that integrates conventional ultrasound (CUS) with two different strain imaging (SI) techniques for diagnosing breast lesions, and to analyze the factors contributing to false negative (FN) and false positive (FP) in the decision tree's outcomes.

MATERIALS AND METHODS

Imaging and clinical data of 796 cases in the training set and 351 cases in the validation set were prospectively collected. A decision tree model that combines two types of SI and CUS was constructed, and its diagnostic performance was analyzed. Univariate analysis and multivariate analysis were applied to identify independent risk factors associated with FP and FN results of the decision tree model.

RESULTS

Size, shape, margin, vascularity, the types of internal calcifications, EI score and VTI pattern were found to be significantly independently associated with the diagnosis of benign and malignant breast lesions. Therefore, size, shape, margin, vascularity, EI score and VTI pattern were used to construct decision tree models. The Tree (EI+VTI) model had the highest AUC. Both in the training and validation groups, the AUC of Tree (EI+VTI) was significantly higher compared with that of EI, VTI, and BI-RADS (all, P < 0.05). Orientation, posterior acoustic features and the types of internal calcifications were significantly positively associated with misdiagnosis results of Tree (EI+VTI) in evaluation of breast lesions (all P < 0.05).

CONCLUSION

The diagnostic model based on a decision tree that integrates two distinct types of SI with CUS enhances the diagnostic accuracy of each method when used individually. This integration lowers the misdiagnosis rate, potentially assisting radiologists in more effective lesion assessments. When applying the decision tree model, attention should be paid to the orientation, posterior acoustic features, and the types of internal calcifications of the lesions.

CT radiomics based on different machine learning models for classifying gross tumor volume and normal liver tissue in hepatocellular carcinoma

BACKGROUND & AIMS

The present study utilized extracted computed tomography radiomics features to classify the gross tumor volume and normal liver tissue in hepatocellular carcinoma by mainstream machine learning methods, aiming to establish an automatic classification model.

METHODS

We recruited 104 pathologically confirmed hepatocellular carcinoma patients for this study. GTV and normal liver tissue samples were manually segmented into regions of interest and randomly divided into five-fold cross-validation groups. Dimensionality reduction using LASSO regression. Radiomics models were constructed via logistic regression, support vector machine (SVM), random forest, Xgboost, and Adaboost algorithms. The diagnostic efficacy, discrimination, and calibration of algorithms were verified using area under the receiver operating characteristic curve (AUC) analyses and calibration plot comparison.

RESULTS

Seven screened radiomics features excelled at distinguishing the gross tumor area. The Xgboost machine learning algorithm had the best discrimination and comprehensive diagnostic performance with an AUC of 0.9975 [95% confidence interval (CI): 0.9973-0.9978] and mean MCC of 0.9369. SVM had the second best discrimination and diagnostic performance with an AUC of 0.9846 (95% CI: 0.9835- 0.9857), mean Matthews correlation coefficient (MCC)of 0.9105, and a better calibration. All other algorithms showed an excellent ability to distinguish between gross tumor area and normal liver tissue (mean AUC 0.9825, 0.9861,0.9727,0.9644 for Adaboost, random forest, logistic regression, naivem Bayes algorithm respectively).

CONCLUSION

CT radiomics based on machine learning algorithms can accurately classify GTV and normal liver tissue, while the Xgboost and SVM algorithms served as the best complementary algorithms.

Discrimination of benign and malignant breast lesions on dynamic contrast-enhanced magnetic resonance imaging using deep learning

PURPOSE

To evaluate the capability of deep transfer learning (DTL) and fine-tuning methods in differentiating malignant from benign lesions in breast dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI).

METHODS

The diagnostic efficiencies of the VGG19, ResNet50, and DenseNet201 models were tested under the same dataset. The model with the highest performance was selected and modified utilizing three fine-tuning strategies (S1-3). Fifty additional lesions were selected to form the validation set to verify the generalization abilities of these models. The accuracy (Ac) of the different models in the training and test sets, as well as the precision (Pr), recall rate (Rc), F1 score (), and area under the receiver operating characteristic curve (AUC), were primary performance indicators. Finally, the kappa test was used to compare the degree of agreement between the DTL models and pathological diagnosis in differentiating malignant from benign breast lesions.

RESULTS

The Pr, Rc, f1, and AUC of VGG19 (86.0%, 0.81, 0.81, and 0.81, respectively) were higher than those of DenseNet201 (70.0%, 0.61, 0.63, and 0.61, respectively) and ResNet50 (61.0%, 0.59, 0.59, and 0.59). After fine-tuning, the Pr, Rc, f1, and AUC of S1 (87.0%, 0.86, 0.86, and 0.86, respectively) were higher than those of VGG19. Notably, the degree of agreement between S1 and pathological diagnosis in differentiating malignant from benign breast lesions was 0.720 (κ = 0.720), which was higher than that of DenseNet201 (κ = 0.440), VGG19 (κ = 0.640), and ResNet50 (κ = 0.280).

CONCLUSION

The VGG19 model is an effective method for identifying benign and malignant breast lesions on DCE-MRI, and its performance can be further improved via fine-tuning. Overall, our findings insinuate that this technique holds potential clinical application value.

Deep learning-based multimodal fusion network for segmentation and classification of breast cancers using B-mode and elastography ultrasound images

Ultrasonography is one of the key medical imaging modalities for evaluating breast lesions. For differentiating benign from malignant lesions, computer-aided diagnosis (CAD) systems have greatly assisted radiologists by automatically segmenting and identifying features of lesions. Here, we present deep learning (DL)-based methods to segment the lesions and then classify benign from malignant, utilizing both B-mode and strain elastography (SE-mode) images. We propose a weighted multimodal U-Net (W-MM-U-Net) model for segmenting lesions where optimum weight is assigned on different imaging modalities using a weighted-skip connection method to emphasize its importance. We design a multimodal fusion framework (MFF) on cropped B-mode and SE-mode ultrasound (US) lesion images to classify benign and malignant lesions. The MFF consists of an integrated feature network (IFN) and a decision network (DN). Unlike other recent fusion methods, the proposed MFF method can simultaneously learn complementary information from convolutional neural networks (CNNs) trained using B-mode and SE-mode US images. The features from the CNNs are ensembled using the multimodal EmbraceNet model and DN classifies the images using those features. The experimental results (sensitivity of 100 ± 0.00% and specificity of 94.28 ± 7.00%) on the real-world clinical data showed that the proposed method outperforms the existing single- and multimodal methods. The proposed method predicts seven benign patients as benign three times out of five trials and six malignant patients as malignant five out of five trials. The proposed method would potentially enhance the classification accuracy of radiologists for breast cancer detection in US images.

Ultrasound-based radiomics combined with immune status to predict sentinel lymph node metastasis in primary breast cancer

In the past few years, the axillary lymph node dissection technique has been steadily replaced by sentinel lymph node biopsy for treating and diagnosing breast cancer, thereby minimizing the complications and sequelae of the patients. Nevertheless, sentinel lymph node biopsy still presents limitations, such as high operation requirements, prolonged surgical duration, and adverse reactions to tracer agents. This study developed a novel non-invasive method to predict sentinel lymph node metastasis in breast cancer by analyzing the ultrasound imaging characteristics of the primary tumor, combined with the analysis of peripheral blood T-cell subsets that reflect the immune status of the body. The radiomic features analyzed in this study were extracted from preoperative ultrasound images of 199 solitary breast cancer patients, who were undergoing surgery and were pathologically diagnosed at the Yancheng First People's Hospital. All cases were randomly categorized in a 4:1 ratio to the training (n = 159) and validation (n = 40) cohorts. The extracted radiomics features were subjected to dimensional reduction with the help of the least absolute shrinkage and selection operator technique, resulting in the inclusion of 19 radiomics features. Four classifiers, including naïve Bayesian, logistic regression, classification decision tree, and support vector machine, were utilized to model the radiomics features, conventional ultrasound features, and peripheral blood T cell subsets in the training dataset, and validated using the validation dataset. The best-performing model was chosen for constructing the combined model. The radiomics model constructed using the logistic regression showed the best performance, with the training and validation cohorts showing areas under the curve (AUCs) of 0.77 and 0.68, respectively. The conventional ultrasound and peripheral blood T cell models constructed using the classification decision tree showed the best performance, wherein the training cohort presented AUCs of 0.71 and 0.81, respectively, while the validation cohort presented AUCs of 0.68 and 0.69, respectively. The combined model constructed by logistic regression showed AUCs of 0.91 and 0.79 in the training and validation datasets, respectively. The resulting combined model can be considered a simple, non-invasive method with strong reproducibility and clinical significance. Thus, it can be utilized to predict sentinel lymph node metastasis in breast cancer. Furthermore, the combined model can be effectively used to guide clinical decisions related to the selection of surgical procedures in breast surgery.

Artificial intelligence - based ultrasound elastography for disease evaluation - a narrative review

Ultrasound elastography (USE) provides complementary information of tissue stiffness and elasticity to conventional ultrasound imaging. It is noninvasive and free of radiation, and has become a valuable tool to improve diagnostic performance with conventional ultrasound imaging. However, the diagnostic accuracy will be reduced due to high operator-dependence and intra- and inter-observer variability in visual observations of radiologists. Artificial intelligence (AI) has great potential to perform automatic medical image analysis tasks to provide a more objective, accurate and intelligent diagnosis. More recently, the enhanced diagnostic performance of AI applied to USE have been demonstrated for various disease evaluations. This review provides an overview of the basic concepts of USE and AI techniques for clinical radiologists and then introduces the applications of AI in USE imaging that focus on the following anatomical sites: liver, breast, thyroid and other organs for lesion detection and segmentation, machine learning (ML) - assisted classification and prognosis prediction. In addition, the existing challenges and future trends of AI in USE are also discussed.

High-throughput precision MRI assessment with integrated stack-ensemble deep learning can enhance the preoperative prediction of prostate cancer Gleason grade

BACKGROUND

To develop and test a Prostate Imaging Stratification Risk (PRISK) tool for precisely assessing the International Society of Urological Pathology Gleason grade (ISUP-GG) of prostate cancer (PCa).

METHODS

This study included 1442 patients with prostate biopsy from two centres (training, n = 672; internal test, n = 231 and external test, n = 539). PRISK is designed to classify ISUP-GG 0 (benign), ISUP-GG 1, ISUP-GG 2, ISUP-GG 3 and ISUP GG 4/5. Clinical indicators and high-throughput MRI features of PCa were integrated and modelled with hybrid stacked-ensemble learning algorithms.

RESULTS

PRISK achieved a macro area-under-curve of 0.783, 0.798 and 0.762 for the classification of ISUP-GGs in training, internal and external test data. Permitting error ±1 in grading ISUP-GGs, the overall accuracy of PRISK is nearly comparable to invasive biopsy (train: 85.1% vs 88.7%; internal test: 85.1% vs 90.4%; external test: 90.4% vs 94.2%). PSA ≥ 20 ng/ml (odds ratio [OR], 1.58; p = 0.001) and PRISK ≥ GG 3 (OR, 1.45; p = 0.005) were two independent predictors of biochemical recurrence (BCR)-free survival, with a C-index of 0.76 (95% CI, 0.73-0.79) for BCR-free survival prediction.

CONCLUSIONS

PRISK might offer a potential alternative to non-invasively assess ISUP-GG of PCa.

Determining whether the diagnostic value of B-ultrasound combined with contrast-enhanced ultrasound and shear wave elastography in breast mass-like and non-mass-like lesions differs: a diagnostic test

Background

Mass-like (ML) and non-mass-like (NML) are two manifestations of breast lesions on ultrasound. Contrast-enhanced ultrasound (CEUS) can make up for the limitation of B-ultrasound (US) in the observation of focal blood flow, and shear wave elastography (SWE) can supplement the hardness information of the lesion. The present study aimed to analyze the characteristic manifestations of US, CEUS, and SWE in NML and ML breast and evaluate whether the diagnostic performance of these three ultrasound techniques differs in terms of differentiating between benign and malignant breast lesions.

Methods

From January to August 2021, 382 patients (417 breast lesions) underwent US, CEUS, and SWE examinations. Of these, 204 women (218 breast lesions) were included in our study due to subsequent biopsy or surgery with pathological findings. The patients were divided into ML and NML groups according to the ultrasound characteristics, and the differences in multimodal ultrasound performance between benign and malignant NML and benign and malignant ML breast lesions were compared. The diagnostic performance of US, US + CEUS, US + SWE, US + CEUS + SWE for ML, NML and all breast lesions was evaluated by analyzing sensitivity, specificity and area under receiver operating characteristic (ROC) curve (AUC).

Results

Pathologically, the 218 lesions included 96 malignant and 122 benign breast lesions. The sensitivity and specificity of US + CEUS + SWE in all lesion groups, ML group and NML group were 92.7% and 90.2%, 95.9% and 90.3%, 91.3% and 79.3%, respectively. In all breast group, AUCs of US + CEUS, US + SWE, US + CEUS + SWE were statistically different from AUC of US (P=0.0010, 0.0001, 0.0001). In the ML group, the AUC of US + CEUS, US + SWE, US + CEUS + SWE were statistically different from that of US (P=0.0120, 0.0008, 0.0002). In the NML group, there was a statistical difference between US + SWE and US AUC (P=0.0149).

Conclusions

US, CEUS, and SWE have an important diagnostic value for benign and malignant ML and NML breast lesions. Multimodal ultrasound combined with US, CEUS, and SWE can improve the diagnostic efficacy in distinguishing between benign and malignant ML and NML lesions.

Convolutional neural network based on automatic segmentation of peritumoral shear-wave elastography images for predicting breast cancer

Objective

Our aim was to develop dual-modal CNN models based on combining conventional ultrasound (US) images and shear-wave elastography (SWE) of peritumoral region to improve prediction of breast cancer.

Method

We retrospectively collected US images and SWE data of 1271 ACR- BIRADS 4 breast lesions from 1116 female patients (mean age ± standard deviation, 45.40 ± 9.65 years). The lesions were divided into three subgroups based on the maximum diameter (MD): ≤15 mm; >15 mm and ≤25 mm; >25 mm. We recorded lesion stiffness (SWV1) and 5-point average stiffness of the peritumoral tissue (SWV5). The CNN models were built based on the segmentation of different widths of peritumoral tissue (0.5 mm, 1.0 mm, 1.5 mm, 2.0 mm) and internal SWE image of the lesions. All single-parameter CNN models, dual-modal CNN models, and quantitative SWE parameters in the training cohort (971 lesions) and the validation cohort (300 lesions) were assessed by receiver operating characteristic (ROC) curve.

Results

The US + 1.0 mm SWE model achieved the highest area under the ROC curve (AUC) in the subgroup of lesions with MD ≤15 mm in both the training (0.94) and the validation cohorts (0.91). In the subgroups with MD between15 and 25 mm and above 25 mm, the US + 2.0 mm SWE model achieved the highest AUCs in both the training cohort (0.96 and 0.95, respectively) and the validation cohort (0.93 and 0.91, respectively).

Conclusion

The dual-modal CNN models based on the combination of US and peritumoral region SWE images allow accurate prediction of breast cancer.

Prediction of the Malignancy of a Breast Lesion Detected on Breast Ultrasound: Radiomics Applied to Clinical Practice

The study aimed to evaluate the performance of radiomics features and one ultrasound CAD (computer-aided diagnosis) in the prediction of the malignancy of a breast lesion detected with ultrasound and to develop a nomogram incorporating radiomic score and available information on CAD performance, conventional Breast Imaging Reporting and Data System evaluation (BI-RADS), and clinical information. Data on 365 breast lesions referred for breast US with subsequent histologic analysis between January 2020 and March 2022 were retrospectively collected. Patients were randomly divided into a training group (n = 255) and a validation test group (n = 110). A radiomics score was generated from the US image. The CAD was performed in a subgroup of 209 cases. The radiomics score included seven radiomics features selected with the LASSO logistic regression model. The multivariable logistic model incorporating CAD performance, BI-RADS evaluation, clinical information, and radiomic score as covariates showed promising results in the prediction of the malignancy of breast lesions: Area under the receiver operating characteristic curve, [AUC]: 0.914; 95% Confidence Interval, [CI]: 0.876-0.951. A nomogram was developed based on these results for possible future applications in clinical practice.

Research on automatic classification technology of kidney tumor and normal kidney tissue based on computed tomography radiomics

Purpose

By using a radiomics-based approach, multiple radiomics features can be extracted from regions of interest in computed tomography (CT) images, which may be applied to automatically classify kidney tumors and normal kidney tissues. The study proposes a method based on CT radiomics and aims to use extracted radiomics features to automatically classify of kidney tumors and normal kidney tissues and to establish an automatic classification model.

Methods

CT data were retrieved from the 2019 Kidney and Kidney Tumor Segmentation Challenge (KiTS19) in The Cancer Imaging Archive (TCIA) open access database. Arterial phase-enhanced CT images from 210 cases were used to establish an automatic classification model. These CT images of patients were randomly divided into training (168 cases) and test (42 cases) sets. Furthermore, the radiomics features of gross tumor volume (GTV) and normal kidney tissues in the training set were extracted and screened, and a binary logistic regression model was established. For the test set, the radiomic features and cutoff value of P were consistent with the training set.

Results

Three radiomics features were selected to establish the binary logistic regression model. The accuracy (ACC), sensitivity (SENS), specificity (SPEC), area under the curve (AUC), and Youden index of the training and test sets based on the CT radiomics classification model were all higher than 0.85.

Conclusion

The automatic classification model of kidney tumors and normal kidney tissues based on CT radiomics exhibited good classification ability. Kidney tumors could be distinguished from normal kidney tissues. This study may complement automated tumor delineation techniques and warrants further research.

Artificial Intelligence in Breast Ultrasound: From Diagnosis to Prognosis-A Rapid Review

BACKGROUND

Ultrasound (US) is a fundamental diagnostic tool in breast imaging. However, US remains an operator-dependent examination. Research into and the application of artificial intelligence (AI) in breast US are increasing. The aim of this rapid review was to assess the current development of US-based artificial intelligence in the field of breast cancer.

METHODS

Two investigators with experience in medical research performed literature searching and data extraction on PubMed. The studies included in this rapid review evaluated the role of artificial intelligence concerning BC diagnosis, prognosis, molecular subtypes of breast cancer, axillary lymph node status, and the response to neoadjuvant chemotherapy. The mean values of sensitivity, specificity, and AUC were calculated for the main study categories with a meta-analytical approach.

RESULTS

A total of 58 main studies, all published after 2017, were included. Only 9/58 studies were prospective (15.5%); 13/58 studies (22.4%) used an ML approach. The vast majority (77.6%) used DL systems. Most studies were conducted for the diagnosis or classification of BC (55.1%). At present, all the included studies showed that AI has excellent performance in breast cancer diagnosis, prognosis, and treatment strategy.

CONCLUSIONS

US-based AI has great potential and research value in the field of breast cancer diagnosis, treatment, and prognosis. More prospective and multicenter studies are needed to assess the potential impact of AI in breast ultrasound.

Quantitative Assessment of Breast-Tumor Stiffness Using Shear-Wave Elastography Histograms

Purpose: Shear-wave elastography (SWE) measures tissue elasticity using ultrasound waves. This study proposes a histogram-based SWE analysis to improve breast malignancy detection. Methods: N = 22/32 (patients/tumors) benign and n = 51/64 malignant breast tumors with histological ground truth. Colored SWE heatmaps were adjusted to a 0−180 kPa scale. Normalized, 250-binned RGB histograms were used as image descriptors based on skewness and area under curve (AUC). The histogram method was compared to conventional SWE metrics, such as (1) the qualitative 5-point scale classification and (2) average stiffness (SWEavg)/maximal tumor stiffness (SWEmax) within the tumor B-mode boundaries. Results: The SWEavg and SWEmax did not discriminate malignant lesions in this database, p > 0.05, rank sum test. RGB histograms, however, differed between malignant and benign tumors, p < 0.001, Kolmogorov−Smirnoff test. The AUC analysis of histograms revealed the reduction of soft-tissue components as a significant SWE biomarker (p = 0.03, rank sum). The diagnostic accuracy of the suggested method is still low (Se = 0.30 for Se = 0.90) and a subject for improvement in future studies. Conclusions: Histogram-based SWE quantitation improved the diagnostic accuracy for malignancy compared to conventional average SWE metrics. The sensitivity is a subject for improvement in future studies.

Clinical use and adjustment of ultrasound elastography for breast lesions followed WFUMB guidelines and recommendations in the real world

Objective

This study aimed to explore the value of strain elastography (SE) and shear wave elastography (SWE) following the World Federation of Ultrasound in Medicine and Biology (WFUMB) guidelines and recommendations in the real world in distinguishing benign and malignant breast lesions and reducing biopsy of BI-RADS (Breast Imaging Reporting and Data System) 4a lesions.

Methods

This prospective study included 274 breast lesions. The elastography score (ES) by the Tsukuba score, the strain ratio (SR) for SE, and Emax for SWE of the lesion(A) and the regions(A') included the lesion and the margin (0.5-5 mm) surrounding the lesion were measured. The sensitivity, specificity, and AUC were calculated and compared by the cutoff values recommended by WFUMB guidelines.

Results

When scores of 1 to 3 were classified as probably benign by WFUMB recommendation, the ES was significantly higher in malignant lesions compared to benign lesions (p < 0.05) in all lesions. For the cohort by size >20 mm, the sensitivity was 100%, and the specificity was 45.5%. ES had the highest AUC: 0.79(95% CI 0.72-0.86) with a sensitivity of 96.2%, and a specificity of 61.8% for the cohort by size ≤20 mm. For the Emax-A'-S2.5mm, when the high stiffness would be considered with Emax above 80 kPa in SWE, the malignant lesions were diagnosed with a sensitivity of 95.8%, a specificity of 43.3% for all lesions, a sensitivity of 88.5% for lesions with size ≤20 mm, and sensitivity of 100.0% for lesions with size >20 mm. In 84 lesions of BI-RADS category 4a, if category 4a lesions with ES of 1-3 points or Emax-A'-S2.5 less than 80 kPa could be downgraded to category 3, 52 (61.9%) lesions could be no biopsy, including two malignancies. If category 4a lesions with ES of 1-3 points and Emax-A'-S2.5 less than 80kPa could be downgraded to category 3, 23 (27.4%) lesions could be no biopsy, with no malignancy.

Conclusions

The elastography score for SE and Emax-A' for SWE after our modification were beneficial in the diagnosis of breast cancer. The combination of SWE and SE could effectively reduce the biopsy rate of BI-RADS category 4a lesions.

Deep learning radiomics of ultrasonography for differentiating sclerosing adenosis from breast cancer

OBJECTIVES: The purpose of our study is to present a method combining radiomics with deep learning and clinical data for improved differential diagnosis of sclerosing adenosis (SA)and breast cancer (BC). METHODS: A total of 97 patients with SA and 100 patients with BC were included in this study. The best model for classification was selected from among four different convolutional neural network (CNN) models, including Vgg16, Resnet18, Resnet50, and Desenet121. The intra-/inter-class correlation coefficient and least absolute shrinkage and selection operator method were used for radiomics feature selection. The clinical features selected were patient age and nodule size. The overall accuracy, sensitivity, specificity, Youden index, positive predictive value, negative predictive value, and area under curve (AUC) value were calculated for comparison of diagnostic efficacy. RESULTS: All the CNN models combined with radiomics and clinical data were significantly superior to CNN models only. The Desenet121+radiomics+clinical data model showed the best classification performance with an accuracy of 86.80%, sensitivity of 87.60%, specificity of 86.20% and AUC of 0.915, which was better than that of the CNN model only, which had an accuracy of 85.23%, sensitivity of 85.48%, specificity of 85.02%, and AUC of 0.870. In comparison, the diagnostic accuracy, sensitivity, specificity, and AUC value for breast radiologists were 72.08%, 100%, 43.30%, and 0.716, respectively. CONCLUSIONS: A combination of the CNN-radiomics model and clinical data could be a helpful auxiliary diagnostic tool for distinguishing between SA and BC.

The Role of Deep Learning in Advancing Breast Cancer Detection Using Different Imaging Modalities: A Systematic Review

Breast cancer is among the most common and fatal diseases for women, and no permanent treatment has been discovered. Thus, early detection is a crucial step to control and cure breast cancer that can save the lives of millions of women. For example, in 2020, more than 65% of breast cancer patients were diagnosed in an early stage of cancer, from which all survived. Although early detection is the most effective approach for cancer treatment, breast cancer screening conducted by radiologists is very expensive and time-consuming. More importantly, conventional methods of analyzing breast cancer images suffer from high false-detection rates. Different breast cancer imaging modalities are used to extract and analyze the key features affecting the diagnosis and treatment of breast cancer. These imaging modalities can be divided into subgroups such as mammograms, ultrasound, magnetic resonance imaging, histopathological images, or any combination of them. Radiologists or pathologists analyze images produced by these methods manually, which leads to an increase in the risk of wrong decisions for cancer detection. Thus, the utilization of new automatic methods to analyze all kinds of breast screening images to assist radiologists to interpret images is required. Recently, artificial intelligence (AI) has been widely utilized to automatically improve the early detection and treatment of different types of cancer, specifically breast cancer, thereby enhancing the survival chance of patients. Advances in AI algorithms, such as deep learning, and the availability of datasets obtained from various imaging modalities have opened an opportunity to surpass the limitations of current breast cancer analysis methods. In this article, we first review breast cancer imaging modalities, and their strengths and limitations. Then, we explore and summarize the most recent studies that employed AI in breast cancer detection using various breast imaging modalities. In addition, we report available datasets on the breast-cancer imaging modalities which are important in developing AI-based algorithms and training deep learning models. In conclusion, this review paper tries to provide a comprehensive resource to help researchers working in breast cancer imaging analysis.

Convolutional Neural Network for Predicting Thyroid Cancer Based on Ultrasound Elastography Image of Perinodular Region

We aimed to develop deep learning models based on perinodular regions' shear-wave elastography (SWE) images and ultrasound (US) images of thyroid nodules (TNs) and determine their performances in predicting thyroid cancer. A total of 1747 American College of Radiology Thyroid Imaging Reporting & Data System 4 (TR4) thyroid nodules (TNs) in 1582 patients were included in this retrospective study. US images, SWE images, and 2 quantitative SWE parameters (maximum elasticity of TNs; 5-point average maximum elasticity of TNs) were obtained. Based on US and SWE images of TNs and perinodular tissue, respectively, 7 single-image convolutional neural networks (CNN) models [US, internal SWE, 0.5 mm SWE, 1.0 mm SWE, 1.5 mm SWE, 2.0 mm SWE of perinodular tissue, and whole SWE region of interest (ROI) image] and another 6 fusional-image CNN models (US + internal SWE, US + 0.5 mm SWE, US + 1.0 mm SWE, US + 1.5 mm SWE, US + 2.0 mm SWE, US + ROI SWE) were established using RestNet18. All of the CNN models and quantitative SWE parameters were built on a training cohort (1247 TNs) and evaluated on a validation cohort (500 TNs). In predicting thyroid cancer, US + 2.0 mm SWE image CNN model obtained the highest area under the curve in 10 mm < TNs ≤ 20 mm (0.95 for training; 0.92 for validation) and TNs > 20 mm (0.95 for training; 0.92 for validation), while US + 1.0 mm SWE image CNN model obtained the highest area under the curve in TNs ≤ 10 mm (0.95 for training; 0.92 for validation). The CNN models based on the fusion of SWE segmentation images and US images improve the radiological diagnostic accuracy of thyroid cancer.

Ultrasound radiomics in personalized breast management: Current status and future prospects

Breast cancer is the most common cancer in women worldwide. Providing accurate and efficient diagnosis, risk stratification and timely adjustment of treatment strategies are essential steps in achieving precision medicine before, during and after treatment. Radiomics provides image information that cannot be recognized by the naked eye through deep mining of medical images. Several studies have shown that radiomics, as a second reader of medical images, can assist physicians not only in the detection and diagnosis of breast lesions but also in the assessment of risk stratification and prediction of treatment response. Recently, more and more studies have focused on the application of ultrasound radiomics in breast management. We summarized recent research advances in ultrasound radiomics for the diagnosis of benign and malignant breast lesions, prediction of molecular subtype, assessment of lymph node status, prediction of neoadjuvant chemotherapy response, and prediction of survival. In addition, we discuss the current challenges and future prospects of ultrasound radiomics.

Using an Improved Residual Network to Identify PIK3CA Mutation Status in Breast Cancer on Ultrasound Image

Background

The detection of phosphatidylinositol-3 kinase catalytic alpha (PIK3CA) gene mutations in breast cancer is a key step to design personalizing an optimal treatment strategy. Traditional genetic testing methods are invasive and time-consuming. It is urgent to find a non-invasive method to estimate the PIK3CA mutation status. Ultrasound (US), one of the most common methods for breast cancer screening, has the advantages of being non-invasive, fast imaging, and inexpensive. In this study, we propose to develop a deep convolutional neural network (DCNN) to identify PIK3CA mutations in breast cancer based on US images.

Materials and Methods

We retrospectively collected 312 patients with pathologically confirmed breast cancer who underwent genetic testing. All US images (n=800) of breast cancer patients were collected and divided into the training set (n=600) and test set (n=200). A DCNN-Improved Residual Network (ImResNet) was designed to identify the PIK3CA mutations. We also compared the ImResNet model with the original ResNet50 model, classical machine learning models, and other deep learning models.

Results

The proposed ImResNet model has the ability to identify PIK3CA mutations in breast cancer based on US images. Notably, our ImResNet model outperforms the original ResNet50, DenseNet201, Xception, MobileNetv2, and two machine learning models (SVM and KNN), with an average area under the curve (AUC) of 0.775. Moreover, the overall accuracy, average precision, recall rate, and F1-score of the ImResNet model achieved 74.50%, 74.17%, 73.35%, and 73.76%, respectively. All of these measures were significantly higher than other models.

Conclusion

The ImResNet model gives an encouraging performance in predicting PIK3CA mutations based on breast US images, providing a new method for noninvasive gene prediction. In addition, this model could provide the basis for clinical adjustments and precision treatment.

CVD-HNet: Classifying Pneumonia and COVID-19 in Chest X-ray Images Using Deep Network

The use of computer-assisted analysis to improve image interpretation has been a long-standing challenge in the medical imaging industry. In terms of image comprehension, Continuous advances in AI (Artificial Intelligence), predominantly in DL (Deep Learning) techniques, are supporting in the classification, Detection, and quantification of anomalies in medical images. DL techniques are the most rapidly evolving branch of AI, and it's recently been successfully pragmatic in a variety of fields, including medicine. This paper provides a classification method for COVID 19 infected X-ray images based on new novel deep CNN model. For COVID19 specified pneumonia analysis, two new customized CNN architectures, CVD-HNet1 (COVID-HybridNetwork1) and CVD-HNet2 (COVID-HybridNetwork2), have been designed. The suggested method utilizes operations based on boundaries and regions, as well as convolution processes, in a systematic manner. In comparison to existing CNNs, the suggested classification method achieves excellent Accuracy 98 percent, F Score 0.99 and MCC 0.97. These results indicate impressive classification accuracy on a limited dataset, with more training examples, much better results can be achieved. Overall, our CVD-HNet model could be a useful tool for radiologists in diagnosing and detecting COVID 19 instances early.

Machine learning-based diagnostic evaluation of shear-wave elastography in BI-RADS category 4 breast cancer screening: a multicenter, retrospective study

BACKGROUND

Ultrasound is commonly used in breast cancer screening but lacks quantification ability and diagnostic power due to its low specificity, which can lead to overdiagnosis and unnecessary biopsies. This study evaluated the diagnostic efficacy and clinical utility of adding shear-wave elastography (SWE) to the screening of the Breast Imaging Reporting and Data System (BI-RADS) category 4 breast cancer.

METHODS

A machine learning-based diagnostic model was constructed using data retrospectively collected from 3 independent cohorts with features selected using lasso regression and support vector machine-recursive feature elimination algorithms. Propensity score matching (PSM) was used to preclude confounding baseline characteristics between malignant and benign lesions. A decision curve analysis (DCA) was used to evaluate the clinical benefit of the diagnostic model in identifying high-risk tumor patients for intervention while simultaneously avoiding overtreatment of low-risk patients with integrative evaluation using a net benefit value and treatment reduction rate.

RESULTS

In our training center, a total of 122 patients were enrolled, and 577 breast tumors were collected. The comparison between malignant and benign lesions revealed significant differences in patient age, tumor size, resistance index (RI), and elasticity values. The maximum elasticity value (Emax) was identified as an independent diagnostic feature and was included in the diagnostic model. The combination of Emax with BI-RADS category 4 demonstrated a significantly better diagnostic efficacy than the BI-RADS category alone [BI-RADS+Emax: AUC =0.908, 95% confidence interval (CI): 0.842-0.974; BI-RADS: AUC =0.862, 95% CI: 0.784-0.94; P=0.024] and significantly increased the clinical benefit for patients and policy makers by effectively reducing overdiagnosis and biopsy rates. In the BI-RADS category 4A subgroup, adding Emax to breast cancer screening benefited patients and showed a greater absolute benefit than did the BI-RADS category alone when used for patients with a higher probability of cancer (>0.403), demonstrating a 50% overtreatment reduction.

CONCLUSIONS

Adding Emax to BI-RADS category 4 breast cancer screening using SWE significantly reduced overdiagnosis and biopsy rates compared with the BI-RADS category alone, especially for BI-RADS 4A patients.

Usefulness of new shear wave elastography in early predicting the efficacy of neoadjuvant chemotherapy for patients with breast cancer: where and when to measure is optimal?

BACKGROUND

The aim of this study was to investigate the diagnosis performance of new shear wave elastography (sound touch elastography, STE) in the prediction of neoadjuvant chemotherapy (NAC) response at an early stage in breast cancer patients and to determine the optimal measurement locations around the lesion in different ranges.

METHODS

One hundred and eight patients were analyzed in this prospective study from November 2018 to December 2020. All patients completed NAC treatment and underwent STE examination at three time points [the day before NAC (t0); the day before the second course (t1); the day before third course (t2)]. The stiffness of the whole lesion (G), 1-mm shell (S1) and 2-mm shell (S2) around the lesion was expressed by STE parameters. The relative changes (∆stiffness) of STE parameters after the first and second course of NAC were calculated and shown as the variables [Δ(t1) and Δ(t2)]. The diagnostic accuracy of STE was evaluated by means of receiver operating characteristic curve analysis.

RESULTS

The ∆stiffness (%) including ∆G_mean(t2), ∆S1_mean(t2) and ∆S2_mean(t2) all showed significant differences between pathological complete response (pCR) and non-pCR groups. ∆S2_mean(t2) displayed the best predictive performance for pCR (AUC = 0.842) with an ideal ∆stiffness threshold value - 26%.

CONCLUSIONS

Measuring the relative changes in the stiffness of surrounding tissue or entire lesion with STE holds promise for effectively predicting the response to NAC at its early stage for breast cancer patients and ∆stiffness of shell 2 mm after the second course of NAC may be a potential prediction parameter.

Breast Tumour Classification Using Ultrasound Elastography with Machine Learning: A Systematic Scoping Review

Ultrasound elastography can quantify stiffness distribution of tissue lesions and complements conventional B-mode ultrasound for breast cancer screening. Recently, the development of computer-aided diagnosis has improved the reliability of the system, whilst the inception of machine learning, such as deep learning, has further extended its power by facilitating automated segmentation and tumour classification. The objective of this review was to summarize application of the machine learning model to ultrasound elastography systems for breast tumour classification. Review databases included PubMed, Web of Science, CINAHL, and EMBASE. Thirteen (n = 13) articles were eligible for review. Shear-wave elastography was investigated in six articles, whereas seven studies focused on strain elastography (5 freehand and 2 Acoustic Radiation Force). Traditional computer vision workflow was common in strain elastography with separated image segmentation, feature extraction, and classifier functions using different algorithm-based methods, neural networks or support vector machines (SVM). Shear-wave elastography often adopts the deep learning model, convolutional neural network (CNN), that integrates functional tasks. All of the reviewed articles achieved sensitivity ³ 80%, while only half of them attained acceptable specificity ³ 95%. Deep learning models did not necessarily perform better than traditional computer vision workflow. Nevertheless, there were inconsistencies and insufficiencies in reporting and calculation, such as the testing dataset, cross-validation, and methods to avoid overfitting. Most of the studies did not report loss or hyperparameters. Future studies may consider using the deep network with an attention layer to locate the targeted object automatically and online training to facilitate efficient re-training for sequential data.

Bi-Modal Transfer Learning for Classifying Breast Cancers via Combined B-Mode and Ultrasound Strain Imaging

Although accurate detection of breast cancer still poses significant challenges, deep learning (DL) can support more accurate image interpretation. In this study, we develop a highly robust DL model based on combined B-mode ultrasound (B-mode) and strain elastography ultrasound (SE) images for classifying benign and malignant breast tumors. This study retrospectively included 85 patients, including 42 with benign lesions and 43 with malignancies, all confirmed by biopsy. Two deep neural network models, AlexNet and ResNet, were separately trained on combined 205 B-mode and 205 SE images (80% for training and 20% for validation) from 67 patients with benign and malignant lesions. These two models were then configured to work as an ensemble using both image-wise and layer-wise and tested on a dataset of 56 images from the remaining 18 patients. The ensemble model captures the diverse features present in the B-mode and SE images and also combines semantic features from AlexNet and ResNet models to classify the benign from the malignant tumors. The experimental results demonstrate that the accuracy of the proposed ensemble model is 90%, which is better than the individual models and the model trained using B-mode or SE images alone. Moreover, some patients that were misclassified by the traditional methods were correctly classified by the proposed ensemble method. The proposed ensemble DL model will enable radiologists to achieve superior detection efficiency owing to enhance classification accuracy for breast cancers in ultrasound (US) images.

A pre-operative MRI-based brain metastasis risk-prediction model for triple-negative breast cancer

BACKGROUND

Triple-negative breast cancer (TNBC) patients have a high 2-year post-operative incidence of brain metastasis (BM). Currently, there is no early prediction tool to predict the risk of BM in TNBC patients.

METHODS

Data of breast cancer patients, who had been scanned, resected, and pathologically diagnosed at a local hospital from May 2012 to June 2018 were collected. Primary and radiological secondary exclusion criteria were used to determine patients' eligibility for inclusion in the study. Data for the TNBC cohort included qualified 2-year post-operative follow-up information, BM status, and pre-operative MRI data. Age-based propensity score matching (PSM) was used to build a comparable study cohort. The tumor regions of interest were segmented and used for lattice radiomics feature extraction. The filtered and normalized lattice radiomics features were then trained with BM status using the random forest (RF), support vector machine (SVM), k-nearest neighbor, least absolute shrinkage and selection operator regression, naïve Bayesian, and neural network algorithms. The generated prediction models were evaluated using 10-fold cross verification, and the areas under the curve (AUCs), accuracy, sensitivity, and specificity were reported.

RESULTS

Data from 643 breast cancer patients were collected. Among these, 84 TNBC cases (comprising 42 pairs) were included in this study after primary exclusion, radiological secondary exclusion, and PSM. We extracted 3,854 lattice radiomics features from the pre-operative MRI. Of these, 2,480 were used for model training after filtration. The 10-fold verification results showed that the BM risk-prediction model, which was based on the normalized and filtered lattice radiomics features of collected cases trained by naïve Bayesian algorithm, had a high AUC (0.878), accuracy (0.786), specificity (81.0%), and sensitivity (76.2%).

CONCLUSIONS

The pre-operative MRI data of TNBC patients can be used to predict 2-year BM risk. This application could help to achieve better early stratification, BM screening, and the overall prognosis.

Future artificial intelligence tools and perspectives in medicine

PURPOSE OF REVIEW

Artificial intelligence has become popular in medical applications, specifically as a clinical support tool for computer-aided diagnosis. These tools are typically employed on medical data (i.e., image, molecular data, clinical variables, etc.) and used the statistical and machine-learning methods to measure the model performance. In this review, we summarized and discussed the most recent radiomic pipeline used for clinical analysis.

RECENT FINDINGS

Currently, limited management of cancers benefits from artificial intelligence, mostly related to a computer-aided diagnosis that avoids a biopsy analysis that presents additional risks and costs. Most artificial intelligence tools are based on imaging features, known as radiomic analysis that can be refined into predictive models in noninvasively acquired imaging data. This review explores the progress of artificial intelligence-based radiomic tools for clinical applications with a brief description of necessary technical steps. Explaining new radiomic approaches based on deep-learning techniques will explain how the new radiomic models (deep radiomic analysis) can benefit from deep convolutional neural networks and be applied on limited data sets.

SUMMARY

To consider the radiomic algorithms, further investigations are recommended to involve deep learning in radiomic models with additional validation steps on various cancer types.

Radiomics: A Primer for Breast Radiologists

Radiomics has a long-standing history in breast imaging with computer-aided detection (CAD) for screening mammography developed in the late 20th century. Although conventional CAD had widespread adoption, the clinical benefits for experienced breast radiologists were debatable due to high false-positive marks and subsequent increased recall rates. The dramatic growth in recent years of artificial intelligence-based analysis, including machine learning and deep learning, has provided numerous opportunities for improved modern radiomics work in breast imaging. There has been extensive radiomics work in mammography, digital breast tomosynthesis, MRI, ultrasound, PET-CT, and combined multimodality imaging. Specific radiomics outcomes of interest have been diverse, including CAD, prediction of response to neoadjuvant therapy, lesion classification, and survival, among other outcomes. Additionally, the radiogenomics subfield that correlates radiomics features with genetics has been very proliferative, in parallel with the clinical validation of breast cancer molecular subtypes and gene expression assays. Despite the promise of radiomics, there are important challenges related to image normalization, limited large unbiased data sets, and lack of external validation. Much of the radiomics work to date has been exploratory using single-institution retrospective series for analysis, but several promising lines of investigation have made the leap to clinical practice with commercially available products. As a result, breast radiologists will increasingly be incorporating radiomics-based tools into their daily practice in the near future. Therefore, breast radiologists must have a broad understanding of the scope, applications, and limitations of radiomics work.

Images Are Data: A Breast Imaging Perspective on a Contemporary Paradigm

BACKGROUND

 Considering radiological examinations not as mere images, but as a source of data, has become the key paradigm in the diagnostic imaging field. This change of perspective is particularly popular in breast imaging. It allows breast radiologists to apply algorithms derived from computer science, to realize innovative clinical applications, and to refine already established methods. In this context, the terminology "imaging biomarker", "radiomics", and "artificial intelligence" are of pivotal importance. These methods promise noninvasive, low-cost (e. g., in comparison to multigene arrays), and workflow-friendly (automated, only one examination, instantaneous results, etc.) delivery of clinically relevant information.

METHODS AND RESULTS

 This paper is designed as a narrative review on the previously mentioned paradigm. The focus is on key concepts in breast imaging and important buzzwords are explained. For all areas of breast imaging, exemplary studies and potential clinical use cases are discussed.

CONCLUSION

 Considering radiological examination as a source of data may optimize patient management by guiding individualized breast cancer diagnosis and oncologic treatment in the age of precision medicine.

KEY POINTS

  · In conventional breast imaging, examinations are interpreted based on patterns perceivable by visual inspection.. · The radiomics paradigm treats breast images as a source of data, containing information beyond what is visible to our eyes.. · This results in radiomic signatures that may be considered as imaging biomarkers, as they provide diagnostic, predictive, and prognostic information.. · Radiomics derived imaging biomarkers may be used to individualize breast cancer treatment in the era of precision medicine.. · The concept and key research of radiomics in the field of breast imaging will be discussed in this narrative review..

CITATION FORMAT

· Dietzel M, Clauser P, Kapetas P et al. Images Are Data: A Breast Imaging Perspective on a Contemporary Paradigm. Fortschr Röntgenstr 2021; 193: 898 - 908.