Decoding the molecular subtypes of breast cancer seen on multimodal ultrasound images using an assembled convolutional neural network model: A prospective and multicentre study

Use of ultrasound imaging Omics in predicting molecular typing and assessing the risk of postoperative recurrence in breast cancer

BACKGROUND

The aim of this study is to assess the efficacy of a multiparametric ultrasound imaging omics model in predicting the risk of postoperative recurrence and molecular typing of breast cancer.

METHODS

A retrospective analysis was conducted on 534 female patients diagnosed with breast cancer through preoperative ultrasonography and pathology, from January 2018 to June 2023 at the Affiliated Cancer Hospital of Xinjiang Medical University. Univariate analysis and multifactorial logistic regression modeling were used to identify independent risk factors associated with clinical characteristics. The PyRadiomics package was used to delineate the region of interest in selected ultrasound images and extract radiomic features. Subsequently, radiomic scores were established through Least Absolute Shrinkage and Selection Operator (LASSO) regression and Support Vector Machine (SVM) methods. The predictive performance of the model was assessed using the receiver operating characteristic (ROC) curve, and the area under the curve (AUC) was calculated. Evaluation of diagnostic efficacy and clinical practicability was conducted through calibration curves and decision curves.

RESULTS

In the training set, the AUC values for the postoperative recurrence risk prediction model were 0.9489, and for the validation set, they were 0.8491. Regarding the molecular typing prediction model, the AUC values in the training set and validation set were 0.93 and 0.92 for the HER-2 overexpression phenotype, 0.94 and 0.74 for the TNBC phenotype, 1.00 and 0.97 for the luminal A phenotype, and 1.00 and 0.89 for the luminal B phenotype, respectively. Based on a comprehensive analysis of calibration and decision curves, it was established that the model exhibits strong predictive performance and clinical practicability.

CONCLUSION

The use of multiparametric ultrasound imaging omics proves to be of significant value in predicting both the risk of postoperative recurrence and molecular typing in breast cancer. This non-invasive approach offers crucial guidance for the diagnosis and treatment of the condition.

Diagnostic performance of ultrasound-based artificial intelligence for predicting key molecular markers in breast cancer: A systematic review and meta-analysis

BACKGROUND

Breast cancer (BC) diagnosis and treatment rely heavily on molecular markers such as HER2, Ki67, PR, and ER. Currently, these markers are identified by invasive methods.

OBJECTIVE

This meta-analysis investigates the diagnostic accuracy of ultrasound-based radiomics as a novel approach to predicting these markers.

METHODS

A comprehensive search of PubMed, EMBASE, and Web of Science databases was conducted to identify studies evaluating ultrasound-based radiomics in BC. Inclusion criteria encompassed research on HER2, Ki67, PR, and ER as key molecular markers. Quality assessment using Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) and Radiomics Quality Score (RQS) was performed. The data extraction step was performed systematically.

RESULTS

Our meta-analysis quantifies the diagnostic accuracy of ultrasound-based radiomics with a sensitivity and specificity of 0.76 and 0.78 for predicting HER2, 0.80, and 0.76 for Ki67 biomarkers. Studies did not provide sufficient data for quantitative PR and ER prediction analysis. The overall quality of studies based on the RQS tool was moderate. The QUADAS-2 evaluation showed that the studies had an unclear risk of bias regarding the flow and timing domain.

CONCLUSION

Our analysis indicated that AI models have a promising accuracy for predicting key molecular biomarkers' status in BC patients. We performed the quantitative analysis for HER2 and Ki67 biomarkers which yielded a moderate to high accuracy. However, studies did not provide adequate data for meta-analysis of ER and PR prediction accuracy of developed models. The overall quality of the studies was acceptable. In future research, studies need to report the results thoroughly. Also, we suggest more prospective studies from different centers.

Comparative Evaluation of Machine Learning Models for Subtyping Triple-Negative Breast Cancer: A Deep Learning-Based Multi-Omics Data Integration Approach

Objective: Triple-negative breast cancer (TNBC) poses significant diagnostic challenges due to its aggressive nature. This research develops an innovative deep learning (DL) model based on the latest multi-omics data to enhance the accuracy of TNBC subtype and prognosis prediction. The study focuses on addressing the constraints of prior studies by showcasing a model with substantial advancements in data integration, statistical performance, and algorithmic optimization. Methods: Breast cancer-related molecular characteristic data, including mRNA, miRNA, gene mutations, DNA methylation, and magnetic resonance imaging (MRI) images, were retrieved from the TCGA and TCIA databases. This study not only compared single-omics with multi-omics machine learning models but also applied Bayesian optimization to innovatively optimize the neural network structure of a DL model for multi-omics data. Results: The DL model for multi-omics data significantly outperformed single-omics models in subtype prediction, achieving a 98.0% accuracy in cross-validation, 97.0% in the validation set, and 91.0% in an external test set. Additionally, the MRI radiomics model showed promising performance, especially with the training set; however, a decrease in performance during transfer testing underscored the advantages of the DL model for multi-omics data in data consistency and digital processing. Conclusion: Our multi-omics DL model presents notable innovations in statistical performance and transfer learning capability, bearing significant clinical relevance for TNBC classification and prognosis prediction. While the MRI radiomics model proved effective, it requires further optimization for cross-dataset application to enhance accuracy and consistency. Our findings offer new insights into improving TNBC classification and prognosis through multi-omics data and DL algorithms.

New Frontiers in Breast Cancer Imaging: The Rise of AI

Artificial intelligence (AI) has been implemented in multiple fields of medicine to assist in the diagnosis and treatment of patients. AI implementation in radiology, more specifically for breast imaging, has advanced considerably. Breast cancer is one of the most important causes of cancer mortality among women, and there has been increased attention towards creating more efficacious methods for breast cancer detection utilizing AI to improve radiologist accuracy and efficiency to meet the increasing demand of our patients. AI can be applied to imaging studies to improve image quality, increase interpretation accuracy, and improve time efficiency and cost efficiency. AI applied to mammography, ultrasound, and MRI allows for improved cancer detection and diagnosis while decreasing intra- and interobserver variability. The synergistic effect between a radiologist and AI has the potential to improve patient care in underserved populations with the intention of providing quality and equitable care for all. Additionally, AI has allowed for improved risk stratification. Further, AI application can have treatment implications as well by identifying upstage risk of ductal carcinoma in situ (DCIS) to invasive carcinoma and by better predicting individualized patient response to neoadjuvant chemotherapy. AI has potential for advancement in pre-operative 3-dimensional models of the breast as well as improved viability of reconstructive grafts.

Diagnostic Performance of Deep Learning in Video-Based Ultrasonography for Breast Cancer: A Retrospective Multicentre Study

OBJECTIVE

Although ultrasound is a common tool for breast cancer screening, its accuracy is often operator-dependent. In this study, we proposed a new automated deep-learning framework that extracts video-based ultrasound data for breast cancer screening.

METHODS

Our framework incorporates DenseNet121, MobileNet, and Xception as backbones for both video- and image-based models. We used data from 3907 patients to train and evaluate the models, which were tested using video- and image-based methods, as well as reader studies with human experts.

RESULTS

This study evaluated 3907 female patients aged 22 to 86 years. The results indicated that the MobileNet video model achieved an AUROC of 0.961 in prospective data testing, surpassing the DenseNet121 video model. In real-world data testing, it demonstrated an accuracy of 92.59%, outperforming both the DenseNet121 and Xception video models, and exceeding the 76.00% to 85.60% accuracy range of human experts. Additionally, the MobileNet video model exceeded the performance of image models and other video models across all evaluation metrics, including accuracy, sensitivity, specificity, F1 score, and AUC. Its exceptional performance, particularly suitable for resource-limited clinical settings, demonstrates its potential for clinical application in breast cancer screening.

CONCLUSIONS

The level of expertise reached by the video models was greater than that achieved by image-based models. We have developed an artificial intelligence framework based on videos that may be able to aid breast cancer diagnosis and alleviate the shortage of experienced experts.

Breast cancer prognosis through the use of multi-modal classifiers: current state of the art and the way forward

We present a survey of the current state-of-the-art in breast cancer detection and prognosis. We analyze the evolution of Artificial Intelligence-based approaches from using just uni-modal information to multi-modality for detection and how such paradigm shift facilitates the efficacy of detection, consistent with clinical observations. We conclude that interpretable AI-based predictions and ability to handle class imbalance should be considered priority.

Sonography-based multimodal information platform for identifying the surgical pathology of ductal carcinoma in situ

BACKGROUND

The risk of ductal carcinoma in situ (DCIS) identified by biopsy often increases during surgery. Therefore, confirming the DCIS grade preoperatively is necessary for clinical decision-making.

PURPOSE

To train a three-classification deep learning (DL) model based on ultrasound (US), combining clinical data, mammography (MG), US, and core needle biopsy (CNB) pathology to predict low-grade DCIS, intermediate-to-high-grade DCIS, and upstaged DCIS.

MATERIALS AND METHODS

Data of 733 patients with 754 DCIS cases confirmed by biopsy were retrospectively collected from May 2013 to June 2022 (N1), and other data (N2) were confirmed by biopsy as low-grade DCIS. The lesions were randomly divided into training (n=471), validation (n=142), and test (n = 141) sets to establish the DCIS-Net. Information on the DCIS-Net, clinical (age and sign), US (size, calcifications, type, breast imaging reporting and data system [BI-RADS]), MG (microcalcifications, BI-RADS), and CNB pathology (nuclear grade, architectural features, and immunohistochemistry) were collected. Logistic regression and random forest analyses were conducted to develop Multimodal DCIS-Net to calculate the specificity, sensitivity, accuracy, receiver operating characteristic curve, and area under the curve (AUC).

RESULTS

In the test set of N1, the accuracy and AUC of the multimodal DCIS-Net were 0.752-0.766 and 0.859-0.907 in the three-classification task, respectively. The accuracy and AUC for discriminating DCIS from upstaged DCIS were 0.751-0.780 and 0.829-0.861, respectively. In the test set of N2, the accuracy and AUC of discriminating low-grade DCIS from upstaged low-grade DCIS were 0.769-0.987 and 0.818-0.939, respectively. DL was ranked from one to five in the importance of features in the multimodal-DCIS-Net.

CONCLUSION

By developing the DCIS-Net and integrating it with multimodal information, diagnosing low-grade DCIS, intermediate-to high-grade DCIS, and upstaged DCIS is possible. It can also be used to distinguish DCIS from upstaged DCIS and low-grade DCIS from upstaged low-grade DCIS, which could pave the way for the DCIS clinical workflow.

Artificial intelligence in breast ultrasound: application in clinical practice

Ultrasound (US) is a widely accessible and extensively used tool for breast imaging. It is commonly used as an additional screening tool, especially for women with dense breast tissue. Advances in artificial intelligence (AI) have led to the development of various AI systems that assist radiologists in identifying and diagnosing breast lesions using US. This article provides an overview of the background and supporting evidence for the use of AI in hand held breast US. It discusses the impact of AI on clinical workflow, covering breast cancer detection, diagnosis, prediction of molecular subtypes, evaluation of axillary lymph node status, and response to neoadjuvant chemotherapy. Additionally, the article highlights the potential significance of AI in breast US for low and middle income countries.

Exploring the Potential of Artificial Intelligence in Breast Ultrasound

Breast ultrasound has emerged as a valuable imaging modality in the detection and characterization of breast lesions, particularly in women with dense breast tissue or contraindications for mammography. Within this framework, artificial intelligence (AI) has garnered significant attention for its potential to improve diagnostic accuracy in breast ultrasound and revolutionize the workflow. This review article aims to comprehensively explore the current state of research and development in harnessing AI's capabilities for breast ultrasound. We delve into various AI techniques, including machine learning, deep learning, as well as their applications in automating lesion detection, segmentation, and classification tasks. Furthermore, the review addresses the challenges and hurdles faced in implementing AI systems in breast ultrasound diagnostics, such as data privacy, interpretability, and regulatory approval. Ethical considerations pertaining to the integration of AI into clinical practice are also discussed, emphasizing the importance of maintaining a patient-centered approach. The integration of AI into breast ultrasound holds great promise for improving diagnostic accuracy, enhancing efficiency, and ultimately advancing patient's care. By examining the current state of research and identifying future opportunities, this review aims to contribute to the understanding and utilization of AI in breast ultrasound and encourage further interdisciplinary collaboration to maximize its potential in clinical practice.

Artificial intelligence in breast imaging: potentials and challenges

Breast cancer, which is the most common type of malignant tumor among humans, is a leading cause of death in females. Standard treatment strategies, including neoadjuvant chemotherapy, surgery, postoperative chemotherapy, targeted therapy, endocrine therapy, and radiotherapy, are tailored for individual patients. Such personalized therapies have tremendously reduced the threat of breast cancer in females. Furthermore, early imaging screening plays an important role in reducing the treatment cycle and improving breast cancer prognosis. The recent innovative revolution in artificial intelligence (AI) has aided radiologists in the early and accurate diagnosis of breast cancer. In this review, we introduce the necessity of incorporating AI into breast imaging and the applications of AI in mammography, ultrasonography, magnetic resonance imaging, and positron emission tomography/computed tomography based on published articles since 1994. Moreover, the challenges of AI in breast imaging are discussed.

Prediction of sentinel lymph node metastasis in breast cancer by using deep learning radiomics based on ultrasound images

Sentinel lymph node metastasis (SLNM) is a crucial predictor for breast cancer treatment and survival. This study was designed to propose deep learning (DL) models based on grayscale ultrasound, color Doppler flow imaging (CDFI), and elastography images, and to evaluate how DL radiomics can be used to classify SLNM in breast cancer. Clinical and ultrasound data of 317 patients diagnosed with breast cancer at the Second Affiliated Hospital of Nanchang University were collected from January 2018 to December 2021 and randomly divided into training and internal validation cohorts at a ratio of 7:3. An external validation cohort comprising data from Nanchang Third Hospital with 42 patients collected. Three DL models, namely DL-grayscale, DL-CDFI, and DL-elastography, were proposed to predict SLNM by analyzing grayscale ultrasound, CDFI, and elastography images. Three DL models were compared and evaluated to assess diagnostic performance based on the area under the curve (AUC). The AUCs of the DL-grayscale were 0.855 and 0.788 in the internal and external validation cohorts, respectively. For the DL-CDFI model, the AUCs were 0.761 and 0.728, respectively. The diagnostic performance of DL-elastography was superior to that of the DL-grayscale and DL-CDFI. The AUC of the DL-elastography model was 0.879 in the internal validation cohort, with a classification accuracy of 86.13%, sensitivity of 91.60%, and specificity of 82.79%. The generalization capability of DL-elastography remained high in the external cohort, with an AUC of 0.876, and an accuracy of 85.00%. DL radiomics can be used to classify SLNM in breast cancer using ultrasound images. The proposed DL-elastography model based on elastography images achieved the best diagnostic performance and holds good potential for the management of patients with SLNM.

Artificial intelligence in breast imaging: Current situation and clinical challenges

Breast cancer ranks among the most prevalent malignant tumours and is the primary contributor to cancer-related deaths in women. Breast imaging is essential for screening, diagnosis, and therapeutic surveillance. With the increasing demand for precision medicine, the heterogeneous nature of breast cancer makes it necessary to deeply mine and rationally utilize the tremendous amount of breast imaging information. With the rapid advancement of computer science, artificial intelligence (AI) has been noted to have great advantages in processing and mining of image information. Therefore, a growing number of scholars have started to focus on and research the utility of AI in breast imaging. Here, an overview of breast imaging databases and recent advances in AI research are provided, the challenges and problems in this field are discussed, and then constructive advice is further provided for ongoing scientific developments from the perspective of the National Natural Science Foundation of China.

Deep Learning-Based Evaluation of Ultrasound Images for Benign Skin Tumors

In this study, a combined convolutional neural network for the diagnosis of three benign skin tumors was designed, and its effectiveness was verified through quantitative and statistical analysis. To this end, 698 sonographic images were taken and diagnosed at the Department of Dermatology at Severance Hospital in Seoul, Korea, between 10 November 2017 and 17 January 2020. Through an empirical process, a convolutional neural network combining two structures, which consist of a residual structure and an attention-gated structure, was designed. Five-fold cross-validation was applied, and the train set for each fold was augmented by the Fast AutoAugment technique. As a result of training, for three benign skin tumors, an average accuracy of 95.87%, an average sensitivity of 90.10%, and an average specificity of 96.23% were derived. Also, through statistical analysis using a class activation map and physicians' findings, it was found that the judgment criteria of physicians and the trained combined convolutional neural network were similar. This study suggests that the model designed and trained in this study can be a diagnostic aid to assist physicians and enable more efficient and accurate diagnoses.

Deep learning radiopathomics based on preoperative US images and biopsy whole slide images can distinguish between luminal and non-luminal tumors in early-stage breast cancers

BACKGROUND

For patients with early-stage breast cancers, neoadjuvant treatment is recommended for non-luminal tumors instead of luminal tumors. Preoperative distinguish between luminal and non-luminal cancers at early stages will facilitate treatment decisions making. However, the molecular immunohistochemical subtypes based on biopsy specimens are not always consistent with final results based on surgical specimens due to the high intra-tumoral heterogeneity. Given that, we aimed to develop and validate a deep learning radiopathomics (DLRP) model to preoperatively distinguish between luminal and non-luminal breast cancers at early stages based on preoperative ultrasound (US) images, and hematoxylin and eosin (H&E)-stained biopsy slides.

METHODS

This multicentre study included three cohorts from a prospective study conducted by our team and registered on the Chinese Clinical Trial Registry (ChiCTR1900027497). Between January 2019 and August 2021, 1809 US images and 603 H&E-stained whole slide images (WSIs) from 603 patients with early-stage breast cancers were obtained. A Resnet18 model pre-trained on ImageNet and a multi-instance learning based attention model were used to extract the features of US and WSIs, respectively. An US-guided Co-Attention module (UCA) was designed for feature fusion of US and WSIs. The DLRP model was constructed based on these three feature sets including deep learning US feature, deep learning WSIs feature and UCA-fused feature from a training cohort (1467 US images and 489 WSIs from 489 patients). The DLRP model's diagnostic performance was validated in an internal validation cohort (342 US images and 114 WSIs from 114 patients) and an external test cohort (270 US images and 90 WSIs from 90 patients). We also compared diagnostic efficacy of the DLRP model with that of deep learning radiomics model and deep learning pathomics model in the external test cohort.

FINDINGS

The DLRP yielded high performance with area under the curve (AUC) values of 0.929 (95% CI 0.865-0.968) in the internal validation cohort, and 0.900 (95% CI 0.819-0.953) in the external test cohort. The DLRP also outperformed deep learning radiomics model based on US images only (AUC 0.815 [0.719-0.889], p = 0.027) and deep learning pathomics model based on WSIs only (AUC 0.802 [0.704-0.878], p = 0.013) in the external test cohort.

INTERPRETATION

The DLRP can effectively distinguish between luminal and non-luminal breast cancers at early stages before surgery based on pretherapeutic US images and biopsy H&E-stained WSIs, providing a tool to facilitate treatment decision making in early-stage breast cancers.

FUNDING

Natural Science Foundation of Guangdong Province (No. 2023A1515011564), and National Natural Science Foundation of China (No. 91959127; No. 81971631).

A ultrasonic nomogram of quantitative parameters for diagnosing breast cancer

This study aimed to develop a nomogram through the collection of quantitative ultrasound parameters to predict breast cancer. From March 2021 to September 2022, a total of 313 breast tumors were included with pathological results. Through collecting quantitative ultrasound parameters of breast tumors and multivariate regression analysis, a nomogram was developed. The diagnostic performances, calibration and clinical usefulness of the nomogram for predicting breast cancer were assessed. A total of 182 benign and 131 malignant breast tumors were included in this study. The nomogram indicated excellent predictive properties with an AUC of 0.934, sensitivity of 0.881, specificity of 0.848, PPV of 0.795 and NPV of 0.841. The calibration curve showed the predicted values are basically consistent with the actual observed values. The optimum cut-off for the nomogram was 0.310 for predicting cancer. The decision curve analysis results corroborated good clinical usefulness. The model including BI-RADS score, SWE and VI is potentially useful for predicting breast cancer.

Combining conventional ultrasound and ultrasound elastography to predict HER2 status in patients with breast cancer

Introduction: Identifying the HER2 status of breast cancer patients is important for treatment options. Previous studies have shown that ultrasound features are closely related to the subtype of breast cancer. Methods: In this study, we used features of conventional ultrasound and ultrasound elastography to predict HER2 status. Results and Discussion: The performance of model (AUROC) with features of conventional ultrasound and ultrasound elastography is higher than that of the model with features of conventional ultrasound (0.82 vs. 0.53). The SHAP method was used to explore the interpretability of the models. Compared with HER2- tumors, HER2+ tumors usually have greater elastic modulus parameters and microcalcifications. Therefore, we concluded that the features of conventional ultrasound combined with ultrasound elastography could improve the accuracy for predicting HER2 status.

Three-dimensional convolutional neural network model to identify clinically significant prostate cancer in transrectal ultrasound videos: a prospective, multi-institutional, diagnostic study

Background

Identifying patients with clinically significant prostate cancer (csPCa) before biopsy helps reduce unnecessary biopsies and improve patient prognosis. The diagnostic performance of traditional transrectal ultrasound (TRUS) for csPCa is relatively limited. This study was aimed to develop a high-performance convolutional neural network (CNN) model (P-Net) based on a TRUS video of the entire prostate and investigate its efficacy in identifying csPCa.

Methods

Between January 2021 and December 2022, this study prospectively evaluated 832 patients from four centres who underwent prostate biopsy and/or radical prostatectomy. All patients had a standardised TRUS video of the whole prostate. A two-dimensional CNN (2D P-Net) and three-dimensional CNN (3D P-Net) were constructed using the training cohort (559 patients) and tested on the internal validation cohort (140 patients) as well as on the external validation cohort (133 patients). The performance of 2D P-Net and 3D P-Net in predicting csPCa was assessed in terms of the area under the receiver operating characteristic curve (AUC), biopsy rate, and unnecessary biopsy rate, and compared with the TRUS 5-point Likert score system as well as multiparametric magnetic resonance imaging (mp-MRI) prostate imaging reporting and data system (PI-RADS) v2.1. Decision curve analyses (DCAs) were used to determine the net benefits associated with their use. The study is registered at https://www.chictr.org.cn with the unique identifier ChiCTR2200064545.

Findings

The diagnostic performance of 3D P-Net (AUC: 0.85-0.89) was superior to TRUS 5-point Likert score system (AUC: 0.71-0.78, P = 0.003-0.040), and similar to mp-MRI PI-RADS v2.1 score system interpreted by experienced radiologists (AUC: 0.83-0.86, P = 0.460-0.732) and 2D P-Net (AUC: 0.79-0.86, P = 0.066-0.678) in the internal and external validation cohorts. The biopsy rate decreased from 40.3% (TRUS 5-point Likert score system) and 47.6% (mp-MRI PI-RADS v2.1 score system) to 35.5% (2D P-Net) and 34.0% (3D P-Net). The unnecessary biopsy rate decreased from 38.1% (TRUS 5-point Likert score system) and 35.2% (mp-MRI PI-RADS v2.1 score system) to 32.0% (2D P-Net) and 25.8% (3D P-Net). 3D P-Net yielded the highest net benefit according to the DCAs.

Interpretation

3D P-Net based on a prostate grayscale TRUS video achieved satisfactory performance in identifying csPCa and potentially reducing unnecessary biopsies. More studies to determine how AI models better integrate into routine practice and randomized controlled trials to show the values of these models in real clinical applications are warranted.

Funding

The National Natural Science Foundation of China (Grants 82202174 and 82202153), the Science and Technology Commission of Shanghai Municipality (Grants 18441905500 and 19DZ2251100), Shanghai Municipal Health Commission (Grants 2019LJ21 and SHSLCZDZK03502), Shanghai Science and Technology Innovation Action Plan (21Y11911200), and Fundamental Research Funds for the Central Universities (ZD-11-202151), Scientific Research and Development Fund of Zhongshan Hospital of Fudan University (Grant 2022ZSQD07).

Predicting breast cancer types on and beyond molecular level in a multi-modal fashion

Accurately determining the molecular subtypes of breast cancer is important for the prognosis of breast cancer patients and can guide treatment selection. In this study, we develop a deep learning-based model for predicting the molecular subtypes of breast cancer directly from the diagnostic mammography and ultrasound images. Multi-modal deep learning with intra- and inter-modality attention modules (MDL-IIA) is proposed to extract important relations between mammography and ultrasound for this task. MDL-IIA leads to the best diagnostic performance compared to other cohort models in predicting 4-category molecular subtypes with Matthews correlation coefficient (MCC) of 0.837 (95% confidence interval [CI]: 0.803, 0.870). The MDL-IIA model can also discriminate between Luminal and Non-Luminal disease with an area under the receiver operating characteristic curve of 0.929 (95% CI: 0.903, 0.951). These results significantly outperform clinicians' predictions based on radiographic imaging. Beyond molecular-level test, based on gene-level ground truth, our method can bypass the inherent uncertainty from immunohistochemistry test. This work thus provides a noninvasive method to predict the molecular subtypes of breast cancer, potentially guiding treatment selection for breast cancer patients and providing decision support for clinicians.

Quantitative evaluation of gastrocnemius medialis mass in patients with chronic heart failure by gray-scale ultrasound and shear wave elastography

Objective

To assess the usefulness of gray-scale ultrasound (US) and shear wave elastography (SWE) in assessing the condition of the skeletal muscles in patients with chronic heart failure (CHF).

Methods

We prospectively compared 20 patients with clinically diagnosed CHF and a control population of 20 normal volunteers. The gastrocnemius medialis (GM) of each individual in the rest and the contraction position was assessed using gray-scale US and SWE. The quantitative US parameters including the fascicle length (FL), pinnation angle (PA), echo intensity (EI), and Young's modulus of the muscle were measured.

Results

In the CHF group compared with the control group, in the rest position, there was a significant difference in EI, PA, and FL of the GM (P < 0.001), but no statistically significant difference in Young's modulus values (P > 0.05); however, in the contraction position, all parameters were statistically different between the two groups (P < 0.001). In the different subgroups of the CHF group grouped according to New York Heart Association staging (NYHA) or left ventricular ejection fraction (LVEF), there were no significant differences in ultrasound parameters in the rest position. However, during the contraction of GM, the smaller the FL and Young's modulus, the larger the PA and EI with the increase of NYHA grade or the decrease of LVEF (P < 0.001).

Conclusion

The gray-scale US and SWE can provide an objective assessment of skeletal muscle status for CHF patients and are expected to be used to guide their early rehabilitation training and improve their prognosis.

5G-Based Telerobotic Ultrasound System Improves Access to Breast Examination in Rural and Remote Areas: A Prospective and Two-Scenario Study

OBJECTIVE

Ultrasound (US) plays an important role in the diagnosis and management of breast diseases; however, effective breast US screening is lacking in rural and remote areas. To alleviate this issue, we prospectively evaluated the clinical availability of 5G-based telerobotic US technology for breast examinations in rural and remote areas.

METHODS

Between September 2020 and March 2021, 63 patients underwent conventional and telerobotic US examinations in a rural island (Scenario A), while 20 patients underwent telerobotic US examination in a mobile car located in a remote county (Scenario B) in May 2021. The safety, duration, US image quality, consistency, and acceptability of the 5G-based telerobotic US were assessed.

RESULTS

In Scenario A, the average duration of the telerobotic US procedure was longer than that of conventional US (10.3 ± 3.3 min vs. 7.6 ± 3.0 min, p = 0.017), but their average imaging scores were similar (4.86 vs. 4.90, p = 0.159). Two cases of gynecomastia, one of lactation mastitis, and one of postoperative breast effusion were diagnosed and 32 nodules were detected using the two US methods. There was good interobserver agreement between the US features and BI-RADS categories of the identical nodules (ICC = 0.795-1.000). In Scenario B, breast nodules were detected in 65% of the patients using telerobotic US. Its average duration was 10.1 ± 2.3 min, and the average imaging score was 4.85. Overall, 90.4% of the patients were willing to choose telerobotic US in the future, and tele-sonologists were satisfied with 85.5% of the examinations.

CONCLUSION

The 5G-based telerobotic US system is feasible for providing effective breast examinations in rural and remote areas.

Deep learning-based system for automatic prediction of triple-negative breast cancer from ultrasound images

To develop a deep-learning system for the automatic identification of triple-negative breast cancer (TNBC) solely from ultrasound images. A total of 145 patients and 831 images were retrospectively enrolled at Peking Union College Hospital from April 2018 to March 2019. Ultrasound images and clinical information were collected accordingly. Molecular subtypes were determined from immunohistochemical (IHC) results. A CNN with VGG-based architecture was then used to predict TNBC. The model's performance was evaluated using randomized k-fold stratified cross-validation. A t-SNE analysis and saliency maps were used for model visualization. TNBC was identified in 16 of 145 (11.03%) patients. One hundred fifteen (80%) patients, 15 (10%) patients, and 15 (10%) patients formed the train, validation, and test set respectively. The deep learning system exhibits good efficacy, with an AUC of 0.86 (95% CI: 0.64, 0.95), an accuracy of 85%, a sensitivity of 86%, a specificity of 86%, and an F1-score of 0.74. In addition, the internal representation features learned by the model showed clear differentiation across molecular subtype groups. Such a deep learning system can automatically predict triple-negative breast cancer preoperatively and accurately. It may help to get to more precise and comprehensive management.

Artificial Intelligence in Breast Ultrasound: From Diagnosis to Prognosis-A Rapid Review

BACKGROUND

Ultrasound (US) is a fundamental diagnostic tool in breast imaging. However, US remains an operator-dependent examination. Research into and the application of artificial intelligence (AI) in breast US are increasing. The aim of this rapid review was to assess the current development of US-based artificial intelligence in the field of breast cancer.

METHODS

Two investigators with experience in medical research performed literature searching and data extraction on PubMed. The studies included in this rapid review evaluated the role of artificial intelligence concerning BC diagnosis, prognosis, molecular subtypes of breast cancer, axillary lymph node status, and the response to neoadjuvant chemotherapy. The mean values of sensitivity, specificity, and AUC were calculated for the main study categories with a meta-analytical approach.

RESULTS

A total of 58 main studies, all published after 2017, were included. Only 9/58 studies were prospective (15.5%); 13/58 studies (22.4%) used an ML approach. The vast majority (77.6%) used DL systems. Most studies were conducted for the diagnosis or classification of BC (55.1%). At present, all the included studies showed that AI has excellent performance in breast cancer diagnosis, prognosis, and treatment strategy.

CONCLUSIONS

US-based AI has great potential and research value in the field of breast cancer diagnosis, treatment, and prognosis. More prospective and multicenter studies are needed to assess the potential impact of AI in breast ultrasound.

Application of Deep Learning to Reduce the Rate of Malignancy Among BI-RADS 4A Breast Lesions Based on Ultrasonography

The aim of the work described here was to develop an ultrasound (US) image-based deep learning model to reduce the rate of malignancy among breast lesions diagnosed as category 4A of the Breast Imaging-Reporting and Data System (BI-RADS) during the pre-operative US examination. A total of 479 breast lesions diagnosed as BI-RADS 4A in pre-operative US examination were enrolled. There were 362 benign lesions and 117 malignant lesions confirmed by postoperative pathology with a malignancy rate of 24.4%. US images were collected from the database server. They were then randomly divided into training and testing cohorts at a ratio of 4:1. To correctly classify malignant and benign tumors diagnosed as BI-RADS 4A in US, four deep learning models, including MobileNet, DenseNet121, Xception and Inception V3, were developed. The performance of deep learning models was compared using the area under the receiver operating characteristic curve (AUROC), accuracy, sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV). Meanwhile, the robustness of the models was evaluated by five-fold cross-validation. Among the four models, the MobileNet model turned to be the optimal model with the best performance in classifying benign and malignant lesions among BI-RADS 4A breast lesions. The AUROC, accuracy, sensitivity, specificity, PPV and NPV of the optimal model in the testing cohort were 0.897, 0.913, 0.926, 0.899, 0.958 and 0.784, respectively. About 14.4% of patients were expected to be upgraded to BI-RADS 4B in US with the assistance of the MobileNet model. The deep learning model MobileNet can help to reduce the rate of malignancy among BI-RADS 4A breast lesions in pre-operative US examinations, which is valuable to clinicians in tailoring treatment for suspicious breast lesions identified on US.

Intercellular Communication Reveals Therapeutic Potential of Epithelial-Mesenchymal Transition in Triple-Negative Breast Cancer

(1) Background: Triple-negative breast cancer (TNBC) is an aggressive subtype of breast cancer with high intra-tumoral heterogeneity. The epithelial-mesenchymal transition (EMT) is one of the inducers of cancer metastasis and migration. However, the description of the EMT process in TNBC using single-cell RNA sequencing (scRNA-seq) remains unclear. (2) Methods: In this study, we analyzed 8938 cellular gene expression profiles from five TNBC patients. We first scored each malignant cell based on functional pathways to determine its EMT characteristics. Then, a pseudo-time trajectory analysis was employed to characterize the cell trajectories. Furthermore, CellChat was used to identify the cellular communications. (3) Results: We identified 888 epithelium-like and 846 mesenchyme-like malignant cells, respectively. A further pseudo-time trajectory analysis indicated the transition trends from epithelium-like to mesenchyme-like in malignant cells. To characterize the potential regulators of the EMT process, we identified 10 dysregulated transcription factors (TFs) between epithelium-like and mesenchyme-like malignant cells, in which overexpressed forkhead box protein A1 (FOXA1) was recognized as a poor prognosis marker of TNBC. Furthermore, we dissected the cell-cell communications via ligand-receptor (L-R) interactions. We observed that tumor-associated macrophages (TAMs) may support the invasion of malignant epithelial cells, based on CXCL-CXCR2 signaling. The tumor necrosis factor (TNF) signaling pathway secreted by TAMs was identified as an outgoing communication pattern, mediating the communications between monocytes/TAMs and malignant epithelial cells. Alternatively, the TNF-related ligand-receptor (L-R) pairs showed promising clinical implications. Some immunotherapy and anti-neoplastic drugs could interact with the L-R pairs as a potential strategy for the treatment of TNBC. In summary, this study enhances the understanding of the EMT process in the TNBC microenvironment, and dissections of EMT-related cell communications also provided us with potential treatment targets.

A Comparative Study of Multiple Deep Learning Models Based on Multi-Input Resolution for Breast Ultrasound Images

Purpose

The purpose of this study was to explore the performance of different parameter combinations of deep learning (DL) models (Xception, DenseNet121, MobileNet, ResNet50 and EfficientNetB0) and input image resolutions (REZs) (224 × 224, 320 × 320 and 488 × 488 pixels) for breast cancer diagnosis.

Methods

This multicenter study retrospectively studied gray-scale ultrasound breast images enrolled from two Chinese hospitals. The data are divided into training, validation, internal testing and external testing set. Three-hundreds images were randomly selected for the physician-AI comparison. The Wilcoxon test was used to compare the diagnose error of physicians and models under P=0.05 and 0.10 significance level. The specificity, sensitivity, accuracy, area under the curve (AUC) were used as primary evaluation metrics.

Results

A total of 13,684 images of 3447 female patients are finally included. In external test the 224 and 320 REZ achieve the best performance in MobileNet and EfficientNetB0 respectively (AUC: 0.893 and 0.907). Meanwhile, 448 REZ achieve the best performance in Xception, DenseNet121 and ResNet50 (AUC: 0.900, 0.883 and 0.871 respectively). In physician-AI test set, the 320 REZ for EfficientNetB0 (AUC: 0.896, P < 0.1) is better than senior physicians. Besides, the 224 REZ for MobileNet (AUC: 0.878, P < 0.1), 448 REZ for Xception (AUC: 0.895, P < 0.1) are better than junior physicians. While the 448 REZ for DenseNet121 (AUC: 0.880, P < 0.05) and ResNet50 (AUC: 0.838, P < 0.05) are only better than entry physicians.

Conclusion

Based on the gray-scale ultrasound breast images, we obtained the best DL combination which was better than the physicians.