Breast Cancer Classification from Ultrasound Images Using Probability-Based Optimal Deep Learning Feature Fusion

Deep learning radiomics based on multimodal imaging for distinguishing benign and malignant breast tumours

Objectives

This study aimed to develop a deep learning radiomic model using multimodal imaging to differentiate benign and malignant breast tumours.

Methods

Multimodality imaging data, including ultrasonography (US), mammography (MG), and magnetic resonance imaging (MRI), from 322 patients (112 with benign breast tumours and 210 with malignant breast tumours) with histopathologically confirmed breast tumours were retrospectively collected between December 2018 and May 2023. Based on multimodal imaging, the experiment was divided into three parts: traditional radiomics, deep learning radiomics, and feature fusion. We tested the performance of seven classifiers, namely, SVM, KNN, random forest, extra trees, XGBoost, LightGBM, and LR, on different feature models. Through feature fusion using ensemble and stacking strategies, we obtained the optimal classification model for benign and malignant breast tumours.

Results

In terms of traditional radiomics, the ensemble fusion strategy achieved the highest accuracy, AUC, and specificity, with values of 0.892, 0.942 [0.886-0.996], and 0.956 [0.873-1.000], respectively. The early fusion strategy with US, MG, and MRI achieved the highest sensitivity of 0.952 [0.887-1.000]. In terms of deep learning radiomics, the stacking fusion strategy achieved the highest accuracy, AUC, and sensitivity, with values of 0.937, 0.947 [0.887-1.000], and 1.000 [0.999-1.000], respectively. The early fusion strategies of US+MRI and US+MG achieved the highest specificity of 0.954 [0.867-1.000]. In terms of feature fusion, the ensemble and stacking approaches of the late fusion strategy achieved the highest accuracy of 0.968. In addition, stacking achieved the highest AUC and specificity, which were 0.997 [0.990-1.000] and 1.000 [0.999-1.000], respectively. The traditional radiomic and depth features of US+MG + MR achieved the highest sensitivity of 1.000 [0.999-1.000] under the early fusion strategy.

Conclusion

This study demonstrated the potential of integrating deep learning and radiomic features with multimodal images. As a single modality, MRI based on radiomic features achieved greater accuracy than US or MG. The US and MG models achieved higher accuracy with transfer learning than the single-mode or radiomic models. The traditional radiomic and depth features of US+MG + MR achieved the highest sensitivity under the early fusion strategy, showed higher diagnostic performance, and provided more valuable information for differentiation between benign and malignant breast tumours.

Improving the Automated Diagnosis of Breast Cancer with Mesh Reconstruction of Ultrasound Images Incorporating 3D Mesh Features and a Graph Attention Network

This study proposes a novel approach for breast tumor classification from ultrasound images into benign and malignant by converting the region of interest (ROI) of a 2D ultrasound image into a 3D representation using the point-e system, allowing for in-depth analysis of underlying characteristics. Instead of relying solely on 2D imaging features, this method extracts 3D mesh features that describe tumor patterns more precisely. Ten informative and medically relevant mesh features are extracted and assessed with two feature selection techniques. Additionally, a feature pattern analysis has been conducted to determine the feature's significance. A feature table with dimensions of 445 × 12 is generated and a graph is constructed, considering the rows as nodes and the relationships among the nodes as edges. The Spearman correlation coefficient method is employed to identify edges between the strongly connected nodes (with a correlation score greater than or equal to 0.7), resulting in a graph containing 56,054 edges and 445 nodes. A graph attention network (GAT) is proposed for the classification task and the model is optimized with an ablation study, resulting in the highest accuracy of 99.34%. The performance of the proposed model is compared with ten machine learning (ML) models and one-dimensional convolutional neural network where the test accuracy of these models ranges from 73 to 91%. Our novel 3D mesh-based approach, coupled with the GAT, yields promising performance for breast tumor classification, outperforming traditional models, and has the potential to reduce time and effort of radiologists providing a reliable diagnostic system.

Machine-learning models for diagnosis of rotator cuff tears in osteoporosis patients based on anteroposterior X-rays of the shoulder joint

OBJECTIVE

This study aims to diagnose Rotator Cuff Tears (RCT) and classify the severity of RCT in patients with Osteoporosis (OP) through the analysis of shoulder joint anteroposterior (AP) X-ray-based localized proximal humeral bone mineral density (BMD) measurements and clinical information based on machine learning (ML) models.

METHODS

A retrospective cohort of 89 patients was analyzed, including 63 with both OP and RCT (OPRCT) and 26 with OP only. The study analyzed a series of shoulder radiographs from April 2021 to April 2023. Grayscale values were measured after plotting ROIs based on AP X-rays of shoulder joint. Five kinds of ML models were developed and compared based on their performance in predicting the occurrence and severity of RCT from ROIs' greyscale values and clinical information (age, gender, advantage side, lumbar BMD, and acromion morphology (AM)). Further analysis using SHAP values illustrated the significant impact of selected features on model predictions.

RESULTS

R1-6 had a positive correlation with BMD respectively. The nine variables, including greyscale R1-6, age, BMD, and AM, were used in the prediction models. The RF model was determined to be superior in effectively diagnosing RCT in OP patients, with high AUC scores of 0.998, 0.889, and 0.95 in the training, validation, and testing sets, respectively. SHAP values revealed that the most influential factors on the diagnostic outcomes were the grayscale values of all cancellous bones in ROIs. A column-line graph prediction model based on nine variables was constructed, and DCA curves indicated that RCT prediction in OP patients was favored based on this model. Furthermore, the RF model was also the most superior in predicting the types of RCT within the OPRCT group, with an accuracy of 86.364% and 73.684% in the training and test sets, respectively. SHAP values indicated that the most significant factor affecting the predictive outcomes was the AM, followed by the grayscale values of the greater tubercle, among others.

CONCLUSIONS

ML models, particularly the RF algorithm, show significant promise in diagnosing RCT occurrence and severity in OP patients using conventional shoulder X-rays based on the nine variables. This method presents a cost-effective, accessible, and non-invasive diagnostic strategy that has the potential to substantially enhance the early detection and management of RCT in OP patient population.

A Review of Artificial Intelligence in Breast Imaging

With the increasing dominance of artificial intelligence (AI) techniques, the important prospects for their application have extended to various medical fields, including domains such as in vitro diagnosis, intelligent rehabilitation, medical imaging, and prognosis. Breast cancer is a common malignancy that critically affects women's physical and mental health. Early breast cancer screening-through mammography, ultrasound, or magnetic resonance imaging (MRI)-can substantially improve the prognosis for breast cancer patients. AI applications have shown excellent performance in various image recognition tasks, and their use in breast cancer screening has been explored in numerous studies. This paper introduces relevant AI techniques and their applications in the field of medical imaging of the breast (mammography and ultrasound), specifically in terms of identifying, segmenting, and classifying lesions; assessing breast cancer risk; and improving image quality. Focusing on medical imaging for breast cancer, this paper also reviews related challenges and prospects for AI.

CT-based radiomics research for discriminating the risk stratification of pheochromocytoma using different machine learning models: a multi-center study

OBJECTIVES

The purpose of this study was to explore and verify the value of various machine learning models in preoperative risk stratification of pheochromocytoma.

METHODS

A total of 155 patients diagnosed with pheochromocytoma through surgical pathology were included in this research (training cohort: n = 105; test cohort: n = 50); the risk stratification scoring system classified a PASS score of < 4 as low risk and a PASS score of ≥ 4 as high risk. From CT images captured during the non-enhanced, arterial, and portal venous phase, radiomic features were extracted. After reducing dimensions and selecting features, Logistic Regression (LR), Extra Trees, and K-Nearest Neighbor (KNN) were utilized to construct the radiomics models. By adopting ROC curve analysis, the optimal radiomics model was selected. Univariate and multivariate logistic regression analyses of clinical radiological features were used to determine the variables and establish a clinical model. The integration of radiomics and clinical features resulted in the creation of a combined model. ROC curve analysis was used to evaluate the performance of the model, while decision curve analysis (DCA) was employed to assess its clinical value.

RESULTS

3591 radiomics features were extracted from the region of interest in unenhanced and dual-phase (arterial and portal venous phase) CT images. 13 radiomics features were deemed to be valuable. The LR model demonstrated the highest prediction efficiency and robustness among the tested radiomics models, with an AUC of 0.877 in the training cohort and 0.857 in the test cohort. Ultimately, the composite of clinical features was utilized to formulate the clinical model. The combined model demonstrated the best discriminative ability (AUC, training cohort: 0.887; test cohort: 0.874). The DCA of the combined model showed the best clinical efficacy.

CONCLUSION

The combined model integrating radiomics and clinical features had an outstanding performance in differentiating the risk of pheochromocytoma and could offer a non-intrusive and effective approach for making clinical decisions.

Offloading the computational complexity of transfer learning with generic features

Deep learning approaches are generally complex, requiring extensive computational resources and having high time complexity. Transfer learning is a state-of-the-art approach to reducing the requirements of high computational resources by using pre-trained models without compromising accuracy and performance. In conventional studies, pre-trained models are trained on datasets from different but similar domains with many domain-specific features. The computational requirements of transfer learning are directly dependent on the number of features that include the domain-specific and the generic features. This article investigates the prospects of reducing the computational requirements of the transfer learning models by discarding domain-specific features from a pre-trained model. The approach is applied to breast cancer detection using the dataset curated breast imaging subset of the digital database for screening mammography and various performance metrics such as precision, accuracy, recall, F1-score, and computational requirements. It is seen that discarding the domain-specific features to a specific limit provides significant performance improvements as well as minimizes the computational requirements in terms of training time (reduced by approx. 12%), processor utilization (reduced approx. 25%), and memory usage (reduced approx. 22%). The proposed transfer learning strategy increases accuracy (approx. 7%) and offloads computational complexity expeditiously.

A Comprehensive Review on Synergy of Multi-Modal Data and AI Technologies in Medical Diagnosis

Disease diagnosis represents a critical and arduous endeavor within the medical field. Artificial intelligence (AI) techniques, spanning from machine learning and deep learning to large model paradigms, stand poised to significantly augment physicians in rendering more evidence-based decisions, thus presenting a pioneering solution for clinical practice. Traditionally, the amalgamation of diverse medical data modalities (e.g., image, text, speech, genetic data, physiological signals) is imperative to facilitate a comprehensive disease analysis, a topic of burgeoning interest among both researchers and clinicians in recent times. Hence, there exists a pressing need to synthesize the latest strides in multi-modal data and AI technologies in the realm of medical diagnosis. In this paper, we narrow our focus to five specific disorders (Alzheimer's disease, breast cancer, depression, heart disease, epilepsy), elucidating advanced endeavors in their diagnosis and treatment through the lens of artificial intelligence. Our survey not only delineates detailed diagnostic methodologies across varying modalities but also underscores commonly utilized public datasets, the intricacies of feature engineering, prevalent classification models, and envisaged challenges for future endeavors. In essence, our research endeavors to contribute to the advancement of diagnostic methodologies, furnishing invaluable insights for clinical decision making.

Malignancy pattern analysis of breast ultrasound images using clinical features and a graph convolutional network

Objective

Early diagnosis of breast cancer can lead to effective treatment, possibly increase long-term survival rates, and improve quality of life. The objective of this study is to present an automated analysis and classification system for breast cancer using clinical markers such as tumor shape, orientation, margin, and surrounding tissue. The novelty and uniqueness of the study lie in the approach of considering medical features based on the diagnosis of radiologists.

Methods

Using clinical markers, a graph is generated where each feature is represented by a node, and the connection between them is represented by an edge which is derived through Pearson's correlation method. A graph convolutional network (GCN) model is proposed to classify breast tumors into benign and malignant, using the graph data. Several statistical tests are performed to assess the importance of the proposed features. The performance of the proposed GCN model is improved by experimenting with different layer configurations and hyper-parameter settings.

Results

Results show that the proposed model has a 98.73% test accuracy. The performance of the model is compared with a graph attention network, a one-dimensional convolutional neural network, and five transfer learning models, ten machine learning models, and three ensemble learning models. The performance of the model was further assessed with three supplementary breast cancer ultrasound image datasets, where the accuracies are 91.03%, 94.37%, and 89.62% for Dataset A, Dataset B, and Dataset C (combining Dataset A and Dataset B) respectively. Overfitting issues are assessed through k-fold cross-validation.

Conclusion

Several variants are utilized to present a more rigorous and fair evaluation of our work, especially the importance of extracting clinically relevant features. Moreover, a GCN model using graph data can be a promising solution for an automated feature-based breast image classification system.

Edge-Driven Multi-Agent Reinforcement Learning: A Novel Approach to Ultrasound Breast Tumor Segmentation

A segmentation model of the ultrasound (US) images of breast tumors based on virtual agents trained using reinforcement learning (RL) is proposed. The agents, living in the edge map, are able to avoid false boundaries, connect broken parts, and finally, accurately delineate the contour of the tumor. The agents move similarly to robots navigating in the unknown environment with the goal of maximizing the rewards. The individual agent does not know the goal of the entire population. However, since the robots communicate, the model is able to understand the global information and fit the irregular boundaries of complicated objects. Combining the RL with a neural network makes it possible to automatically learn and select the local features. In particular, the agents handle the edge leaks and artifacts typical for the US images. The proposed model outperforms 13 state-of-the-art algorithms, including selected deep learning models and their modifications.

State-of-the-Art of Breast Cancer Diagnosis in Medical Images via Convolutional Neural Networks (CNNs)

Early detection of breast cancer is crucial for a better prognosis. Various studies have been conducted where tumor lesions are detected and localized on images. This is a narrative review where the studies reviewed are related to five different image modalities: histopathological, mammogram, magnetic resonance imaging (MRI), ultrasound, and computed tomography (CT) images, making it different from other review studies where fewer image modalities are reviewed. The goal is to have the necessary information, such as pre-processing techniques and CNN-based diagnosis techniques for the five modalities, readily available in one place for future studies. Each modality has pros and cons, such as mammograms might give a high false positive rate for radiographically dense breasts, while ultrasounds with low soft tissue contrast result in early-stage false detection, and MRI provides a three-dimensional volumetric image, but it is expensive and cannot be used as a routine test. Various studies were manually reviewed using particular inclusion and exclusion criteria; as a result, 91 recent studies that classify and detect tumor lesions on breast cancer images from 2017 to 2022 related to the five image modalities were included. For histopathological images, the maximum accuracy achieved was around 99 % , and the maximum sensitivity achieved was 97.29 % by using DenseNet, ResNet34, and ResNet50 architecture. For mammogram images, the maximum accuracy achieved was 96.52 % using a customized CNN architecture. For MRI, the maximum accuracy achieved was 98.33 % using customized CNN architecture. For ultrasound, the maximum accuracy achieved was around 99 % by using DarkNet-53, ResNet-50, G-CNN, and VGG. For CT, the maximum sensitivity achieved was 96 % by using Xception architecture. Histopathological and ultrasound images achieved higher accuracy of around 99 % by using ResNet34, ResNet50, DarkNet-53, G-CNN, and VGG compared to other modalities for either of the following reasons: use of pre-trained architectures with pre-processing techniques, use of modified architectures with pre-processing techniques, use of two-stage CNN, and higher number of studies available for Artificial Intelligence (AI)/machine learning (ML) researchers to reference. One of the gaps we found is that only a single image modality is used for CNN-based diagnosis; in the future, a multiple image modality approach can be used to design a CNN architecture with higher accuracy.

Graph neural network-based breast cancer diagnosis using ultrasound images with optimized graph construction integrating the medically significant features

PURPOSE

An automated computerized approach can aid radiologists in the early diagnosis of breast cancer. In this study, a novel method is proposed for classifying breast tumors into benign and malignant, based on the ultrasound images through a Graph Neural Network (GNN) model utilizing clinically significant features.

METHOD

Ten informative features are extracted from the region of interest (ROI), based on the radiologists' diagnosis markers. The significance of the features is evaluated using density plot and T test statistical analysis method. A feature table is generated where each row represents individual image, considered as node, and the edges between the nodes are denoted by calculating the Spearman correlation coefficient. A graph dataset is generated and fed into the GNN model. The model is configured through ablation study and Bayesian optimization. The optimized model is then evaluated with different correlation thresholds for getting the highest performance with a shallow graph. The performance consistency is validated with k-fold cross validation. The impact of utilizing ROIs and handcrafted features for breast tumor classification is evaluated by comparing the model's performance with Histogram of Oriented Gradients (HOG) descriptor features from the entire ultrasound image. Lastly, a clustering-based analysis is performed to generate a new filtered graph, considering weak and strong relationships of the nodes, based on the similarities.

RESULTS

The results indicate that with a threshold value of 0.95, the GNN model achieves the highest test accuracy of 99.48%, precision and recall of 100%, and F1 score of 99.28%, reducing the number of edges by 85.5%. The GNN model's performance is 86.91%, considering no threshold value for the graph generated from HOG descriptor features. Different threshold values for the Spearman's correlation score are experimented with and the performance is compared. No significant differences are observed between the previous graph and the filtered graph.

CONCLUSION

The proposed approach might aid the radiologists in effective diagnosing and learning tumor pattern of breast cancer.

Employing Atrous Pyramid Convolutional Deep Learning Approach for Detection to Diagnose Breast Cancer Tumors

Breast cancer is among the most common diseases and one of the most common causes of death in the female population worldwide. Early identification of breast cancer improves survival. Therefore, radiologists will be able to make more accurate diagnoses if a computerized system is developed to detect breast cancer. Computer-aided design techniques have the potential to help medical professionals to determine the specific location of breast tumors and better manage this disease more rapidly and accurately. MIAS datasets were used in this study. The aim of this study is to evaluate a noise reduction for mammographic pictures and to identify salt and pepper, Gaussian, and Poisson so that precise mass detection operations can be estimated. As a result, it provides a method for noise reduction known as quantum wavelet transform (QWT) filtering and an image morphology operator for precise mass segmentation in mammographic images by utilizing an Atrous pyramid convolutional neural network as the deep learning model for classification of mammographic images. The hybrid methodology dubbed QWT-APCNN is compared to earlier methods in terms of peak signal-to-noise ratio (PSNR) and mean square error (MSE) in noise reduction and detection accuracy for mass area recognition. Compared to state-of-the-art approaches, the proposed method performed better at noise reduction and segmentation according to different evaluation criteria such as an accuracy rate of 98.57%, 92% sensitivity, 88% specificity, 90% DSS, and ROC and AUC rate of 88.77.

Evaluation of Breast Cancer Tumor-Infiltrating Lymphocytes on Ultrasound Images Based on a Novel Multi-Cascade Residual U-Shaped Network

OBJECTIVE

Breast cancer has become the leading cancer of the 21st century. Tumor-infiltrating lymphocytes (TILs) have emerged as effective biomarkers for predicting treatment response and prognosis in breast cancer. The work described here was aimed at designing a novel deep learning network to assess the levels of TILs in breast ultrasound images.

METHODS

We propose the Multi-Cascade Residual U-Shaped Network (MCRUNet), which incorporates a gray feature enhancement (GFE) module for image reconstruction and normalization to achieve data synergy. Additionally, multiple residual U-shaped (RSU) modules are cascaded as the backbone network to maximize the fusion of global and local features, with a focus on the tumor's location and surrounding regions. The development of MCRUNet is based on data from two hospitals and uses a publicly available ultrasound data set for transfer learning.

RESULTS

MCRUNet exhibits excellent performance in assessing TILs levels, achieving an area under the receiver operating characteristic curve of 0.8931, an accuracy of 85.71%, a sensitivity of 83.33%, a specificity of 88.64% and an F1 score of 86.54% in the test group. It outperforms six state-of-the-art networks in terms of performance.

CONCLUSION

The MCRUNet network based on breast ultrasound images of breast cancer patients holds promise for non-invasively predicting TILs levels and aiding personalized treatment decisions.

Automated breast tumor ultrasound image segmentation with hybrid UNet and classification using fine-tuned CNN model

Introduction

Breast cancer stands as the second most deadly form of cancer among women worldwide. Early diagnosis and treatment can significantly mitigate mortality rates.

Purpose

The study aims to classify breast ultrasound images into benign and malignant tumors. This approach involves segmenting the breast's region of interest (ROI) employing an optimized UNet architecture and classifying the ROIs through an optimized shallow CNN model utilizing an ablation study.

Method

Several image processing techniques are utilized to improve image quality by removing text, artifacts, and speckle noise, and statistical analysis is done to check the enhanced image quality is satisfactory. With the processed dataset, the segmentation of breast tumor ROI is carried out, optimizing the UNet model through an ablation study where the architectural configuration and hyperparameters are altered. After obtaining the tumor ROIs from the fine-tuned UNet model (RKO-UNet), an optimized CNN model is employed to classify the tumor into benign and malignant classes. To enhance the CNN model's performance, an ablation study is conducted, coupled with the integration of an attention unit. The model's performance is further assessed by classifying breast cancer with mammogram images.

Result

The proposed classification model (RKONet-13) results in an accuracy of 98.41 %. The performance of the proposed model is further compared with five transfer learning models for both pre-segmented and post-segmented datasets. K-fold cross-validation is done to assess the proposed RKONet-13 model's performance stability. Furthermore, the performance of the proposed model is compared with previous literature, where the proposed model outperforms existing methods, demonstrating its effectiveness in breast cancer diagnosis. Lastly, the model demonstrates its robustness for breast cancer classification, delivering an exceptional performance of 96.21 % on a mammogram dataset.

Conclusion

The efficacy of this study relies on image pre-processing, segmentation with hybrid attention UNet, and classification with fine-tuned robust CNN model. This comprehensive approach aims to determine an effective technique for detecting breast cancer within ultrasound images.

Role of long non-coding RNAs and TGF-β signaling in the regulation of breast cancer pathogenesis and therapeutic targets

The cytokine known as transforming growth factor (TGF) is essential for cell development, differentiation, and apoptosis in BC. TGF-β dysregulation can either promote or inhibit tumor development, and it is a key signaling pathway in BC spread. A recently identified family of ncRNAs known as lncRNAs has received a great deal of effort and is an important regulator of many cellular processes, including transcription of genes, chromatin remodeling, progression of the cell cycle, and posttranscriptional processing. Furthermore, both TGF-β signaling and lncRNAs serve as important early-stage biomarkers for BC diagnosis and prognosis and also play a significant role in BC drug resistance. According to recent studies, lncRNAs can regulate TGF-β by modulating its cofactors in BC. However, the particular functions of lncRNAs and the TGF-β pathway in controlling BC progression are not well understood yet. This review explores the lncRNAs' functional properties in BC as tumor suppressors or oncogenes in the regulation of genes, with a focus on dysregulated TGF-β signaling. Further, we emphasize the functional roles of lncRNAs and TGF-β pathway in the progression of BC to discover new treatment strategies and better comprehend the fundamental cellular pathways.

A Novel Fuzzy Relative-Position-Coding Transformer for Breast Cancer Diagnosis Using Ultrasonography

Breast cancer is a leading cause of death in women worldwide, and early detection is crucial for successful treatment. Computer-aided diagnosis (CAD) systems have been developed to assist doctors in identifying breast cancer on ultrasound images. In this paper, we propose a novel fuzzy relative-position-coding (FRPC) Transformer to classify breast ultrasound (BUS) images for breast cancer diagnosis. The proposed FRPC Transformer utilizes the self-attention mechanism of Transformer networks combined with fuzzy relative-position-coding to capture global and local features of the BUS images. The performance of the proposed method is evaluated on one benchmark dataset and compared with those obtained by existing Transformer approaches using various metrics. The experimental outcomes distinctly establish the superiority of the proposed method in achieving elevated levels of accuracy, sensitivity, specificity, and F1 score (all at 90.52%), as well as a heightened area under the receiver operating characteristic (ROC) curve (0.91), surpassing those attained by the original Transformer model (at 89.54%, 89.54%, 89.54%, and 0.89, respectively). Overall, the proposed FRPC Transformer is a promising approach for breast cancer diagnosis. It has potential applications in clinical practice and can contribute to the early detection of breast cancer.

Efficient and Automatic Breast Cancer Early Diagnosis System Based on the Hierarchical Extreme Learning Machine

Breast cancer is the leading type of cancer in women, causing nearly 600,000 deaths every year, globally. Although the tumors can be localized within the breast, they can spread to other body parts, causing more harm. Therefore, early diagnosis can help reduce the risks of this cancer. However, a breast cancer diagnosis is complicated, requiring biopsy by various methods, such as MRI, ultrasound, BI-RADS, or even needle aspiration and cytology with the suggestions of specialists. On certain occasions, such as body examinations of a large number of people, it is also a large workload to check the images. Therefore, in this work, we present an efficient and automatic diagnosis system based on the hierarchical extreme learning machine (H-ELM) for breast cancer ultrasound results with high efficiency and make a primary diagnosis of the images. To make it compatible to use, this system consists of PNG images and general medical software within the H-ELM framework, which is easily trained and applied. Furthermore, this system only requires ultrasound images on a small scale, of 28×28 pixels, reducing the resources and fulfilling the application with low-resolution images. The experimental results show that the system can achieve 86.13% in the classification of breast cancer based on ultrasound images from the public breast ultrasound images (BUSI) dataset, without other relative information and supervision, which is higher than the conventional deep learning methods on the same dataset. Moreover, the training time is highly reduced, to only 5.31 s, and consumes few resources. The experimental results indicate that this system could be helpful for precise and efficient early diagnosis of breast cancers with primary examination results.

Deep Learning-assisted Diagnosis of Breast Lesions on US Images: A Multivendor, Multicenter Study

Purpose

To evaluate the diagnostic performance of a deep learning (DL) model for breast US across four hospitals and assess its value to readers with different levels of experience.

Materials and Methods

In this retrospective study, a dual attention-based convolutional neural network was built and validated to discriminate malignant tumors from benign tumors by using B-mode and color Doppler US images (n = 45 909, March 2011-August 2018), acquired with 42 types of US machines, of 9895 pathologic analysis-confirmed breast lesions in 8797 patients (27 men and 8770 women; mean age, 47 years ± 12 [SD]). With and without assistance from the DL model, three novice readers with less than 5 years of US experience and two experienced readers with 8 and 18 years of US experience, respectively, interpreted 1024 randomly selected lesions. Differences in the areas under the receiver operating characteristic curves (AUCs) were tested using the DeLong test.

Results

The DL model using both B-mode and color Doppler US images demonstrated expert-level performance at the lesion level, with an AUC of 0.94 (95% CI: 0.92, 0.95) for the internal set. In external datasets, the AUCs were 0.92 (95% CI: 0.90, 0.94) for hospital 1, 0.91 (95% CI: 0.89, 0.94) for hospital 2, and 0.96 (95% CI: 0.94, 0.98) for hospital 3. DL assistance led to improved AUCs (P < .001) for one experienced and three novice radiologists and improved interobserver agreement. The average false-positive rate was reduced by 7.6% (P = .08).

Conclusion

The DL model may help radiologists, especially novice readers, improve accuracy and interobserver agreement of breast tumor diagnosis using US.Keywords: Ultrasound, Breast, Diagnosis, Breast Cancer, Deep Learning, Ultrasonography Supplemental material is available for this article. © RSNA, 2023.

Automatic Breast Volume Scanner and B-Ultrasound-Based Radiomics Nomogram for Clinician Management of BI-RADS 4A Lesions

RATIONALE AND OBJECTIVES

To develop and validate a nomogram for predicting the risk of malignancy of breast imaging reporting and data system (BI-RADS) 4A lesions to reduce unnecessary invasive examinations.

MATERIALS AND METHODS

From January 2017 to July 2021, 190 cases of 4A lesions included in this study were divided into training and validation sets in a ratio of 8:2. Radiomics features were extracted from sonograms by Automatic Breast Volume Scanner (ABVS) and B-ultrasound. We constructed the radiomics model and calculated the rad-scores. Univariate and multivariate logistic regressions were used to assess demographics and lesion elastography values (virtual touch tissue image, shear wave velocity) and to develop clinical model. A clinical radiomics model was developed using rad-score and independent clinical factors, and a nomogram was plotted. Nomogram performance was evaluated using discrimination, calibration, and clinical utility.

RESULTS

The nomogram included rad-score, age, and elastography, and showed good calibration. In the training set, the area under the receiver operating characteristic curve (AUC) of the clinical radiomics model (0.900, 95% confidence interval (CI): 0.843-0.958) was superior to that of the radiomics model (0.860, 95% CI: 0.799-0.921) and clinical model (0.816, 95% CI: 0.735-0.958) (p = 0.024 and 0.008, respectively). The decision curve analysis showed that the clinical radiomics model had the highest net benefit in most threshold probability ranges.

CONCLUSION

ABVS and B-ultrasound-based radiomics nomograms have satisfactory performance in differentiating benign and malignant 4A lesions. This can help clinicians make an accurate diagnosis of 4A lesions and reduce unnecessary biopsy.

The effect of image resolution on convolutional neural networks in breast ultrasound

Purpose

The objective of this research was to investigate the efficacy of various parameter combinations of Convolutional Neural Networks (CNNs) models, namely MobileNet and DenseNet121, and different input image resolutions (REZs) ranging from 64×64 to 512×512 pixels, for diagnosing breast cancer.

Materials and methods

During the period of June 2015 to November 2020, two hospitals were involved in the collection of two-dimensional ultrasound breast images for this retrospective multicenter study. The diagnostic performance of the computer models MobileNet and DenseNet 121 was compared at different resolutions.

Results

The results showed that MobileNet had the best breast cancer diagnosis performance at 320×320pixel REZ and DenseNet121 had the best breast cancer diagnosis performance at 448×448pixel REZ.

Conclusion

Our study reveals a significant correlation between image resolution and breast cancer diagnosis accuracy. Through the comparison of MobileNet and DenseNet121, it is highlighted that lightweight neural networks (LW-CNNs) can achieve model performance similar to or even slightly better than large neural networks models (HW-CNNs) in ultrasound images, and LW-CNNs' prediction time per image is lower.

Deep Learning for Medical Image-Based Cancer Diagnosis

(1) Background: The application of deep learning technology to realize cancer diagnosis based on medical images is one of the research hotspots in the field of artificial intelligence and computer vision. Due to the rapid development of deep learning methods, cancer diagnosis requires very high accuracy and timeliness as well as the inherent particularity and complexity of medical imaging. A comprehensive review of relevant studies is necessary to help readers better understand the current research status and ideas. (2) Methods: Five radiological images, including X-ray, ultrasound (US), computed tomography (CT), magnetic resonance imaging (MRI), positron emission computed tomography (PET), and histopathological images, are reviewed in this paper. The basic architecture of deep learning and classical pretrained models are comprehensively reviewed. In particular, advanced neural networks emerging in recent years, including transfer learning, ensemble learning (EL), graph neural network, and vision transformer (ViT), are introduced. Five overfitting prevention methods are summarized: batch normalization, dropout, weight initialization, and data augmentation. The application of deep learning technology in medical image-based cancer analysis is sorted out. (3) Results: Deep learning has achieved great success in medical image-based cancer diagnosis, showing good results in image classification, image reconstruction, image detection, image segmentation, image registration, and image synthesis. However, the lack of high-quality labeled datasets limits the role of deep learning and faces challenges in rare cancer diagnosis, multi-modal image fusion, model explainability, and generalization. (4) Conclusions: There is a need for more public standard databases for cancer. The pre-training model based on deep neural networks has the potential to be improved, and special attention should be paid to the research of multimodal data fusion and supervised paradigm. Technologies such as ViT, ensemble learning, and few-shot learning will bring surprises to cancer diagnosis based on medical images.

Breast Cancer Classification through Meta-Learning Ensemble Technique Using Convolution Neural Networks

This study aims to develop an efficient and accurate breast cancer classification model using meta-learning approaches and multiple convolutional neural networks. This Breast Ultrasound Images (BUSI) dataset contains various types of breast lesions. The goal is to classify these lesions as benign or malignant, which is crucial for the early detection and treatment of breast cancer. The problem is that traditional machine learning and deep learning approaches often fail to accurately classify these images due to their complex and diverse nature. In this research, to address this problem, the proposed model used several advanced techniques, including meta-learning ensemble technique, transfer learning, and data augmentation. Meta-learning will optimize the model's learning process, allowing it to adapt to new and unseen datasets quickly. Transfer learning will leverage the pre-trained models such as Inception, ResNet50, and DenseNet121 to enhance the model's feature extraction ability. Data augmentation techniques will be applied to artificially generate new training images, increasing the size and diversity of the dataset. Meta ensemble learning techniques will combine the outputs of multiple CNNs, improving the model's classification accuracy. The proposed work will be investigated by pre-processing the BUSI dataset first, then training and evaluating multiple CNNs using different architectures and pre-trained models. Then, a meta-learning algorithm will be applied to optimize the learning process, and ensemble learning will be used to combine the outputs of multiple CNN. Additionally, the evaluation results indicate that the model is highly effective with high accuracy. Finally, the proposed model's performance will be compared with state-of-the-art approaches in other existing systems' accuracy, precision, recall, and F1 score.

Benign and Malignant Breast Tumor Classification in Ultrasound and Mammography Images via Fusion of Deep Learning and Handcraft Features

Breast cancer is a disease that affects women in different countries around the world. The real cause of breast cancer is particularly challenging to determine, and early detection of the disease is necessary for reducing the death rate, due to the high risks associated with breast cancer. Treatment in the early period can increase the life expectancy and quality of life for women. CAD (Computer Aided Diagnostic) systems can perform the diagnosis of the benign and malignant lesions of breast cancer using technologies and tools based on image processing, helping specialist doctors to obtain a more precise point of view with fewer processes when making their diagnosis by giving a second opinion. This study presents a novel CAD system for automated breast cancer diagnosis. The proposed method consists of different stages. In the preprocessing stage, an image is segmented, and a mask of a lesion is obtained; during the next stage, the extraction of the deep learning features is performed by a CNN-specifically, DenseNet 201. Additionally, handcrafted features (Histogram of Oriented Gradients (HOG)-based, ULBP-based, perimeter area, area, eccentricity, and circularity) are obtained from an image. The designed hybrid system uses CNN architecture for extracting deep learning features, along with traditional methods which perform several handcraft features, following the medical properties of the disease with the purpose of later fusion via proposed statistical criteria. During the fusion stage, where deep learning and handcrafted features are analyzed, the genetic algorithms as well as mutual information selection algorithm, followed by several classifiers (XGBoost, AdaBoost, Multilayer perceptron (MLP)) based on stochastic measures, are applied to choose the most sensible information group among the features. In the experimental validation of two modalities of the CAD design, which performed two types of medical studies-mammography (MG) and ultrasound (US)-the databases mini-DDSM (Digital Database for Screening Mammography) and BUSI (Breast Ultrasound Images Dataset) were used. Novel CAD systems were evaluated and compared with recent state-of-the-art systems, demonstrating better performance in commonly used criteria, obtaining ACC of 97.6%, PRE of 98%, Recall of 98%, F1-Score of 98%, and IBA of 95% for the abovementioned datasets.

Breast Cancer Diagnosis Based on IoT and Deep Transfer Learning Enabled by Fog Computing

Across all countries, both developing and developed, women face the greatest risk of breast cancer. Patients who have their breast cancer diagnosed and staged early have a better chance of receiving treatment before the disease spreads. The automatic analysis and classification of medical images are made possible by today's technology, allowing for quicker and more accurate data processing. The Internet of Things (IoT) is now crucial for the early and remote diagnosis of chronic diseases. In this study, mammography images from the publicly available online repository The Cancer Imaging Archive (TCIA) were used to train a deep transfer learning (DTL) model for an autonomous breast cancer diagnostic system. The data were pre-processed before being fed into the model. A popular deep learning (DL) technique, i.e., convolutional neural networks (CNNs), was combined with transfer learning (TL) techniques such as ResNet50, InceptionV3, AlexNet, VGG16, and VGG19 to boost prediction accuracy along with a support vector machine (SVM) classifier. Extensive simulations were analyzed by employing a variety of performances and network metrics to demonstrate the viability of the proposed paradigm. Outperforming some current works based on mammogram images, the experimental accuracy, precision, sensitivity, specificity, and f1-scores reached 97.99%, 99.51%, 98.43%, 80.08%, and 98.97%, respectively, on the huge dataset of mammography images categorized as benign and malignant, respectively. Incorporating Fog computing technologies, this model safeguards the privacy and security of patient data, reduces the load on centralized servers, and increases the output.

Deep Learning in Different Ultrasound Methods for Breast Cancer, from Diagnosis to Prognosis: Current Trends, Challenges, and an Analysis

Breast cancer is the second-leading cause of mortality among women around the world. Ultrasound (US) is one of the noninvasive imaging modalities used to diagnose breast lesions and monitor the prognosis of cancer patients. It has the highest sensitivity for diagnosing breast masses, but it shows increased false negativity due to its high operator dependency. Underserved areas do not have sufficient US expertise to diagnose breast lesions, resulting in delayed management of breast lesions. Deep learning neural networks may have the potential to facilitate early decision-making by physicians by rapidly yet accurately diagnosing and monitoring their prognosis. This article reviews the recent research trends on neural networks for breast mass ultrasound, including and beyond diagnosis. We discussed original research recently conducted to analyze which modes of ultrasound and which models have been used for which purposes, and where they show the best performance. Our analysis reveals that lesion classification showed the highest performance compared to those used for other purposes. We also found that fewer studies were performed for prognosis than diagnosis. We also discussed the limitations and future directions of ongoing research on neural networks for breast ultrasound.

Analyzing Histological Images Using Hybrid Techniques for Early Detection of Multi-Class Breast Cancer Based on Fusion Features of CNN and Handcrafted

Breast cancer is the second most common type of cancer among women, and it can threaten women's lives if it is not diagnosed early. There are many methods for detecting breast cancer, but they cannot distinguish between benign and malignant tumors. Therefore, a biopsy taken from the patient's abnormal tissue is an effective way to distinguish between malignant and benign breast cancer tumors. There are many challenges facing pathologists and experts in diagnosing breast cancer, including the addition of some medical fluids of various colors, the direction of the sample, the small number of doctors and their differing opinions. Thus, artificial intelligence techniques solve these challenges and help clinicians resolve their diagnostic differences. In this study, three techniques, each with three systems, were developed to diagnose multi and binary classes of breast cancer datasets and distinguish between benign and malignant types with 40× and 400× factors. The first technique for diagnosing a breast cancer dataset is using an artificial neural network (ANN) with selected features from VGG-19 and ResNet-18. The second technique for diagnosing breast cancer dataset is by ANN with combined features for VGG-19 and ResNet-18 before and after principal component analysis (PCA). The third technique for analyzing breast cancer dataset is by ANN with hybrid features. The hybrid features are a hybrid between VGG-19 and handcrafted; and a hybrid between ResNet-18 and handcrafted. The handcrafted features are mixed features extracted using Fuzzy color histogram (FCH), local binary pattern (LBP), discrete wavelet transform (DWT) and gray level co-occurrence matrix (GLCM) methods. With the multi classes data set, ANN with the hybrid features of the VGG-19 and handcrafted reached a precision of 95.86%, an accuracy of 97.3%, sensitivity of 96.75%, AUC of 99.37%, and specificity of 99.81% with images at magnification factor 400×. Whereas with the binary classes data set, ANN with the hybrid features of the VGG-19 and handcrafted reached a precision of 99.74%, an accuracy of 99.7%, sensitivity of 100%, AUC of 99.85%, and specificity of 100% with images at a magnification factor 400×.

BRMI-Net: Deep Learning Features and Flower Pollination-Controlled Regula Falsi-Based Feature Selection Framework for Breast Cancer Recognition in Mammography Images

The early detection of breast cancer using mammogram images is critical for lowering women's mortality rates and allowing for proper treatment. Deep learning techniques are commonly used for feature extraction and have demonstrated significant performance in the literature. However, these features do not perform well in several cases due to redundant and irrelevant information. We created a new framework for diagnosing breast cancer using entropy-controlled deep learning and flower pollination optimization from the mammogram images. In the proposed framework, a filter fusion-based method for contrast enhancement is developed. The pre-trained ResNet-50 model is then improved and trained using transfer learning on both the original and enhanced datasets. Deep features are extracted and combined into a single vector in the following phase using a serial technique known as serial mid-value features. The top features are then classified using neural networks and machine learning classifiers in the following stage. To accomplish this, a technique for flower pollination optimization with entropy control has been developed. The exercise used three publicly available datasets: CBIS-DDSM, INbreast, and MIAS. On these selected datasets, the proposed framework achieved 93.8, 99.5, and 99.8% accuracy, respectively. Compared to the current methods, the increase in accuracy and decrease in computational time are explained.

Qureshi A, Li J, Mahmood T. On the Analyses of Medical Images Using Traditional Machine Learning Techniques and Convolutional Neural Networks

Convolutional neural network (CNN) has shown dissuasive accomplishment on different areas especially Object Detection, Segmentation, Reconstruction (2D and 3D), Information Retrieval, Medical Image Registration, Multi-lingual translation, Local language Processing, Anomaly Detection on video and Speech Recognition. CNN is a special type of Neural Network, which has compelling and effective learning ability to learn features at several steps during augmentation of the data. Recently, different interesting and inspiring ideas of Deep Learning (DL) such as different activation functions, hyperparameter optimization, regularization, momentum and loss functions has improved the performance, operation and execution of CNN Different internal architecture innovation of CNN and different representational style of CNN has significantly improved the performance. This survey focuses on internal taxonomy of deep learning, different models of vonvolutional neural network, especially depth and width of models and in addition CNN components, applications and current challenges of deep learning.

BC2NetRF: Breast Cancer Classification from Mammogram Images Using Enhanced Deep Learning Features and Equilibrium-Jaya Controlled Regula Falsi-Based Features Selection

One of the most frequent cancers in women is breast cancer, and in the year 2022, approximately 287,850 new cases have been diagnosed. From them, 43,250 women died from this cancer. An early diagnosis of this cancer can help to overcome the mortality rate. However, the manual diagnosis of this cancer using mammogram images is not an easy process and always requires an expert person. Several AI-based techniques have been suggested in the literature. However, still, they are facing several challenges, such as similarities between cancer and non-cancer regions, irrelevant feature extraction, and weak training models. In this work, we proposed a new automated computerized framework for breast cancer classification. The proposed framework improves the contrast using a novel enhancement technique called haze-reduced local-global. The enhanced images are later employed for the dataset augmentation. This step aimed at increasing the diversity of the dataset and improving the training capability of the selected deep learning model. After that, a pre-trained model named EfficientNet-b0 was employed and fine-tuned to add a few new layers. The fine-tuned model was trained separately on original and enhanced images using deep transfer learning concepts with static hyperparameters’ initialization. Deep features were extracted from the average pooling layer in the next step and fused using a new serial-based approach. The fused features were later optimized using a feature selection algorithm known as Equilibrium-Jaya controlled Regula Falsi. The Regula Falsi was employed as a termination function in this algorithm. The selected features were finally classified using several machine learning classifiers. The experimental process was conducted on two publicly available datasets—CBIS-DDSM and INbreast. For these datasets, the achieved average accuracy is 95.4% and 99.7%. A comparison with state-of-the-art (SOTA) technology shows that the obtained proposed framework improved the accuracy. Moreover, the confidence interval-based analysis shows consistent results of the proposed framework.

Genetic hyperparameter optimization with Modified Scalable-Neighbourhood Component Analysis for breast cancer prognostication

Breast cancer is common among women resulting in mortality when left untreated. Early detection is vital so that suitable treatment could assist cancer from spreading further and save people's life. The traditional way of detection is a time-consuming process. With the evolvement of DM (Data Mining), the healthcare industry could be benefitted in predicting the disease as it permits the physicians to determine the significant attributes for diagnosis. Though, conventional techniques have used DM-based methods to identify breast cancer, they lacked in terms of prediction rate. Moreover, parametric-Softmax classifiers have been a general option by conventional works with fixed classes, particularly when huge labelled data are present during training. Nevertheless, this turns into an issue for open set cases where new classes are encountered along with few instances to learn a generalized parametric classifier. Thus, the present study aims to implement a non-parametric strategy by optimizing the embedding of a feature rather than parametric classifiers. This research utilizes Deep CNN (Deep Convolutional Neural Network) and Inception V3 for learning visual features which preserve neighbourhood outline in semantic space relying on NCA (Neighbourhood Component Analysis) criteria. Delimited by its bottleneck, the study proposes MS-NCA (Modified Scalable-Neighbourhood Component Analysis) that relies on a non-linear objective function to perform feature fusion by optimizing the distance-learning objective due to which it gains the capability of computing inner feature products without performing mapping which increases the scalability of MS-NCA. Finally, G-HPO (Genetic-Hyper-parameter Optimization) is proposed. In this case, the new stage in the algorithm simply denotes the enhancement in the length of chromosome bringing several hyperparameters into subsequent XGBoost, NB and RF models having numerous layers for identifying the normal and affected cases of breast cancer for which optimized hyper-parameter values of RF (Random Forest), NB (Naïve Bayes), and XGBoost (eXtreme Gradient Boosting) are determined. This process helps in improvising the classification rate which is confirmed through analytical results.

An Efficient USE-Net Deep Learning Model for Cancer Detection

Breast cancer (BrCa) is the most common disease in women worldwide. Classifying the BrCa image is extremely important for finding BrCa at an earlier stage and monitoring BrCa during treatment. The computer-aided detection methods have been used to interpret BrCa and improve the detection of BrCa during the screening and treatment stages. However, if a new BrCa image is generated for the treatment, it will not classify correctly. The main objective of this research is to classify the BrCa images for newly generated images. The model performs preprocessing, segmentation, feature extraction, and classification. In preprocessing, a hybrid median filtering (HMF) is used to eliminate the noise in the images. The contrast of the images is enhanced using quadrant dynamic histogram equalization (QDHE). Then, ROI segmentation is performed using the USE-Net deep learning model. The CaffeNet model is used for feature extraction on the segmented images, and finally, classification is made using the improved random forest (IRF) with extreme gradient boosting (XGB). The model obtained 97.87% accuracy, 98.45% sensitivity, 95.24% specificity, 98.96% precision, and 98.70% f1-score for ultrasound images. The model gives 98.31% accuracy, 99.29% sensitivity, 90.20% specificity, 98.82% precision, and 99.05% f1-score for mammogram images.

Automatic Intelligent System Using Medical of Things for Multiple Sclerosis Detection

Malfunctions in the immune system cause multiple sclerosis (MS), which initiates mild to severe nerve damage. MS will disturb the signal communication between the brain and other body parts, and early diagnosis will help reduce the harshness of MS in humankind. Magnetic resonance imaging (MRI) supported MS detection is a standard clinical procedure in which the bio-image recorded with a chosen modality is considered to assess the severity of the disease. The proposed research aims to implement a convolutional neural network (CNN) supported scheme to detect MS lesions in the chosen brain MRI slices. The stages of this framework include (i) image collection and resizing, (ii) deep feature mining, (iii) hand-crafted feature mining, (iii) feature optimization with firefly algorithm, and (iv) serial feature integration and classification. In this work, five-fold cross-validation is executed, and the final result is considered for the assessment. The brain MRI slices with/without the skull section are examined separately, presenting the attained results. The experimental outcome of this study confirms that the VGG16 with random forest (RF) classifier offered a classification accuracy of >98% MRI with skull, and VGG16 with K-nearest neighbor (KNN) provided an accuracy of >98% without the skull.

Convolutional neural network based on automatic segmentation of peritumoral shear-wave elastography images for predicting breast cancer

Objective

Our aim was to develop dual-modal CNN models based on combining conventional ultrasound (US) images and shear-wave elastography (SWE) of peritumoral region to improve prediction of breast cancer.

Method

We retrospectively collected US images and SWE data of 1271 ACR- BIRADS 4 breast lesions from 1116 female patients (mean age ± standard deviation, 45.40 ± 9.65 years). The lesions were divided into three subgroups based on the maximum diameter (MD): ≤15 mm; >15 mm and ≤25 mm; >25 mm. We recorded lesion stiffness (SWV1) and 5-point average stiffness of the peritumoral tissue (SWV5). The CNN models were built based on the segmentation of different widths of peritumoral tissue (0.5 mm, 1.0 mm, 1.5 mm, 2.0 mm) and internal SWE image of the lesions. All single-parameter CNN models, dual-modal CNN models, and quantitative SWE parameters in the training cohort (971 lesions) and the validation cohort (300 lesions) were assessed by receiver operating characteristic (ROC) curve.

Results

The US + 1.0 mm SWE model achieved the highest area under the ROC curve (AUC) in the subgroup of lesions with MD ≤15 mm in both the training (0.94) and the validation cohorts (0.91). In the subgroups with MD between15 and 25 mm and above 25 mm, the US + 2.0 mm SWE model achieved the highest AUCs in both the training cohort (0.96 and 0.95, respectively) and the validation cohort (0.93 and 0.91, respectively).

Conclusion

The dual-modal CNN models based on the combination of US and peritumoral region SWE images allow accurate prediction of breast cancer.

Using Deep Learning with Bayesian–Gaussian Inspired Convolutional Neural Architectural Search for Cancer Recognition and Classification from Histopathological Image Frames

We propose a neural architectural search model which examines histopathological images to detect the presence of cancer in both lung and colon tissues. In recent times, deep artificial neural networks have made tremendous impacts in healthcare. However, obtaining an optimal artificial neural network model that could yield excellent performance during training, evaluation, and inferencing has been a bottleneck for researchers. Our method uses a Bayesian convolutional neural architectural search algorithm in collaboration with Gaussian processes to provide an efficient neural network architecture for efficient colon and lung cancer classification and recognition. The proposed model learns by using the Gaussian process to estimate the required optimal architectural values by choosing a set of model parameters through the exploitation of the expected improvement (EI) values, thereby minimizing the number of sampled trials and suggesting the best model architecture. Several experiments were conducted, and a landmark performance was obtained in both validation and test data through the evaluation of the proposed model on a dataset consisting of 25,000 images of five different classes with convergence and F1-score matrices.

Degree of Accuracy With Which Deep Learning for Ultrasound Images Identifies Osteochondritis Dissecans of the Humeral Capitellum

BACKGROUND

Medical screening using ultrasonography (US) has been performed on young baseball players for early detection of osteochondritis dissecans (OCD) of the humeral capitellum. Deep learning (DL) and artificial intelligence (AI) techniques are widely adopted in the medical imaging research field.

PURPOSE/HYPOTHESIS

The purpose of this study was to calculate the diagnostic accuracy using DL for US images of OCD. We hypothesized that using DL for US imaging would improve the prediction accuracy of OCD.

STUDY DESIGN

Cohort study (Diagnosis); Level of evidence, 2.

METHODS

A total of 40 elbows (mean age of patients, 12.1 years) that were suspected of having OCD at a medical checkup and later confirmed by radiographs and magnetic resonance imaging were included in the study. The affected elbows were used as the OCD group and the contralateral elbows as the control group. From US videos, 100 images per elbow were captured from different angles, and 4000 images of the elbows were prepared for both groups. Of these, 80% were randomly selected by DL models and used as training data; the remaining were used as test data. Transfer learning was conducted using 3 pretrained DL models. The confusion matrix and the area under the receiver operating characteristic curve (AUC) were used to evaluate the model, and the visualization of the areas deemed important by the DL models was also performed. Furthermore, OCD regions were detected using an automatic image recognition model based on DL.

RESULTS

Classification of the OCD image by the DL model was performed; the best accuracy score was 0.87; the recall was 1.00. AUC was high for all DL models. Visualization of important features showed that AI predicted the presence of OCD by focusing on the irregularity or discontinuity of the surface of subchondral bone. In the detection of OCD task, the mean average precision was 0.83.

CONCLUSION

The DL on US images identified OCD with high accuracy. The important features detected by the DL models correspond to the areas used by clinicians in screening the US images. The OCD was also detected with high accuracy using the object detection model. The AI model may be used in medical screening for OCD.

Overview of Artificial Intelligence in Breast Cancer Medical Imaging

The heavy global burden and mortality of breast cancer emphasize the importance of early diagnosis and treatment. Imaging detection is one of the main tools used in clinical practice for screening, diagnosis, and treatment efficacy evaluation, and can visualize changes in tumor size and texture before and after treatment. The overwhelming number of images, which lead to a heavy workload for radiologists and a sluggish reporting period, suggests the need for computer-aid detection techniques and platform. In addition, complex and changeable image features, heterogeneous quality of images, and inconsistent interpretation by different radiologists and medical institutions constitute the primary difficulties in breast cancer screening and imaging diagnosis. The advancement of imaging-based artificial intelligence (AI)-assisted tumor diagnosis is an ideal strategy for improving imaging diagnosis efficient and accuracy. By learning from image data input and constructing algorithm models, AI is able to recognize, segment, and diagnose tumor lesion automatically, showing promising application prospects. Furthermore, the rapid advancement of "omics" promotes a deeper and more comprehensive recognition of the nature of cancer. The fascinating relationship between tumor image and molecular characteristics has attracted attention to the radiomic and radiogenomics, which allow us to perform analysis and detection on the molecular level with no need for invasive operations. In this review, we integrate the current developments in AI-assisted imaging diagnosis and discuss the advances of AI-based breast cancer precise diagnosis from a clinical point of view. Although AI-assisted imaging breast cancer screening and detection is an emerging field and draws much attention, the clinical application of AI in tumor lesion recognition, segmentation, and diagnosis is still limited to research or in limited patients' cohort. Randomized clinical trials based on large and high-quality cohort are lacking. This review aims to describe the progress of the imaging-based AI application in breast cancer screening and diagnosis for clinicians.

Efficient Breast Cancer Diagnosis from Complex Mammographic Images Using Deep Convolutional Neural Network

Medical image analysis places a significant focus on breast cancer, which poses a significant threat to women's health and contributes to many fatalities. An early and precise diagnosis of breast cancer through digital mammograms can significantly improve the accuracy of disease detection. Computer-aided diagnosis (CAD) systems must analyze the medical imagery and perform detection, segmentation, and classification processes to assist radiologists with accurately detecting breast lesions. However, early-stage mammography cancer detection is certainly difficult. The deep convolutional neural network has demonstrated exceptional results and is considered a highly effective tool in the field. This study proposes a computational framework for diagnosing breast cancer using a ResNet-50 convolutional neural network to classify mammogram images. To train and classify the INbreast dataset into benign or malignant categories, the framework utilizes transfer learning from the pretrained ResNet-50 CNN on ImageNet. The results revealed that the proposed framework achieved an outstanding classification accuracy of 93%, surpassing other models trained on the same dataset. This novel approach facilitates early diagnosis and classification of malignant and benign breast cancer, potentially saving lives and resources. These outcomes highlight that deep convolutional neural network algorithms can be trained to achieve highly accurate results in various mammograms, along with the capacity to enhance medical tools by reducing the error rate in screening mammograms.

Breast Cancer Classification by Using Multi-Headed Convolutional Neural Network Modeling

Breast cancer is one of the most widely recognized diseases after skin cancer. Though it can occur in all kinds of people, it is undeniably more common in women. Several analytical techniques, such as Breast MRI, X-ray, Thermography, Mammograms, Ultrasound, etc., are utilized to identify it. In this study, artificial intelligence was used to rapidly detect breast cancer by analyzing ultrasound images from the Breast Ultrasound Images Dataset (BUSI), which consists of three categories: Benign, Malignant, and Normal. The relevant dataset comprises grayscale and masked ultrasound images of diagnosed patients. Validation tests were accomplished for quantitative outcomes utilizing the exhibition measures for each procedure. The proposed framework is discovered to be effective, substantiating outcomes with only raw image evaluation giving a 78.97% test accuracy and masked image evaluation giving 81.02% test precision, which could decrease human errors in the determination cycle. Additionally, our described framework accomplishes higher accuracy after using multi-headed CNN with two processed datasets based on masked and original images, where the accuracy hopped up to 92.31% (±2) with a Mean Squared Error (MSE) loss of 0.05. This work primarily contributes to identifying the usefulness of multi-headed CNN when working with two different types of data inputs. Finally, a web interface has been made to make this model usable for non-technical personals.

Research on Classification Method of Medical Ultrasound Image Processing Based on Neural Network

In clinical applications, the classification of ultrasound images needs to be processed as an aid to diagnosis. Based on this, a hybrid model of cascaded deep convolutional neural network consisting of two different CNNs and a new classification method are designed and evaluated for its feasibility and effectiveness in ultrasound image classification. A total of 1000 pathological slides of patients with thyroid nodular lesions kept in the Department of Pathology of the First Affiliated Hospital of Lanzhou University, China, were retrospectively collected. After image acquisition, the images were randomly divided into training set, validation set, and test set in the ratio of 4 : 3 : 3. Three convolutional neural network models (VGG 19 model, Inception V3 model, and DenseNet 161 model) with pretraining parameters acquired on the training set were trained, and the models were combined to construct an integrated learning model, and the performance of the models in recognizing pathological images was evaluated based on the test set data. The experimental results show that the VGG 19 model is less effective in classification, with a correct rate of 88.20%, which is lower than that of Inception V3 and DenseNet161 models (92.87% and 92.95%). InceptionV3 and DenseNet161 models have significant advantages in terms of accuracy, number of parameters, and training efficiency, where the DenseNet161 model has faster convergence and better generalization performance, but occupies more video memory in the operation; moreover, the DenseNet161 operation time (1986.48 s) and response time (16 s) are better than the other two models. In addition, the integrated learning of InceptionV3 and DenseNet161 can improve the recognition of pathological images by a single model. Compared with other methods, the performance of the cascaded CNNs proposed in this study is significantly improved, and the multiview strategy can improve the performance of cascaded CNNs. The experimental results demonstrate the potential clinical application of cascaded CNNs, which can provide physicians with an objective second opinion and reduce their heavy workload, in addition to making the diagnosis of thyroid nodules easy and reproducible for people without medical expertise.

Applying Deep Learning for Breast Cancer Detection in Radiology

Recent advances in deep learning have enhanced medical imaging research. Breast cancer is the most prevalent cancer among women, and many applications have been developed to improve its early detection. The purpose of this review is to examine how various deep learning methods can be applied to breast cancer screening workflows. We summarize deep learning methods, data availability and different screening methods for breast cancer including mammography, thermography, ultrasound and magnetic resonance imaging. In this review, we will explore deep learning in diagnostic breast imaging and describe the literature review. As a conclusion, we discuss some of the limitations and opportunities of integrating artificial intelligence into breast cancer clinical practice.

Critical element prediction of tracheal intubation difficulty: Automatic Mallampati classification by jointly using handcrafted and attention-based deep features

Preoperative assessment of the difficulty of tracheal intubation is of great importance in anesthesia practice because failed intubation can lead to severe complications and even death. The Mallampati score is widely used as a critical assessment criterion in combination with other measures to assess the difficulty of tracheal intubation. The performance of existing methods for Mallampati classification with artificial intelligence (AI) is unreliable to the extent that the current clinical judgment of the Mallampati score relies entirely on doctors' experience. In this paper, we propose a new method for automatic Mallampati classification. Our method extracts deep features that are more favorable for the Mallampati classification task by introducing an attention mechanism into the basic deep convolutional neural network (DCNN) and then further improves the classification performance by jointly using attention-based deep features with handcrafted features. We conducted experiments on a dataset consisting of 321 oral images collected online. The proposed method has a classification accuracy of 97.50%, a sensitivity of 96.52%, a specificity of 98.05%, and an F1 score of 96.52% after five-fold cross-validation. The experimental results show that our proposed method is superior to other methods, can assist doctors in determining Mallampati class objectively and accurately, and provide an essential reference element for assessing the difficulty of tracheal intubation.

A Novel CNN pooling layer for breast cancer segmentation and classification from thermograms

Breast cancer is the second most frequent cancer worldwide, following lung cancer and the fifth leading cause of cancer death and a major cause of cancer death among women. In recent years, convolutional neural networks (CNNs) have been successfully applied for the diagnosis of breast cancer using different imaging modalities. Pooling is a main data processing step in CNN that decreases the feature maps’ dimensionality without losing major patterns. However, the effect of pooling layer was not studied efficiently in literature. In this paper, we propose a novel design for the pooling layer called vector pooling block (VPB) for the CCN algorithm. The proposed VPB consists of two data pathways, which focus on extracting features along horizontal and vertical orientations. The VPB makes the CNNs able to collect both global and local features by including long and narrow pooling kernels, which is different from the traditional pooling layer, that gathers features from a fixed square kernel. Based on the novel VPB, we proposed a new pooling module called AVG-MAX VPB. It can collect informative features by using two types of pooling techniques, maximum and average pooling. The VPB and the AVG-MAX VPB are plugged into the backbone CNNs networks, such as U-Net, AlexNet, ResNet18 and GoogleNet, to show the advantages in segmentation and classification tasks associated with breast cancer diagnosis from thermograms. The proposed pooling layer was evaluated using a benchmark thermogram database (DMR-IR) and its results compared with U-Net results which was used as base results. The U-Net results were as follows: global accuracy = 96.6%, mean accuracy = 96.5%, mean IoU = 92.07%, and mean BF score = 78.34%. The VBP-based results were as follows: global accuracy = 98.3%, mean accuracy = 97.9%, mean IoU = 95.87%, and mean BF score = 88.68% while the AVG-MAX VPB-based results were as follows: global accuracy = 99.2%, mean accuracy = 98.97%, mean IoU = 98.03%, and mean BF score = 94.29%. Other network architectures also demonstrate superior improvement considering the use of VPB and AVG-MAX VPB.

A Comprehensive Review on Smart Health Care: Applications, Paradigms, and Challenges with Case Studies

Growth and advancement of the Deep Learning (DL) and the Internet of Things (IoT) are figuring out their way over the modern contemporary world through integrating various technologies in distinct fields viz, agriculture, manufacturing, energy, transportation, supply chains, cities, healthcare, and so on. Researchers had identified the feasibility of integrating deep learning, cloud, and IoT to enhance the overall automation, where IoT may prolong its application area through utilizing cloud services and the cloud can even prolong its applications through data acquired by IoT devices like sensors and deep learning for disease detection and diagnosis. This study explains a summary of various techniques utilized in smart healthcare, i.e., deep learning, cloud-based-IoT applications in smart healthcare, fog computing in smart healthcare, and challenges and issues faced by smart healthcare and it presents a wider scope as it is not intended for a particular application such aspatient monitoring, disease detection, and diagnosing and the technologies used for developing this smart systems are outlined. Smart health bestows the quality of life. Convenient and comfortable living is made possible by the services provided by smart healthcare systems (SHSs). Since healthcare is a massive area with enormous data and a broad spectrum of diseases associated with different organs, immense research can be done to overcome the drawbacks of traditional healthcare methods. Deep learning with IoT can effectively be applied in the healthcare sector to automate the diagnosing and treatment process even in rural areas remotely. Applications may include disease prevention and diagnosis, fitness and patient monitoring, food monitoring, mobile health, telemedicine, emergency systems, assisted living, self-management of chronic diseases, and so on.

Predicting Breast Cancer Leveraging Supervised Machine Learning Techniques

Breast cancer is one of the leading causes of increasing deaths in women worldwide. The complex nature (microcalcification and masses) of breast cancer cells makes it quite difficult for radiologists to diagnose it properly. Subsequently, various computer-aided diagnosis (CAD) systems have previously been developed and are being used to aid radiologists in the diagnosis of cancer cells. However, due to intrinsic risks associated with the delayed and/or incorrect diagnosis, it is indispensable to improve the developed diagnostic systems. In this regard, machine learning has recently been playing a potential role in the early and precise detection of breast cancer. This paper presents a new machine learning-based framework that utilizes the Random Forest, Gradient Boosting, Support Vector Machine, Artificial Neural Network, and Multilayer Perception approaches to efficiently predict breast cancer from the patient data. For this purpose, the Wisconsin Diagnostic Breast Cancer (WDBC) dataset has been utilized and classified using a hybrid Multilayer Perceptron Model (MLP) and 5-fold cross-validation framework as a working prototype. For the improved classification, a connection-based feature selection technique has been used that also eliminates the recursive features. The proposed framework has been validated on two separate datasets, i.e., the Wisconsin Prognostic dataset (WPBC) and Wisconsin Original Breast Cancer (WOBC) datasets. The results demonstrate improved accuracy of 99.12% due to efficient data preprocessing and feature selection applied to the input data.

Classification of breast cancer using a manta-ray foraging optimized transfer learning framework

Due to its high prevalence and wide dissemination, breast cancer is a particularly dangerous disease. Breast cancer survival chances can be improved by early detection and diagnosis. For medical image analyzers, diagnosing is tough, time-consuming, routine, and repetitive. Medical image analysis could be a useful method for detecting such a disease. Recently, artificial intelligence technology has been utilized to help radiologists identify breast cancer more rapidly and reliably. Convolutional neural networks, among other technologies, are promising medical image recognition and classification tools. This study proposes a framework for automatic and reliable breast cancer classification based on histological and ultrasound data. The system is built on CNN and employs transfer learning technology and metaheuristic optimization. The Manta Ray Foraging Optimization (MRFO) approach is deployed to improve the framework's adaptability. Using the Breast Cancer Dataset (two classes) and the Breast Ultrasound Dataset (three-classes), eight modern pre-trained CNN architectures are examined to apply the transfer learning technique. The framework uses MRFO to improve the performance of CNN architectures by optimizing their hyperparameters. Extensive experiments have recorded performance parameters, including accuracy, AUC, precision, F1-score, sensitivity, dice, recall, IoU, and cosine similarity. The proposed framework scored 97.73% on histopathological data and 99.01% on ultrasound data in terms of accuracy. The experimental results show that the proposed framework is superior to other state-of-the-art approaches in the literature review.

Diagnostic Strategies for Breast Cancer Detection: From Image Generation to Classification Strategies Using Artificial Intelligence Algorithms

Breast cancer is one the main death causes for women worldwide, as 16% of the diagnosed malignant lesions worldwide are its consequence. In this sense, it is of paramount importance to diagnose these lesions in the earliest stage possible, in order to have the highest chances of survival. While there are several works that present selected topics in this area, none of them present a complete panorama, that is, from the image generation to its interpretation. This work presents a comprehensive state-of-the-art review of the image generation and processing techniques to detect Breast Cancer, where potential candidates for the image generation and processing are presented and discussed. Novel methodologies should consider the adroit integration of artificial intelligence-concepts and the categorical data to generate modern alternatives that can have the accuracy, precision and reliability expected to mitigate the misclassifications.

Joint deep learning for batch effect removal and classification toward MALDI MS based metabolomics

Background

Metabolomics is a primary omics topic, which occupies an important position in both clinical applications and basic researches for metabolic signatures and biomarkers. Unfortunately, the relevant studies are challenged by the batch effect caused by many external factors. In last decade, the technique of deep learning has become a dominant tool in data science, such that one may train a diagnosis network from a known batch and then generalize it to a new batch. However, the batch effect inevitably hinders such efforts, as the two batches under consideration can be highly mismatched.

Results

We propose an end-to-end deep learning framework, for joint batch effect removal and then classification upon metabolomics data. We firstly validate the proposed deep learning framework on a public CyTOF dataset as a simulated experiment. We also visually compare the t-SNE distribution and demonstrate that our method effectively removes the batch effects in latent space. Then, for a private MALDI MS dataset, we have achieved the highest diagnostic accuracy, with about 5.1 ~ 7.9% increase on average over state-of-the-art methods.

Conclusions

Both experiments conclude that our method performs significantly better in classification than conventional methods benefitting from the effective removal of batch effect.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12859-022-04758-z.

Framework to Segment and Evaluate Multiple Sclerosis Lesion in MRI Slices Using VGG-UNet

Multiple sclerosis (MS) is an autoimmune disease that causes mild to severe issues in the central nervous system (CNS). Early detection and treatment are necessary to reduce the harshness of the disease in individuals. The proposed work aims to implement a convolutional neural network (CNN) segmentation scheme to extract the MS lesion in a 2D brain MRI slice. To achieve a better MS detection, this work implemented the VGG-UNet scheme in which the pretrained VGG19 is considered as the encoder section. This scheme is tested on 30 patient images (600 images with dimension 512 × 512 × 3 pixels), and the experimental outcome confirms that this scheme provides a better result compared to traditional UNet, SegNet, VGG-UNet, and VGG-SegNet. The experimental investigation implemented on axial, coronal and sagittal plane 2D slices of Flair modality confirms that this work provides a better value of Jaccard (>85%), Dice (>92%), and accuracy (>98%).

Deep Transfer Learning-Based Breast Cancer Detection and Classification Model Using Photoacoustic Multimodal Images

The rapid development of technologies in biomedical research has enriched and broadened the range of medical equipment. Magnetic resonance imaging, ultrasonic imaging, and optical imaging have been discovered by diverse research communities to design multimodal systems, which is essential for biomedical applications. One of the important tools is photoacoustic multimodal imaging (PAMI) which combines the concepts of optics and ultrasonic systems. At the same time, earlier detection of breast cancer becomes essential to reduce mortality. The recent advancements of deep learning (DL) models enable detection and classification the breast cancer using biomedical images. This article introduces a novel social engineering optimization with deep transfer learning-based breast cancer detection and classification (SEODTL-BDC) model using PAI. The intention of the SEODTL-BDC technique is to detect and categorize the presence of breast cancer using ultrasound images. Primarily, bilateral filtering (BF) is applied as an image preprocessing technique to remove noise. Besides, a lightweight LEDNet model is employed for the segmentation of biomedical images. In addition, residual network (ResNet-18) model can be utilized as a feature extractor. Finally, SEO with recurrent neural network (RNN) model, named SEO-RNN classifier, is applied to allot proper class labels to the biomedical images. The performance validation of the SEODTL-BDC technique is carried out using benchmark dataset and the experimental outcomes pointed out the supremacy of the SEODTL-BDC approach over the existing methods.