Semi-supervised GAN-based Radiomics Model for Data Augmentation in Breast Ultrasound Mass Classification

Towards robust multimodal ultrasound classification for liver tumor diagnosis: A generative approach to modality missingness

BACKGROUND AND OBJECTIVE

In medical image analysis, combining multiple imaging modalities enhances diagnostic accuracy by providing complementary information. However, missing modalities are common in clinical settings, limiting the effectiveness of multimodal models. This study addresses the challenge of missing modalities in liver tumor diagnosis by proposing a generative model-based method for cross-modality reconstruction and classification. The dataset for this study comprises 359 case data from a hospital, with each case including three modality data: B-mode ultrasound images, Color Doppler Flow Imaging (CDFI), and clinical data. Only cases with one missing image modality are considered, excluding those with missing clinical data.

METHODS

We developed a multimodal classification framework specifically for liver tumor diagnosis, employing various feature extraction networks to explore the impact of different modality combinations on classification performance when only available modalities are used. DenseNet extracts CDFI features, while EfficientNet is employed for B-mode ultrasound image feature extraction. These features are then flattened and concatenated with clinical data using feature-level fusion to obtain a full-modality model. Modality weight parameters are introduced to emphasize the importance of different modalities, yielding Model_D, which serves as the classification model after subsequent image modality supplementation. In cases of missing modalities, generative models, including U-GAT-IT and MSA-GAN, are utilized for cross-modal reconstruction of missing B-mode ultrasound or CDFI images (e.g., reconstructing CDFI from B-mode ultrasound when CDFI is missing). After evaluating the usability of the generated images, they are input into Model_D as supplementary images for the missing modalities.

RESULTS

Model performance and modality supplementation effects were evaluated through accuracy, precision, recall, F1 score, and AUC metrics. The results demonstrate that the proposed Model_D, which introduces modality weights, achieves an accuracy of 88.57 %, precision of 87.97 %, recall of 82.32 %, F1 score of 0.87, and AUC of 0.95 in the full-modality classification task for liver tumors. Moreover, images reconstructed using U-GAT-IT and MSA-GAN across modalities exhibit PSNR > 20 and multi-scale structural similarity > 0.7, indicating moderate image quality with well-preserved overall structures, suitable for input into the model as supplementary images in cases of missing modalities. The supplementary CDFI or B-mode ultrasound images achieve 87.10 % and 86.43 % accuracy, respectively, with AUC values of 0.92 and 0.95. This proves that even in the absence of certain modalities, the generative models can effectively reconstruct missing images, maintaining high classification performance comparable to that in complete modality scenarios.

CONCLUSIONS

The generative model-based approach for modality reconstruction significantly improves the robustness of multimodal classification models, particularly in the context of liver tumor diagnosis. This method enhances the clinical applicability of multimodal models by ensuring high diagnostic accuracy despite missing modalities. Future work will explore further improvements in modality reconstruction techniques to increase the generalization and reliability of the model in various clinical settings.

A radiomics approach to distinguish Progressive Supranuclear Palsy Richardson's syndrome from other phenotypes starting from MR images

BACKGROUND AND OBJECTIVE

Progressive Supranuclear Palsy (PSP) is an uncommon neurodegenerative disorder with different clinical onset, including Richardson's syndrome (PSP-RS) and other variant phenotypes (vPSP). Recognising the clinical progression of different phenotypes would enhance the accuracy of detection and treatment of PSP. The study goal was to identify radiomic biomarkers for distinguishing PSP phenotypes extracted from T1-weighted magnetic resonance images (MRI).

METHODS

Forty PSP patients (20 PSP-RS and 20 vPSP) took part in the present work. Radiomic features were collected from 21 regions of interest (ROIs) mainly from frontal cortex, supratentorial white matter, basal nuclei, brainstem, cerebellum, 3rd and 4th ventricles. After features selection, three tree-based machine learning (ML) classifiers were implemented to classify PSP phenotypes.

RESULTS

10 out of 21 ROIs performed best about sensitivity, specificity, accuracy and area under the receiver operating characteristic curve (AUCROC). Particularly, features extracted from the pons region obtained the best accuracy (0.92) and AUCROC (0.83) values while by using the other 10 ROIs, evaluation metrics range from 0.67 to 0.83. Eight features of the Gray Level Dependence Matrix were recurrently extracted for the 10 ROIs. Furthermore, by combining these ROIs, the results exceeded 0.83 in phenotypes classification and the selected areas were brain stem, pons, occipital white matter, precentral gyrus and thalamus regions.

CONCLUSIONS

Based on the achieved results, our proposed approach could represent a promising tool for distinguishing PSP-RS from vPSP.

Generative AI for synthetic data across multiple medical modalities: A systematic review of recent developments and challenges

This paper presents a comprehensive systematic review of generative models (GANs, VAEs, DMs, and LLMs) used to synthesize various medical data types, including imaging (dermoscopic, mammographic, ultrasound, CT, MRI, and X-ray), text, time-series, and tabular data (EHR). Unlike previous narrowly focused reviews, our study encompasses a broad array of medical data modalities and explores various generative models. Our aim is to offer insights into their current and future applications in medical research, particularly in the context of synthesis applications, generation techniques, and evaluation methods, as well as providing a GitHub repository as a dynamic resource for ongoing collaboration and innovation. Our search strategy queries databases such as Scopus, PubMed, and ArXiv, focusing on recent works from January 2021 to November 2023, excluding reviews and perspectives. This period emphasizes recent advancements beyond GANs, which have been extensively covered in previous reviews. The survey also emphasizes the aspect of conditional generation, which is not focused on in similar work. Key contributions include a broad, multi-modality scope that identifies cross-modality insights and opportunities unavailable in single-modality surveys. While core generative techniques are transferable, we find that synthesis methods often lack sufficient integration of patient-specific context, clinical knowledge, and modality-specific requirements tailored to the unique characteristics of medical data. Conditional models leveraging textual conditioning and multimodal synthesis remain underexplored but offer promising directions for innovation. Our findings are structured around three themes: (1) Synthesis applications, highlighting clinically valid synthesis applications and significant gaps in using synthetic data beyond augmentation, such as for validation and evaluation; (2) Generation techniques, identifying gaps in personalization and cross-modality innovation; and (3) Evaluation methods, revealing the absence of standardized benchmarks, the need for large-scale validation, and the importance of privacy-aware, clinically relevant evaluation frameworks. These findings emphasize the need for benchmarking and comparative studies to promote openness and collaboration.

Synthetic Lung Ultrasound Data Generation Using Autoencoder With Generative Adversarial Network

Class imbalance is a significant challenge in medical image analysis, particularly in lung ultrasound (LUS), where severe patterns are often underrepresented. Traditional oversampling techniques, which simply duplicate original data, have limited effectiveness in addressing this issue. To overcome these limitations, this study introduces a novel supervised autoencoder generative adversarial network (SA-GAN) for data augmentation, leveraging advanced generative artificial intelligence (AI) to create high-quality synthetic samples for minority classes. In addition, the traditional data augmentation technique is used for comparison. The SA-GAN incorporates an autoencoder to develop a conditional latent space, effectively addressing weight clipping issues and ensuring higher quality synthetic data. The generated samples are evaluated using similarity metrics and expert analysis to validate their utility. Furthermore, state-of-the-art neural networks are used for multiclass classification, and their performance is compared when trained with GAN-based augmentation versus traditional data augmentation techniques. These contributions enhance the robustness and reliability of AI models in mitigating class imbalance in LUS analysis.

A semi-supervised prototypical network for prostate lesion segmentation from multimodality MRI

Objective.Prostate lesion segmentation from multiparametric magnetic resonance images is particularly challenging due to the limited availability of labeled data. This scarcity of annotated images makes it difficult for supervised models to learn the complex features necessary for accurate lesion detection and segmentation.Approach.We proposed a novel semi-supervised algorithm that embeds prototype learning into mean-teacher (MT) training to improve the feature representation for unlabeled data. In this method, pseudo-labels generated by the teacher network simultaneously serve as supervision for unlabeled prototype-based segmentation. By enabling prototype segmentation to operate across labeled and unlabeled data, the network enriches the pool of "lesion representative prototypes", and allows prototypes to flow bidirectionally-from support-to-query and query-to-support paths. This intersected, bidirectional information flow strengthens the model's generalization ability. This approach is distinct from the MT algorithm as it involves few-shot training and differs from prototypical learning for adopting unlabeled data for training.Main results.This study evaluated multiple datasets with 767 patients from three different institutions, including the publicly available PROSTATEx/PROSTATEx2 datasets as the holdout institute for reproducibility. The experimental results showed that the proposed algorithm outperformed state-of-the-art semi-supervised methods with limited labeled data, observing an improvement in Dice similarity coefficient with increasing labeled data, ranging from 0.04 to 0.09.Significance.Our method shows promise in improving segmentation outcomes with limited labeled data and potentially aiding clinicians in making informed patient treatment and management decisions88The algorithm implementation has been made available on GitHub viagit@github.com:yanwenCi/semi-proto-seg.git...

Enhanced breast cancer diagnosis using modified InceptionNet-V3: a deep learning approach for ultrasound image classification

Introduction

Breast cancer (BC) is a malignant neoplasm that originates in the mammary gland's cellular structures and remains one of the most prevalent cancers among women, ranking second in cancer-related mortality after lung cancer. Early and accurate diagnosis is crucial due to the heterogeneous nature of breast cancer and its rapid progression. However, manual detection and classification are often time-consuming and prone to errors, necessitating the development of automated and reliable diagnostic approaches.

Methods

Recent advancements in deep learning have significantly improved medical image analysis, demonstrating superior predictive performance in breast cancer detection using ultrasound images. Despite these advancements, training deep learning models from scratch can be computationally expensive and data-intensive. Transfer learning, leveraging pre-trained models on large-scale datasets, offers an effective solution to mitigate these challenges. In this study, we investigate and compare multiple deep-learning models for breast cancer classification using transfer learning. The evaluated architectures include modified InceptionV3, GoogLeNet, ShuffleNet, AlexNet, VGG-16, and SqueezeNet. Additionally, we propose a deep neural network model that integrates features from modified InceptionV3 to further enhance classification performance.

Results

The experimental results demonstrate that the modified InceptionV3 model achieves the highest classification accuracy of 99.10%, with a recall of 98.90%, precision of 99.00%, and an F1-score of 98.80%, outperforming all other evaluated models on the given datasets.

Discussion

The achieved findings underscore the potential of the proposed approach in enhancing diagnostic precision and confirm the superiority of the modified InceptionV3 model in breast cancer classification tasks.

Progress in the Application of Artificial Intelligence in Ultrasound-Assisted Medical Diagnosis

The integration of artificial intelligence (AI) into ultrasound medicine has revolutionized medical imaging, enhancing diagnostic accuracy and clinical workflows. This review focuses on the applications, challenges, and future directions of AI technologies, particularly machine learning (ML) and its subset, deep learning (DL), in ultrasound diagnostics. By leveraging advanced algorithms such as convolutional neural networks (CNNs), AI has significantly improved image acquisition, quality assessment, and objective disease diagnosis. AI-driven solutions now facilitate automated image analysis, intelligent diagnostic assistance, and medical education, enabling precise lesion detection across various organs while reducing physician workload. AI's error detection capabilities further enhance diagnostic accuracy. Looking ahead, the integration of AI with ultrasound is expected to deepen, promoting trends in standardization, personalized treatment, and intelligent healthcare, particularly in underserved areas. Despite its potential, comprehensive assessments of AI's diagnostic accuracy and ethical implications remain limited, necessitating rigorous evaluations to ensure effectiveness in clinical practice. This review provides a systematic evaluation of AI technologies in ultrasound medicine, highlighting their transformative potential to improve global healthcare outcomes.

Patch-based feature mapping with generative adversarial networks for auxiliary hip fracture detection

BACKGROUND

Hip fractures are a significant public health issue, particularly among the elderly population. Pelvic radiographs (PXRs) play a crucial role in diagnosing hip fractures and are commonly used for their evaluation. Previous research has demonstrated promising performance in classification models for hip fracture detection. However, these models sometimes focus on the images' non-fracture regions, reducing their explainability. This study applies weakly supervised learning techniques to address this issue and improve the model's focus on the fracture region. Additionally, we introduce a method to quantitatively evaluate the model's focus on the region of interest (ROI).

METHODS

We propose a new auxiliary module called the patch-auxiliary generative adversarial network (PAGAN) for weakly supervised learning tasks. PAGAN can be integrated with any state-of-the-art (SOTA) classification model, such as EfficientNetB0, ResNet50, and DenseNet121, to enhance hip fracture detection. This training strategy incorporates global information (the entire PXR image) and local information (the hip region patch) for more effective learning. Furthermore, we employ GradCAM to generate attention heatmaps, highlighting the focus areas within the classification model. The intersection over union (IOU) and dice coefficient (Dise) are then computed between the attention heatmap and the fracture area, enabling a quantitative assessment of the model's explainability.

RESULTS AND CONCLUSIONS

Incorporating PAGAN improved the performance of the classification models. The accuracy of EfficientNetB0 increased from 93.61 % to 95.97 %, ResNet50 improved from 90.66 % to 94.89 %, and DenseNet121 saw an increase from 93.51 % to 94.49 %. Regarding model explainability, the integration of PAGAN into classification models led to a more pronounced attention to ROI. The average IOU improved from 0.32 to 0.54 for EfficientNetB0, from 0.28 to 0.40 for ResNet50, and from 0.37 to 0.51 for DenseNet121. These results indicate that PAGAN improves hip fracture classification performance and substantially enhances the model's focus on the fracture region, thereby increasing its explainability.

Current status and future direction of cancer research using artificial intelligence for clinical application

The expectations for artificial intelligence (AI) technology have increased considerably in recent years, mainly due to the emergence of deep learning. At present, AI technology is being used for various purposes and has brought about change in society. In particular, the rapid development of generative AI technology, exemplified by ChatGPT, has amplified the societal impact of AI. The medical field is no exception, with a wide range of AI technologies being introduced for basic and applied research. Further, AI-equipped software as a medical device (AI-SaMD) is also being approved by regulatory bodies. Combined with the advent of big data, data-driven research utilizing AI is actively pursued. Nevertheless, while AI technology has great potential, it also presents many challenges that require careful consideration. In this review, we introduce the current status of AI-based cancer research, especially from the perspective of clinical application, and discuss the associated challenges and future directions, with the aim of helping to promote cancer research that utilizes effective AI technology.

Deep Learning in Breast Cancer Imaging: A Decade of Progress and Future Directions

Breast cancer has reached the highest incidence rate worldwide among all malignancies since 2020. Breast imaging plays a significant role in early diagnosis and intervention to improve the outcome of breast cancer patients. In the past decade, deep learning has shown remarkable progress in breast cancer imaging analysis, holding great promise in interpreting the rich information and complex context of breast imaging modalities. Considering the rapid improvement in deep learning technology and the increasing severity of breast cancer, it is critical to summarize past progress and identify future challenges to be addressed. This paper provides an extensive review of deep learning-based breast cancer imaging research, covering studies on mammograms, ultrasound, magnetic resonance imaging, and digital pathology images over the past decade. The major deep learning methods and applications on imaging-based screening, diagnosis, treatment response prediction, and prognosis are elaborated and discussed. Drawn from the findings of this survey, we present a comprehensive discussion of the challenges and potential avenues for future research in deep learning-based breast cancer imaging.

MITS-GAN: Safeguarding medical imaging from tampering with generative adversarial networks

The progress in generative models, particularly Generative Adversarial Networks (GANs), opened new possibilities for image generation but raised concerns about potential malicious uses, especially in sensitive areas like medical imaging. This study introduces MITS-GAN, a novel approach to prevent tampering in medical images, with a specific focus on CT scans. The approach disrupts the output of the attacker's CT-GAN architecture by introducing finely tuned perturbations that are imperceptible to the human eye. Specifically, the proposed approach involves the introduction of appropriate Gaussian noise to the input as a protective measure against various attacks. Our method aims to enhance tamper resistance, comparing favorably to existing techniques. Experimental results on a CT scan demonstrate MITS-GAN's superior performance, emphasizing its ability to generate tamper-resistant images with negligible artifacts. As image tampering in medical domains poses life-threatening risks, our proactive approach contributes to the responsible and ethical use of generative models. This work provides a foundation for future research in countering cyber threats in medical imaging. Models and codes are publicly available.1.

An updated overview of radiomics-based artificial intelligence (AI) methods in breast cancer screening and diagnosis

Current imaging methods for diagnosing breast cancer (BC) are associated with limited sensitivity and specificity and modest positive predictive power. The recent progress in image analysis using artificial intelligence (AI) has created great promise to improve BC diagnosis and subtype differentiation. In this case, novel quantitative computational methods, such as radiomics, have been developed to enhance the sensitivity and specificity of early BC diagnosis and classification. The potential of radiomics in improving the diagnostic efficacy of imaging studies has been shown in several studies. In this review article, we discuss the radiomics workflow and current handcrafted radiomics methods in the diagnosis and classification of BC based on the most recent studies on different imaging modalities, e.g., MRI, mammography, contrast-enhanced spectral mammography (CESM), ultrasound imaging, and digital breast tumosynthesis (DBT). We also discuss current challenges and potential strategies to improve the specificity and sensitivity of radiomics in breast cancer to help achieve a higher level of BC classification and diagnosis in the clinical setting. The growing field of AI incorporation with imaging information has opened a great opportunity to provide a higher level of care for BC patients.

Automatic Segmentation of Abdominal Aortic Aneurysms From Time-Resolved 3-D Ultrasound Images Using Deep Learning

Abdominal aortic aneurysms (AAAs) are rupture-prone dilatations of the aorta. In current clinical practice, the maximal diameter of AAAs is monitored with 2-D ultrasound to estimate their rupture risk. Recent studies have shown that 3-D and mechanical AAA parameters might be better predictors for aneurysm growth and rupture than the diameter. These parameters can be obtained with time-resolved 3-D ultrasound (3-D + t US), which requires robust and automatic segmentation of AAAs from 3-D + t US. This study proposes and validates a deep learning (DL) approach for the automatic segmentation of AAAs. A group of 500 patients was included for follow-up 3-D + t US imaging, resulting in 2495 3-D + t US images. Segmentation masks for model training were obtained using a conventional automatic segmentation algorithm (nonDL). Four different DL models were trained and validated by 1) comparison to CT and 2) reader scoring. Performance of the nonDL and different DL segmentation strategies were evaluated by comparing Hausdorff distance, Dice scores, accuracy, sensitivity, and specificity with a sign test. All DL models had higher median Dice scores, accuracy, and sensitivity (all ) compared to nonDL segmentation. The full image-resolution model without data augmentation showed the highest median Dice score and sensitivity ( ). Applying the DL model on an independent test group produced fewer poor segmentation scores of 1 to 2 on a five-point scale (8% for DL, 18% for nonDL). This demonstrates that a robust and automatic segmentation algorithm for segmenting AAAs from 3-D + t US images was developed, showing improved performance compared to conventional segmentation.

Deep learning-based segmentation of abdominal aortic aneurysms and intraluminal thrombus in 3D ultrasound images

Ultrasound (US)-based patient-specific rupture risk analysis of abdominal aortic aneurysms (AAAs) has shown promising results. Input for these models is the patient-specific geometry of the AAA. However, segmentation of the intraluminal thrombus (ILT) remains challenging in US images due to the low ILT-blood contrast. This study aims to improve AAA and ILT segmentation in time-resolved three-dimensional (3D + t) US images using a deep learning approach. In this study a "no new net" (nnU-Net) model was trained on 3D + t US data using either US-based or (co-registered) computed tomography (CT)-based annotations. The optimal training strategy for this low-contrast data was determined for a limited dataset. The merit of augmentation was investigated, as well as the inclusion of low-contrast areas. Segmentation results were validated with CT-based geometries as the ground truth. The model trained on CT-based masks showed the best performance in terms of DICE index, Hausdorff distance, and diameter differences, covering a larger part of the AAA. With a higher accuracy and less manual input the model outperforms conventional methods, with a mean Hausdorff distance of 4.4 mm for the vessel and 7.8 mm for the lumen. However, visibility of the lumen-ILT interface remains the limiting factor, necessitating improvements in image acquisition to ensure broader patient inclusion and enable rupture risk assessment of AAAs in the future.

CPSS: Fusing consistency regularization and pseudo-labeling techniques for semi-supervised deep cardiovascular disease detection using all unlabeled electrocardiograms

BACKGROUND AND OBJECTIVE

Deep learning usually achieves good performance in the supervised way, which requires a large amount of labeled data. However, manual labeling of electrocardiograms (ECGs) is laborious that requires much medical knowledge. Semi-supervised learning (SSL) provides an effective way of leveraging unlabeled data to improve model performance, providing insight for solving this problem. The objective of this study is to improve the performance of cardiovascular disease (CVD) detection by fully utilizing unlabeled ECG.

METHODS

A novel SSL algorithm fusing consistency regularization and pseudo-labeling techniques (CPSS) is proposed. CPSS consists of supervised learning and unsupervised learning. For supervised learning, the labeled ECGs are mapped into prediction vectors by the classifier. The cross-entropy loss function is used to optimize the classifier. For unsupervised learning, the unlabeled ECGs are weakly and strongly augmented, and a consistency loss is used to minimize the difference between the classifier's predictions for the two augmentations. Pseudo-labeling techniques include positive pseudo-labeling (PL) and ranking-based negative pseudo-labeling (RNL). PL introduces pseudo-labels for data with high prediction confidence. RNL assigns negative pseudo-labels to the lower-ranked categories in the prediction vectors to leverage data with low prediction confidence. In this study, VGGNet and ResNet are used as classifiers, which are jointly optimized by labeled and unlabeled ECGs.

RESULTS

CPSS has been validated on several databases. With the same number of labeled ECGs (10%), it improves the accuracies over pure supervised learning by 13.59%, 4.60%, and 5.38% in the CPSC2018, PTB-XL, and Chapman databases, respectively. CPSS achieves comparable results to the fully supervised method with only 10% of labeled ECGs, which reduces the labeling workload by 90%. In addition, to verify the practicality of CPSS, a cardiovascular disease monitoring system is designed by heterogeneously deploying the trained classifiers on an SoC (system-on-a-chip), which can detect CVD in real time.

CONCLUSION

The results of this study indicate that the proposed CPSS can significantly improve the performance of CVD detection using unlabeled ECG, which reduces the burden of ECG labeling in deep learning. In addition, the designed monitoring system makes the proposed CPSS promising for real-world applications.

BraNet: a mobil application for breast image classification based on deep learning algorithms

Mobile health apps are widely used for breast cancer detection using artificial intelligence algorithms, providing radiologists with second opinions and reducing false diagnoses. This study aims to develop an open-source mobile app named "BraNet" for 2D breast imaging segmentation and classification using deep learning algorithms. During the phase off-line, an SNGAN model was previously trained for synthetic image generation, and subsequently, these images were used to pre-trained SAM and ResNet18 segmentation and classification models. During phase online, the BraNet app was developed using the react native framework, offering a modular deep-learning pipeline for mammography (DM) and ultrasound (US) breast imaging classification. This application operates on a client-server architecture and was implemented in Python for iOS and Android devices. Then, two diagnostic radiologists were given a reading test of 290 total original RoI images to assign the perceived breast tissue type. The reader's agreement was assessed using the kappa coefficient. The BraNet App Mobil exhibited the highest accuracy in benign and malignant US images (94.7%/93.6%) classification compared to DM during training I (80.9%/76.9%) and training II (73.7/72.3%). The information contrasts with radiological experts' accuracy, with DM classification being 29%, concerning US 70% for both readers, because they achieved a higher accuracy in US ROI classification than DM images. The kappa value indicates a fair agreement (0.3) for DM images and moderate agreement (0.4) for US images in both readers. It means that not only the amount of data is essential in training deep learning algorithms. Also, it is vital to consider the variety of abnormalities, especially in the mammography data, where several BI-RADS categories are present (microcalcifications, nodules, mass, asymmetry, and dense breasts) and can affect the API accuracy model.

Investigating the effects of artificial intelligence on the personalization of breast cancer management: a systematic study

BACKGROUND

Providing appropriate specialized treatment to the right patient at the right time is considered necessary in cancer management. Targeted therapy tailored to the genetic changes of each breast cancer patient is a desirable feature of precision oncology, which can not only reduce disease progression but also potentially increase patient survival. The use of artificial intelligence alongside precision oncology can help physicians by identifying and selecting more effective treatment factors for patients.

METHOD

A systematic review was conducted using the PubMed, Embase, Scopus, and Web of Science databases in September 2023. We performed the search strategy with keywords, namely: Breast Cancer, Artificial intelligence, and precision Oncology along with their synonyms in the article titles. Descriptive, qualitative, review, and non-English studies were excluded. The quality assessment of the articles and evaluation of bias were determined based on the SJR journal and JBI indices, as well as the PRISMA2020 guideline.

RESULTS

Forty-six studies were selected that focused on personalized breast cancer management using artificial intelligence models. Seventeen studies using various deep learning methods achieved a satisfactory outcome in predicting treatment response and prognosis, contributing to personalized breast cancer management. Two studies utilizing neural networks and clustering provided acceptable indicators for predicting patient survival and categorizing breast tumors. One study employed transfer learning to predict treatment response. Twenty-six studies utilizing machine-learning methods demonstrated that these techniques can improve breast cancer classification, screening, diagnosis, and prognosis. The most frequent modeling techniques used were NB, SVM, RF, XGBoost, and Reinforcement Learning. The average area under the curve (AUC) for the models was 0.91. Moreover, the average values for accuracy, sensitivity, specificity, and precision were reported to be in the range of 90-96% for the models.

CONCLUSION

Artificial intelligence has proven to be effective in assisting physicians and researchers in managing breast cancer treatment by uncovering hidden patterns in complex omics and genetic data. Intelligent processing of omics data through protein and gene pattern classification and the utilization of deep neural patterns has the potential to significantly transform the field of complex disease management.

Ultrasound-based radiomics for the differential diagnosis of breast masses: A systematic review and meta-analysis

OBJECTIVES

Ultrasound-based radiomics has demonstrated excellent diagnostic performance in differentiating benign and malignant breast masses. Given a few clinical studies on their diagnostic role, we conducted a meta-analysis of the potential effects of ultrasound-based radiomics for the differential diagnosis of breast masses, aiming to provide evidence-based medical basis for clinical research.

MATERIALS AND METHODS

We searched Embase, Web of Science, Cochrane Library, and PubMed databases from inception through to February 2023. The methodological quality assessment of the included studies was performed according to Quality Assessment of Diagnostic Accuracy Studies checklist. A diagnostic test accuracy systematic review and meta-analysis was performed in accordance with PRISMA guidelines. Sensitivity, specificity, and area under curve delineating benign and malignant lesions were recorded. We also used sensitivity analysis and subgroup analysis to explore potential sources of heterogeneity. Deeks' funnel plots was used to examine the publication bias.

RESULTS

A total of 11 studies were included in this meta-analysis. For the diagnosis of malignant breast masses worldwide, the overall mean rates of sensitivity and specificity of ultrasound-based radiomics were 0.90 (95% confidence interval [CI], 0.83-0.95) and 0.89 (95% CI, 0.82-0.94), respectively. The summary diagnostic odds ratio was 76 (95% CI, 26-219), and the area under the curve for the summary receiver operating characteristic curve was 0.95 (95% CI, 0.93-0.97).

CONCLUSION

Ultrasound-based radiomics has the potential to improve diagnostic accuracy to discriminate between benign and malignant breast masses, and could reduce unnecessary biopsies.

Enhancing Histopathological Image Classification Performance through Synthetic Data Generation with Generative Adversarial Networks

Breast cancer is the second most common cancer worldwide, primarily affecting women, while histopathological image analysis is one of the possibile methods used to determine tumor malignancy. Regarding image analysis, the application of deep learning has become increasingly prevalent in recent years. However, a significant issue is the unbalanced nature of available datasets, with some classes having more images than others, which may impact the performance of the models due to poorer generalizability. A possible strategy to avoid this problem is downsampling the class with the most images to create a balanced dataset. Nevertheless, this approach is not recommended for small datasets as it can lead to poor model performance. Instead, techniques such as data augmentation are traditionally used to address this issue. These techniques apply simple transformations such as translation or rotation to the images to increase variability in the dataset. Another possibility is using generative adversarial networks (GANs), which can generate images from a relatively small training set. This work aims to enhance model performance in classifying histopathological images by applying data augmentation using GANs instead of traditional techniques.

A Critical Analysis of Deep Semi-Supervised Learning Approaches for Enhanced Medical Image Classification

Deep semi-supervised learning (DSSL) is a machine learning paradigm that blends supervised and unsupervised learning techniques to improve the performance of various models in computer vision tasks. Medical image classification plays a crucial role in disease diagnosis, treatment planning, and patient care. However, obtaining labeled medical image data is often expensive and time-consuming for medical practitioners, leading to limited labeled datasets. DSSL techniques aim to address this challenge, particularly in various medical image tasks, to improve model generalization and performance. DSSL models leverage both the labeled information, which provides explicit supervision, and the unlabeled data, which can provide additional information about the underlying data distribution. That offers a practical solution to resource-intensive demands of data annotation, and enhances the model’s ability to generalize across diverse and previously unseen data landscapes. The present study provides a critical review of various DSSL approaches and their effectiveness and challenges in enhancing medical image classification tasks. The study categorized DSSL techniques into six classes: consistency regularization method, deep adversarial method, pseudo-learning method, graph-based method, multi-label method, and hybrid method. Further, a comparative analysis of performance for six considered methods is conducted using existing studies. The referenced studies have employed metrics such as accuracy, sensitivity, specificity, AUC-ROC, and F1 score to evaluate the performance of DSSL methods on different medical image datasets. Additionally, challenges of the datasets, such as heterogeneity, limited labeled data, and model interpretability, were discussed and highlighted in the context of DSSL for medical image classification. The current review provides future directions and considerations to researchers to further address the challenges and take full advantage of these methods in clinical practices.

ADGAN: Attribute-Driven Generative Adversarial Network for Synthesis and Multiclass Classification of Pulmonary Nodules

Lung cancer is the leading cause of cancer-related deaths worldwide. According to the American Cancer Society, early diagnosis of pulmonary nodules in computed tomography (CT) scans can improve the five-year survival rate up to 70% with proper treatment planning. In this article, we propose an attribute-driven Generative Adversarial Network (ADGAN) for synthesis and multiclass classification of Pulmonary Nodules. A self-attention U-Net (SaUN) architecture is proposed to improve the generation mechanism of the network. The generator is designed with two modules, namely, self-attention attribute module (SaAM) and a self-attention spatial module (SaSM). SaAM generates a nodule image based on given attributes whereas SaSM specifies the nodule region of the input image to be altered. A reconstruction loss along with an attention localization loss (AL) is used to produce an attention map prioritizing the nodule regions. To avoid resemblance between a generated image and a real image, we further introduce an adversarial loss containing a regularization term based on KL divergence. The discriminator part of the proposed model is designed to achieve the multiclass nodule classification task. Our proposed approach is validated over two challenging publicly available datasets, namely LIDC-IDRI and LUNGX. Exhaustive experimentation on these two datasets clearly indicate that we have achieved promising classification accuracy as compared to other state-of-the-art methods.

Scoping Review of Deep Learning Techniques for Diagnosis, Drug Discovery, and Vaccine Development in Leishmaniasis

Leishmania, a single-cell parasite prevalent in tropical and subtropical regions worldwide, can cause varying degrees of leishmaniasis, ranging from self-limiting skin lesions to potentially fatal visceral complications. As such, the parasite has been the subject of much interest in the scientific community. In recent years, advances in diagnostic techniques such as flow cytometry, molecular biology, proteomics, and nanodiagnosis have contributed to progress in the diagnosis of this deadly disease. Additionally, the emergence of artificial intelligence (AI), including its subbranches such as machine learning and deep learning, has revolutionized the field of medicine. The high accuracy of AI and its potential to reduce human and laboratory errors make it an especially promising tool in diagnosis and treatment. Despite the promising potential of deep learning in the medical field, there has been no review study on the applications of this technology in the context of leishmaniasis. To address this gap, we provide a scoping review of deep learning methods in the diagnosis of the disease, drug discovery, and vaccine development. In conducting a thorough search of available literature, we analyzed articles in detail that used deep learning methods for various aspects of the disease, including diagnosis, drug discovery, vaccine development, and related proteins. Each study was individually analyzed, and the methodology and results were presented. As the first and only review study on this topic, this paper serves as a quick and comprehensive resource and guide for the future research in this field.

Recent advancements in artificial intelligence for breast cancer: Image augmentation, segmentation, diagnosis, and prognosis approaches

Breast cancer is a significant global health burden, with increasing morbidity and mortality worldwide. Early screening and accurate diagnosis are crucial for improving prognosis. Radiographic imaging modalities such as digital mammography (DM), digital breast tomosynthesis (DBT), magnetic resonance imaging (MRI), ultrasound (US), and nuclear medicine techniques, are commonly used for breast cancer assessment. And histopathology (HP) serves as the gold standard for confirming malignancy. Artificial intelligence (AI) technologies show great potential for quantitative representation of medical images to effectively assist in segmentation, diagnosis, and prognosis of breast cancer. In this review, we overview the recent advancements of AI technologies for breast cancer, including 1) improving image quality by data augmentation, 2) fast detection and segmentation of breast lesions and diagnosis of malignancy, 3) biological characterization of the cancer such as staging and subtyping by AI-based classification technologies, 4) prediction of clinical outcomes such as metastasis, treatment response, and survival by integrating multi-omics data. Then, we then summarize large-scale databases available to help train robust, generalizable, and reproducible deep learning models. Furthermore, we conclude the challenges faced by AI in real-world applications, including data curating, model interpretability, and practice regulations. Besides, we expect that clinical implementation of AI will provide important guidance for the patient-tailored management.

2S-BUSGAN: A Novel Generative Adversarial Network for Realistic Breast Ultrasound Image with Corresponding Tumor Contour Based on Small Datasets

Deep learning (DL) models in breast ultrasound (BUS) image analysis face challenges with data imbalance and limited atypical tumor samples. Generative Adversarial Networks (GAN) address these challenges by providing efficient data augmentation for small datasets. However, current GAN approaches fail to capture the structural features of BUS and generated images lack structural legitimacy and are unrealistic. Furthermore, generated images require manual annotation for different downstream tasks before they can be used. Therefore, we propose a two-stage GAN framework, 2s-BUSGAN, for generating annotated BUS images. It consists of the Mask Generation Stage (MGS) and the Image Generation Stage (IGS), generating benign and malignant BUS images using corresponding tumor contours. Moreover, we employ a Feature-Matching Loss (FML) to enhance the quality of generated images and utilize a Differential Augmentation Module (DAM) to improve GAN performance on small datasets. We conduct experiments on two datasets, BUSI and Collected. Moreover, results indicate that the quality of generated images is improved compared with traditional GAN methods. Additionally, our generated images underwent evaluation by ultrasound experts, demonstrating the possibility of deceiving doctors. A comparative evaluation showed that our method also outperforms traditional GAN methods when applied to training segmentation and classification models. Our method achieved a classification accuracy of 69% and 85.7% on two datasets, respectively, which is about 3% and 2% higher than that of the traditional augmentation model. The segmentation model trained using the 2s-BUSGAN augmented datasets achieved DICE scores of 75% and 73% on the two datasets, respectively, which were higher than the traditional augmentation methods. Our research tackles imbalanced and limited BUS image data challenges. Our 2s-BUSGAN augmentation method holds potential for enhancing deep learning model performance in the field.

GSDA: Generative adversarial network-based semi-supervised data augmentation for ultrasound image classification

Medical Ultrasound (US) is one of the most widely used imaging modalities in clinical practice, but its usage presents unique challenges such as variable imaging quality. Deep Learning (DL) models can serve as advanced medical US image analysis tools, but their performance is greatly limited by the scarcity of large datasets. To solve the common data shortage, we develop GSDA, a Generative Adversarial Network (GAN)-based semi-supervised data augmentation method. GSDA consists of the GAN and Convolutional Neural Network (CNN). The GAN synthesizes and pseudo-labels high-resolution, high-quality US images, and both real and synthesized images are then leveraged to train the CNN. To address the training challenges of both GAN and CNN with limited data, we employ transfer learning techniques during their training. We also introduce a novel evaluation standard that balances classification accuracy with computational time. We evaluate our method on the BUSI dataset and GSDA outperforms existing state-of-the-art methods. With the high-resolution and high-quality images synthesized, GSDA achieves a 97.9% accuracy using merely 780 images. Given these promising results, we believe that GSDA holds potential as an auxiliary tool for medical US analysis.

Skin Lesion Synthesis and Classification Using an Improved DCGAN Classifier

The prognosis for patients with skin cancer improves with regular screening and checkups. Unfortunately, many people with skin cancer do not receive a diagnosis until the disease has advanced beyond the point of effective therapy. Early detection is critical, and automated diagnostic technologies like dermoscopy, an imaging device that detects skin lesions early in the disease, are a driving factor. The lack of annotated data and class-imbalance datasets makes using automated diagnostic methods challenging for skin lesion classification. In recent years, deep learning models have performed well in medical diagnosis. Unfortunately, such models require a substantial amount of annotated data for training. Applying a data augmentation method based on generative adversarial networks (GANs) to classify skin lesions is a plausible solution by generating synthetic images to address the problem. This article proposes a skin lesion synthesis and classification model based on an Improved Deep Convolutional Generative Adversarial Network (DCGAN). The proposed system generates realistic images using several convolutional neural networks, making training easier. Scaling, normalization, sharpening, color transformation, and median filters enhance image details during training. The proposed model uses generator and discriminator networks, global average pooling with 2 × 2 fractional-stride, backpropagation with a constant learning rate of 0.01 instead of 0.0002, and the most effective hyperparameters for optimization to efficiently generate high-quality synthetic skin lesion images. As for the classification, the final layer of the Discriminator is labeled as a classifier for predicting the target class. This study deals with a binary classification predicting two classes-benign and malignant-in the ISIC2017 dataset: accuracy, recall, precision, and F1-score model classification performance. BAS measures classifier accuracy on imbalanced datasets. The DCGAN Classifier model demonstrated superior performance with a notable accuracy of 99.38% and 99% for recall, precision, F1 score, and BAS, outperforming the state-of-the-art deep learning models. These results show that the DCGAN Classifier can generate high-quality skin lesion images and accurately classify them, making it a promising tool for deep learning-based medical image analysis.

A Review Paper about Deep Learning for Medical Image Analysis

Medical imaging refers to the process of obtaining images of internal organs for therapeutic purposes such as discovering or studying diseases. The primary objective of medical image analysis is to improve the efficacy of clinical research and treatment options. Deep learning has revamped medical image analysis, yielding excellent results in image processing tasks such as registration, segmentation, feature extraction, and classification. The prime motivations for this are the availability of computational resources and the resurgence of deep convolutional neural networks. Deep learning techniques are good at observing hidden patterns in images and supporting clinicians in achieving diagnostic perfection. It has proven to be the most effective method for organ segmentation, cancer detection, disease categorization, and computer-assisted diagnosis. Many deep learning approaches have been published to analyze medical images for various diagnostic purposes. In this paper, we review the work exploiting current state-of-the-art deep learning approaches in medical image processing. We begin the survey by providing a synopsis of research works in medical imaging based on convolutional neural networks. Second, we discuss popular pretrained models and general adversarial networks that aid in improving convolutional networks' performance. Finally, to ease direct evaluation, we compile the performance metrics of deep learning models focusing on COVID-19 detection and child bone age prediction.

Generative adversarial network based synthetic data training model for lightweight convolutional neural networks

Inadequate training data is a significant challenge for deep learning techniques, particularly in applications where data is difficult to get, and publicly available datasets are uncommon owing to ethical and privacy concerns. Various approaches, such as data augmentation and transfer learning, are employed to address this problem, which help to some extent in removing this limitation. However, after a certain amount of data augmentation, the quality of the generated data stalls, and transfer learning suffers from the issue of negative transfer. This paper proposes a novel generative adversarial network-based synthetic data training (GAN-ST) model to generate synthetic data for training a lightweight convolutional neural network (CNN). An enhanced generator is proposed to quickly saturate and cover the colour space of the training distribution. The GAN-ST model is based on Deep Convolutional Generative Adversarial Network(s) (DCGAN) and Conditional Generative Adversarial Network(s) (CGAN) models, which consist of an enhanced generator. The study evaluates the accuracy of a CNN model on the MNIST and CIFAR 10 datasets using both original and synthetic data. The results revealed an impressive classifier accuracy on the MNIST dataset, achieving an accuracy of 99.38% on GAN-ST-generated synthetic training data, which is only 0.05% lower than the performance on original data-based training. The classifier performance on the CIFAR dataset is also remarkable, achieving an accuracy of 90.23%. The performance of CNN trained using GAN-ST-based synthetic data is notable, with the most considerable improvement of 0.66% and 7.06%, over a single GAN-based synthetic data training for the MNIST and CIFAR datasets, respectively. By training two GANs independently, the GAN-ST model covers different parts of the original data distribution, resulting in a more diverse and realistic training data set for the classifier. This diverse set of synthetic data, when used to train a CNN, shows better generalization to new data, leading to improved classification accuracy.

Deep Learning Approaches for Data Augmentation in Medical Imaging: A Review

Deep learning has become a popular tool for medical image analysis, but the limited availability of training data remains a major challenge, particularly in the medical field where data acquisition can be costly and subject to privacy regulations. Data augmentation techniques offer a solution by artificially increasing the number of training samples, but these techniques often produce limited and unconvincing results. To address this issue, a growing number of studies have proposed the use of deep generative models to generate more realistic and diverse data that conform to the true distribution of the data. In this review, we focus on three types of deep generative models for medical image augmentation: variational autoencoders, generative adversarial networks, and diffusion models. We provide an overview of the current state of the art in each of these models and discuss their potential for use in different downstream tasks in medical imaging, including classification, segmentation, and cross-modal translation. We also evaluate the strengths and limitations of each model and suggest directions for future research in this field. Our goal is to provide a comprehensive review about the use of deep generative models for medical image augmentation and to highlight the potential of these models for improving the performance of deep learning algorithms in medical image analysis.

Synthetic data accelerates the development of generalizable learning-based algorithms for X-ray image analysis

Artificial intelligence (AI) now enables automated interpretation of medical images. However, AI's potential use for interventional image analysis remains largely untapped. This is because the post hoc analysis of data collected during live procedures has fundamental and practical limitations, including ethical considerations, expense, scalability, data integrity and a lack of ground truth. Here we demonstrate that creating realistic simulated images from human models is a viable alternative and complement to large-scale in situ data collection. We show that training AI image analysis models on realistically synthesized data, combined with contemporary domain generalization techniques, results in machine learning models that on real data perform comparably to models trained on a precisely matched real data training set. We find that our model transfer paradigm for X-ray image analysis, which we refer to as SyntheX, can even outperform real-data-trained models due to the effectiveness of training on a larger dataset. SyntheX provides an opportunity to markedly accelerate the conception, design and evaluation of X-ray-based intelligent systems. In addition, SyntheX provides the opportunity to test novel instrumentation, design complementary surgical approaches, and envision novel techniques that improve outcomes, save time or mitigate human error, free from the ethical and practical considerations of live human data collection.

Chronic Wound Image Augmentation and Assessment Using Semi-Supervised Progressive Multi-Granularity EfficientNet

Goal: Augment a small, imbalanced, wound dataset by using semi-supervised learning with a secondary dataset. Then utilize the augmented wound dataset for deep learning-based wound assessment. Methods: The clinically-validated Photographic Wound Assessment Tool (PWAT) scores eight wound attributes: Size, Depth, Necrotic Tissue Type, Necrotic Tissue Amount, Granulation Tissue type, Granulation Tissue Amount, Edges, Periulcer Skin Viability to comprehensively assess chronic wound images. A small corpus of 1639 wound images labeled with ground truth PWAT scores was used as reference. A Semi-Supervised learning and Progressive Multi-Granularity training mechanism were used to leverage a secondary corpus of 9870 unlabeled wound images. Wound scoring utilized the EfficientNet Convolutional Neural Network on the augmented wound corpus. Results: Our proposed Semi-Supervised PMG EfficientNet (SS-PMG-EfficientNet) approach estimated all 8 PWAT sub-scores with classification accuracies and F1 scores of about 90% on average, and outperformed a comprehensive list of baseline models and had a 7% improvement over the prior state-of-the-art (without data augmentation). We also demonstrate that synthetic wound image generation using Generative Adversarial Networks (GANs) did not improve wound assessment. Conclusions: Semi-supervised learning on unlabeled wound images in a secondary dataset achieved impressive performance for deep learning-based wound grading.

Data synthesis and adversarial networks: A review and meta-analysis in cancer imaging

Despite technological and medical advances, the detection, interpretation, and treatment of cancer based on imaging data continue to pose significant challenges. These include inter-observer variability, class imbalance, dataset shifts, inter- and intra-tumour heterogeneity, malignancy determination, and treatment effect uncertainty. Given the recent advancements in image synthesis, Generative Adversarial Networks (GANs), and adversarial training, we assess the potential of these technologies to address a number of key challenges of cancer imaging. We categorise these challenges into (a) data scarcity and imbalance, (b) data access and privacy, (c) data annotation and segmentation, (d) cancer detection and diagnosis, and (e) tumour profiling, treatment planning and monitoring. Based on our analysis of 164 publications that apply adversarial training techniques in the context of cancer imaging, we highlight multiple underexplored solutions with research potential. We further contribute the Synthesis Study Trustworthiness Test (SynTRUST), a meta-analysis framework for assessing the validation rigour of medical image synthesis studies. SynTRUST is based on 26 concrete measures of thoroughness, reproducibility, usefulness, scalability, and tenability. Based on SynTRUST, we analyse 16 of the most promising cancer imaging challenge solutions and observe a high validation rigour in general, but also several desirable improvements. With this work, we strive to bridge the gap between the needs of the clinical cancer imaging community and the current and prospective research on data synthesis and adversarial networks in the artificial intelligence community.

Data augmentation for medical imaging: A systematic literature review

Recent advances in Deep Learning have largely benefited from larger and more diverse training sets. However, collecting large datasets for medical imaging is still a challenge due to privacy concerns and labeling costs. Data augmentation makes it possible to greatly expand the amount and variety of data available for training without actually collecting new samples. Data augmentation techniques range from simple yet surprisingly effective transformations such as cropping, padding, and flipping, to complex generative models. Depending on the nature of the input and the visual task, different data augmentation strategies are likely to perform differently. For this reason, it is conceivable that medical imaging requires specific augmentation strategies that generate plausible data samples and enable effective regularization of deep neural networks. Data augmentation can also be used to augment specific classes that are underrepresented in the training set, e.g., to generate artificial lesions. The goal of this systematic literature review is to investigate which data augmentation strategies are used in the medical domain and how they affect the performance of clinical tasks such as classification, segmentation, and lesion detection. To this end, a comprehensive analysis of more than 300 articles published in recent years (2018-2022) was conducted. The results highlight the effectiveness of data augmentation across organs, modalities, tasks, and dataset sizes, and suggest potential avenues for future research.

A survey on deep learning applied to medical images: from simple artificial neural networks to generative models

Deep learning techniques, in particular generative models, have taken on great importance in medical image analysis. This paper surveys fundamental deep learning concepts related to medical image generation. It provides concise overviews of studies which use some of the latest state-of-the-art models from last years applied to medical images of different injured body areas or organs that have a disease associated with (e.g., brain tumor and COVID-19 lungs pneumonia). The motivation for this study is to offer a comprehensive overview of artificial neural networks (NNs) and deep generative models in medical imaging, so more groups and authors that are not familiar with deep learning take into consideration its use in medicine works. We review the use of generative models, such as generative adversarial networks and variational autoencoders, as techniques to achieve semantic segmentation, data augmentation, and better classification algorithms, among other purposes. In addition, a collection of widely used public medical datasets containing magnetic resonance (MR) images, computed tomography (CT) scans, and common pictures is presented. Finally, we feature a summary of the current state of generative models in medical image including key features, current challenges, and future research paths.

Contrastive learning–guided multi-meta attention network for breast ultrasound video diagnosis

Breast cancer is the most common cause of cancer death in women. Early screening and treatment can effectively improve the success rate of treatment. Ultrasound imaging technology, as the preferred modality for breast cancer screening, provides an essential reference for early diagnosis. Existing computer-aided ultrasound imaging diagnostic techniques mainly rely on the selected key frames for breast cancer lesion diagnosis. In this paper, we first collected and annotated a dataset of ultrasound video sequences of 268 cases of breast lesions. Moreover, we propose a contrastive learning–guided multi-meta attention network (CLMAN) by combining a deformed feature extraction module and a multi-meta attention module to address breast lesion diagnosis in ultrasound sequence. The proposed feature extraction module can autonomously acquire key information of the feature map in the spatial dimension, whereas the designed multi-meta attention module is dedicated to effective information aggregation in the temporal dimension. In addition, we utilize a contrast learning strategy to alleviate the problem of high imaging variability within ultrasound lesion videos. The experimental results on our collected dataset show that our CLMAN significantly outperforms existing advanced methods for video classification.

Improved self-attention generative adversarial adaptation network-based melanoma classification

Melanoma is one of the widespread skin cancers that has affected millions in past decades. Detection of skin cancer at preliminary stages may become a source of reducing mortality rates. Hence, it is required to develop an autonomous system of reliable type for the detection of melanoma via image processing. This paper develops an independent medical imaging technique using Self-Attention Adaptation Generative Adversarial Network (SAAGAN). The entire processing model involves the process of pre-processing, feature extraction using Scale Invariant Feature Transform (SIFT), and finally, classification using SAAGAN. The simulation is conducted on ISIC 2016/PH2 datasets, where 10-fold cross-validation is undertaken on a high-end computing platform. The simulation is performed to test the model efficacy against various images on several performance metrics that include accuracy, precision, recall, f-measure, percentage error, Matthews Correlation Coefficient, and Jaccard Index. The simulation shows that the proposed SAAGAN is more effective in detecting the test images than the existing GAN protocols.

Synthetic MRI improves radiomics-based glioblastoma survival prediction

Glioblastoma is an aggressive and fast-growing brain tumor with poor prognosis. Predicting the expected survival of patients with glioblastoma is a key task for efficient treatment and surgery planning. Survival predictions could be enhanced by means of a radiomic system. However, these systems demand high numbers of multicontrast images, the acquisitions of which are time consuming, giving rise to patient discomfort and low healthcare system efficiency. Synthetic MRI could favor deployment of radiomic systems in the clinic by allowing practitioners not only to reduce acquisition time, but also to retrospectively complete databases or to replace artifacted images. In this work we analyze the replacement of an actually acquired MR weighted image by a synthesized version to predict survival of glioblastoma patients with a radiomic system. Each synthesized version was realistically generated from two acquired images with a deep learning synthetic MRI approach based on a convolutional neural network. Specifically, two weighted images were considered for the replacement one at a time, a T2w and a FLAIR, which were synthesized from the pairs T1w and FLAIR, and T1w and T2w, respectively. Furthermore, a radiomic system for survival prediction, which can classify patients into two groups (survival >480 days and ≤ 480 days), was built. Results show that the radiomic system fed with the synthesized image achieves similar performance compared with using the acquired one, and better performance than a model that does not include this image. Hence, our results confirm that synthetic MRI does add to glioblastoma survival prediction within a radiomics-based approach.

Cyclic GAN Model to Classify Breast Cancer Data for Pathological Healthcare Task

An algorithm framework based on CycleGAN and an upgraded dual-path network (DPN) is suggested to address the difficulties of uneven staining in pathological pictures and difficulty of discriminating benign from malignant cells. CycleGAN is used for color normalization in pathological pictures to tackle the problem of uneven staining. However, the resultant detection model is ineffective. By overlapping the images, the DPN uses the addition of small convolution, deconvolution, and attention mechanisms to enhance the model's ability to classify the texture features of pathological images on the BreaKHis dataset. The parameters that are taken into consideration for measuring the accuracy of the proposed model are false-positive rate, false-negative rate, recall, precision, and F1 score. Several experiments are carried out over the selected parameters, such as making comparisons between benign and malignant classification accuracy under different normalization methods, comparison of accuracy of image level and patient level using different CNN models, correlating the correctness of DPN68-A network with different deep learning models and other classification algorithms at all magnifications. The results thus obtained have proved that the proposed model DPN68-A network can effectively classify the benign and malignant breast cancer pathological images at various magnifications. The proposed model also is able to better assist the pathologists in diagnosing the patients by synthesizing the images of different magnifications in the clinical stage.

Collaborative learning of graph generation, clustering and classification for brain networks diagnosis

PURPOSE

Accurate diagnosis of autism spectrum disorder (ASD) plays a key role in improving the condition and quality of life for patients. In this study, we mainly focus on ASD diagnosis with functional brain networks (FBNs). The major challenge for brain networks modeling is the high dimensional connectivity in brain networks and limited number of subjects, which hinders the classification capability of graph convolutional networks (GCNs).

METHOD

To alleviate the influence of the limited data and high dimensional connectivity, we introduce a unified three-stage graph learning framework for brain network classification, involving multi-graph clustering, graph generation and graph classification. The framework combining Graph Generation, Clustering and Classification Networks (GraphCGC-Net) enhances the critical connections by multi-graph clustering (MGC) with a supervision scheme, and generates realistic brain networks by simultaneously preserving the global consistent distribution and local topology properties.

RESULTS

To demonstrate the effectiveness of our approach, we evaluate the performance of the proposed method on the Autism Brain Imaging Data Exchange (ABIDE) dataset and conduct extensive experiments on the ASD classification problem. Our proposed method achieves an average accuracy of 70.45% and an AUC of 72.76% on ABIDE. Compared with the traditional GCN model, the proposed GraphCGC-Net obtains 9.3%, and 10.64% improvement in terms of accuracy and AUC metrics, respectively.

CONCLUSION

The comprehensive experiments demonstrate that our GraphCGC-Net is effective for graph classification in brain disorders diagnosis. Moreover, we find that MGC can generate biologically meaningful subnetworks, which is highly consistent with the previous neuroimaging-derived biomarker evidence of ASD. More importantly, the promising results suggest that applying generative adversarial networks (GANs) in brain networks to improve the classification performance is worth further investigation.

Predicting Breast Tumor Malignancy Using Deep ConvNeXt Radiomics and Quality-Based Score Pooling in Ultrasound Sequences

Breast cancer needs to be detected early to reduce mortality rate. Ultrasound imaging (US) could significantly enhance diagnosing cases with dense breasts. Most of the existing computer-aided diagnosis (CAD) systems employ a single ultrasound image for the breast tumor to extract features to classify it as benign or malignant. However, the accuracy of such CAD system is limited due to the large tumor size and shape variation, irregular and ambiguous tumor boundaries, and low signal-to-noise ratio in ultrasound images due to their noisy nature and the significant similarity between normal and abnormal tissues. To handle these issues, we propose a deep-learning-based radiomics method based on breast US sequences in this paper. The proposed approach involves three main components: radiomic features extraction based on a deep learning network, so-called ConvNeXt, a malignancy score pooling mechanism, and visual interpretations. Specifically, we employ the ConvNeXt network, a deep convolutional neural network (CNN) trained using the vision transformer style. We also propose an efficient pooling mechanism to fuse the malignancy scores of each breast US sequence frame based on image-quality statistics. The ablation study and experimental results demonstrate that our method achieves competitive results compared to other CNN-based methods.

Augmentation-Consistent Clustering Network for Diabetic Retinopathy Grading with Fewer Annotations

Diabetic retinopathy (DR) is currently one of the severe complications leading to blindness, and computer-aided, diagnosis technology-assisted DR grading has become a popular research trend especially for the development of deep learning methods. However, most deep learning-based DR grading models require a large number of annotations to provide data guidance, and it is laborious for experts to find subtle lesion areas from fundus images, making accurate annotation more expensive than other vision tasks. In contrast, large-scale unlabeled data are easily accessible, becoming a potential solution to reduce the annotating workload in DR grading. Thus, this paper explores the internal correlations from unknown fundus images assisted by limited labeled fundus images to solve the semisupervised DR grading problem and proposes an augmentation-consistent clustering network (ACCN) to address the above-mentioned challenges. Specifically, the augmentation provides an efficient cue for the similarity information of unlabeled fundus images, assisting the supervision from the labeled data. By mining the consistent correlations from augmentation and raw images, the ACCN can discover subtle lesion features by clustering with fewer annotations. Experiments on Messidor and APTOS 2019 datasets show that the ACCN surpasses many state-of-the-art methods in a semisupervised manner.

Generative Adversarial Networks in Medical Image augmentation: A review

OBJECT

With the development of deep learning, the number of training samples for medical image-based diagnosis and treatment models is increasing. Generative Adversarial Networks (GANs) have attracted attention in medical image processing due to their excellent image generation capabilities and have been widely used in data augmentation. In this paper, a comprehensive and systematic review and analysis of medical image augmentation work are carried out, and its research status and development prospects are reviewed.

METHOD

This paper reviews 105 medical image augmentation related papers, which mainly collected by ELSEVIER, IEEE Xplore, and Springer from 2018 to 2021. We counted these papers according to the parts of the organs corresponding to the images, and sorted out the medical image datasets that appeared in them, the loss function in model training, and the quantitative evaluation metrics of image augmentation. At the same time, we briefly introduce the literature collected in three journals and three conferences that have received attention in medical image processing.

RESULT

First, we summarize the advantages of various augmentation models, loss functions, and evaluation metrics. Researchers can use this information as a reference when designing augmentation tasks. Second, we explore the relationship between augmented models and the amount of the training set, and tease out the role that augmented models may play when the quality of the training set is limited. Third, the statistical number of papers shows that the development momentum of this research field remains strong. Furthermore, we discuss the existing limitations of this type of model and suggest possible research directions.

CONCLUSION

We discuss GAN-based medical image augmentation work in detail. This method effectively alleviates the challenge of limited training samples for medical image diagnosis and treatment models. It is hoped that this review will benefit researchers interested in this field.

Deep Learning With Radiomics for Disease Diagnosis and Treatment: Challenges and Potential

The high-throughput extraction of quantitative imaging features from medical images for the purpose of radiomic analysis, i.e., radiomics in a broad sense, is a rapidly developing and emerging research field that has been attracting increasing interest, particularly in multimodality and multi-omics studies. In this context, the quantitative analysis of multidimensional data plays an essential role in assessing the spatio-temporal characteristics of different tissues and organs and their microenvironment. Herein, recent developments in this method, including manually defined features, data acquisition and preprocessing, lesion segmentation, feature extraction, feature selection and dimension reduction, statistical analysis, and model construction, are reviewed. In addition, deep learning-based techniques for automatic segmentation and radiomic analysis are being analyzed to address limitations such as rigorous workflow, manual/semi-automatic lesion annotation, and inadequate feature criteria, and multicenter validation. Furthermore, a summary of the current state-of-the-art applications of this technology in disease diagnosis, treatment response, and prognosis prediction from the perspective of radiology images, multimodality images, histopathology images, and three-dimensional dose distribution data, particularly in oncology, is presented. The potential and value of radiomics in diagnostic and therapeutic strategies are also further analyzed, and for the first time, the advances and challenges associated with dosiomics in radiotherapy are summarized, highlighting the latest progress in radiomics. Finally, a robust framework for radiomic analysis is presented and challenges and recommendations for future development are discussed, including but not limited to the factors that affect model stability (medical big data and multitype data and expert knowledge in medical), limitations of data-driven processes (reproducibility and interpretability of studies, different treatment alternatives for various institutions, and prospective researches and clinical trials), and thoughts on future directions (the capability to achieve clinical applications and open platform for radiomics analysis).

Breast Cancer Classification from Ultrasound Images Using Probability-Based Optimal Deep Learning Feature Fusion

After lung cancer, breast cancer is the second leading cause of death in women. If breast cancer is detected early, mortality rates in women can be reduced. Because manual breast cancer diagnosis takes a long time, an automated system is required for early cancer detection. This paper proposes a new framework for breast cancer classification from ultrasound images that employs deep learning and the fusion of the best selected features. The proposed framework is divided into five major steps: (i) data augmentation is performed to increase the size of the original dataset for better learning of Convolutional Neural Network (CNN) models; (ii) a pre-trained DarkNet-53 model is considered and the output layer is modified based on the augmented dataset classes; (iii) the modified model is trained using transfer learning and features are extracted from the global average pooling layer; (iv) the best features are selected using two improved optimization algorithms known as reformed differential evaluation (RDE) and reformed gray wolf (RGW); and (v) the best selected features are fused using a new probability-based serial approach and classified using machine learning algorithms. The experiment was conducted on an augmented Breast Ultrasound Images (BUSI) dataset, and the best accuracy was 99.1%. When compared with recent techniques, the proposed framework outperforms them.