Dataset of breast ultrasound images

Human visual perception-inspired medical image segmentation network with multi-feature compression

Medical image segmentation is crucial for computer-aided diagnosis and treatment planning, directly influencing clinical decision-making. To enhance segmentation accuracy, existing methods typically fuse local, global, and various other features. However, these methods often ignore the negative impact of noise on the results during the feature fusion process. In contrast, certain regions of the human visual system, such as the inferotemporal cortex and parietal cortex, effectively suppress irrelevant noise while integrating multiple features-a capability lacking in current methods. To address this gap, we propose MS-Net, a medical image segmentation network inspired by human visual perception. MS-Net incorporates a multi-feature compression (MFC) module that mimics the human visual system's processing of complex images, first learning various feature types and subsequently filtering out irrelevant ones. Additionally, MS-Net features a segmentation refinement (SR) module that emulates how physicians segment lesions. This module initially performs coarse segmentation to capture the lesion's approximate location and shape, followed by a refinement step to achieve precise boundary delineation. Experimental results demonstrate that MS-Net not only attains state-of-the-art segmentation performance across three public datasets but also significantly reduces the number of parameters compared to existing models. Code is available at https://github.com/guangguangLi/MS-Net.

BUS-M2AE: Multi-scale Masked Autoencoder for Breast Ultrasound Image Analysis

Masked AutoEncoder (MAE) has demonstrated significant potential in medical image analysis by reducing the cost of manual annotations. However, MAE and its recent variants are not well-developed for ultrasound images in breast cancer diagnosis, as they struggle to generalize to the task of distinguishing ultrasound breast tumors of varying sizes. This limitation hinders the model's ability to adapt to the diverse morphological characteristics of breast tumors. In this paper, we propose a novel Breast UltraSound Multi-scale Masked AutoEncoder (BUS-M2AE) model to address the limitations of the general MAE. BUS-M2AE incorporates multi-scale masking methods at both the token level during the image patching stage and the feature level during the feature learning stage. These two multi-scale masking methods enable flexible strategies to match the explicit masked patches and the implicit features with varying tumor scales. By introducing these multi-scale masking methods in the image patching and feature learning phases, BUS-M2AE allows the pre-trained vision transformer to adaptively perceive and accurately distinguish breast tumors of different sizes, thereby improving the model's overall performance in handling diverse tumor morphologies. Comprehensive experiments demonstrate that BUS-M2AE outperforms recent MAE variants and commonly used supervised learning methods in breast cancer classification and tumor segmentation tasks.

One-shot learning for generalization in medical image classification across modalities

Generalizability is one of the biggest challenges hindering the advancement of medical sensing technologies across multiple imaging modalities. This issue is further impaired when the imaging data is limited in scope or of poor quality. To tackle this, we propose a generalized and robust, lightweight one-shot learning method for medical image classification across various imaging modalities, including X-ray, microscopic, and CT scans. Our model introduces a collaborative one-shot training (COST) approach, incorporating both meta-learning and metric-learning. This approach allows for effective training on only one image per class. To ensure generalization with fewer epochs, we employ gradient generalization at dense and fully connected layers, utilizing a lightweight Siamese network with triplet loss and shared parameters. The proposed model was evaluated on 12 medical image datasets from MedMNIST2D, achieving an average accuracy of 91.5 % and area under the curve (AUC) of 0.89, outperforming state-of-the-art models such as ResNet-50 and AutoML by over 10 % on certain datasets. Further, in the OCTMNIST dataset, our model achieved an AUC of 0.91 compared to ResNet-50's 0.77. Ablation studies further validate the superiority of our approach, with the COST method showing significant improvement in convergence speed and accuracy when compared to traditional one-shot learning setups. Additionally, our model's lightweight architecture requires only 0.15 million parameters, making it well-suited for deployment on resource-constrained devices.

DEMAC-Net: A Dual-Encoder Multiattention Collaborative Network for Cervical Nerve Pathway and Adjacent Anatomical Structure Segmentation

OBJECTIVE

Currently, cervical anesthesia is performed using three main approaches: superficial cervical plexus block, deep cervical plexus block, and intermediate plexus nerve block. However, each technique carries inherent risks and demands significant clinical expertise. Ultrasound imaging, known for its real-time visualization capabilities and accessibility, is widely used in both diagnostic and interventional procedures. Nevertheless, accurate segmentation of small and irregularly shaped structures such as the cervical and brachial plexuses remains challenging due to image noise, complex anatomical morphology, and limited annotated training data. This study introduces DEMAC-Net-a dual-encoder, multiattention collaborative network-to significantly improve the segmentation accuracy of these neural structures. By precisely identifying the cervical nerve pathway (CNP) and adjacent anatomical tissues, DEMAC-Net aims to assist clinicians, especially those less experienced, in effectively guiding anesthesia procedures and accurately identifying optimal needle insertion points. Consequently, this improvement is expected to enhance clinical safety, reduce procedural risks, and streamline decision-making efficiency during ultrasound-guided regional anesthesia.

METHODS

DEMAC-Net combines a dual-encoder architecture with the Spatial Understanding Convolution Kernel (SUCK) and the Spatial-Channel Attention Module (SCAM) to extract multi-scale features effectively. Additionally, a Global Attention Gate (GAG) and inter-layer fusion modules refine relevant features while suppressing noise. A novel dataset, Neck Ultrasound Dataset (NUSD), was introduced, containing 1,500 annotated ultrasound images across seven anatomical regions. Extensive experiments were conducted on both NUSD and the BUSI public dataset, comparing DEMAC-Net to state-of-the-art models using metrics such as Dice Similarity Coefficient (DSC) and Intersection over Union (IoU).

RESULTS

On the NUSD dataset, DEMAC-Net achieved a mean DSC of 93.3%, outperforming existing models. For external validation on the BUSI dataset, it demonstrated superior generalization, achieving a DSC of 87.2% and a mean IoU of 77.4%, surpassing other advanced methods. Notably, DEMAC-Net displayed consistent segmentation stability across all tested structures.

CONCLUSION

The proposed DEMAC-Net significantly improves segmentation accuracy for small nerves and complex anatomical structures in ultrasound images, outperforming existing methods in terms of accuracy and computational efficiency. This framework holds great potential for enhancing ultrasound-guided procedures, such as peripheral nerve blocks, by providing more precise anatomical localization, ultimately improving clinical outcomes.

Flip Learning: Weakly supervised erase to segment nodules in breast ultrasound

Accurate segmentation of nodules in both 2D breast ultrasound (BUS) and 3D automated breast ultrasound (ABUS) is crucial for clinical diagnosis and treatment planning. Therefore, developing an automated system for nodule segmentation can enhance user independence and expedite clinical analysis. Unlike fully-supervised learning, weakly-supervised segmentation (WSS) can streamline the laborious and intricate annotation process. However, current WSS methods face challenges in achieving precise nodule segmentation, as many of them depend on inaccurate activation maps or inefficient pseudo-mask generation algorithms. In this study, we introduce a novel multi-agent reinforcement learning-based WSS framework called Flip Learning, which relies solely on 2D/3D boxes for accurate segmentation. Specifically, multiple agents are employed to erase the target from the box to facilitate classification tag flipping, with the erased region serving as the predicted segmentation mask. The key contributions of this research are as follows: (1) Adoption of a superpixel/supervoxel-based approach to encode the standardized environment, capturing boundary priors and expediting the learning process. (2) Introduction of three meticulously designed rewards, comprising a classification score reward and two intensity distribution rewards, to steer the agents' erasing process precisely, thereby avoiding both under- and over-segmentation. (3) Implementation of a progressive curriculum learning strategy to enable agents to interact with the environment in a progressively challenging manner, thereby enhancing learning efficiency. Extensively validated on the large in-house BUS and ABUS datasets, our Flip Learning method outperforms state-of-the-art WSS methods and foundation models, and achieves comparable performance as fully-supervised learning algorithms.

FET-UNet: Merging CNN and transformer architectures for superior breast ultrasound image segmentation

PURPOSE

Breast cancer remains a significant cause of mortality among women globally, highlighting the critical need for accurate diagnosis. Although Convolutional Neural Networks (CNNs) have shown effectiveness in segmenting breast ultrasound images, they often face challenges in capturing long-range dependencies, particularly for lesions with similar intensity distributions, irregular shapes, and blurred boundaries. To overcome these limitations, we introduce FET-UNet, a novel hybrid framework that integrates CNNs and Swin Transformers within a UNet-like architecture.

METHODS

FET-UNet features parallel branches for feature extraction: one utilizes ResNet34 blocks, and the other employs Swin Transformer blocks. These branches are fused using an advanced feature aggregation module (AFAM), enabling the network to effectively combine local details and global context. Additionally, we include a multi-scale upsampling mechanism in the decoder to ensure precise segmentation outputs. This design enhances the capture of both local details and long-range dependencies.

RESULTS

Extensive evaluations on the BUSI, UDIAT, and BLUI datasets demonstrate the superior performance of FET-UNet compared to state-of-the-art methods. The model achieves Dice coefficients of 82.9% on BUSI, 88.9% on UDIAT, and 90.1% on BLUI.

CONCLUSION

FET-UNet shows great potential to advance breast ultrasound image segmentation and support more precise clinical diagnoses. Further research could explore the application of this framework to other medical imaging modalities and its integration into clinical workflows.

Multilevel perception boundary-guided network for breast lesion segmentation in ultrasound images

BACKGROUND

Automatic segmentation of breast tumors from the ultrasound images is essential for the subsequent clinical diagnosis and treatment plan. Although the existing deep learning-based methods have achieved considerable progress in automatic segmentation of breast tumors, their performance on tumors with similar intensity to the normal tissues is still not satisfactory, especially for the tumor boundaries.

PURPOSE

To accurately segment the non-enhanced lesions with more accurate boundaries, a novel multilevel perception boundary-guided network (PBNet) is proposed to segment breast tumors from ultrasound images.

METHODS

PBNet consists of a multilevel global perception module (MGPM) and a boundary guided module (BGM). MGPM models long-range spatial dependencies by fusing both intra- and inter-level semantic information to enhance tumor recognition. In BGM, the tumor boundaries are extracted from the high-level semantic maps using the dilation and erosion effects of max pooling; such boundaries are then used to guide the fusion of low- and high-level features. Additionally, a multi-level boundary-enhanced segmentation (BS) loss is introduced to improve boundary segmentation performance. To evaluate the effectiveness of the proposed method, we compared it with state-of-the-art methods on two datasets, one publicly available datasets BUSI containing 780 images and one in-house dataset containing 995 images. To verify the robustness of each method, a 5-fold cross-validation method was used to train and test the models. Dice score (Dice), Jaccard coefficients (Jac), Hausdorff Distance (HD), Sensitivity (Sen), and specificity(Spe) were used to evaluate the segmentation performance quantitatively. The Wilcoxon test and Benjamini-Hochberg false discovery rate (FDR) multi-comparison correction were then performed to assess whether the proposed method presents statistically significant performance (p ≤ 0.05 $p\le 0.05$ ) difference comparing with existing methods. In addition, to comprehensively demonstrate the difference between different methods, the Cohen's d effect size and compound p-value (c-Pvalue) obtained with Fisher's method were also calculated.

RESULTS

On the BUSI dataset, the mean Dice and Sen of PBNet was increased by 0.93% (p ≤ 0.01 $p\le 0.01$ ) and 1.42% (p ≤ 0.05 $p\le 0.05$ ), respectively, comparing against the corresponding suboptimal methods. On the in-house dataset, PBNet improved Dice, Jac and Spe by approximately 0.86% (p ≤ 0.01 $p\le 0.01$ ), 1.42% (p ≤ 0.01 $p\le 0.01$ ), and 0.1%, respectively, and reduced HD by 1.7% (p ≤ 0.01 $p\le 0.01$ ) compared to the sub-optimal model. Comprehensively, in terms of all the evaluation metics, the performance of the proposed method significantly (c-Pvalue≤ 0.05 $\le 0.05$ ) outperformed the others but the effect size was smaller than 0.2. Ablation results confirmed that MGPM is effective in distinguishing non-enhanced tumors, while BGM and BS loss are beneficial for refining tumor segmentation contours.

CONCLUSIONS

The proposed PBNet allows us to segment the non-enhanced breast lesions from ultrasound images with more accurate boundaries, which provides a valuable means for the subsequent clinical applications.

AI in Breast Cancer Imaging: An Update and Future Trends

Breast cancer is one of the most common types of cancer affecting women worldwide. Artificial intelligence (AI) is transforming breast cancer imaging by enhancing diagnostic capabilities across multiple imaging modalities including mammography, digital breast tomosynthesis, ultrasound, magnetic resonance imaging, and nuclear medicines techniques. AI is being applied to diverse tasks such as breast lesion detection and classification, risk stratification, molecular subtyping, gene mutation status prediction, and treatment response assessment, with emerging research demonstrating performance levels comparable to or potentially exceeding those of radiologists. The large foundation models are showing remarkable potential in different breast cancer imaging tasks. Self-supervised learning gives an insight into data inherent correlation, and federated learning is an alternative way to maintain data privacy. While promising results have been obtained so far, data standardization from source, large-scale annotated multimodal datasets, and extensive prospective clinical trials are still needed to fully explore and validate deep learning's clinical utility and address the legal and ethical considerations, which will ultimately determine its widespread adoption in breast cancer care. We hereby provide a review of the most up-to-date knowledge on AI in breast cancer imaging.

Large-scale benchmarking and boosting transfer learning for medical image analysis

Transfer learning, particularly fine-tuning models pretrained on photographic images to medical images, has proven indispensable for medical image analysis. There are numerous models with distinct architectures pretrained on various datasets using different strategies. But, there is a lack of up-to-date large-scale evaluations of their transferability to medical imaging, posing a challenge for practitioners in selecting the most proper pretrained models for their tasks at hand. To fill this gap, we conduct a comprehensive systematic study, focusing on (i) benchmarking numerous conventional and modern convolutional neural network (ConvNet) and vision transformer architectures across various medical tasks; (ii) investigating the impact of fine-tuning data size on the performance of ConvNets compared with vision transformers in medical imaging; (iii) examining the impact of pretraining data granularity on transfer learning performance; (iv) evaluating transferability of a wide range of recent self-supervised methods with diverse training objectives to a variety of medical tasks across different modalities; and (v) delving into the efficacy of domain-adaptive pretraining on both photographic and medical datasets to develop high-performance models for medical tasks. Our large-scale study (∼5,000 experiments) yields impactful insights: (1) ConvNets demonstrate higher transferability than vision transformers when fine-tuning for medical tasks; (2) ConvNets prove to be more annotation efficient than vision transformers when fine-tuning for medical tasks; (3) Fine-grained representations, rather than high-level semantic features, prove pivotal for fine-grained medical tasks; (4) Self-supervised models excel in learning holistic features compared with supervised models; and (5) Domain-adaptive pretraining leads to performant models via harnessing knowledge acquired from ImageNet and enhancing it through the utilization of readily accessible expert annotations associated with medical datasets. As open science, all codes and pretrained models are available at GitHub.com/JLiangLab/BenchmarkTransferLearning (Version 2).

Multimodal Distillation Pre-Training Model for Ultrasound Dynamic Images Annotation

With the development of medical technology, ultrasonography has become an important diagnostic method in doctors' clinical work. However, compared with the static medical image processing work such as CT, MRI, etc., which has more research bases, ultrasonography is a dynamic medical image similar to video, which is captured and generated by a real-time moving probe, so how to deal with the video data in the medical field and cross modal extraction of the textual semantics in the medical video is a difficult problem that needs to be researched. For this reason, this paper proposes a pre-training model of multimodal distillation and fusion coding for processing the semantic relationship between ultrasound dynamic Images and text. Firstly, by designing the fusion encoder, the visual geometric features of tissues and organs in ultrasound dynamic images, the overall visual appearance descriptive features and the named entity linguistic features are fused to form a unified visual-linguistic feature, so that the model obtains richer visual, linguistic cues aggregation and alignment ability. Then, the pre-training model is augmented by multimodal knowledge distillation to improve the learning ability of the model. The final experimental results on multiple datasets show that the multimodal distillation pre-training model generally improves the fusion ability of various types of features in ultrasound dynamic images, and realizes the automated and accurate annotation of ultrasound dynamic images.

A Dirichlet Distribution-Based Complex Ensemble Approach for Breast Cancer Classification from Ultrasound Images with Transfer Learning and Multiphase Spaced Repetition Method

Breast ultrasound is a useful and rapid diagnostic tool for the early detection of breast cancer. Artificial intelligence-supported computer-aided decision systems, which assist expert radiologists and clinicians, provide reliable and rapid results. Deep learning methods and techniques are widely used in the field of health for early diagnosis, abnormality detection, and disease diagnosis. Therefore, in this study, a deep ensemble learning model based on Dirichlet distribution using pre-trained transfer learning models for breast cancer classification from ultrasound images is proposed. In the study, experiments were conducted using the Breast Ultrasound Images Dataset (BUSI). The dataset, which had an imbalanced class structure, was balanced using data augmentation techniques. DenseNet201, InceptionV3, VGG16, and ResNet152 models were used for transfer learning with fivefold cross-validation. Statistical analyses, including the ANOVA test and Tukey HSD test, were applied to evaluate the model's performance and ensure the reliability of the results. Additionally, Grad-CAM (Gradient-weighted Class Activation Mapping) was used for explainable AI (XAI), providing visual explanations of the deep learning model's decision-making process. The spaced repetition method, commonly used to improve the success of learners in educational sciences, was adapted to artificial intelligence in this study. The results of training with transfer learning models were used as input for further training, and spaced repetition was applied using previously learned information. The use of the spaced repetition method led to increased model success and reduced learning times. The weights obtained from the trained models were input into an ensemble learning system based on Dirichlet distribution with different variations. The proposed model achieved 99.60% validation accuracy on the dataset, demonstrating its effectiveness in breast cancer classification.

MED-NCA: Bio-inspired medical image segmentation

The reliance on computationally intensive U-Net and Transformer architectures significantly limits their accessibility in low-resource environments, creating a technological divide that hinders global healthcare equity, especially in medical diagnostics and treatment planning. This divide is most pronounced in low- and middle-income countries, primary care facilities, and conflict zones. We introduced MED-NCA, Neural Cellular Automata (NCA) based segmentation models characterized by their low parameter count, robust performance, and inherent quality control mechanisms. These features drastically lower the barriers to high-quality medical image analysis in resource-constrained settings, allowing the models to run efficiently on hardware as minimal as a Raspberry Pi or a smartphone. Building upon the foundation laid by MED-NCA, this paper extends its validation across eight distinct anatomies, including the hippocampus and prostate (MRI, 3D), liver and spleen (CT, 3D), heart and lung (X-ray, 2D), breast tumor (Ultrasound, 2D), and skin lesion (Image, 2D). Our comprehensive evaluation demonstrates the broad applicability and effectiveness of MED-NCA in various medical imaging contexts, matching the performance of two magnitudes larger UNet models. Additionally, we introduce NCA-VIS, a visualization tool that gives insight into the inference process of MED-NCA and allows users to test its robustness by applying various artifacts. This combination of efficiency, broad applicability, and enhanced interpretability makes MED-NCA a transformative solution for medical image analysis, fostering greater global healthcare equity by making advanced diagnostics accessible in even the most resource-limited environments.

Variational mode directed deep learning framework for breast lesion classification using ultrasound imaging

Breast cancer is the most prevalent cancer and the second cause of cancer related death among women in the United States. Accurate and early detection of breast cancer can reduce the number of mortalities. Recent works explore deep learning techniques with ultrasound for detecting malignant breast lesions. However, the lack of explanatory features, need for segmentation, and high computational complexity limit their applicability in this detection. Therefore, we propose a novel ultrasound-based breast lesion classification framework that utilizes two-dimensional variational mode decomposition (2D-VMD) which provides self-explanatory features for guiding a convolutional neural network (CNN) with mixed pooling and attention mechanisms. The visual inspection of these features demonstrates their explainability in terms of discriminative lesion-specific boundary and texture in the decomposed modes of benign and malignant images, which further guide the deep learning network for enhanced classification. The proposed framework can classify the lesions with accuracies of 98% and 93% in two public breast ultrasound datasets and 89% in an in-house dataset without having to segment the lesions unlike existing techniques, along with an optimal trade-off between the sensitivity and specificity. 2D-VMD improves the areas under the receiver operating characteristics and precision-recall curves by 5% and 10% respectively. The proposed method achieves relative improvement of 14.47%(8.42%) (mean (SD)) in accuracy over state-of-the-art methods for one public dataset, and 5.75%(4.52%) for another public dataset with comparable performance to two existing methods. Further, it is computationally efficient with a reduction of [Formula: see text] in floating point operations as compared to existing methods.

Enhanced breast cancer diagnosis using modified InceptionNet-V3: a deep learning approach for ultrasound image classification

Introduction

Breast cancer (BC) is a malignant neoplasm that originates in the mammary gland's cellular structures and remains one of the most prevalent cancers among women, ranking second in cancer-related mortality after lung cancer. Early and accurate diagnosis is crucial due to the heterogeneous nature of breast cancer and its rapid progression. However, manual detection and classification are often time-consuming and prone to errors, necessitating the development of automated and reliable diagnostic approaches.

Methods

Recent advancements in deep learning have significantly improved medical image analysis, demonstrating superior predictive performance in breast cancer detection using ultrasound images. Despite these advancements, training deep learning models from scratch can be computationally expensive and data-intensive. Transfer learning, leveraging pre-trained models on large-scale datasets, offers an effective solution to mitigate these challenges. In this study, we investigate and compare multiple deep-learning models for breast cancer classification using transfer learning. The evaluated architectures include modified InceptionV3, GoogLeNet, ShuffleNet, AlexNet, VGG-16, and SqueezeNet. Additionally, we propose a deep neural network model that integrates features from modified InceptionV3 to further enhance classification performance.

Results

The experimental results demonstrate that the modified InceptionV3 model achieves the highest classification accuracy of 99.10%, with a recall of 98.90%, precision of 99.00%, and an F1-score of 98.80%, outperforming all other evaluated models on the given datasets.

Discussion

The achieved findings underscore the potential of the proposed approach in enhancing diagnostic precision and confirm the superiority of the modified InceptionV3 model in breast cancer classification tasks.

A comprehensive and easy-to-use multi-domain multi-task medical imaging meta-dataset

While the field of medical image analysis has undergone a transformative shift with the integration of machine learning techniques, the main challenge of these techniques is often the scarcity of large, diverse, and well-annotated datasets. Medical images vary in format, size, and other parameters and therefore require extensive preprocessing and standardization, for usage in machine learning. Addressing these challenges, we introduce the Medical Imaging Meta-Dataset (MedIMeta), a novel multi-domain, multi-task meta-dataset. MedIMeta contains 19 medical imaging datasets spanning 10 different domains and encompassing 54 distinct medical tasks, all of which are standardized to the same format and readily usable in PyTorch or other ML frameworks. We perform a technical validation of MedIMeta, demonstrating its utility through fully supervised and cross-domain few-shot learning baselines.

Automatic joint segmentation and classification of breast ultrasound images via multi-task learning with object contextual attention

The segmentation and classification of breast ultrasound (BUS) images are crucial for the early diagnosis of breast cancer and remain a key focus in BUS image processing. Numerous machine learning and deep learning algorithms have shown their effectiveness in the segmentation and diagnosis of BUS images. In this work, we propose a multi-task learning network with an object contextual attention module (MTL-OCA) for the segmentation and classification of BUS images. The proposed method utilizes the object contextual attention module to capture pixel-region relationships, enhancing the quality of segmentation masks. For classification, the model leverages high-level features extracted from unenhanced segmentation masks to improve accuracy. Cross-validation on a public BUS dataset demonstrates that MTL-OCA outperforms several current state-of-the-art methods, achieving superior results in both classification and segmentation tasks.

M2UNet: Multi-Scale Feature Acquisition and Multi-Input Edge Supplement Based on UNet for Efficient Segmentation of Breast Tumor in Ultrasound Images

Background/Objectives: The morphological characteristics of breast tumors play a crucial role in the preliminary diagnosis of breast cancer. However, malignant tumors often exhibit rough, irregular edges and unclear, boundaries in ultrasound images. Additionally, variations in tumor size, location, and shape further complicate the accurate segmentation of breast tumors from ultrasound images. Methods: For these difficulties, this paper introduces a breast ultrasound tumor segmentation network comprising a multi-scale feature acquisition (MFA) module and a multi-input edge supplement (MES) module. The MFA module effectively incorporates dilated convolutions of various sizes in a serial-parallel fashion to capture tumor features at diverse scales. Then, the MES module is employed to enhance the output of each decoder layer by supplementing edge information. This process aims to improve the overall integrity of tumor boundaries, contributing to more refined segmentation results. Results: The mean Dice (mDice), Pixel Accuracy (PA), Intersection over Union (IoU), Recall, and Hausdorff Distance (HD) of this method for the publicly available breast ultrasound image (BUSI) dataset were 79.43%, 96.84%, 83.00%, 87.17%, and 19.71 mm, respectively, and for the dataset of Fujian Cancer Hospital, 90.45%, 97.55%, 90.08%, 93.72%, and 11.02 mm, respectively. In the BUSI dataset, compared to the original UNet, the Dice for malignant tumors increased by 14.59%, and the HD decreased by 17.13 mm. Conclusions: Our method is capable of accurately segmenting breast tumor ultrasound images, which provides very valuable edge information for subsequent diagnosis of breast cancer. The experimental results show that our method has made substantial progress in improving accuracy.

Manifold Topological Deep Learning for Biomedical Data

Recently, topological deep learning (TDL), which integrates algebraic topology with deep neural networks, has achieved tremendous success in processing point-cloud data, emerging as a promising paradigm in data science. However, TDL has not been developed for data on differentiable manifolds, including images, due to the challenges posed by differential topology. We address this challenge by introducing manifold topological deep learning (MTDL) for the first time. To highlight the power of Hodge theory rooted in differential topology, we consider a simple convolutional neural network (CNN) in MTDL. In this novel framework, original images are represented as smooth manifolds with vector fields that are decomposed into three orthogonal components based on Hodge theory. These components are then concatenated to form an input image for the CNN architecture. The performance of MTDL is evaluated using the MedMNIST v2 benchmark database, which comprises 717,287 biomedical images from eleven 2D and six 3D datasets. MTDL significantly outperforms other competing methods, extending TDL to a wide range of data on smooth manifolds.

MM-UKAN++: A Novel Kolmogorov-Arnold Network-Based U-Shaped Network for Ultrasound Image Segmentation

Ultrasound (US) imaging is an important and commonly used medical imaging modality. Accurate and fast automatic segmentation of regions of interest (ROIs) in US images is essential for enhancing the efficiency of clinical and robot-assisted diagnosis. However, US images suffer from low contrast, fuzzy boundaries, and significant scale variations in ROIs. Existing convolutional neural network (CNN)-based and transformer-based methods struggle with model efficiency and explainability. To address these challenges, we introduce MM-UKAN++, a novel U-shaped network based on Kolmogorov-Arnold networks (KANs). MM-UKAN++ leverages multilevel KAN layers as the encoder and decoder within the U-network architecture and incorporates an innovative multidimensional attention mechanism to refine skip connections by weighting features from frequency-channel and spatial perspectives. In addition, the network effectively integrates multiscale information, fusing outputs from different scale decoders to generate precise segmentation predictions. MM-UKAN++ achieves higher segmentation accuracy with lower computational cost and outperforms other mainstream methods on several open-source datasets for US image segmentation tasks, including achieving 69.42% IoU, 81.30% Dice, and 3.31 mm HD in the BUSI dataset with 3.17 G floating point of operations (FLOPs) and 9.90 M parameters. The excellent performance on our automatic carotid artery US scanning and diagnostic system further proves the speed and accuracy of MM-UKAN++. Besides, the good performance in other medical image segmentation tasks reveals the promising applications of MM-UKAN++. The code is available on GitHub.

Vital Characteristics Cellular Neural Network (VCeNN) for Melanoma Lesion Segmentation: A Biologically Inspired Deep Learning Approach

Cutaneous melanoma is a highly lethal form of cancer. Developing a medical image segmentation model capable of accurately delineating melanoma lesions with high robustness and generalization presents a formidable challenge. This study draws inspiration from cellular functional characteristics and natural selection, proposing a novel medical segmentation model named the vital characteristics cellular neural network. This model incorporates vital characteristics observed in multicellular organisms, including memory, adaptation, apoptosis, and division. Memory module enables the network to rapidly adapt to input data during the early stages of training, accelerating model convergence. Adaptation module allows neurons to select the appropriate activation function based on varying environmental conditions. Apoptosis module reduces the risk of overfitting by pruning neurons with low activation values. Division module enhances the network's learning capacity by duplicating neurons with high activation values. Experimental evaluations demonstrate the efficacy of this model in enhancing the performance of neural networks for medical image segmentation. The proposed method achieves outstanding results across numerous publicly available datasets, indicating its potential to contribute significantly to the field of medical image analysis and facilitating accurate and efficient segmentation of medical imagery. The proposed method achieves outstanding results across numerous publicly available datasets, with an F1 score of 0.901, Intersection over Union of 0.841, and Dice coefficient of 0.913, indicating its potential to contribute significantly to the field of medical image analysis and facilitating accurate and efficient segmentation of medical imagery.

Multi-scale multi-object semi-supervised consistency learning for ultrasound image segmentation

Manual annotation of ultrasound images relies on expert knowledge and requires significant time and financial resources. Semi-supervised learning (SSL) exploits large amounts of unlabeled data to improve model performance under limited labeled data. However, it faces two challenges: fusion of contextual information at multiple scales and bias of spatial information between multiple objects. We propose a consistency learning-based multi-scale multi-object (MSMO) semi-supervised framework for ultrasound image segmentation. MSMO addresses these challenges by employing a contextual-aware encoder coupled with a multi-object semantic calibration and fusion decoder. First, the encoder extracts multi-scale multi-objects context-aware features, and introduces attention module to refine the feature map and enhance channel information interaction. Then, the decoder uses HConvLSTM to calibrate the output features of the current object by using the hidden state of the previous object, and recursively fuses multi-object semantics at different scales. Finally, MSMO further reduces variations among multiple decoders in different perturbations through consistency constraints, thereby producing consistent predictions for highly uncertain areas. Extensive experiments show that proposed MSMO outperforms the SSL baseline on four benchmark datasets, whether for single-object or multi-object ultrasound image segmentation. MSMO significantly reduces the burden of manual analysis of ultrasound images and holds great potential as a clinical tool. The source code is accessible to the public at: https://github.com/lol88/MSMO.

A novel framework for segmentation of small targets in medical images

Medical image segmentation represents a pivotal and intricate procedure in the domain of medical image processing and analysis. With the progression of artificial intelligence in recent years, the utilization of deep learning techniques for medical image segmentation has witnessed escalating popularity. Nevertheless, the intricate nature of medical image poses challenges on the segmentation of diminutive targets is still in its early stages. Current networks encounter difficulties in addressing the segmentation of exceedingly small targets, especially when the number of training samples is limited. To overcome this constraint, we have implemented a proficient strategy to enhance lesion images containing small targets and constrained samples. We introduce a segmentation framework termed STS-Net, specifically designed for small target segmentation. This framework leverages the established capacity of convolutional neural networks to acquire effective image representations. The proposed STS-Net network adopts a ResNeXt50-32x4d architecture as the encoder, integrating attention mechanisms during the encoding phase to amplify the feature representation capabilities of the network. We evaluated the proposed network on four publicly available datasets. Experimental results underscore the superiority of our approach in the domain of medical image segmentation, particularly for small target segmentation. The codes are available at https://github.com/zlxokok/STSNet .

Adaptive image encryption approach using an enhanced swarm intelligence algorithm

Chaos-based encryption methods have gained popularity due to the unique properties of chaos. The performance of chaos-based encryption methods is highly impacted by the values of initial and control parameters. Therefore, this work proposes Iterative Cosine operator-based Hippopotamus Optimization (ICO-HO) to select optimal parameters for chaotic maps, which is further used to design an adaptive image encryption approach. ICO-HO algorithm improves the Hippopotamus Optimization (HO) by integrating a new phase (Phase 4) to update the position of the hippopotamus. ICO-HO updates the position of hippopotamuses using ICO and opposition-based learning, which enhances the exploration and exploitation capabilities of the HO algorithm. ICO-HO algorithm's better performance is signified by the Friedman mean rank test applied to mean values obtained on the CEC-2017 benchmark functions. The ICO-HO algorithm is utilized to optimize the parameters of PWLCM and PWCM chaotic maps to generate a secret key in the confusion and diffusion phases of image encryption. The performance of the proposed encryption approach is evaluated on grayscale, RGB, and hyperspectral medical images of different modalities, bit depth, and sizes. Different analyses, such as visual analysis, statistical attack analysis, differential attack analysis, and quantitative analysis, have been utilized to assess the effectiveness of the proposed encryption approach. The higher NPCR and UACI values, i.e., 99.60% and 33.40%, respectively, ensure security against differential attacks. Furthermore, the proposed encryption approach is compared with five state-of-the-art encryption techniques available in the literature and six similar metaheuristic techniques using NPCR, UACI, entropy, and correlation coefficient. The proposed methods exhibit 7.9995 and 15.8124 entropy values on 8-bit and 16-bit images, respectively, which is better than all other stated methods, resulting in improved image encryption with high randomness.

Dual branch segment anything model-transformer fusion network for accurate breast ultrasound image segmentation

BACKGROUND

Precise and rapid ultrasound-based breast cancer diagnosis is essential for effective treatment. However, existing ultrasound image segmentation methods often fail to capture both global contextual features and fine-grained boundary details.

PURPOSE

This study proposes a dual-branch network architecture that combines the Swin Transformer and Segment Anything Model (SAM) to enhance breast ultrasound image (BUSI) segmentation accuracy and reliability.

METHODS

Our network integrates the global attention mechanism of the Swin Transformer with fine-grained boundary detection from SAM through a multi-stage feature fusion module. We evaluated our method against state-of-the-art methods on two datasets: the Breast Ultrasound Images dataset from Wuhan University (BUSI-WHU), which contains 927 images (560 benign and 367 malignant) with ground truth masks annotated by radiologists, and the public BUSI dataset. Performance was evaluated using mean Intersection-over-Union (mIoU), 95th percentile Hausdorff Distance (HD95) and Dice Similarity coefficients, with statistical significance assessed using two-tailed independent t-tests with Holm-Bonferroni correction (α = 0.05 $\alpha =0.05$ ).

RESULTS

On our proposed dataset, the network achieved a mIoU of 90.82% and a HD95 of 23.50 pixels, demonstrating significant improvements over current state-of-the-art methods with effect sizes for mIoU ranging from 0.38 to 0.61 (p < $<$ 0.05). On the BUSI dataset, the network achieved a mIoU of 82.83% and a HD95 of 71.13 pixels, demonstrating comparable improvements with effect sizes for mIoU ranging from 0.45 to 0.58 (p < $<$ 0.05).

CONCLUSIONS

Our dual-branch network leverages the complementary strengths of Swin Transformer and SAM through a fusion mechanism, demonstrating superior breast ultrasound segmentation performance. Our code is publicly available at https://github.com/Skylanding/DSATNet.

Rethinking model prototyping through the MedMNIST+ dataset collection

The integration of deep learning based systems in clinical practice is often impeded by challenges rooted in limited and heterogeneous medical datasets. In addition, the field has increasingly prioritized marginal performance gains on a few, narrowly scoped benchmarks over clinical applicability, slowing down meaningful algorithmic progress. This trend often results in excessive fine-tuning of existing methods on selected datasets rather than fostering clinically relevant innovations. In response, this work introduces a comprehensive benchmark for the MedMNIST+ dataset collection, designed to diversify the evaluation landscape across several imaging modalities, anatomical regions, classification tasks and sample sizes. We systematically reassess commonly used Convolutional Neural Networks (CNNs) and Vision Transformer (ViT) architectures across distinct medical datasets, training methodologies, and input resolutions to validate and refine existing assumptions about model effectiveness and development. Our findings suggest that computationally efficient training schemes and modern foundation models offer viable alternatives to costly end-to-end training. Additionally, we observe that higher image resolutions do not consistently improve performance beyond a certain threshold. This highlights the potential benefits of using lower resolutions, particularly in prototyping stages, to reduce computational demands without sacrificing accuracy. Notably, our analysis reaffirms the competitiveness of CNNs compared to ViTs, emphasizing the importance of comprehending the intrinsic capabilities of different architectures. Finally, by establishing a standardized evaluation framework, we aim to enhance transparency, reproducibility, and comparability within the MedMNIST+ dataset collection as well as future research. Code is available at (https://github.com/sdoerrich97/rethinking-model-prototyping-MedMNISTPlus).

Multi-task interaction learning for accurate segmentation and classification of breast tumors in ultrasound images

Objective.In breast diagnostic imaging, the morphological variability of breast tumors and the inherent ambiguity of ultrasound images pose significant challenges. Moreover, multi-task computer-aided diagnosis systems in breast imaging may overlook inherent relationships between pixel-wise segmentation and categorical classification tasks.Approach.In this paper, we propose a multi-task learning network with deep inter-task interactions that exploits the inherently relations between two tasks. First, we fuse self-task attention and cross-task attention mechanisms to explore the two types of interaction information, location and semantic, between tasks. In addition, a feature aggregation block is developed based on the channel attention mechanism, which reduces the semantic differences between the decoder and the encoder. To exploit inter-task further, our network uses an circle training strategy to refine heterogeneous feature with the help of segmentation maps obtained from previous training.Main results.The experimental results show that our method achieved excellent performance on the BUSI and BUS-B datasets, with DSCs of 81.95% and 86.41% for segmentation tasks, and F1 scores of 82.13% and 69.01% for classification tasks, respectively.Significance.The proposed multi-task interaction learning not only enhances the performance of all tasks related to breast tumor segmentation and classification but also promotes research in multi-task learning, providing further insights for clinical applications.

Evaluating text and visual diagnostic capabilities of large language models on questions related to the Breast Imaging Reporting and Data System Atlas 5th edition

PURPOSE

This study aimed to evaluate the performance of large language models (LLMs) and multimodal LLMs in interpreting the Breast Imaging Reporting and Data System (BI-RADS) categories and providing clinical management recommendations for breast radiology in text-based and visual questions.

METHODS

This cross-sectional observational study involved two steps. In the first step, we compared ten LLMs (namely ChatGPT 4o, ChatGPT 4, ChatGPT 3.5, Google Gemini 1.5 Pro, Google Gemini 1.0, Microsoft Copilot, Perplexity, Claude 3.5 Sonnet, Claude 3 Opus, and Claude 3 Opus 200K), general radiologists, and a breast radiologist using 100 text-based multiple-choice questions (MCQs) related to the BI-RADS Atlas 5th edition. In the second step, we assessed the performance of five multimodal LLMs (ChatGPT 4o, ChatGPT 4V, Claude 3.5 Sonnet, Claude 3 Opus, and Google Gemini 1.5 Pro) in assigning BI-RADS categories and providing clinical management recommendations on 100 breast ultrasound images. The comparison of correct answers and accuracy by question types was analyzed using McNemar's and chi-squared tests. Management scores were analyzed using the Kruskal- Wallis and Wilcoxon tests.

RESULTS

Claude 3.5 Sonnet achieved the highest accuracy in text-based MCQs (90%), followed by ChatGPT 4o (89%), outperforming all other LLMs and general radiologists (78% and 76%) (P < 0.05), except for the Claude 3 Opus models and the breast radiologist (82%) (P > 0.05). Lower-performing LLMs included Google Gemini 1.0 (61%) and ChatGPT 3.5 (60%). Performance across different categories of showed no significant variation among LLMs or radiologists (P > 0.05). For breast ultrasound images, Claude 3.5 Sonnet achieved 59% accuracy, significantly higher than other multimodal LLMs (P < 0.05). Management recommendations were evaluated using a 3-point Likert scale, with Claude 3.5 Sonnet scoring the highest (mean: 2.12 ± 0.97) (P < 0.05). Accuracy varied significantly across BI-RADS categories, except Claude 3 Opus (P < 0.05). Gemini 1.5 Pro failed to answer any BI-RADS 5 questions correctly. Similarly, ChatGPT 4V failed to answer any BI-RADS 1 questions correctly, making them the least accurate in these categories (P < 0.05).

CONCLUSION

Although LLMs such as Claude 3.5 Sonnet and ChatGPT 4o show promise in text-based BI-RADS assessments, their limitations in visual diagnostics suggest they should be used cautiously and under radiologists' supervision to avoid misdiagnoses.

CLINICAL SIGNIFICANCE

This study demonstrates that while LLMs exhibit strong capabilities in text-based BI-RADS assessments, their visual diagnostic abilities are currently limited, necessitating further development and cautious application in clinical practice.

Ultrasound-based deep learning radiomics for multi-stage assisted diagnosis in reducing unnecessary biopsies of BI-RADS 4A lesions

Background

Even with the Breast Imaging Reporting and Data System (BI-RADS) guiding risk stratification on ultrasound (US) images, inconsistencies in diagnostic accuracy still exist, leading patients being subjected to unnecessary biopsies in clinical practice. This study investigated the construction of deep learning radiomics (DLR) models to improve the diagnostic consistency and reduce the unnecessary biopsies for BI-RADS 4A lesions.

Methods

A total of 746 patients with breast lesions were enrolled in this retrospective study. Two DLR models based on US images and clinical variables were developed to conduct breast lesion risk re-stratification as BI-RADS 3 or lower and BI-RADS 4A or higher (DLR_LH), while simultaneously identifying BI-RADS 4A lesions with low malignancy probabilities to avoid unnecessary biopsy (DLR_BM). A three-round reader study with a two-stage artificial intelligence (AI)-assisted diagnosis process was performed to verify the assistive capability and practical benefits of the models in clinical applications.

Results

The DLR_LH model achieved areas under the receiver operating characteristic curve (AUCs) of 0.963 and 0.889 with sensitivities of 92.0% and 83.3%, in the internal and external validation cohorts, respectively. The DLR_BM model exhibited AUCs of 0.977 and 0.942, with sensitivities of 94.1% and 86.4%, respectively. Both models were evaluated using integrated features of US images and clinical variables. Ultimately, 27.7% of BI-RADS 4A lesions avoided unnecessary biopsies. In the three-round reader study, all readers achieved significantly higher diagnostic accuracy and specificity, while maintaining outstanding sensitivity comparable to human experts, both before and after model assistance (P<0.05). These findings demonstrate the positive impact of the DLR models in assisting radiologists to enhance their diagnostic capabilities.

Conclusions

The models performed well in breast US imaging interpretation and BI-RADS risk re-stratification, and demonstrated potential in reducing unnecessary biopsies of BI-RADS 4A lesions, indicating the promising applicability of the DLR models in clinical diagnosis.

Rad4XCNN: A new agnostic method for post-hoc global explanation of CNN-derived features by means of Radiomics

BACKGROUND AND OBJECTIVE

In recent years, machine learning-based clinical decision support systems (CDSS) have played a key role in the analysis of several medical conditions. Despite their promising capabilities, the lack of transparency in AI models poses significant challenges, particularly in medical contexts where reliability is a mandatory aspect. However, it appears that explainability is inversely proportional to accuracy. For this reason, achieving transparency without compromising predictive accuracy remains a key challenge.

METHODS

This paper presents a novel method, namely Rad4XCNN, to enhance the predictive power of CNN-derived features with the inherent interpretability of radiomic features. Rad4XCNN diverges from conventional methods based on saliency maps, by associating intelligible meaning to CNN-derived features by means of Radiomics, offering new perspectives on explanation methods beyond visualization maps.

RESULTS

Using a breast cancer classification task as a case study, we evaluated Rad4XCNN on ultrasound imaging datasets, including an online dataset and two in-house datasets for internal and external validation. Some key results are: (i) CNN-derived features guarantee more robust accuracy when compared against ViT-derived and radiomic features; (ii) conventional visualization map methods for explanation present several pitfalls; (iii) Rad4XCNN does not sacrifice model accuracy for their explainability; (iv) Rad4XCNN provides a global explanation enabling the physician to extract global insights and findings.

CONCLUSIONS

Our method can mitigate some concerns related to the explainability-accuracy trade-off. This study highlighted the importance of proposing new methods for model explanation without affecting their accuracy.

SAM-MedUS: a foundational model for universal ultrasound image segmentation

Purpose

Segmentation of ultrasound images for medical diagnosis, monitoring, and research is crucial, and although existing methods perform well, they are limited by specific organs, tumors, and image devices. Applications of the Segment Anything Model (SAM), such as SAM-med2d, use a large number of medical datasets that contain only a small fraction of the ultrasound medical images.

Approach

In this work, we proposed a SAM-MedUS model for generic ultrasound image segmentation that utilizes the latest publicly available ultrasound image dataset to create a diverse dataset containing eight site categories for training and testing. We integrated ConvNext V2 and CM blocks in the encoder for better global context extraction. In addition, a boundary loss function is used to improve the segmentation of fuzzy boundaries and low-contrast ultrasound images.

Results

Experimental results show that SAM-MedUS outperforms recent methods on multiple ultrasound datasets. For the more easily datasets such as the adult kidney, it achieves 87.93% IoU and 93.58% dice, whereas for more complex ones such as the infant vein, IoU and dice reach 62.31% and 78.93%, respectively.

Conclusions

We collected and collated an ultrasound dataset of multiple different site types to achieve uniform segmentation of ultrasound images. In addition, the use of additional auxiliary branches ConvNext V2 and CM block enhances the ability of the model to extract global information and the use of boundary loss allows the model to exhibit robust performance and excellent generalization ability.

A multi-task framework for breast cancer segmentation and classification in ultrasound imaging

BACKGROUND

Ultrasound (US) is a medical imaging modality that plays a crucial role in the early detection of breast cancer. The emergence of numerous deep learning systems has offered promising avenues for the segmentation and classification of breast cancer tumors in US images. However, challenges such as the absence of data standardization, the exclusion of non-tumor images during training, and the narrow view of single-task methodologies have hindered the practical applicability of these systems, often resulting in biased outcomes. This study aims to explore the potential of multi-task systems in enhancing the detection of breast cancer lesions.

METHODS

To address these limitations, our research introduces an end-to-end multi-task framework designed to leverage the inherent correlations between breast cancer lesion classification and segmentation tasks. Additionally, a comprehensive analysis of a widely utilized public breast cancer ultrasound dataset named BUSI was carried out, identifying its irregularities and devising an algorithm tailored for detecting duplicated images in it.

RESULTS

Experiments are conducted utilizing the curated dataset to minimize potential biases in outcomes. Our multi-task framework exhibits superior performance in breast cancer respecting single-task approaches, achieving improvements close to 15% in segmentation and classification. Moreover, a comparative analysis against the state-of-the-art reveals statistically significant enhancements across both tasks.

CONCLUSION

The experimental findings underscore the efficacy of multi-task techniques, showcasing better generalization capabilities when considering all image types: benign, malignant, and non-tumor images. Consequently, our methodology represents an advance towards more general architectures with real clinical applications in the breast cancer field.

QMaxViT-Unet+: A query-based MaxViT-Unet with edge enhancement for scribble-supervised segmentation of medical images

The deployment of advanced deep learning models for medical image segmentation is often constrained by the requirement for extensively annotated datasets. Weakly-supervised learning, which allows less precise labels, has become a promising solution to this challenge. Building on this approach, we propose QMaxViT-Unet+, a novel framework for scribble-supervised medical image segmentation. This framework is built on the U-Net architecture, with the encoder and decoder replaced by Multi-Axis Vision Transformer (MaxViT) blocks. These blocks enhance the model's ability to learn local and global features efficiently. Additionally, our approach integrates a query-based Transformer decoder to refine features and an edge enhancement module to compensate for the limited boundary information in the scribble label. We evaluate the proposed QMaxViT-Unet+ on four public datasets focused on cardiac structures, colorectal polyps, and breast cancer: ACDC, MS-CMRSeg, SUN-SEG, and BUSI. Evaluation metrics include the Dice similarity coefficient (DSC) and the 95th percentile of Hausdorff distance (HD95). Experimental results show that QMaxViT-Unet+ achieves 89.1% DSC and 1.316 mm HD95 on ACDC, 88.4% DSC and 2.226 mm HD95 on MS-CMRSeg, 71.4% DSC and 4.996 mm HD95 on SUN-SEG, and 69.4% DSC and 50.122 mm HD95 on BUSI. These results demonstrate that our method outperforms existing approaches in terms of accuracy, robustness, and efficiency while remaining competitive with fully-supervised learning approaches. This makes it ideal for medical image analysis, where high-quality annotations are often scarce and require significant effort and expense. The code is available at https://github.com/anpc849/QMaxViT-Unet.

Interactively Fusing Global and Local Features for Benign and Malignant Classification of Breast Ultrasound Images

OBJECTIVE

Breast ultrasound (BUS) is used to classify benign and malignant breast tumors, and its automatic classification can reduce subjectivity. However, current convolutional neural networks (CNNs) face challenges in capturing global features, while vision transformer (ViT) networks have limitations in effectively extracting local features. Therefore, this study aimed to develop a deep learning method that enables the interaction and updating of intermediate features between CNN and ViT to achieve high-accuracy BUS image classification.

METHODS

This study introduced the CNN and transformer multi-stage fusion network (CTMF-Net) consisting of two branches: a CNN branch and a transformer branch. The CNN branch employs visual geometry group as its backbone, while the transformer branch utilizes ViT as its base network. Both branches were divided into four stages. At the end of each stage, a proposed feature interaction module facilitated feature interaction and fusion between the two branches. Additionally, the convolutional block attention module was employed to enhance relevant features after each stage of the CNN branch. Extensive experiments were conducted using various state-of-the-art deep-learning classification methods on three public breast ultrasound datasets (SYSU, UDIAT and BUSI).

RESULTS

For the internal validation on SYSU and UDIAT, our proposed method CTMF-Net achieved the highest accuracy of 90.14 ± 0.58% on SYSU and 92.04 ± 4.90% on UDIAT, which showed superior classification performance over other state-of-art networks (p < 0.05). Additionally, for external validation on BUSI, CTMF-Net showed outstanding performance, achieving the highest area under the curve score of 0.8704 when trained on SYSU, marking a 0.0126 improvement over the second-best visual geometry group attention ViT method. Similarly, when applied to UDIAT, CTMF-Net achieved an area under the curve score of 0.8505, surpassing the second-best global context ViT method by 0.0130.

CONCLUSION

Our proposed method, CTMF-Net, outperforms all existing methods and can effectively assist doctors in achieving more accurate classification performance of breast tumors.

Syn-Net: A Synchronous Frequency-Perception Fusion Network for Breast Tumor Segmentation in Ultrasound Images

Accurate breast tumor segmentation in ultrasound images is a crucial step in medical diagnosis and locating the tumor region. However, segmentation faces numerous challenges due to the complexity of ultrasound images, similar intensity distributions, variable tumor morphology, and speckle noise. To address these challenges and achieve precise segmentation of breast tumors in complex ultrasound images, we propose a Synchronous Frequency-perception Fusion Network (Syn-Net). Initially, we design a synchronous dual-branch encoder to extract local and global feature information simultaneously from complex ultrasound images. Secondly, we introduce a novel Frequency- perception Cross-Feature Fusion (FrCFusion) Block, which utilizes Discrete Cosine Transform (DCT) to learn all-frequency features and effectively fuse local and global features while mitigating issues arising from similar intensity distributions. In addition, we develop a Full-Scale Deep Supervision method that not only corrects the influence of speckle noise on segmentation but also effectively guides decoder features towards the ground truth. We conduct extensive experiments on three publicly available ultrasound breast tumor datasets. Comparison with 14 state-of-the-art deep learning segmentation methods demonstrates that our approach exhibits superior sensitivity to different ultrasound images, variations in tumor size and shape, speckle noise, and similarity in intensity distribution between surrounding tissues and tumors. On the BUSI and Dataset B datasets, our method achieves better Dice scores compared to state-of-the-art methods, indicating superior performance in ultrasound breast tumor segmentation.

Attention-based Fusion Network for Breast Cancer Segmentation and Classification Using Multi-modal Ultrasound Images

OBJECTIVE

Breast cancer is one of the most commonly occurring cancers in women. Thus, early detection and treatment of cancer lead to a better outcome for the patient. Ultrasound (US) imaging plays a crucial role in the early detection of breast cancer, providing a cost-effective, convenient, and safe diagnostic approach. To date, much research has been conducted to facilitate reliable and effective early diagnosis of breast cancer through US image analysis. Recently, with the introduction of machine learning technologies such as deep learning (DL), automated lesion segmentation and classification, the identification of malignant masses in US breasts has progressed, and computer-aided diagnosis (CAD) technology is being applied in clinics effectively. Herein, we propose a novel deep learning-based "segmentation + classification" model based on B- and SE-mode images.

METHODS

For the segmentation task, we propose a Multi-Modal Fusion U-Net (MMF-U-Net), which segments lesions by mixing B- and SE-mode information through fusion blocks. After segmenting, the lesion area from the B- and SE-mode images is cropped using a predicted segmentation mask. The encoder part of the pre-trained MMF-U-Net model is then used on the cropped B- and SE-mode breast US images to classify benign and malignant lesions.

RESULTS

The experimental results using the proposed method showed good segmentation and classification scores. The dice score, intersection over union (IoU), precision, and recall are 78.23%, 68.60%, 82.21%, and 80.58%, respectively, using the proposed MMF-U-Net on real-world clinical data. The classification accuracy is 98.46%.

CONCLUSION

Our results show that the proposed method will effectively segment the breast lesion area and can reliably classify the benign from malignant lesions.

[A joint distillation model for the tumor segmentation using breast ultrasound images]

The accurate segmentation of breast ultrasound images is an important precondition for the lesion determination. The existing segmentation approaches embrace massive parameters, sluggish inference speed, and huge memory consumption. To tackle this problem, we propose T 2KD Attention U-Net (dual-Teacher Knowledge Distillation Attention U-Net), a lightweight semantic segmentation method combined double-path joint distillation in breast ultrasound images. Primarily, we designed two teacher models to learn the fine-grained features from each class of images according to different feature representation and semantic information of benign and malignant breast lesions. Then we leveraged the joint distillation to train a lightweight student model. Finally, we constructed a novel weight balance loss to focus on the semantic feature of small objection, solving the unbalance problem of tumor and background. Specifically, the extensive experiments conducted on Dataset BUSI and Dataset B demonstrated that the T 2KD Attention U-Net outperformed various knowledge distillation counterparts. Concretely, the accuracy, recall, precision, Dice, and mIoU of proposed method were 95.26%, 86.23%, 85.09%, 83.59%and 77.78% on Dataset BUSI, respectively. And these performance indexes were 97.95%, 92.80%, 88.33%, 88.40% and 82.42% on Dataset B, respectively. Compared with other models, the performance of this model was significantly improved. Meanwhile, compared with the teacher model, the number, size, and complexity of student model were significantly reduced (2.2×10 6 vs. 106.1×10 6, 8.4 MB vs. 414 MB, 16.59 GFLOPs vs. 205.98 GFLOPs, respectively). Indeedy, the proposed model guarantees the performances while greatly decreasing the amount of computation, which provides a new method for the deployment of clinical medical scenarios.

NMTNet: A Multi-task Deep Learning Network for Joint Segmentation and Classification of Breast Tumors

Segmentation and classification of breast tumors are two critical tasks since they provide significant information for computer-aided breast cancer diagnosis. Combining these tasks leverages their intrinsic relevance to enhance performance, but the variability and complexity of tumor characteristics remain challenging. We propose a novel multi-task deep learning network (NMTNet) for the joint segmentation and classification of breast tumors, which is based on a convolutional neural network (CNN) and U-shaped architecture. It mainly comprises a shared encoder, a multi-scale fusion channel refinement (MFCR) module, a segmentation branch, and a classification branch. First, ResNet18 is used as the backbone network in the encoding part to enhance the feature representation capability. Then, the MFCR module is introduced to enrich the feature depth and diversity. Besides, the segmentation branch combines a lesion region enhancement (LRE) module between the encoder and decoder parts, aiming to capture more detailed texture and edge information of irregular tumors to improve segmentation accuracy. The classification branch incorporates a fine-grained classifier that reuses valuable segmentation information to discriminate between benign and malignant tumors. The proposed NMTNet is evaluated on both ultrasound and magnetic resonance imaging datasets. It achieves segmentation dice scores of 90.30% and 91.50%, and Jaccard indices of 84.70% and 88.10% for each dataset, respectively. And the classification accuracy scores are 87.50% and 99.64% for the corresponding datasets, respectively. Experimental results demonstrate the superiority of NMTNet over state-of-the-art methods on breast tumor segmentation and classification tasks.

Identification of benign and malignant breast nodules on ultrasound: comparison of multiple deep learning models and model interpretation

Background and Purpose

Deep learning (DL) algorithms generally require full supervision of annotating the region of interest (ROI), a process that is both labor-intensive and susceptible to bias. We aimed to develop a weakly supervised algorithm to differentiate between benign and malignant breast tumors in ultrasound images without image annotation.

Methods

We developed and validated the models using two publicly available datasets: breast ultrasound image (BUSI) and GDPH&SYSUCC breast ultrasound datasets. After removing the poor quality images, a total of 3049 images were included, divided into two classes: benign (N = 1320 images) and malignant (N = 1729 images). Weakly-supervised DL algorithms were implemented with four networks (DenseNet121, ResNet50, EffientNetb0, and Vision Transformer) and trained using 2136 unannotated breast ultrasound images. 609 and 304 images were used for validation and test sets, respectively. Diagnostic performances were calculated as the area under the receiver operating characteristic curve (AUC). Using the class activation map to interpret the prediction results of weakly supervised DL algorithms.

Results

The DenseNet121 model, utilizing complete image inputs without ROI annotations, demonstrated superior diagnostic performance in distinguishing between benign and malignant breast nodules when compared to ResNet50, EfficientNetb0, and Vision Transformer models. DenseNet121 achieved the highest AUC, with values of 0.94 on the validation set and 0.93 on the test set, significantly surpassing the performance of the other models across both datasets (all P < 0.05).

Conclusion

The weakly supervised DenseNet121 model developed in this study demonstrated feasibility for ultrasound diagnosis of breast tumor and showed good capabilities in differential diagnosis. This model may help radiologists, especially novice doctors, to improve the accuracy of breast tumor diagnosis using ultrasound.

BUS-UCLM: Breast ultrasound lesion segmentation dataset

This dataset comprises 38 breast ultrasound scans from patients, encompassing a total of 683 images. The scans were conducted using a Siemens ACUSON S2000TM Ultrasound System from 2022 to 2023. The dataset is specifically created for the purpose of segmenting breast lesions, with the goal of identifying the area and contour of the lesion, as well as classifying it as either benign or malignant. The images can be classified into three categories based on their findings: 419 are normal, 174 are benign, and 90 are malignant. The ground truth is given as RGB segmentation masks in individual files, with black indicating normal breast tissue and green and red indicating benign and malignant lesions, respectively. This dataset enables researchers to construct and evaluate machine learning models for identifying between benign and malignant tumours in authentic breast ultrasound images. The segmentation annotations provided by expert radiologists enable accurate model training and evaluation, making this dataset a valuable asset in the field of computer vision and public health.

A feature fusion method based on radiomic features and revised deep features for improving tumor prediction in ultrasound images

BACKGROUND

Radiomic features and deep features are both vitally helpful for the accurate prediction of tumor information in breast ultrasound. However, whether integrating radiomic features and deep features can improve the prediction performance of tumor information is unclear.

METHODS

A feature fusion method based on radiomic features and revised deep features was proposed to predict tumor information. Radiomic features were extracted from the tumor region on ultrasound images, and the optimal radiomic features were subsequently selected based on Gini score. Revised deep features, which were extracted using the revised CNN models integrating prior information, were combined with radiomic features to build a logistic regression classifier for tumor prediction. The performance was evaluated using area under the receiver operating characteristic (ROC) curve (AUC).

RESULTS

The results showed that the proposed feature fusion method (AUC = 0.9845) obtained better prediction performance than that based on radiomic features (AUC = 0.9796) or deep features (AUC = 0.9342).

CONCLUSIONS

Our results demonstrate that the proposed feature fusion framework integrating the radiomic features and revised deep features is an efficient method to improve the prediction performance of tumor information.

Multi-Organ Foundation Model for Universal Ultrasound Image Segmentation With Task Prompt and Anatomical Prior

Semantic segmentation of ultrasound (US) images with deep learning has played a crucial role in computer-aided disease screening, diagnosis and prognosis. However, due to the scarcity of US images and small field of view, resulting segmentation models are tailored for a specific single organ and may lack robustness, overlooking correlations among anatomical structures of multiple organs. To address these challenges, we propose the Multi-Organ FOundation (MOFO) model for universal US image segmentation. The MOFO is optimized jointly from multiple organs across various anatomical regions to overcome the data scarcity and explore correlations between multiple organs. The MOFO extracts organ-invariant representations from US images. Simultaneously, the task prompt is employed to refine organ-specific representations for segmentation predictions. Moreover, the anatomical prior is incorporated to enhance the consistency of the anatomical structures. A multi-organ US database with segmentation labels, comprising 7039 images from 10 organs across various regions of the human body, has been established to develop and evaluate our model. Results demonstrate that the MOFO outperforms single-organ methods in terms of the Dice coefficient, 95% Hausdorff distance and average symmetric surface distance with statistically sufficient margins. Our experiments in multi-organ universal segmentation for US images serve as a pioneering exploration of improving segmentation performance by leveraging semantic and anatomical relationships within US images of multiple organs.

EDSRNet: An Enhanced Decoder Semantic Recovery Network for 2D Medical Image Segmentation

In recent years, with the advancement of medical imaging technology, medical image segmentation has played a key role in assisting diagnosis and treatment planning. Current deep learning-based medical image segmentation methods mainly adopt encoder-decoder architecture design and have received wide attention. However, these methods still have some limitations, including: (1) Existing methods are often influenced by the significant semantic information gap when supplementing features for the decoder. (2) Existing methods do not simultaneously consider global and local information interaction during decoding, resulting in ineffective semantic recovery. Therefore, this paper proposes a novel Enhanced Decoder Semantic Recovery Network to address these challenges. Firstly, the Multi-Level Semantic Fusion (MLSF) module is introduced, which effectively fuses low-level features of the original image, encoder features, high-level features of the deepest network layer, and decoder features, and assigns weights based on semantic gaps. Secondly, the Multiscale Spatial Attention (MSSA) and Cross Convolution Channel Attention (CCCA) modules are employed to obtain richer feature information. Finally, the Global-Local Semantic Recovery (GLSR) module is designed to achieve better semantic recovery. Experiments on public datasets such as BUSI, CVC-ClinicDB, and Kvasir-SEG demonstrate that the proposed model improves IoU compared to suboptimal algorithms by 0.81%, 0.85% and 1.98%, respectively, significantly enhancing the performance of 2D medical image segmentation. This method provides effective technical support for further development in the field of medical image.

HSC-T: B-Ultrasound-to-Elastography Translation via Hierarchical Structural Consistency Learning for Thyroid Cancer Diagnosis

Elastography ultrasound imaging is increasingly important in the diagnosis of thyroid cancer and other diseases, but its reliance on specialized equipment and techniques limits widespread adoption. This paper proposes a novel multimodal ultrasound diagnostic pipeline that expands the application of elastography ultrasound by translating B-ultrasound (BUS) images into elastography images (EUS). Additionally, to address the limitations of existing image-to-image translation methods, which struggle to effectively model inter-sample variations and accurately capture regional-scale structural consistency, we propose a BUS-to-EUS translation method based on hierarchical structural consistency. By incorporating domain-level, sample-level, patch-level, and pixel-level constraints, our approach guides the model in learning a more precise mapping from BUS to EUS, thereby enhancing diagnostic accuracy. Experimental results demonstrate that the proposed method significantly improves the accuracy of BUS-to-EUS translation on the MTUSI dataset and that the generated elastography images enhance nodule diagnostic accuracy compared to solely using BUS images on the STUSI and the BUSI datasets. This advancement highlights the potential for broader application of elastography in clinical practice.

BCT-Net: semantic-guided breast cancer segmentation on BUS

Accurately and swiftly segmenting breast tumors is significant for cancer diagnosis and treatment. Ultrasound imaging stands as one of the widely employed methods in clinical practice. However, due to challenges such as low contrast, blurred boundaries, and prevalent shadows in ultrasound images, tumor segmentation remains a daunting task. In this study, we propose BCT-Net, a network amalgamating CNN and transformer components for breast tumor segmentation. BCT-Net integrates a dual-level attention mechanism to capture more features and redefines the skip connection module. We introduce the utilization of a classification task as an auxiliary task to impart additional semantic information to the segmentation network, employing supervised contrastive learning. A hybrid objective loss function is proposed, which combines pixel-wise cross-entropy, binary cross-entropy, and supervised contrastive learning loss. Experimental results demonstrate that BCT-Net achieves high precision, with Pre and DSC indices of 86.12% and 88.70%, respectively. Experiments conducted on the BUSI dataset of breast ultrasound images manifest that this approach exhibits high accuracy in breast tumor segmentation.

Applying YOLOv6 as an ensemble federated learning framework to classify breast cancer pathology images

The most common carcinoma-related cause of death among women is breast cancer. Early detection is crucial, and the manual screening method may lead to a delayed diagnosis, which would delay treatment and put lives at risk. Mammography imaging is advised for routine screening to diagnose breast cancer at an early stage. To improve generalizability, this study examines the implementation of Federated Learning (FedL) to detect breast cancer. Its performance is compared to a centralized training technique that diagnoses breast cancer. Although FedL has been famous as a safeguarding privacy algorithm, its similarities to ensemble learning methods, such as federated averaging (FEDAvrg), still need to be thoroughly investigated. This study examines explicitly how a YOLOv6 model trained with FedL performs across several clients. A new homomorphic encryption and decryption algorithm is also proposed to retain data privacy. A novel pruned YOLOv6 model with FedL is introduced in this study to differentiate benign and malignant tissues. The model is trained on the breast cancer pathological dataset BreakHis and BUSI. The proposed model achieved a validation accuracy of 98% on BreakHis dataset and 97% on BUSI dataset. The results are compared with the VGG-19, ResNet-50, and InceptionV3 algorithms, showing that the proposed model achieved better results. The tests reveal that federated learning is feasible, as FedAvrg trains models of outstanding quality with only a few communication rounds, as shown by the results on a range of model topologies such as ResNet50, VGG-19, InceptionV3, and the proposed Ensembled FedL YOLOv6.

Adaptive ensemble loss and multi-scale attention in breast ultrasound segmentation with UMA-Net

The generalization of deep learning (DL) models is critical for accurate lesion segmentation in breast ultrasound (BUS) images. Traditional DL models often struggle to generalize well due to the high frequency and scale variations inherent in BUS images. Moreover, conventional loss functions used in these models frequently result in imbalanced optimization, either prioritizing region overlap or boundary accuracy, which leads to suboptimal segmentation performance. To address these issues, we propose UMA-Net, an enhanced UNet architecture specifically designed for BUS image segmentation. UMA-Net integrates residual connections, attention mechanisms, and a bottleneck with atrous convolutions to effectively capture multi-scale contextual information without compromising spatial resolution. Additionally, we introduce an adaptive ensemble loss function that dynamically balances the contributions of different loss components during training, ensuring optimization across key segmentation metrics. This novel approach mitigates the imbalances found in conventional loss functions. We validate UMA-Net on five diverse BUS datasets-BUET, BUSI, Mendeley, OMI, and UDIAT-demonstrating superior performance. Our findings highlight the importance of addressing frequency and scale variations, confirming UMA-Net as a robust and generalizable solution for BUS image segmentation.

A Cross-scale Attention-Based U-Net for Breast Ultrasound Image Segmentation

Breast cancer remains a significant global health concern and is a leading cause of mortality among women. The accuracy of breast cancer diagnosis can be greatly improved with the assistance of automatic segmentation of breast ultrasound images. Research has demonstrated the effectiveness of convolutional neural networks (CNNs) and transformers in segmenting these images. Some studies combine transformers and CNNs, using the transformer's ability to exploit long-distance dependencies to address the limitations inherent in convolutional neural networks. Many of these studies face limitations due to the forced integration of transformer blocks into CNN architectures. This approach often leads to inconsistencies in the feature extraction process, ultimately resulting in suboptimal performance for the complex task of medical image segmentation. This paper presents CSAU-Net, a cross-scale attention-guided U-Net, which is a combined CNN-transformer structure that leverages the local detail depiction of CNNs and the ability of transformers to handle long-distance dependencies. To integrate global context data, we propose a cross-scale cross-attention transformer block that is embedded within the skip connections of the U-shaped architectural network. To further enhance the effectiveness of the segmentation process, we incorporated a gated dilated convolution (GDC) module and a lightweight channel self-attention transformer (LCAT) on the encoder side. Extensive experiments conducted on three open-source datasets demonstrate that our CSAU-Net surpasses state-of-the-art techniques in segmenting ultrasound breast lesions.

LightweightUNet: Multimodal Deep Learning with GAN-Augmented Imaging Data for Efficient Breast Cancer Detection

Breast cancer ranks as the second most prevalent cancer globally and is the most frequently diagnosed cancer among women; therefore, early, automated, and precise detection is essential. Most AI-based techniques for breast cancer detection are complex and have high computational costs. Hence, to overcome this challenge, we have presented the innovative LightweightUNet hybrid deep learning (DL) classifier for the accurate classification of breast cancer. The proposed model boasts a low computational cost due to its smaller number of layers in its architecture, and its adaptive nature stems from its use of depth-wise separable convolution. We have employed a multimodal approach to validate the model's performance, using 13,000 images from two distinct modalities: mammogram imaging (MGI) and ultrasound imaging (USI). We collected the multimodal imaging datasets from seven different sources, including the benchmark datasets DDSM, MIAS, INbreast, BrEaST, BUSI, Thammasat, and HMSS. Since the datasets are from various sources, we have resized them to the uniform size of 256 × 256 pixels and normalized them using the Box-Cox transformation technique. Since the USI dataset is smaller, we have applied the StyleGAN3 model to generate 10,000 synthetic ultrasound images. In this work, we have performed two separate experiments: the first on a real dataset without augmentation and the second on a real + GAN-augmented dataset using our proposed method. During the experiments, we used a 5-fold cross-validation method, and our proposed model obtained good results on the real dataset (87.16% precision, 86.87% recall, 86.84% F1-score, and 86.87% accuracy) without adding any extra data. Similarly, the second experiment provides better performance on the real + GAN-augmented dataset (96.36% precision, 96.35% recall, 96.35% F1-score, and 96.35% accuracy). This multimodal approach, which utilizes LightweightUNet, enhances the performance by 9.20% in precision, 9.48% in recall, 9.51% in F1-score, and a 9.48% increase in accuracy on the combined dataset. The LightweightUNet model we proposed works very well thanks to a creative network design, adding fake images to the data, and a multimodal training method. These results show that the model has a lot of potential for use in clinical settings.

On the Readiness of Scientific Data Papers for a Fair and Transparent Use in Machine Learning

To ensure the fairness and trustworthiness of machine learning (ML) systems, recent legislative initiatives and relevant research in the ML community have pointed out the need to document the data used to train ML models. Besides, data-sharing practices in many scientific domains have evolved in recent years for reproducibility purposes. In this sense, academic institutions' adoption of these practices has encouraged researchers to publish their data and technical documentation in peer-reviewed publications such as data papers. In this study, we analyze how this broader scientific data documentation meets the needs of the ML community and regulatory bodies for its use in ML technologies. We examine a sample of 4041 data papers of different domains, assessing their coverage and trends in the requested dimensions and comparing them to those from an ML-focused venue (NeurIPS D&B), which publishes papers describing datasets. As a result, we propose a set of recommendation guidelines for data creators and scientific data publishers to increase their data's preparedness for its transparent and fairer use in ML technologies.