An effective approach of lesion segmentation within the breast ultrasound image based on the cellular automata principle

MFA-ICPS: Semi-supervised medical image segmentation with improved cross pseudo supervision and multi-dimensional feature attention

BACKGROUND

In the medical field, medical image segmentation plays a pivotal role in facilitating disease evaluation and supporting treatment decision-making for doctors. Recently, deep learning methods have been employed in the field of medical image segmentation. However, the manual annotation of a large number of reliable labels is a costly and time-consuming process.

PURPOSE

To address this challenge, a semi-supervised learning framework is required to alleviate the burden of reliable labeling and enhance segmentation accuracy in challenging areas of medical images.

METHODS

Therefore, this paper presents MFA-ICPS framework, a semi-supervised learning framework based on the improved cross pseudo supervision (ICPS) and multi-dimensional feature attention (MFA) modules. Medical images inevitably contain some noise that may affect the segmentation accuracy, so the proposed framework addresses this challenge by introducing noise disturbance, combining ICPS and MFA modules, and using pseudo-segmentation maps and MFA maps to maintain the consistency at both the output and feature levels.

RESULTS

In the experiments, MFA-ICPS framework accurately obtains the following performances on the left atrial dataset: Dice, Jaccard, 95HD, and ASD values are90.89 % $90.89\%$ ,83.40 % $83.40\%$ , 6.00 and 1.94 mm, respectively. And on the pancreas-CT dataset, the following performances are accurately obtained: Dice, Jaccard, 95HD, and ASD values are79.55 % $79.55\%$ ,66.87 % $66.87\%$ , 7.67 and 1.65 mm, respectively.

CONCLUSIONS

The segmentation performance of MFA-ICPS framework on different medical datasets demonstrates its remarkable capability to significantly enhance medical image segmentation.

Trustworthy Breast Ultrasound Image Semantic Segmentation Based on Fuzzy Uncertainty Reduction

Medical image semantic segmentation is essential in computer-aided diagnosis systems. It can separate tissues and lesions in the image and provide valuable information to radiologists and doctors. The breast ultrasound (BUS) images have advantages: no radiation, low cost, portable, etc. However, there are two unfavorable characteristics: (1) the dataset size is often small due to the difficulty in obtaining the ground truths, and (2) BUS images are usually in poor quality. Trustworthy BUS image segmentation is urgent in breast cancer computer-aided diagnosis systems, especially for fully understanding the BUS images and segmenting the breast anatomy, which supports breast cancer risk assessment. The main challenge for this task is uncertainty in both pixels and channels of the BUS images. In this paper, we propose a Spatial and Channel-wise Fuzzy Uncertainty Reduction Network (SCFURNet) for BUS image semantic segmentation. The proposed architecture can reduce the uncertainty in the original segmentation frameworks. We apply the proposed method to four datasets: (1) a five-category BUS image dataset with 325 images, and (2) three BUS image datasets with only tumor category (1830 images in total). The proposed approach compares state-of-the-art methods such as U-Net with VGG-16, ResNet-50/ResNet-101, Deeplab, FCN-8s, PSPNet, U-Net with information extension, attention U-Net, and U-Net with the self-attention mechanism. It achieves 2.03%, 1.84%, and 2.88% improvements in the Jaccard index on three public BUS datasets, and 6.72% improvement in the tumor category and 4.32% improvement in the overall performance on the five-category dataset compared with that of the original U-shape network with ResNet-101 since it can handle the uncertainty effectively and efficiently.

A contrastive consistency semi-supervised left atrium segmentation model

Accurate segmentation for the left atrium (LA) is a key process of clinical diagnosis and therapy for atrial fibrillation. In clinical, the semantic-level segmentation of LA consumes much time and labor. Although supervised deep learning methods can somewhat solve this problem, a high-efficient deep learning model requires abundant labeled data that is hard to acquire. Therefore, the research on automatic LA segmentation of leveraging unlabeled data is highly required. In this paper, we propose a semi-supervised LA segmentation framework including a segmentation model and a classification model. The segmentation model takes volumes from both labeled and unlabeled data as input and generates predictions of LAs. And then, a classification model maps these predictions to class-vectors for each input. Afterward, to leverage the class information, we construct a contrastive consistency loss function based on these class-vectors, so that the model can enlarge the discrepancy of the inter-class and compact the similarity of the intra-class for learning more distinguishable representation. Moreover, we set the class-vectors from the labeled data as references to the class-vectors from the unlabeled data to relieve the influence of the unreliable prediction for the unlabeled data. At last, we evaluate our semi-supervised LA segmentation framework on a public LA dataset using four universal metrics and compare it with recent state-of-the-art models. The proposed model achieves the best performance on all metrics with a Dice Score of 89.81 %, Jaccard of 81.64 %, 95 % Hausdorff distance of 7.15 mm, and Average Surface Distance of 1.82 mm. The outstanding performance of the proposed framework shows that it may have a significant contribution to assisting the therapy of patients with atrial fibrillation. Code is available at: https://github.com/PerceptionComputingLab/SCC.

BUSIS: A Benchmark for Breast Ultrasound Image Segmentation

Breast ultrasound (BUS) image segmentation is challenging and critical for BUS computer-aided diagnosis (CAD) systems. Many BUS segmentation approaches have been studied in the last two decades, but the performances of most approaches have been assessed using relatively small private datasets with different quantitative metrics, which results in a discrepancy in performance comparison. Therefore, there is a pressing need for building a benchmark to compare existing methods using a public dataset objectively, to determine the performance of the best breast tumor segmentation algorithm available today, and to investigate what segmentation strategies are valuable in clinical practice and theoretical study. In this work, a benchmark for B-mode breast ultrasound image segmentation is presented. In the benchmark, (1) we collected 562 breast ultrasound images and proposed standardized procedures to obtain accurate annotations using four radiologists; (2) we extensively compared the performance of 16 state-of-the-art segmentation methods and demonstrated that most deep learning-based approaches achieved high dice similarity coefficient values (DSC ≥ 0.90) and outperformed conventional approaches; (3) we proposed the losses-based approach to evaluate the sensitivity of semi-automatic segmentation to user interactions; and (4) the successful segmentation strategies and possible future improvements were discussed in details.

Fully multi-target segmentation for breast ultrasound image based on fully convolutional network

Ultrasound image segmentation plays an important role in computer-aided diagnosis of breast cancer. Existing approaches focused on extracting the tumor tissue to characterize the tumor class. However, other tissues are also helpful for providing the references. In this paper, a multi-target semantic segmentation approach is proposed based on the fully convolutional network for segmenting the breast ultrasound image into different target tissue regions. For handling the uncertain affiliation of pixels in blurry boundaries, the certain outputs of pixel characteristics in AlexNet are transformed into the fuzzy decision expression. For improving the image detail representation, the AlexNet network structure of fully convolutional network is optimized with fully connected skip structure. In addition, the output of net model is optimized with fully connected conditional random field to improve the characterization of spatial consistency and pixels' correlation of the image. Moreover, a data training optimization method is developed for improving the efficiency of network training. In the experiment, 325 ultrasound images and four error metrics are utilized for validating the segmentation performance. Comparing with existing methods, experimental results show that the proposed approach is effective for handling the breast ultrasound images accurately and reliably. Graphical abstract.

A direct volume rendering visualization approach for serial PET-CT scans that preserves anatomical consistency

PURPOSE

Our aim was to develop an interactive 3D direct volume rendering (DVR) visualization solution to interpret and analyze complex, serial multi-modality imaging datasets from positron emission tomography-computed tomography (PET-CT).

METHODS

Our approach uses: (i) a serial transfer function (TF) optimization to automatically depict particular regions of interest (ROIs) over serial datasets with consistent anatomical structures; (ii) integration of a serial segmentation algorithm to interactively identify and track ROIs on PET; and (iii) parallel graphics processing unit (GPU) implementation for interactive visualization.

RESULTS

Our DVR visualization more easily identifies changes in ROIs in serial scans in an automated fashion and parallel GPU computation which enables interactive visualization.

CONCLUSIONS

Our approach provides a rapid 3D visualization of relevant ROIs over multiple scans, and we suggest that it can be used as an adjunct to conventional 2D viewing software from scanner vendors.

Automated and real-time segmentation of suspicious breast masses using convolutional neural network

In this work, a computer-aided tool for detection was developed to segment breast masses from clinical ultrasound (US) scans. The underlying Multi U-net algorithm is based on convolutional neural networks. Under the Mayo Clinic Institutional Review Board protocol, a prospective study of the automatic segmentation of suspicious breast masses was performed. The cohort consisted of 258 female patients who were clinically identified with suspicious breast masses and underwent clinical US scan and breast biopsy. The computer-aided detection tool effectively segmented the breast masses, achieving a mean Dice coefficient of 0.82, a true positive fraction (TPF) of 0.84, and a false positive fraction (FPF) of 0.01. By avoiding positioning of an initial seed, the algorithm is able to segment images in real time (13–55 ms per image), and can have potential clinical applications. The algorithm is at par with a conventional seeded algorithm, which had a mean Dice coefficient of 0.84 and performs significantly better (P< 0.0001) than the original U-net algorithm.

Family of boundary overlap metrics for the evaluation of medical image segmentation

All medical image segmentation algorithms need to be validated and compared, yet no evaluation framework is widely accepted within the imaging community. None of the evaluation metrics that are popular in the literature are consistent in the way they rank segmentation results: they tend to be sensitive to one or another type of segmentation error (size, location, and shape) but no single metric covers all error types. We introduce a family of metrics, with hybrid characteristics. These metrics quantify the similarity or difference of segmented regions by considering their average overlap in fixed-size neighborhoods of points on the boundaries of those regions. Our metrics are more sensitive to combinations of segmentation error types than other metrics in the existing literature. We compare the metric performance on collections of segmentation results sourced from carefully compiled two-dimensional synthetic data and three-dimensional medical images. We show that our metrics: (1) penalize errors successfully, especially those around region boundaries; (2) give a low similarity score when existing metrics disagree, thus avoiding overly inflated scores; and (3) score segmentation results over a wider range of values. We analyze a representative metric from this family and the effect of its free parameter on error sensitivity and running time.

The diagnostic performance of leak-plugging automated segmentation versus manual tracing of breast lesions on ultrasound images

PURPOSE

To assess the diagnostic performance of a leak-plugging segmentation method that we have developed for delineating breast masses on ultrasound images.

MATERIALS AND METHODS

Fifty-two biopsy-proven breast lesion images were analyzed by three observers using the leak-plugging and manual segmentation methods. From each segmentation method, grayscale and morphological features were extracted and classified as malignant or benign by logistic regression analysis. The performance of leak-plugging and manual segmentations was compared by: size of the lesion, overlap area (O_a ) between the margins, and area under the ROC curves (A_z ).

RESULTS

The lesion size from leak-plugging segmentation correlated closely with that from manual tracing (R² of 0.91). O_a was higher for leak plugging, 0.92 ± 0.01 and 0.86 ± 0.06 for benign and malignant masses, respectively, compared to 0.80 ± 0.04 and 0.73 ± 0.02 for manual tracings. Overall O_a between leak-plugging and manual segmentations was 0.79 ± 0.14 for benign and 0.73 ± 0.14 for malignant lesions. A_z for leak plugging was consistently higher (0.910 ± 0.003) compared to 0.888 ± 0.012 for manual tracings. The coefficient of variation of A_z between three observers was 0.29% for leak plugging compared to 1.3% for manual tracings.

CONCLUSION

The diagnostic performance, size measurements, and observer variability for automated leak-plugging segmentations were either comparable to or better than those of manual tracings.

Cellular automata and anisotropic diffusion filter based interactive tumor segmentation for positron emission tomography

Tumor segmentation in positron emission tomography (PET) aids clinical diagnosis and in assessing treatment response. However, the low resolution and signal-to-noise inherent in PET images, makes accurate tumor segmentation challenging. Manual delineation is time-consuming and subjective, whereas fully automated algorithms are often limited to particular tumor types, and have difficulties in segmenting small and low-contrast tumors. Interactive segmentation may reduce the inter-observer variability and minimize the user input. In this study, we present a new interactive PET tumor segmentation method based on cellular automata (CA) and a nonlinear anisotropic diffusion filter (ADF). CA is tolerant of noise and image pattern complexity while ADF reduces noise while preserving edges. By coupling CA with ADF, our proposed approach was robust and accurate in detecting and segmenting noisy tumors. We evaluated our method with computer simulation and clinical data and it outperformed other common interactive PET segmentation algorithms.

Synthetic Brainbows

Brainbow is a genetic engineering technique that randomly colorizes cells. Biological samples processed with this technique and imaged with confocal microscopy have distinctive colors for individual cells. Complex cellular structures can then be easily visualized. However, the complexity of the Brainbow technique limits its applications. In practice, most confocal microscopy scans use different florescence staining with typically at most three distinct cellular structures. These structures are often packed and obscure each other in rendered images making analysis difficult. In this paper, we leverage a process known as GPU framebuffer feedback loops to synthesize Brainbow-like images. In addition, we incorporate ID shuffing and Monte-Carlo sampling into our technique, so that it can be applied to single-channel confocal microscopy data. The synthesized Brainbow images are presented to domain experts with positive feedback. A user survey demonstrates that our synthetic Brainbow technique improves visualizations of volume data with complex structures for biologists.

Computer aided diagnosis system for breast cancer based on color Doppler flow imaging

Color Doppler flow imaging takes a great value in diagnosing and classifying benign and malignant breast lesions. However, scanning of color Doppler sonography is operator-dependent and ineffective. In this paper, a novel breast classification system based on B-Mode ultrasound and color Doppler flow imaging is proposed. First, different feature extraction methods were used to obtain the texture and geometric features from B-Mode ultrasound images. In color Doppler feature extraction stage, several spectrum features are extracted by applying blood flow velocity analysis to Doppler signals. Moreover, a velocity coherent vector method is proposed based on color coherence vector, which is helpful for designing to the optimize detection of flow indices from different blood flow velocity fields automatically. Finally, a support vector machine classifier with selected feature vectors is used to classify breast tumors into benign and malignant. The experimental results demonstrate that the proposed computer-aided diagnosis system is useful for reducing the unnecessary biopsy and death rate.