Contrastive rendering with semi-supervised learning for ovary and follicle segmentation from 3D ultrasound

Semi-supervised learning framework with shape encoding for neonatal ventricular segmentation from 3D ultrasound

BACKGROUND

Three-dimensional (3D) ultrasound (US) imaging has shown promise in non-invasive monitoring of changes in the lateral brain ventricles of neonates suffering from intraventricular hemorrhaging. Due to the poorly defined anatomical boundaries and low signal-to-noise ratio, fully supervised methods for segmentation of the lateral ventricles in 3D US images require a large dataset of annotated images by trained physicians, which is tedious, time-consuming, and expensive. Training fully supervised segmentation methods on a small dataset may lead to overfitting and hence reduce its generalizability. Semi-supervised learning (SSL) methods for 3D US segmentation may be able to address these challenges but most existing SSL methods have been developed for magnetic resonance or computed tomography (CT) images.

PURPOSE

To develop a fast, lightweight, and accurate SSL method, specifically for 3D US images, that will use unlabeled data towards improving segmentation performance.

METHODS

We propose an SSL framework that leverages the shape-encoding ability of an autoencoder network to enforce complex shape and size constraints on a 3D U-Net segmentation model. The autoencoder created pseudo-labels, based on the 3D U-Net predicted segmentations, that enforces shape constraints. An adversarial discriminator network then determined whether images came from the labeled or unlabeled data distributions. We used 887 3D US images, of which 87 had manually annotated labels and 800 images were unlabeled. Training/validation/testing sets of 25/12/50, 25/12/25 and 50/12/25 images were used for model experimentation. The Dice similarity coefficient (DSC), mean absolute surface distance (MAD), and absolute volumetric difference (VD) were used as metrics for comparing to other benchmarks. The baseline benchmark was the fully supervised vanilla 3D U-Net while dual task consistency, shape-aware semi-supervised network, correlation-aware mutual learning, and 3D U-Net Ensemble models were used as state-of-the-art benchmarks with DSC, MAD, and VD as comparison metrics. The Wilcoxon signed-rank test was used to test statistical significance between algorithms for DSC and VD with the threshold being p < 0.05 and corrected to p < 0.01 using the Bonferroni correction. The random-access memory (RAM) trace and number of trainable parameters were used to compare the computing efficiency between models.

RESULTS

Relative to the baseline 3D U-Net model, our shape-encoding SSL method reported a mean DSC improvement of 6.5%, 7.7%, and 4.1% with a 95% confidence interval of 4.2%, 5.7%, and 2.1% using image data splits of 25/12/50, 25/12/25, and 50/12/25, respectively. Our method only used a 1GB increase in RAM compared to the baseline 3D U-Net and required less than half the RAM and trainable parameters compared to the 3D U-Net ensemble method.

CONCLUSIONS

Based on our extensive literature survey, this is one of the first reported works to propose an SSL method designed for segmenting organs in 3D US images and specifically one that incorporates unlabeled data for segmenting neonatal cerebral lateral ventricles. When compared to the state-of-the-art SSL and fully supervised learning methods, our method yielded the highest DSC and lowest VD while being computationally efficient.

Boundary-Repairing Dual-Path Network for Retinal Layer Segmentation in OCT Image with Pigment Epithelial Detachment

Automatic retinal layer segmentation in optical coherence tomography (OCT) images is crucial for the diagnosis of ocular diseases. Currently, automatic retinal layer segmentation works well with normal OCT images. However, pigment epithelial detachment (PED) dramatically alters the retinal structure, causing blurred boundaries and partial disappearance of the Bruch's Membrane (BM), thus posing challenges to the segmentation. To tackle these problems, we propose a novel dual-path U-shaped network for simultaneous layer segmentation and boundary regression. This network first designs a feature interaction fusion (FIF) module to strengthen the boundary shape constraints in the layer path. To address the challenge posed by partial BM disappearance and boundary-blurring, we propose a layer boundary repair (LBR) module. This module aims to use contrastive loss to enhance the confidence of blurred boundary regions and refine the segmentation of layer boundaries through the re-prediction head. In addition, we introduce a novel bilateral threshold distance map (BTDM) designed for the boundary path. The BTDM serves to emphasize information within boundary regions. This map, combined with the updated probability map, culminates in topology-guaranteed segmentation results achieved through a topology correction (TC) module. We investigated the proposed network on two severely deformed datasets (i.e., OCTA-500 and Aier-PED) and one slightly deformed dataset (i.e., DUKE). The proposed method achieves an average Dice score of 94.26% on the OCTA-500 dataset, which was 1.5% higher than BAU-Net and outperformed other methods. In the DUKE and Aier-PED datasets, the proposed method achieved average Dice scores of 91.65% and 95.75%, respectively.

Semi-supervised liver segmentation based on local regions self-supervision

BACKGROUND

Semi-supervised learning has gained popularity in medical image segmentation due to its ability to reduce reliance on image annotation. A typical approach in semi-supervised learning is to select reliable predictions as pseudo-labels and eliminate unreliable predictions. Contrastive learning helps prevent the insufficient utilization of unreliable predictions, but neglecting the anatomical structure of medical images can lead to suboptimal optimization results.

PURPOSE

We propose a novel approach for semi-supervised liver segmentation using contrastive learning, which leverages unlabeled data and enhances the suitability of contrastive learning for liver segmentation.

METHOD AND MATERIALS

Contrastive learning helps prevent the inappropriate utilization of unreliable predictions, but neglecting the anatomical structure of medical images can lead to suboptimal optimization results. Therefore, we propose a semi-supervised contrastive learning method with local regions self-supervision (LRS2). On one side, we employ Shannon entropy to distinguish between reliable and unreliable predictions and reduce the dissimilarity between their representations within regional artificial units. Within each unit of the liver image, we strongly encourage unreliable predictions to acquire valuable information pertaining to the correct category by leveraging the representations of reliable predictions in their vicinity. On the other side, we introduce a dynamic reliability threshold based on the Shannon entropy of each prediction, gradually evaluating the confidence threshold of reliable predictions as predictive accuracy improves. After selecting reliable predictions, we sequentially apply erosion and dilation to refine them for better selection of qualified positive and negative samples. We evaluate our proposed method on abdominal CT images, including 131 images (train data: 77, validation data: 26, and testing data: 28) from 2017 ISBI Liver Tumors Segmentation Challenge.

RESULTS

Our method obtains satisfactory performance in different proportion by exploiting the unreliable predictions. Compared with the result of VNet only under supervised settings (with 10, 30, 50, 70% and full labeled data), LRS2, respectively, brings an improvement of Dice coefficient by +6.11, +3.55, +4.43, and +2.25%, achieving Dice coefficients of 93.44, 93.31, 94.85, and 95.12%, respectively.

CONCLUSION

In this study, we carefully select appropriate positive and negative samples from reliable regions, ensuring that anchor pixels within unreliable regions are correctly assigned to their respective categories. With a consideration of the anatomical structure present in CT images, we partition the image representations into regional units, enabling anchor pixels to capture more precise sample information. Extensive experiments confirm the effectiveness of our method.

Reviewing 3D convolutional neural network approaches for medical image segmentation

Background

Convolutional neural networks (CNNs) assume pivotal roles in aiding clinicians in diagnosis and treatment decisions. The rapid evolution of imaging technology has established three-dimensional (3D) CNNs as a formidable framework for delineating organs and anomalies in medical images. The prominence of 3D CNN frameworks is steadily growing within medical image segmentation and classification. Thus, our proposition entails a comprehensive review, encapsulating diverse 3D CNN algorithms for the segmentation of medical image anomalies and organs.

Methods

This study systematically presents an exhaustive review of recent 3D CNN methodologies. Rigorous screening of abstracts and titles were carried out to establish their relevance. Research papers disseminated across academic repositories were meticulously chosen, analyzed, and appraised against specific criteria. Insights into the realm of anomalies and organ segmentation were derived, encompassing details such as network architecture and achieved accuracies.

Results

This paper offers an all-encompassing analysis, unveiling the prevailing trends in 3D CNN segmentation. In-depth elucidations encompass essential insights, constraints, observations, and avenues for future exploration. A discerning examination indicates the preponderance of the encoder-decoder network in segmentation tasks. The encoder-decoder framework affords a coherent methodology for the segmentation of medical images.

Conclusion

The findings of this study are poised to find application in clinical diagnosis and therapeutic interventions. Despite inherent limitations, CNN algorithms showcase commendable accuracy levels, solidifying their potential in medical image segmentation and classification endeavors.

Semi-supervised contrast learning-based segmentation of choroidal vessel in optical coherence tomography images

Objective.Choroidal vessels account for 85% of all blood vessels in the eye, and the accurate segmentation of choroidal vessels from optical coherence tomography (OCT) images provides important support for the quantitative analysis of choroid-related diseases and the development of treatment plans. Although deep learning-based methods have great potential for segmentation, these methods rely on large amounts of well-labeled data, and the data collection process is both time-consuming and laborious.Approach.In this paper, we propose a novel asymmetric semi-supervised segmentation framework called SSCR, based on a student-teacher model, to segment choroidal vessels in OCT images. The proposed framework enhances the segmentation results with uncertainty-aware self-integration and transformation consistency techniques. Meanwhile, we designed an asymmetric encoder-decoder network called Pyramid Pooling SegFormer (APP-SFR) for choroidal vascular segmentation. The network combines local attention and global attention information to improve the model's ability to learn complex vascular features. Additionally, we proposed a boundary repair module that enhances boundary confidence by utilizing a repair head to re-predict selected fuzzy points and further refines the segmentation boundary.Main results.We conducted extensive experiments on three different datasets: the ChorVessel dataset with 400 OCT images, the Meibomian Glands (MG) dataset with 400 images, and the U2OS Cell Nucleus Dataset with 200 images. The proposed method achieved an average Dice score of 74.23% on the ChorVessel dataset, which is 2.95% higher than the fully supervised network (U-Net) and outperformed other comparison methods. In both the MG dataset and the U2OS cell nucleus dataset, our proposed SSCR method achieved average Dice scores of 80.10% and 87.26%, respectively.Significance.The experimental results show that our proposed methods achieve better segmentation accuracy than other state-of-the-art methods. The method is designed to help clinicians make rapid diagnoses of ophthalmic diseases and has potential for clinical application.

HaTU-Net: Harmonic Attention Network for Automated Ovarian Ultrasound Quantification in Assisted Pregnancy

Antral follicle Count (AFC) is a non-invasive biomarker used to assess ovarian reserves through transvaginal ultrasound (TVUS) imaging. Antral follicles' diameter is usually in the range of 2-10 mm. The primary aim of ovarian reserve monitoring is to measure the size of ovarian follicles and the number of antral follicles. Manual follicle measurement is inhibited by operator time, expertise and the subjectivity of delineating the two axes of the follicles. This necessitates an automated framework capable of quantifying follicle size and count in a clinical setting. This paper proposes a novel Harmonic Attention-based U-Net network, HaTU-Net, to precisely segment the ovary and follicles in ultrasound images. We replace the standard convolution operation with a harmonic block that convolves the features with a window-based discrete cosine transform (DCT). Additionally, we proposed a harmonic attention mechanism that helps to promote the extraction of rich features. The suggested technique allows for capturing the most relevant features, such as boundaries, shape, and textural patterns, in the presence of various noise sources (i.e., shadows, poor contrast between tissues, and speckle noise). We evaluated the proposed model on our in-house private dataset of 197 patients undergoing TransVaginal UltraSound (TVUS) exam. The experimental results on an independent test set confirm that HaTU-Net achieved a Dice coefficient score of 90% for ovaries and 81% for antral follicles, an improvement of 2% and 10%, respectively, when compared to a standard U-Net. Further, we accurately measure the follicle size, yielding the recall, and precision rates of 91.01% and 76.49%, respectively.

Applications of machine learning techniques for enhancing nondestructive food quality and safety detection

In considering the need of people all over the world for high-quality food, there has been a recent increase in interest in the role of nondestructive and rapid detection technologies in the food industry. Moreover, the analysis of data acquired by most nondestructive technologies is complex, time-consuming, and requires highly skilled operators. Meanwhile, the general applicability of various chemometric or statistical methods is affected by noise, sample, variability, and data complexity that vary under various testing conditions. Nowadays, machine learning (ML) techniques have a wide range of applications in the food industry, especially in nondestructive technology and equipment intelligence, due to their powerful ability in handling irrelevant information, extracting feature variables, and building calibration models. The review provides an introduction and comparison of machine learning techniques, and summarizes these algorithms as traditional machine learning (TML), and deep learning (DL). Moreover, several novel nondestructive technologies, namely acoustic analysis, machine vision (MV), electronic nose (E-nose), and spectral imaging, combined with different advanced ML techniques and their applications in food quality assessment such as variety identification and classification, safety inspection and processing control, are presented. In addition to this, the existing challenges and prospects are discussed. The result of this review indicates that nondestructive testing technologies combined with state-of-the-art machine learning techniques show great potential for monitoring the quality and safety of food products and different machine learning algorithms have their characteristics and applicability scenarios. Due to the nature of feature learning, DL is one of the most promising and powerful techniques for real-time applications, which needs further research for full and wide applications in the food industry.

A two-stage multiresolution neural network for automatic diagnosis of hepatic echinococcosis from ultrasound images: A multicenter study

PURPOSE

Hepatic echinococcosis is a parasitic disease. Ultrasound imaging is a crucially important tool for the diagnosis of this disease. Based on ultrasonic manifestations, hepatic echinococcosis can be classified into many subtypes. However, the subtyping is non-trivial due to the challenges of complex sonographic textures and the large intra-class and small inter-class differences. The purpose of this study is to develop a computer aided diagnosis system for hepatic echinococcosis based on ultrasound images.

METHODS

We collected a multicenter ultrasound dataset containing 9112 images from 5028 patients who were diagnosed with hepatic echinococcosis (the largest cohort to date) and developed a two-stage multiresolution neural network for the automatic diagnosis of hepatic echinococcosis into nine subtypes as suggested by WHO. Our method was based on YOLO3 with two additional strategies to improve its performance: coarse grouping and multiresolution sampling. Considering that some subtypes are inherently very similar and difficult to be differentiated, in the first stage we detected and classified lesions into four coarse groups, instead of making a direct classification into nine classes. In the second stage, we performed fine-grained classification within each coarse group. Multiple views with different resolutions were sampled from the detected lesions and were input into Darknet53. The softmax outputs for the multiresolution views were averaged to generate the final output.

RESULTS

Both the proposed coarse grouping and multiresolution sampling strategies proved to be effective and improved the classification performance by a large margin compared with the setting without using the two strategies. Using five-fold cross validation, our method achieved 87.1%, 86.2%, and 86.5% in the average recall, precision and F1-score, respectively, and outperformed other state-of-the-art methods remarkably.

CONCLUSIONS

The experimental results demonstrate the great promise of our method for classifying hepatic echinococcosis. Our method can be used as an effective tool to facilitate large-scale screening for hepatic echinococcosis in high-risk, resource-poor areas, thus contributing to early diagnosis of this disease and resulting in more successful treatment. This article is protected by copyright. All rights reserved.