Image Quality Improvement of Hand-Held Ultrasound Devices With a Two-Stage Generative Adversarial Network

DF-GAM: Cross-Domain Ultrasound Image High-Quality Reconstruction Using a Dual Frequency-Domain Guided Adaptation Model

OBJECTIVE

To enhance the quality of low-resolution (LR) ultrasound images and mitigate artifacts and speckle noise, which can impede accurate medical diagnosis, a novel method called the dual frequency-domain guided adaptation model (DF-GAM) is proposed. The method aims to achieve high-quality image reconstruction across diverse domains, including different ultrasound machines, diseases and phantom images.

METHODS

DF-GAM utilizes a dual-branch network architecture combined with frequency-domain self-adaptation and self-supervised edge regression. This approach enables cross-domain enhancement by focusing on the reconstruction of clear tissue structures and speckle patterns. The model is designed to adapt to various ultrasound imaging (USI) scenarios, ensuring its applicability in real-world clinical settings.

RESULTS

Experimental evaluations of DF-GAM were conducted using five different datasets. The results demonstrated the method's effectiveness, with DF-GAM outperforming existing enhancement techniques. The average peak signal-to-noise ratio (PSNR) achieved was 34.62, and the structural similarity index (SSIM) was 0.91, indicating a significant improvement in image quality compared to other methods.

CONCLUSION

DF-GAM shows great potential in improving medical image diagnosis and interpretation. Its ability to enhance LR ultrasound images across various domains without the need for extensive training data makes it a valuable tool for clinical use. The high PSNR and SSIM scores validate the method's effectiveness, suggesting that DF-GAM could significantly contribute to the field of USI diagnostics.

Handheld Ultrasound Does not Replace Magnetic Resonance Imaging for Diagnosis of Rotator Cuff Tears

Purpose

The purpose of this study was to examine the reliability and validity of handheld ultrasound (HHUS) alone versus conventional ultrasound (US) or magnetic resonance imaging (MRI) for diagnosis of rotator cuff tears and versus MRI plus computed tomography (CT) for diagnosis of fatty infiltration.

Methods

Adult patients with shoulder complaints were included in this study. HHUS of the shoulder was performed twice by an orthopedic surgeon and once by a radiologist. RCTs, tear width, retraction and FI were measured. Inter- and intrarater reliability of the HHUS was calculated using a Cohen's kappa coefficient. Criterion and concurrent validity were calculated using a Spearman's correlation coefficient.

Results

Sixty-one patients (64 shoulders) were included in this study. Intra-rater agreement of HHUS for assessment of RCTs (к = 0.914, supraspinatus) and FI (к = 0.844, supraspinatus) was moderate to strong. Interrater agreement was none to minimal for the diagnosis of RCTs (к = 0.465, supraspinatus) and FI (к = 0.346, supraspinatus). Concurrent validity of HHUS compared to MRI was fair for diagnosis of RCTs (r = 0.377, supraspinatus) and fair-to-moderate FI (r = 0.608, supraspinatus). HHUS shows a sensitivity of 81.1% and specificity of 62.5% for diagnosis of supraspinatus tears, 60% and 93.1% for subscapularis tears, 55.6% and 88.9% for infraspinatus tears.

Conclusions

On the basis of findings in this study, we conclude that HHUS is an aid in diagnosis of RCTs and higher degrees of FI in patients who are not obese, but it does not replace MRI as the gold standard. Further clinical studies on the application of HHUS comparing HHUS devices in larger patient populations and healthy patients are required to identify its utility in clinical practice.

Level of Evidence

Level III.

Sparse-based Domain Adaptation Network for OCTA Image Super-Resolution Reconstruction

Retinal Optical Coherence Tomography Angiography (OCTA) with high-resolution is important for the quantification and analysis of retinal vasculature. However, the resolution of OCTA images is inversely proportional to the field of view at the same sampling frequency, which is not conducive to clinicians for analyzing larger vascular areas. In this paper, we propose a novel Sparse-based domain Adaptation Super-Resolution network (SASR) for the reconstruction of realistic [Formula: see text]/low-resolution (LR) OCTA images to high-resolution (HR) representations. To be more specific, we first perform a simple degradation of the [Formula: see text]/high-resolution (HR) image to obtain the synthetic LR image. An efficient registration method is then employed to register the synthetic LR with its corresponding [Formula: see text] image region within the [Formula: see text] image to obtain the cropped realistic LR image. We then propose a multi-level super-resolution model for the fully-supervised reconstruction of the synthetic data, guiding the reconstruction of the realistic LR images through a generative-adversarial strategy that allows the synthetic and realistic LR images to be unified in the feature domain. Finally, a novel sparse edge-aware loss is designed to dynamically optimize the vessel edge structure. Extensive experiments on two OCTA sets have shown that our method performs better than state-of-the-art super-resolution reconstruction methods. In addition, we have investigated the performance of the reconstruction results on retina structure segmentations, which further validate the effectiveness of our approach.

Successful Use of a 5G-Based Robot-Assisted Remote Ultrasound System in a Care Center for Disabled Patients in Rural China

Background

Disability has become a global population health challenge. Due to difficulties in self-care or independent living, patients with disability mainly live in community-based care centers or institutions for long-term care. Nonetheless, these settings often lack basic medical resources, such as ultrasonography. Thus, remote ultrasonic robot technology for clinical applications across wide regions is imperative. To date, few experiences of remote diagnostic systems in rural care centers have been reported.

Objective

To assess the feasibility of a fifth-generation cellular technology (5G)-based robot-assisted remote ultrasound system in a care center for disabled patients in rural China.

Methods

Patients underwent remote robot-assisted and bedside ultrasound examinations of the liver, gallbladder, spleen, and kidneys. We compared the diagnostic consistency and differences between the two modalities and evaluated the examination duration, image quality, and safety.

Results

Forty-nine patients were included (21 men; mean age: 61.0 ± 19.0 [range: 19–91] years). Thirty-nine and ten had positive and negative results, respectively; 67 lesions were detected. Comparing the methods, 41 and 8 patients had consistent and inconsistent diagnoses, respectively. The McNemar and kappa values were 0.727 and 0.601, respectively. The mean duration of remote and bedside examinations was 12.2 ± 4.5 (range: 5–26) min and 7.5 ± 1.8 (range: 5–13) min (p < 0.001), respectively. The median image score for original images on the patient side and transmitted images on the doctor side was 5 points (interquartile range: [IQR]: 4.7–5.0) and 4.7 points (IQR: 4.5–5.0) (p = 0.176), respectively. No obvious complications from the examination were reported.

Conclusions

A 5G-based robot-assisted remote ultrasound system is feasible and has comparable diagnostic efficiency to traditional bedside ultrasound. This system may provide a unique solution for basic ultrasound diagnostic services in primary healthcare settings.

Deep learning in ultrasound elastography imaging: A review

It is known that changes in the mechanical properties of tissues are associated with the onset and progression of certain diseases. Ultrasound elastography is a technique to characterize tissue stiffness using ultrasound imaging either by measuring tissue strain using quasi-static elastography or natural organ pulsation elastography, or by tracing a propagated shear wave induced by a source or a natural vibration using dynamic elastography. In recent years, deep learning has begun to emerge in ultrasound elastography research. In this review, several common deep learning frameworks in the computer vision community, such as multilayer perceptron, convolutional neural network, and recurrent neural network are described. Then, recent advances in ultrasound elastography using such deep learning techniques are revisited in terms of algorithm development and clinical diagnosis. Finally, the current challenges and future developments of deep learning in ultrasound elastography are prospected. This article is protected by copyright. All rights reserved.

Deep Denoising of Raw Biomedical Knowledge Graph from COVID-19 Literature, LitCovid and Pubtator

Background

Multiple types of biomedical associations of knowledge graphs, including COVID-19–related ones, are constructed based on co-occurring biomedical entities retrieved from recent literature. However, the applications derived from these raw graphs (eg, association predictions among genes, drugs, and diseases) have a high probability of false-positive predictions as co-occurrences in the literature do not always mean there is a true biomedical association between two entities.

Objective

Data quality plays an important role in training deep neural network models; however, most of the current work in this area has been focused on improving a model’s performance with the assumption that the preprocessed data are clean. Here, we studied how to remove noise from raw knowledge graphs with limited labeled information.

Methods

The proposed framework used generative-based deep neural networks to generate a graph that can distinguish the unknown associations in the raw training graph. Two generative adversarial network models, NetGAN and Cross-Entropy Low-rank Logits (CELL), were adopted for the edge classification (ie, link prediction), leveraging unlabeled link information based on a real knowledge graph built from LitCovid and Pubtator.

Results

The performance of link prediction, especially in the extreme case of training data versus test data at a ratio of 1:9, demonstrated that the proposed method still achieved favorable results (area under the receiver operating characteristic curve >0.8 for the synthetic data set and 0.7 for the real data set), despite the limited amount of testing data available.

Conclusions

Our preliminary findings showed the proposed framework achieved promising results for removing noise during data preprocessing of the biomedical knowledge graph, potentially improving the performance of downstream applications by providing cleaner data.

Auto-Denoising for EEG Signals Using Generative Adversarial Network

The brain-computer interface (BCI) has many applications in various fields. In EEG-based research, an essential step is signal denoising. In this paper, a generative adversarial network (GAN)-based denoising method is proposed to denoise the multichannel EEG signal automatically. A new loss function is defined to ensure that the filtered signal can retain as much effective original information and energy as possible. This model can imitate and integrate artificial denoising methods, which reduces processing time; hence it can be used for a large amount of data processing. Compared to other neural network denoising models, the proposed model has one more discriminator, which always judges whether the noise is filtered out. The generator is constantly changing the denoising way. To ensure the GAN model generates EEG signals stably, a new normalization method called sample entropy threshold and energy threshold-based (SETET) normalization is proposed to check the abnormal signals and limit the range of EEG signals. After the denoising system is established, although the denoising model uses the different subjects' data for training, it can still apply to the new subjects' data denoising. The experiments discussed in this paper employ the HaLT public dataset. Correlation and root mean square error (RMSE) are used as evaluation criteria. Results reveal that the proposed automatic GAN denoising network achieves the same performance as the manual hybrid artificial denoising method. Moreover, the GAN network makes the denoising process automatic, representing a significant reduction in time.

When medical images meet generative adversarial network: recent development and research opportunities

Deep learning techniques have promoted the rise of artificial intelligence (AI) and performed well in computer vision. Medical image analysis is an important application of deep learning, which is expected to greatly reduce the workload of doctors, contributing to more sustainable health systems. However, most current AI methods for medical image analysis are based on supervised learning, which requires a lot of annotated data. The number of medical images available is usually small and the acquisition of medical image annotations is an expensive process. Generative adversarial network (GAN), an unsupervised method that has become very popular in recent years, can simulate the distribution of real data and reconstruct approximate real data. GAN opens some exciting new ways for medical image generation, expanding the number of medical images available for deep learning methods. Generated data can solve the problem of insufficient data or imbalanced data categories. Adversarial training is another contribution of GAN to medical imaging that has been applied to many tasks, such as classification, segmentation, or detection. This paper investigates the research status of GAN in medical images and analyzes several GAN methods commonly applied in this area. The study addresses GAN application for both medical image synthesis and adversarial learning for other medical image tasks. The open challenges and future research directions are also discussed.

DCPR-GAN: Dental Crown Prosthesis Restoration Using Two-stage Generative Adversarial Networks

Restoring the correct masticatory function of broken teeth is the basis of dental crown prosthesis rehabilitation. However, it is a challenging task primarily due to the complex and personalized morphology of the occlusal surface. In this article, we address this problem by designing a new two-stage generative adversarial network (GAN) to reconstruct a dental crown surface in the data-driven perspective. Specifically, in the first stage, a conditional GAN (CGAN) is designed to learn the inherent relationship between the defective tooth and the target crown, which can solve the problem of the occlusal relationship restoration. In the second stage, an improved CGAN is further devised by considering an occlusal groove parsing network (GroNet) and an occlusal fingerprint constraint to enforce the generator to enrich the functional characteristics of the occlusal surface. Experimental results demonstrate that the proposed framework significantly outperforms the state-of-the-art deep learning methods in functional occlusal surface reconstruction using a real-world patient database. Moreover, the standard deviation (SD) and root mean square (RMS) between the generated occlusal surface and the target crown calculated by our method are both less than 0.161mm. Importantly, the designed dental crown has enough anatomical morphology and higher clinical applicability.

There is No Substitute for Human Intelligence

How to cite this article: Kumar V. There is No Substitute for Human Intelligence. Indian J Crit Care Med 2021;25(5):486-488.

Fproi-GAN with Fused Regional Features for the Synthesis of High-Quality Paired Medical Images

The use of medical image synthesis with generative adversarial networks (GAN) is effective for expanding medical samples. The structural consistency between the synthesized and actual image is a key indicator of the quality of the synthesized image, and the region of interest (ROI) of the synthesized image is related to its usability, and these parameters are the two key issues in image synthesis. In this paper, the fusion-ROI patch GAN (Fproi-GAN) model was constructed by incorporating a priori regional feature based on the two-stage cycle consistency mechanism of cycleGAN. This model has improved the tissue contrast of ROI and achieved the pairwise synthesis of high-quality medical images and their corresponding ROIs. The quantitative evaluation results in two publicly available datasets, INbreast and BRATS 2017, show that the synthesized ROI images have a DICE coefficient of 0.981 ± 0.11 and a Hausdorff distance of 4.21 ± 2.84 relative to the original images. The classification experimental results show that the synthesized images can effectively assist in the training of machine learning models, improve the generalization performance of prediction models, and improve the classification accuracy by 4% and sensitivity by 5.3% compared with the cycleGAN method. Hence, the paired medical images synthesized using Fproi-GAN have high quality and structural consistency with real medical images.

A Programmable Platform for Accelerating the Development of Smart Ultrasound Transducer Probe

During the COVID-19 pandemic, an ultraportable ultrasound smart probe has proven to be one of the few practical diagnostic and monitoring tools for doctors who are fully covered with personal protective equipment. The real-time, safety, ease of sanitization, and ultraportability features of an ultrasound smart probe make it extremely suitable for diagnosing COVID-19. In this article, we discuss the implementation of a smart probe designed according to the classic architecture of ultrasound scanners. The design balanced both performance and power consumption. This programmable platform for an ultrasound smart probe supports a 64-channel full digital beamformer. The platform's size is smaller than 10 cm ×5 cm. It achieves a 60-dBFS signal-to-noise ratio (SNR) and an average power consumption of ~4 W with 80% power efficiency. The platform is capable of achieving triplex B-mode, M-mode, color, pulsed-wave Doppler mode imaging in real time. The hardware design files are available for researchers and engineers for further study, improvement or rapid commercialization of ultrasound smart probes to fight COVID-19.

Imaging Study of Pseudo-CT Synthesized From Cone-Beam CT Based on 3D CycleGAN in Radiotherapy

Purpose

To propose a synthesis method of pseudo-CT (CT_CycleGAN) images based on an improved 3D cycle generative adversarial network (CycleGAN) to solve the limitations of cone-beam CT (CBCT), which cannot be directly applied to the correction of radiotherapy plans.

Methods

The improved U-Net with residual connection and attention gates was used as the generator, and the discriminator was a full convolutional neural network (FCN). The imaging quality of pseudo-CT images is improved by adding a 3D gradient loss function. Fivefold cross-validation was performed to validate our model. Each pseudo CT generated is compared against the real CT image (ground truth CT, CT_gt) of the same patient based on mean absolute error (MAE) and structural similarity index (SSIM). The dice similarity coefficient (DSC) coefficient was used to evaluate the segmentation results of pseudo CT and real CT. 3D CycleGAN performance was compared to 2D CycleGAN based on normalized mutual information (NMI) and peak signal-to-noise ratio (PSNR) metrics between the pseudo-CT and CT_gt images. The dosimetric accuracy of pseudo-CT images was evaluated by gamma analysis.

Results

The MAE metric values between the CT_CycleGAN and the real CT in fivefold cross-validation are 52.03 ± 4.26HU, 50.69 ± 5.25HU, 52.48 ± 4.42HU, 51.27 ± 4.56HU, and 51.65 ± 3.97HU, respectively, and the SSIM values are 0.87 ± 0.02, 0.86 ± 0.03, 0.85 ± 0.02, 0.85 ± 0.03, and 0.87 ± 0.03 respectively. The DSC values of the segmentation of bladder, cervix, rectum, and bone between CT_CycleGAN and real CT images are 91.58 ± 0.45, 88.14 ± 1.26, 87.23 ± 2.01, and 92.59 ± 0.33, respectively. Compared with 2D CycleGAN, the 3D CycleGAN based pseudo-CT image is closer to the real image, with NMI values of 0.90 ± 0.01 and PSNR values of 30.70 ± 0.78. The gamma pass rate of the dose distribution between CT_CycleGAN and CT_gt is 97.0% (2%/2 mm).

Conclusion

The pseudo-CT images obtained based on the improved 3D CycleGAN have more accurate electronic density and anatomical structure.

Handheld Ultrasound Video High-Quality Reconstruction Using a Low-Rank Representation Multipathway Generative Adversarial Network

Recently, the use of portable equipment has attracted much attention in the medical ultrasound field. Handheld ultrasound devices have great potential for improving the convenience of diagnosis, but noise-induced artifacts and low resolution limit their application. To enhance the video quality of handheld ultrasound devices, we propose a low-rank representation multipathway generative adversarial network (LRR MPGAN) with a cascade training strategy. This method can directly generate sequential, high-quality ultrasound video with clear tissue structures and details. In the cascade training process, the network is first trained with plane wave (PW) single-/multiangle video pairs to capture dynamic information and then fine-tuned with handheld/high-end image pairs to extract high-quality single-frame information. In the proposed GAN structure, a multipathway generator is applied to implement the cascade training strategy, which can simultaneously extract dynamic information and synthesize multiframe features. The LRR decomposition channel approach guarantees the fine reconstruction of both global features and local details. In addition, a novel ultrasound loss is added to the conventional mean square error (MSE) loss to acquire ultrasound-specific perceptual features. A comprehensive evaluation is conducted in the experiments, and the results confirm that the proposed method can effectively reconstruct high-quality ultrasound videos for handheld devices. With the aid of the proposed method, handheld ultrasound devices can be used to obtain convincing and convenient diagnoses.