Attention gate and dilation U-shaped network (GDUNet): an efficient breast ultrasound image segmentation network with multiscale information extraction

Background

In recent years, computer-aided diagnosis (CAD) systems have played an important role in breast cancer screening and diagnosis. The image segmentation task is the key step in a CAD system for the rapid identification of lesions. Therefore, an efficient breast image segmentation network is necessary for improving the diagnostic accuracy in breast cancer screening. However, due to the characteristics of blurred boundaries, low contrast, and speckle noise in breast ultrasound images, breast lesion segmentation is challenging. In addition, many of the proposed breast tumor segmentation networks are too complex to be applied in practice.

Methods

We developed the attention gate and dilation U-shaped network (GDUNet), a lightweight, breast lesion segmentation model. This model improves the inverted bottleneck, integrating it with tokenized multilayer perceptron (MLP) to construct the encoder. Additionally, we introduce the lightweight attention gate (AG) within the skip connection, which effectively filters noise in low-level semantic information across spatial and channel dimensions, thus attenuating irrelevant features. To further improve performance, we innovated the AG dilation (AGDT) block and embedded it between the encoder and decoder in order to capture critical multiscale contextual information.

Results

We conducted experiments on two breast cancer datasets. The experiment's results show that compared to UNet, GDUNet could reduce the number of parameters by 10 times and the computational complexity by 58 times while providing a double of the inference speed. Moreover, the GDUNet achieved a better segmentation performance than did the state-of-the-art medical image segmentation architecture.

Conclusions

Our proposed GDUNet method can achieve advanced segmentation performance on different breast ultrasound image datasets with high efficiency.

Weakly supervised learning-based 3D bladder reconstruction from 2D ultrasound images for bladder volume measurement

BACKGROUND

Accurate measurement of bladder volume is necessary to maintain the consistency of the patient's anatomy in radiation therapy for pelvic tumors. As the diversity of the bladder shape, traditional methods for bladder volume measurement from 2D ultrasound have been found to produce inaccurate results.

PURPOSE

To improve the accuracy of bladder volume measurement from 2D ultrasound images for patients with pelvic tumors.

METHODS

The bladder ultrasound images from 130 patients with pelvic cancer were collected retrospectively. All data were split into a training set (80 patients), a validation set (20 patients), and a test set (30 patients). A total of 12 transabdominal ultrasound images for one patient were captured by automatically rotating the ultrasonic probe with an angle step of 15°. An incomplete 3D ultrasound volume was synthesized by arranging these 2D ultrasound images in 3D space according to the acquisition angles. With this as input, a weakly supervised learning-based 3D bladder reconstruction neural network model was built to predict the complete 3D bladder. The key point is that we designed a novel loss function, including the supervised loss of bladder segmentation in the ultrasound images at known angles and the compactness loss of the 3D bladder. Bladder volume was calculated by counting the number of voxels belonging to the 3D bladder. The dice similarity coefficient (DSC) was used to evaluate the accuracy of bladder segmentation, and the relative standard deviation (RSD) was used to evaluate the calculation accuracy of bladder volume with that of computed tomography (CT) images as the gold standard.

RESULTS

The results showed that the mean DSC was up to 0.94 and the mean absolute RSD can be reduced to 6.3% when using 12 ultrasound images of one patient. Further, the mean DSC also was up to 0.90 and the mean absolute RSD can be reduced to 9.0% even if only two ultrasound images were used (i.e., the angle step is 90°). Compared with the commercial algorithm in bladder scanners, which has a mean absolute RSD of 13.6%, our proposed method showed a considerably huge improvement.

CONCLUSIONS

The proposed weakly supervised learning-based 3D bladder reconstruction method can greatly improve the accuracy of bladder volume measurement. It has great potential to be used in bladder volume measurement devices in the future.

Real-time reliable semantic segmentation of thyroid nodules in ultrasound images

Objective.Low efficiency in medical image segmentation is a common issue that limits computer-aided diagnosis development. Due to the varying positions and sizes of nodules, it is not easy to accurately segment ultrasound images. This study aims to propose a segmentation model that maintains high efficiency while improving accuracy.Approach. We propose a novel layer that integrates the advantages of dense connectivity, dilated convolution, and factorized filters to maintain excellent efficiency while improving accuracy. Dense connectivity optimizes feature reuse, dilated convolution redesigns layers, and factorized convolution improves efficiency. Moreover, we propose a loss function optimization method from a pixel perspective to increase the network's accuracy further.Main results.Experiments on the Thyroid dataset show that our method achieves 81.70% intersection-over-union (IoU), 90.50% true positive rate (TPR), and 0.25% false positive rate (FPR). In terms of accuracy, our method outperforms the state-of-the-art methods, with twice faster inference and nearly 400 times fewer parameters. Meanwhile, in a test on an External Thyroid dataset, our method achieves 77.03% IoU, 82.10% TPR, and 0.16% FPR, demonstrating our proposed model's robustness.Significance.We propose a real-time semantic segmentation architecture for thyroid nodule segmentation in ultrasound images called fully convolution dense dilated network (FCDDN). Our method runs fast with a few parameters and is suitable for medical devices requiring real-time segmentation.

Multi-Instance Classification of Breast Tumor Ultrasound Images Using Convolutional Neural Networks and Transfer Learning

BACKGROUND

Breast cancer is arguably one of the leading causes of death among women around the world. The automation of the early detection process and classification of breast masses has been a prominent focus for researchers in the past decade. The utilization of ultrasound imaging is prevalent in the diagnostic evaluation of breast cancer, with its predictive accuracy being dependent on the expertise of the specialist. Therefore, there is an urgent need to create fast and reliable ultrasound image detection algorithms to address this issue.

METHODS

This paper aims to compare the efficiency of six state-of-the-art, fine-tuned deep learning models that can classify breast tissue from ultrasound images into three classes: benign, malignant, and normal, using transfer learning. Additionally, the architecture of a custom model is introduced and trained from the ground up on a public dataset containing 780 images, which was further augmented to 3900 and 7800 images, respectively. What is more, the custom model is further validated on another private dataset containing 163 ultrasound images divided into two classes: benign and malignant. The pre-trained architectures used in this work are ResNet-50, Inception-V3, Inception-ResNet-V2, MobileNet-V2, VGG-16, and DenseNet-121. The performance evaluation metrics that are used in this study are as follows: Precision, Recall, F1-Score and Specificity.

RESULTS

The experimental results show that the models trained on the augmented dataset with 7800 images obtained the best performance on the test set, having 94.95 ± 0.64%, 97.69 ± 0.52%, 97.69 ± 0.13%, 97.77 ± 0.29%, 95.07 ± 0.41%, 98.11 ± 0.10%, and 96.75 ± 0.26% accuracy for the ResNet-50, MobileNet-V2, InceptionResNet-V2, VGG-16, Inception-V3, DenseNet-121, and our model, respectively.

CONCLUSION

Our proposed model obtains competitive results, outperforming some state-of-the-art models in terms of accuracy and training time.

Deep learning diagnostic performance and visual insights in differentiating benign and malignant thyroid nodules on ultrasound images

This study aims to construct and evaluate a deep learning model, utilizing ultrasound images, to accurately differentiate benign and malignant thyroid nodules. The objective includes visualizing the model's process for interpretability and comparing its diagnostic precision with a cohort of 80 radiologists. We employed ResNet as the classification backbone for thyroid nodule prediction. The model was trained using 2096 ultrasound images of 655 distinct thyroid nodules. For performance evaluation, an independent test set comprising 100 cases of thyroid nodules was curated. In addition, to demonstrate the superiority of the artificial intelligence (AI) model over radiologists, a Turing test was conducted with 80 radiologists of varying clinical experience. This was meant to assess which group of radiologists' conclusions were in closer alignment with AI predictions. Furthermore, to highlight the interpretability of the AI model, gradient-weighted class activation mapping (Grad-CAM) was employed to visualize the model's areas of focus during its prediction process. In this cohort, AI diagnostics demonstrated a sensitivity of 81.67%, a specificity of 60%, and an overall diagnostic accuracy of 73%. In comparison, the panel of radiologists on average exhibited a diagnostic accuracy of 62.9%. The AI's diagnostic process was significantly faster than that of the radiologists. The generated heat-maps highlighted the model's focus on areas characterized by calcification, solid echo and higher echo intensity, suggesting these areas might be indicative of malignant thyroid nodules. Our study supports the notion that deep learning can be a valuable diagnostic tool with comparable accuracy to experienced senior radiologists in the diagnosis of malignant thyroid nodules. The interpretability of the AI model's process suggests that it could be clinically meaningful. Further studies are necessary to improve diagnostic accuracy and support auxiliary diagnoses in primary care settings.

Pulmonary embolism detection on venous thrombosis ultrasound images with bi-dimensional entropy measures: Preliminary results

BACKGROUND

Venous thromboembolism (VTE) is a common health issue. A clinical expression of VTE is a deep vein thrombosis (DVT) that may lead to pulmonary embolism (PE), a critical illness. When DVT is suspected, an ultrasound exam is performed. However, the characteristics of the clot observed on ultrasound images cannot be linked with the presence of PE. Computed tomography angiography is the gold standard to diagnose PE. Nevertheless, the latter technique is expensive and requires the use of contrast agents.

PURPOSE

In this article, we present an image processing method based on ultrasound images to determine whether PE is associated or not with lower limb DVT. In terms of medical equipment, this new approach (Doppler ultrasound image processing) is inexpensive and quite easy.

METHODS

With the aim to help medical doctors in detecting PE, we herein propose to process ultrasound images of patients with DVT. After a first step based on histogram equalization, the analysis procedure is based on the use of bi-dimensional entropy measures. Two different algorithms are tested: the bi-dimensional dispersion entropy (D i s p E n 2 D $DispEn_{2D}$ ) mesure and the bi-dimensional fuzzy entropy (F u z E n 2 D $FuzEn_{2D}$ ) mesure. Thirty-two patients (12 women and 20 men, 67.63 ± 16.19 years old), split into two groups (16 with and 16 without PE), compose our database of around 1490 ultrasound images (split into seven different sizes from 32× 32 px to 128 × 128 px). p-values, computed with the Mann-Whitney test, are used to determine if entropy values of the two groups are statistically significantly different. Receiver operating characteristic (ROC) curves are plotted and analyzed for the most significant cases to define if entropy values are able to discriminate the two groups.

RESULTS

p-values show that there are statistical differences betweenF u z E n 2 D $FuzEn_{2D}$  of patients with PE and patients without PE for 112× 112 px and 128× 128 px images. Area under the ROC curve (AUC) is higher than 0.7 (threshold for a fair test) for 112× 112 and 128× 128 images. The best value of AUC (0.72) is obtained for 112× 112 px images.

CONCLUSIONS

Bi-dimensional entropy measures applied to ultrasound images seem to offer encouraging perspectives for PE detection: our first experiment, on a small dataset, shows thatF u z E n 2 D $FuzEn_{2D}$  on 112× 112 px images is able to detect PE. The next step of our work will consist in testing this approach on a larger dataset and in integratingF u z E n 2 D $FuzEn_{2D}$  in a machine learning algorithm. Furthermore, this study could also contribute to PE risk prediction for patients with VTE.

Automatic ultrasound diagnosis of thyroid nodules: a combination of deep learning and KWAK TI-RADS

Objective. There has been a considerable amount of computer-aided diagnosis (CAD) methods highlighted in the field of ultrasonic examination (USE) of thyroid nodules. However, few researches focused on the automatic risk classification, which was the basis for determining whether fine needle aspiration (FNA) was needed. The aim of this work was to implement automatic risk level assessment of thyroid nodules.Approach. Firstly, 1862 cases of thyroid nodules with the results of USE and FNA were collected as the dataset. Then, an improved U-Net++ model was utilized for segmenting thyroid nodules in ultrasound images automatically. Finally, the segmentation result was imported into a multi-task convolutional neural network (MT-CNN), the design of which was based on the clinical guideline called KWAK TI-RADS. Apart from the category of benign and malignant, the MT-CNN also exported the classification result of four malignant features, solid component (SC), hypoechogenicity or marked hypoechogenicity (HMH), microlobulated or irregular margin (MIM), microcalcification (MC), which were used for counting the risk level in KWAK TI-RADS.Main results. The performance of the improved U-Net++ was evaluated on our test set, including 302 cases. The Dice coefficient and intersection over union reached 0.899, 0.816, respectively. The classification accuracy rates of SC, HMH, MIM, MC, were 94.5%, 92.8%, 86.1%, 88.9%, while the false positive (FP) rate was 6.0%, 5.6%, 10.6%, 12.9% respectively. As for the category of benign and malignant, the precision and recall rates were 93.7% and 94.4%.Significance. The proposed CAD method showed favourable performance in the diagnosis of thyroid nodules. Compared with other methods, it could provide reports closer to clinical practice for doctors.

Application of ultrasound images-based radiomics in carpal tunnel syndrome: Without measuring the median nerve cross-sectional area

PURPOSE

By constructing a prediction model of carpal tunnel syndrome (CTS) based on ultrasound images, it can automatically and accurately diagnose CTS without measuring the median nerve cross-sectional area (CSA).

METHODS

A total of 268 wrists ultrasound images of 101 patients diagnosed with CTS and 76 controls in Ningbo NO.2 Hospital from December 2021 to August 2022 were retrospectively analyzed. The radiomics method was used to construct the Logistic model through the steps of feature extraction, feature screening, reduction, and modeling. The area under the receiver operating characteristic curve was calculated to evaluate the performance of the model, and the diagnostic efficiency of the radiomics model was compared with two radiologists with different experience.

RESULTS

The 134 wrists in the CTS group included 65 mild CTS, 42 moderate CTS, and 17 severe CTS. In the CTS group, 28 wrists median nerve CSA were less than the cut-off value, 17 wrists were missed by Dr. A, 26 wrists by Dr. B, and only 6 wrists were missed by radiomics model. A total of 335 radiomics features were extracted from each MN, of which 10 features were significantly different between compressed and normal nerves, and were used to construct the model. The area under curve (AUC) value, sensitivity, specificity, and accuracy of the radiomics model in the training set and testing set were 0.939, 86.17%, 87.10%, 86.63%, and 0.891, 87.50%, 80.49%, and 83.95%, respectively. The AUC value, sensitivity, specificity, and accuracy of the two doctors in the diagnosis of CTS were 0.746, 75.37%, 73.88%, 74.63% and 0.679, 68.66%, 67.16%, and 67.91%, respectively. The radiomics model was superior to the two-radiologist diagnosis, especially when there was no significant change in CSA.

CONCLUSION

Radiomics based on ultrasound images can quantitatively analyze the subtle changes in the median nerve, and can automatically and accurately diagnose CTS without measuring CSA, especially when there was no significant change in CSA, which was better than radiologists.

A Deep Learning Framework for the Characterization of Thyroid Nodules from Ultrasound Images Using Improved Inception Network and Multi-Level Transfer Learning

In the past few years, deep learning has gained increasingly widespread attention and has been applied to diagnosing benign and malignant thyroid nodules. It is difficult to acquire sufficient medical images, resulting in insufficient data, which hinders the development of an efficient deep-learning model. In this paper, we developed a deep-learning-based characterization framework to differentiate malignant and benign nodules from the thyroid ultrasound images. This approach improves the recognition accuracy of the inception network by combining squeeze and excitation networks with the inception modules. We have also integrated the concept of multi-level transfer learning using breast ultrasound images as a bridge dataset. This transfer learning approach addresses the issues regarding domain differences between natural images and ultrasound images during transfer learning. This paper aimed to investigate how the entire framework could help radiologists improve diagnostic performance and avoid unnecessary fine-needle aspiration. The proposed approach based on multi-level transfer learning and improved inception blocks achieved higher precision (0.9057 for the benign class and 0.9667 for the malignant class), recall (0.9796 for the benign class and 0.8529 for malignant), and F1-score (0.9412 for benign class and 0.9062 for malignant class). It also obtained an AUC value of 0.9537, which is higher than that of the single-level transfer learning method. The experimental results show that this model can achieve satisfactory classification accuracy comparable to experienced radiologists. Using this model, we can save time and effort as well as deliver potential clinical application value.

A Multiscale Attentional Unet Model for Automatic Segmentation in Medical Ultrasound Images

Ultrasonography has become an essential part of clinical diagnosis owing to its noninvasive, and real-time nature. To assist diagnosis, automatically segmenting a region of interest (ROI) in ultrasound images is becoming a vital part of computer-aided diagnosis (CAD). However, segmenting ROIs on medical images with relatively low contrast is a challenging task. To better achieve medical ROI segmentation, we propose an efficient module denoted as multiscale attentional convolution (MSAC), utilizing cascaded convolutions and a self-attention approach to concatenate features from various receptive field scales. Then, MSAC-Unet is constructed based on Unet, employing MSAC instead of the standard convolution in each encoder and decoder for segmentation. In this study, two representative types of ultrasound images, one of the thyroid nodules and the other of the brachial plexus nerves, were used to assess the effectiveness of the proposed approach. The best segmentation results from MSAC-Unet were achieved on two thyroid nodule datasets (TND-PUH3 and DDTI) and a brachial plexus nerve dataset (NSD) with Dice coefficients of 0.822, 0.792, and 0.746, respectively. The analysis of segmentation results shows that our MSAC-Unet greatly improves the segmentation accuracy with more reliable ROI edges and boundaries, decreasing the number of erroneously segmented ROIs in ultrasound images.

The ability of Segmenting Anything Model (SAM) to segment ultrasound images

Accurate ultrasound (US) image segmentation is important for disease screening, diagnosis, and prognosis assessment. However, US images typically have shadow artifacts and ambiguous boundaries that affect US segmentation. Recently, Segmenting Anything Model (SAM) from Meta AI has demonstrated remarkable potential in a wide range of applications. The purpose of this paper was to conduct an initial evaluation of the ability for SAM to segment US images, particularly in the event of shadow artifacts and ambiguous boundaries. We evaluated SAM's performance on three US datasets of different tissues, including multi-structure cardiac tissue, thyroid nodules, and the fetal head. Results indicated that SAM generally performs well with US images with clear tissue structures, but it has limited performance in the event of shadow artifacts and ambiguous boundaries. Thus, creating an improved SAM that considers the characteristics of US images is significant for automatic and accurate US segmentation.

Detection of Gallbladder Disease Types Using Deep Learning: An Informative Medical Method

Nowadays, despite all the conducted research and the provided efforts in advancing the healthcare sector, there is a strong need to rapidly and efficiently diagnose various diseases. The complexity of some disease mechanisms on one side and the dramatic life-saving potential on the other side raise big challenges for the development of tools for the early detection and diagnosis of diseases. Deep learning (DL), an area of artificial intelligence (AI), can be an informative medical tomography method that can aid in the early diagnosis of gallbladder (GB) disease based on ultrasound images (UI). Many researchers considered the classification of only one disease of the GB. In this work, we successfully managed to apply a deep neural network (DNN)-based classification model to a rich built database in order to detect nine diseases at once and to determine the type of disease using UI. In the first step, we built a balanced database composed of 10,692 UI of the GB organ from 1782 patients. These images were carefully collected from three hospitals over roughly three years and then classified by professionals. In the second step, we preprocessed and enhanced the dataset images in order to achieve the segmentation step. Finally, we applied and then compared four DNN models to analyze and classify these images in order to detect nine GB disease types. All the models produced good results in detecting GB diseases; the best was the MobileNet model, with an accuracy of 98.35%.

What it does and does not do: Effects of ultrasound viewing on women's intention to terminate a pregnancy

Ultrasound imaging is a renowned prenatal technology used globally to assess foetal growth, viability and abnormalities. In South Africa, ultrasound viewing has not been made mandatory for women who want to terminate their pregnancies. The purpose of this study was to provide a deeper understanding of the effects of ultrasound viewing on women's intention to terminate their pregnancies. Fifteen women in their first trimester were recruited for the study from a community health centre mandated for abortion. Van Manen's hermeneutic phenomenological analysis method was adopted for the study. Three major themes emerged from the data analysis: motivation beyond ultrasound viewing, the emotional burden of the experience, and viewing the ultrasound image as punishment. The study concluded that even though most participants reported the ultrasound viewing negatively affected their person, their reason for termination was so strong that they would not change their minds. However, the ultrasound viewing helped three participants to earnestly reflect on their situations, weigh the pros and cons, and subsequently decide to continue their pregnancy.

Boundary-oriented Network for Automatic Breast Tumor Segmentation in Ultrasound Images

Breast cancer is considered as the most prevalent cancer. Using ultrasound images is a momentous clinical diagnosis method to locate breast tumors. However, accurate segmentation of breast tumors remains an open problem due to ultrasound artifacts, low contrast, and complicated tumor shapes in ultrasound images. To address this issue, we proposed a boundary-oriented network (BO-Net) for boosting breast tumor segmentation in ultrasound images. The BO-Net boosts tumor segmentation performance from two perspectives. Firstly, a boundary-oriented module (BOM) was designed to capture the weak boundaries of breast tumors by learning additional breast tumor boundary maps. Second, we focus on enhanced feature extraction, which takes advantage of the Atrous Spatial Pyramid Pooling (ASPP) module and Squeeze-and-Excitation (SE) block to obtain multi-scale and efficient feature information. We evaluate our network on two public datasets: Dataset B and BUSI. For the Dataset B, our network achieves 0.8685 in Dice, 0.7846 in Jaccard, 0.8604 in Precision, 0.9078 in Recall, and 0.9928 in Specificity. For the BUSI dataset, our network achieves 0.7954 in Dice, 0.7033 in Jaccard, 0.8275 in Precision, 0.8251 in Recall, and 0.9814 in Specificity. Experimental results show that BO-Net outperforms the state-of-the-art segmentation methods for breast tumor segmentation in ultrasound images. It demonstrates that focusing on boundary and feature enhancement creates more efficient and robust breast tumor segmentation.

Hepatocellular Carcinoma Recognition from Ultrasound Images Using Combinations of Conventional and Deep Learning Techniques

Hepatocellular Carcinoma (HCC) is the most frequent malignant liver tumor and the third cause of cancer-related deaths worldwide. For many years, the golden standard for HCC diagnosis has been the needle biopsy, which is invasive and carries risks. Computerized methods are due to achieve a noninvasive, accurate HCC detection process based on medical images. We developed image analysis and recognition methods to perform automatic and computer-aided diagnosis of HCC. Conventional approaches that combined advanced texture analysis, mainly based on Generalized Co-occurrence Matrices (GCM) with traditional classifiers, as well as deep learning approaches based on Convolutional Neural Networks (CNN) and Stacked Denoising Autoencoders (SAE), were involved in our research. The best accuracy of 91% was achieved for B-mode ultrasound images through CNN by our research group. In this work, we combined the classical approaches with CNN techniques, within B-mode ultrasound images. The combination was performed at the classifier level. The CNN features obtained at the output of various convolution layers were combined with powerful textural features, then supervised classifiers were employed. The experiments were conducted on two datasets, acquired with different ultrasound machines. The best performance, above 98%, overpassed our previous results, as well as representative state-of-the-art results.

Improving artificial intelligence pipeline for liver malignancy diagnosis using ultrasound images and video frames

Recent developments of deep learning methods have demonstrated their feasibility in liver malignancy diagnosis using ultrasound (US) images. However, most of these methods require manual selection and annotation of US images by radiologists, which limit their practical application. On the other hand, US videos provide more comprehensive morphological information about liver masses and their relationships with surrounding structures than US images, potentially leading to a more accurate diagnosis. Here, we developed a fully automated artificial intelligence (AI) pipeline to imitate the workflow of radiologists for detecting liver masses and diagnosing liver malignancy. In this pipeline, we designed an automated mass-guided strategy that used segmentation information to direct diagnostic models to focus on liver masses, thus increasing diagnostic accuracy. The diagnostic models based on US videos utilized bi-directional convolutional long short-term memory modules with an attention-boosted module to learn and fuse spatiotemporal information from consecutive video frames. Using a large-scale dataset of 50 063 US images and video frames from 11 468 patients, we developed and tested the AI pipeline and investigated its applications. A dataset of annotated US images is available at https://doi.org/10.5281/zenodo.7272660.

Analysis of cranial ultrasound images for newborn

Introduction

Neonatal cranial ultrasound plays an important role in the evaluation of neonatal brain development and related diseases.

Methods

This paper preliminarily explored the analysis and interpretation methods of neonatal brain ultrasound images, and applied the relevant medical image analysis methods to analyze the relevant neonatal brain ultrasound images in more detail.

Results

Compared with other types of imaging methods, ultrasound has its unique advantages and characteristics in such applications as neonatal head imaging.

Discussion

The analysis steps and schemes adopted in this paper have certain reference significance for the analysis of other neonatal brain pictures.

The Effect of Primary Aldosteronism on Carotid Artery Texture in Ultrasound Images

Primary aldosteronism (PA) is the most frequent cause of secondary hypertension. Early diagnoses of PA are essential to avoid the long-term negative effects of elevated aldosterone concentration on the cardiovascular and renal system. In this work, we study the texture of the carotid artery vessel wall from longitudinal ultrasound images in order to automatically distinguish between PA and essential hypertension (EH). The texture is characterized using 140 Haralick and 10 wavelet features evaluated in a region of interest in the vessel wall, followed by the XGBoost classifier. Carotid ultrasound studies were carried out on 33 patients aged 42-72 years with PA, 52 patients with EH, and 33 normotensive controls. For the most clinically relevant task of distinguishing PA and EH classes, we achieved a classification accuracy of 73% as assessed by a leave-one-out procedure. This result is promising even compared to the 57% prediction accuracy using clinical characteristics alone or 63% accuracy using a combination of clinical characteristics and intima-media thickness (IMT) parameters. If the accuracy is improved and the method incorporated into standard clinical procedures, this could eventually lead to an improvement in the early diagnosis of PA and consequently improve the clinical outcome for these patients in future.

Contrasting EfficientNet, ViT, and gMLP for COVID-19 Detection in Ultrasound Imagery

A timely diagnosis of coronavirus is critical in order to control the spread of the virus. To aid in this, we propose in this paper a deep learning-based approach for detecting coronavirus patients using ultrasound imagery. We propose to exploit the transfer learning of a EfficientNet model pre-trained on the ImageNet dataset for the classification of ultrasound images of suspected patients. In particular, we contrast the results of EfficentNet-B2 with the results of ViT and gMLP. Then, we show the results of the three models by learning from scratch, i.e., without transfer learning. We view the detection problem from a multiclass classification perspective by classifying images as COVID-19, pneumonia, and normal. In the experiments, we evaluated the models on a publically available ultrasound dataset. This dataset consists of 261 recordings (202 videos + 59 images) belonging to 216 distinct patients. The best results were obtained using EfficientNet-B2 with transfer learning. In particular, we obtained precision, recall, and F1 scores of 95.84%, 99.88%, and 24 97.41%, respectively, for detecting the COVID-19 class. EfficientNet-B2 with transfer learning presented an overall accuracy of 96.79%, outperforming gMLP and ViT, which achieved accuracies of 93.03% and 92.82%, respectively.

Local and Context-Attention Adaptive LCA-Net for Thyroid Nodule Segmentation in Ultrasound Images

The thyroid nodule segmentation of ultrasound images is a critical step for the early diagnosis of thyroid cancers in clinics. Due to the weak edge of ultrasound images and the complexity of thyroid tissue structure, it is still challenging to accurately segment the delicate contour of thyroid nodules. A local and context-attention adaptive network (LCA-Net) for thyroid nodule segmentation is proposed to address these shortcomings, which leverages both local feature information from convolution neural networks and global context information from transformers. Firstly, since most existing thyroid nodule segmentation models are skilled at local detail features and lose some context information, we propose a transformers-based context-attention module to capture more global associative information for the network and perceive the edge information of the nodule contour. Secondly, a backbone module with 7×1, 1×7 convolutions and the activation function Mish is designed, which enlarges the receptive field and extracts more feature details. Furthermore, a nodule adaptive convolution (NAC) module is introduced to adaptively deal with thyroid nodules of different sizes and positions, thereby improving the generalization performance of the model. Simultaneously, an optimized loss function is proposed to solve the pixels class imbalance problem in segmentation. The proposed LCA-Net, validated on the public TN-SCUI2020 and TN3K datasets, achieves Dice scores of 90.26% and 82.08% and PA scores of 98.87% and 96.97%, respectively, which outperforms other state-of-the-art thyroid nodule segmentation models. This paper demonstrates the superiority of the proposed LCA-Net for thyroid nodule segmentation, which possesses strong generalization performance and promising segmentation accuracy. Consequently, the proposed model has wide application prospects for thyroid nodule diagnosis in clinics.

Liver Fat Assessment in Multiview Sonography Using Transfer Learning With Convolutional Neural Networks

OBJECTIVES

To develop and evaluate deep learning models devised for liver fat assessment based on ultrasound (US) images acquired from four different liver views: transverse plane (hepatic veins at the confluence with the inferior vena cava, right portal vein, right posterior portal vein) and sagittal plane (liver/kidney).

METHODS

US images (four separate views) were acquired from 135 participants with known or suspected nonalcoholic fatty liver disease. Proton density fat fraction (PDFF) values derived from chemical shift-encoded magnetic resonance imaging served as ground truth. Transfer learning with a deep convolutional neural network (CNN) was applied to develop models for diagnosis of fatty liver (PDFF ≥ 5%), diagnosis of advanced steatosis (PDFF ≥ 10%), and PDFF quantification for each liver view separately. In addition, an ensemble model based on all four liver view models was investigated. Diagnostic performance was assessed using the area under the receiver operating characteristics curve (AUC), and quantification was assessed using the Spearman correlation coefficient (SCC).

RESULTS

The most accurate single view was the right posterior portal vein, with an SCC of 0.78 for quantifying PDFF and AUC values of 0.90 (PDFF ≥ 5%) and 0.79 (PDFF ≥ 10%). The ensemble of models achieved an SCC of 0.81 and AUCs of 0.91 (PDFF ≥ 5%) and 0.86 (PDFF ≥ 10%).

CONCLUSION

Deep learning-based analysis of US images from different liver views can help assess liver fat.

Automatic Classification of Hepatic Cystic Echinococcosis Using Ultrasound Images and Deep Learning

BACKGROUND

Hepatic cystic echinococcosis is the main form of hepatic echinococcosis, which is a life-threatening liver disease caused by parasites that requires a precise diagnosis and proper treatment.

OBJECTIVE

This study focuses on the automatic classification system of five different subtypes of hepatic cystic echinococcosis based on ultrasound images and deep learning algorithms.

METHODS

Three popular deep convolutional neural networks (VGG19, Inception-v3, and ResNet18) with and without pretrained weights were selected to test their performance on the classification task, and the experiments were followed by a 5-fold cross-validation process.

RESULTS

A total of 1820 abdominal ultrasound images covering five subtypes of hepatic cystic echinococcosis from 967 patients were used in the study. The classification accuracy for the models with pretrained weights (fine-tuning) ranged from 88.2 to 90.6%. The best accuracy of 90.6% was obtained by VGG19. For comparison, the models without pretrained weights (from scratch) achieved a lower accuracy, ranging from 69.4 to 75.1%.

CONCLUSION

Deep convolutional neural networks with pretrained weights are capable of recognizing different subtypes of hepatic cystic echinococcosis from ultrasound images, which are expected to be applied in the computer-aided diagnosis systems in future work.

A review of artificial intelligence in prostate cancer detection on imaging

A multitude of studies have explored the role of artificial intelligence (AI) in providing diagnostic support to radiologists, pathologists, and urologists in prostate cancer detection, risk-stratification, and management. This review provides a comprehensive overview of relevant literature regarding the use of AI models in (1) detecting prostate cancer on radiology images (magnetic resonance and ultrasound imaging), (2) detecting prostate cancer on histopathology images of prostate biopsy tissue, and (3) assisting in supporting tasks for prostate cancer detection (prostate gland segmentation, MRI-histopathology registration, MRI-ultrasound registration). We discuss both the potential of these AI models to assist in the clinical workflow of prostate cancer diagnosis, as well as the current limitations including variability in training data sets, algorithms, and evaluation criteria. We also discuss ongoing challenges and what is needed to bridge the gap between academic research on AI for prostate cancer and commercial solutions that improve routine clinical care.

Random Fourier Features-Based Deep Learning Improvement with Class Activation Interpretability for Nerve Structure Segmentation

Peripheral nerve blocking (PNB) is a standard procedure to support regional anesthesia. Still, correct localization of the nerve's structure is needed to avoid adverse effects; thereby, ultrasound images are used as an aid approach. In addition, image-based automatic nerve segmentation from deep learning methods has been proposed to mitigate attenuation and speckle noise ultrasonography issues. Notwithstanding, complex architectures highlight the region of interest lacking suitable data interpretability concerning the learned features from raw instances. Here, a kernel-based deep learning enhancement is introduced for nerve structure segmentation. In a nutshell, a random Fourier features-based approach was utilized to complement three well-known semantic segmentation architectures, e.g., fully convolutional network, U-net, and ResUnet. Moreover, two ultrasound image datasets for PNB were tested. Obtained results show that our kernel-based approach provides a better generalization capability from image segmentation-based assessments on different nerve structures. Further, for data interpretability, a semantic segmentation extension of the GradCam++ for class-activation mapping was used to reveal relevant learned features separating between nerve and background. Thus, our proposal favors both straightforward (shallow) and complex architectures (deeper neural networks).

FMRNet: A fused network of multiple tumoral regions for breast tumor classification with ultrasound images

PURPOSE

Recent studies have illustrated that the peritumoral regions of medical images have value for clinical diagnosis. However, the existing approaches using peritumoral regions mainly focus on the diagnostic capability of the single region and ignore the advantages of effectively fusing the intratumoral and peritumoral regions. In addition, these methods need accurate segmentation masks in the testing stage, which are tedious and inconvenient in clinical applications. To address these issues, we construct a deep convolutional neural network that can adaptively fuse the information of multiple tumoral-regions (FMRNet) for breast tumor classification using ultrasound (US) images without segmentation masks in the testing stage.

METHODS

To sufficiently excavate the potential relationship, we design a fused network and two independent modules to extract and fuse features of multiple regions simultaneously. First, we introduce two enhanced combined-tumoral (EC) region modules, aiming to enhance the combined-tumoral features gradually. Then, we further design a three-branch module for extracting and fusing the features of intratumoral, peritumoral, and combined-tumoral regions, denoted as the intratumoral, peritumoral, and combined-tumoral module. Especially, we design a novel fusion module by introducing a channel attention module to adaptively fuse the features of three regions. The model is evaluated on two public datasets including UDIAT and BUSI with breast tumor ultrasound images. Two independent groups of experiments are performed on two respective datasets using the fivefold stratified cross-validation strategy. Finally, we conduct ablation experiments on two datasets, in which BUSI is used as the training set and UDIAT is used as the testing set.

RESULTS

We conduct detailed ablation experiments about the proposed two modules and comparative experiments with other existing representative methods. The experimental results show that the proposed method yields state-of-the-art performance on both two datasets. Especially, in the UDIAT dataset, the proposed FMRNet achieves a high accuracy of 0.945 and a specificity of 0.945, respectively. Moreover, the precision (PRE = 0.909) even dramatically improves by 21.6% on the BUSI dataset compared with the existing method of the best result.

CONCLUSION

The proposed FMRNet shows good performance in breast tumor classification with US images, and proves its capability of exploiting and fusing the information of multiple tumoral-regions. Furthermore, the FMRNet has potential value in classifying other types of cancers using multiple tumoral-regions of other kinds of medical images.

An ultrasound image-based deep multi-scale texture network for liver fibrosis grading in patients with chronic HBV infection

BACKGROUND & AIMS

The evaluation of the stage of liver fibrosis is essential in patients with chronic liver disease. However, due to the low quality of ultrasound images, the non-invasive diagnosis of liver fibrosis based on ultrasound images is still an outstanding question. This study aimed to investigate the diagnostic accuracy of a deep learning-based method in ultrasound images for liver fibrosis staging in multicentre patients.

METHODS

In this study, we proposed a novel deep learning-based approach, named multi-scale texture network (MSTNet), to assess liver fibrosis, which extracted multi-scale texture features from constructed image pyramid patches. Its diagnostic accuracy was investigated by comparing it with APRI, FIB-4, Forns and sonographers. Data of 508 patients who underwent liver biopsy were included from 4 hospitals. The area-under-the ROC curve (AUC) was determined by receiver operating characteristics (ROC) curves for significant fibrosis (≥F2) and cirrhosis (F4).

RESULTS

The AUCs (95% confidence interval) of MSTNet were 0.92 (0.87-0.96) for ≥F2 and 0.89 (0.83-0.95) for F4 on the validation group, which significantly outperformed APRI, FIB-4 and Forns. The sensitivity and specificity of MSTNet (85.1% (74.5%-92.0%) and 87.6% (78.0%-93.6%)) were better than those of three sonographers in assessing ≥F2.

CONCLUSIONS

The proposed MSTNet is a promising ultrasound image-based method for the non-invasive grading of liver fibrosis in patients with chronic HBV infection.

Calcium Identification and Scoring Based on Echocardiography. An Exploratory Study on Aortic Valve Stenosis

Currently, an echocardiography expert is needed to identify calcium in the aortic valve, and a cardiac CT-Scan image is needed for calcium quantification. When performing a CT-scan, the patient is subject to radiation, and therefore the number of CT-scans that can be performed should be limited, restricting the patient's monitoring. Computer Vision (CV) has opened new opportunities for improved efficiency when extracting knowledge from an image. Applying CV techniques on echocardiography imaging may reduce the medical workload for identifying the calcium and quantifying it, helping doctors to maintain a better tracking of their patients. In our approach, a simple technique to identify and extract the calcium pixel count from echocardiography imaging, was developed by using CV. Based on anonymized real patient echocardiographic images, this approach enables semi-automatic calcium identification. As the brightness of echocardiography images (with the highest intensity corresponding to calcium) vary depending on the acquisition settings, echocardiographic adaptive image binarization has been performed. Given that blood maintains the same intensity on echocardiographic images-being always the darker region-blood areas in the image were used to create an adaptive threshold for binarization. After binarization, the region of interest (ROI) with calcium, was interactively selected by an echocardiography expert and extracted, allowing us to compute a calcium pixel count, corresponding to the spatial amount of calcium. The results obtained from these experiments are encouraging. With this technique, from echocardiographic images collected for the same patient with different acquisition settings and different brightness, obtaining a calcium pixel count, where pixel values show an absolute pixel value margin of error of 3 (on a scale from 0 to 255), achieving a Pearson Correlation of 0.92 indicating a strong correlation with the human expert assessment of calcium area for the same images.

Multiple U-Net-Based Automatic Segmentations and Radiomics Feature Stability on Ultrasound Images for Patients With Ovarian Cancer

Few studies have reported the reproducibility and stability of ultrasound (US) images based radiomics features obtained from automatic segmentation in oncology. The purpose of this study is to study the accuracy of automatic segmentation algorithms based on multiple U-net models and their effects on radiomics features from US images for patients with ovarian cancer. A total of 469 US images from 127 patients were collected and randomly divided into three groups: training sets (353 images), validation sets (23 images), and test sets (93 images) for automatic segmentation models building. Manual segmentation of target volumes was delineated as ground truth. Automatic segmentations were conducted with U-net, U-net++, U-net with Resnet as the backbone (U-net with Resnet), and CE-Net. A python 3.7.0 and package Pyradiomics 2.2.0 were used to extract radiomic features from the segmented target volumes. The accuracy of automatic segmentations was evaluated by Jaccard similarity coefficient (JSC), dice similarity coefficient (DSC), and average surface distance (ASD). The reliability of radiomics features were evaluated by Pearson correlation and intraclass correlation coefficients (ICC). CE-Net and U-net with Resnet outperformed U-net and U-net++ in accuracy performance by achieving a DSC, JSC, and ASD of 0.87, 0.79, 8.54, and 0.86, 0.78, 10.00, respectively. A total of 97 features were extracted from the delineated target volumes. The average Pearson correlation was 0.86 (95% CI, 0.83–0.89), 0.87 (95% CI, 0.84–0.90), 0.88 (95% CI, 0.86–0.91), and 0.90 (95% CI, 0.88–0.92) for U-net++, U-net, U-net with Resnet, and CE-Net, respectively. The average ICC was 0.84 (95% CI, 0.81–0.87), 0.85 (95% CI, 0.82–0.88), 0.88 (95% CI, 0.85–0.90), and 0.89 (95% CI, 0.86–0.91) for U-net++, U-net, U-net with Resnet, and CE-Net, respectively. CE-Net based segmentation achieved the best radiomics reliability. In conclusion, U-net based automatic segmentation was accurate enough to delineate the target volumes on US images for patients with ovarian cancer. Radiomics features extracted from automatic segmented targets showed good reproducibility and for reliability further radiomics investigations.

Breathing Exercise Called the Maximal Abdominal Contraction Maneuver

Background and objectives: The maximal abdominal contraction maneuver (MACM) was designed as an effective and efficient breathing exercise to increase the stability of the spinal joint. However, it has not been determined whether MACM is more effective and efficient than the maximal expiration method. Thus, the present study was undertaken to investigate whole abdominal muscle thickness changes after MACM. Materials and Methods: Thirty healthy subjects (17 males and 13 females) participated in this study. An experimental comparison between MACM and the maximal expiration task was conducted by measuring the change of abdominal muscle thickness such as the transverse abdominis (TrA), internal oblique (IO), external oblique (EO) and rectus abdominis (RA) using ultrasound images. Results: The results indicated that MACM resulted in significantly greater muscle thickness increases of the TrA and RA than the maximal expiration exercise (p < 0.05). Conclusion: MACM provided better exercise than the maximal expiration exercise in terms of increasing spine stability, at least from a co-contraction perspective.

[Detection of carotid intima and media thicknesses based on ultrasound B-mode images clustered with Gaussian mixture model]

In clinic, intima and media thickness are the main indicators for evaluating the development of atherosclerosis. At present, these indicators are measured by professional doctors manually marking the boundaries of the inner and media on B-mode images, which is complicated, time-consuming and affected by many artificial factors. A grayscale threshold method based on Gaussian Mixture Model (GMM) clustering is therefore proposed to detect the intima and media thickness in carotid arteries from B-mode images in this paper. Firstly, the B-mode images are clustered based on the GMM, and the boundary between the intima and media of the vessel wall is then detected by the gray threshold method, and finally the thickness of the two is measured. Compared with the measurement technique using the gray threshold method directly, the clustering of B-mode images of carotid artery solves the problem of gray boundary blurring of inner and middle membrane, thereby improving the stability and detection accuracy of the gray threshold method. In the clinical trials of 120 healthy carotid arteries, means of 4 manual measurements obtained by two experts are used as reference values. Experimental results show that the normalized root mean square errors (NRMSEs) of the estimated intima and media thickness after GMM clustering were 0.104 7 ± 0.076 2 and 0.097 4 ± 0.068 3, respectively. Compared with the results of the direct gray threshold estimation, means of NRMSEs are reduced by 19.6% and 22.4%, respectively, which indicates that the proposed method has higher measurement accuracy. The standard deviations are reduced by 17.0% and 21.7%, respectively, which indicates that the proposed method has better stability. In summary, this method is helpful for early diagnosis and monitoring of vascular diseases, such as atherosclerosis.

Non-invasive evaluation of elasticity of skin with the processing of ultrasound images during ultraviolet radiation: An animal photoaging model

OBJECTIVE

The aim of this study was to provide a non-invasive imaging method to evaluate the physical and mechanical parameters as a novelty method during skin photoaging.

METHODS

In order to evaluate the process of skin damage, 25 mice (C57BL6) were exposed to UVB radiation (0.03 mW/cm² ), 5 times a week for 5 weeks. The thickness of the epidermal and dermal layers was measured weekly from the ultrasound images (40 MHz). The elastic parameters of the skin were estimated from the processing of the sequential ultrasound images with the motion detection algorithm during the injury generation process.

RESULTS

The thickening, Young modulus, and shear modulus of the dermal and epidermal layers during the UVB damage process significantly increased during the 5-week study period (P < .05). In addition, the percentage of changes in the thickness of the epidermal layer (0.22 ± 0.01 mm in day 0 to 0.37 ± 0.02 mm in day 35) and dermal layer (0.57 ± 0.05 mm in day 0 to 0.90 ± 0.08 mm in day 35) increased by 68% and 57%, respectively. Furthermore, Young modulus (154.41 ± 8.8 kPa) was 11 times more than that of non-irradiated skin (14.90 ± 2.2 kPa) and the shear modulus (2.33 ± 0.04 kPa) was 2.2 times more than non-irradiated skin (1.06 ± 0.04 kPa).

CONCLUSION

With processing the sequential ultrasound images and extracting the thickening, the elasticity of the skin layers can detect skin lesions by UVB radiation.

Automated location of thyroid nodules in ultrasound images with improved YOLOV3 network

BACKGROUND

Thyroid ultrasonography is widely used to diagnose thyroid nodules in clinics. Automatic localization of nodules can promote the development of intelligent thyroid diagnosis and reduce workload of radiologists. However, besides the ultrasound image has low contrast and high noise, the thyroid nodules are diverse in shape and vary greatly in size. Thus, thyroid nodule detection in ultrasound images is still a challenging task.

OBJECTIVE

This study proposes an automatic detection algorithm to locate nodules in B ultrasound images and Doppler ultrasound images. This method can be used to screen thyroid nodules and provide a basis for subsequent automatic segmentation and intelligent diagnosis.

METHODS

We develop and optimize an improved YOLOV3 model for detecting thyroid nodules in ultrasound images with B-mode and Doppler mode. Improvements include (1) using the high-resolution network (HRNet) as the basic network for gradually extracting high-level semantic features to reduce the missed detection and misdetection, (2) optimizing the loss function for single target detection like nodules, and (3) obtaining the anchor boxes by clustering the candidate frames of real nodules in the dataset.

RESULTS

The experimental results of applying to 8000 clinical ultrasound images show that the new method developed and tested in this study can effectively detect thyroid nodules. The method achieves 94.53% mean precision and 95.00% mean recall.

CONCLUTIONS

The study demonstrates a new automated method that enables to achieve high detection accuracy and effectively locate thyroid nodules in various ultrasound images without any user interaction, which indicates its potential clinical application value for the thyroid nodule screening.

An Ultrasound Image Enhancement Method Using Neutrosophic Similarity Score

Ultrasound images, having low contrast and noise, adversely impact in the detection of abnormalities. In view of this, an enhancement method is proposed in this work to reduce noise and improve contrast of ultrasound images. The proposed method is based on scaling with neutrosophic similarity score (NSS), where an image is represented in the neutrosophic domain through three membership subsets T, I, and F denoting the degree of truth, indeterminacy, and falseness, respectively. The NSS measures the belonging degree of pixel to the texture using multi-criteria that is based on intensity, local mean intensity and edge detection. Then, NSS is utilized to extract the enhanced coefficient and this enhanced coefficient is applied to scale the input image. This scaling reflects contrast improvement and denoising effect on ultrasound images. The performance of proposed enhancement method is evaluated on clinical ultrasound images, using both subjective and objective image quality measures. In subjective evaluation, with proposed method, overall best score of 4.3 was obtained and that was 44% higher than the score of original images. These results were also supported by objective measures. The results demonstrated that the proposed method outperformed the other methods in terms of mean brightness preservation, edge preservation, structural similarity, and human perception-based image quality assessment. Thus, the proposed method can be used in computer-aided diagnosis systems and to visually assist radiologists in their interactive-decision-making task.

Multi-instance Deep Learning of Ultrasound Imaging Data for Pattern Classification of Congenital Abnormalities of the Kidney and Urinary Tract in Children

OBJECTIVE

To reliably and quickly diagnose children with posterior urethral valves (PUV), we developed a multi-instance deep learning method to automate image analysis.

METHODS

We built a robust pattern classifier to distinguish 86 children with PUV from 71 children with mild unilateral hydronephrosis based on ultrasound images (3504 in sagittal view and 2558 in transverse view) obtained during routine clinical care.

RESULTS

The multi-instance deep learning classifier performed better than classifiers built on either single sagittal images or single transverse images. Particularly, the deep learning classifiers built on single images in the sagittal view and single images in the transverse view obtained area under the receiver operating characteristic curve (AUC) values of 0.796 ± 0.064 and 0.815 ± 0.071, respectively. AUC values of the multi-instance deep learning classifiers built on images in the sagittal and transverse views with mean pooling operation were 0.949 ± 0.035 and 0.954 ± 0.033, respectively. The multi-instance deep learning classifiers built on images in both the sagittal and transverse views with a mean pooling operation obtained an AUC of 0.961 ± 0.026 with a classification rate of 0.925 ± 0.060, specificity of 0.986 ± 0.032, and sensitivity of 0.873 ± 0.120, respectively. Discriminative regions of the kidney located using classification activation mapping demonstrated that the deep learning techniques could identify meaningful anatomical features from ultrasound images.

CONCLUSION

The multi-instance deep learning method provides an automatic and accurate means to extract informative features from ultrasound images and discriminate infants with PUV from male children with unilateral hydronephrosis.

Comparison of Deep-Learning and Conventional Machine-Learning Methods for the Automatic Recognition of the Hepatocellular Carcinoma Areas from Ultrasound Images

The emergence of deep-learning methods in different computer vision tasks has proved to offer increased detection, recognition or segmentation accuracy when large annotated image datasets are available. In the case of medical image processing and computer-aided diagnosis within ultrasound images, where the amount of available annotated data is smaller, a natural question arises: are deep-learning methods better than conventional machine-learning methods? How do the conventional machine-learning methods behave in comparison with deep-learning methods on the same dataset? Based on the study of various deep-learning architectures, a lightweight multi-resolution Convolutional Neural Network (CNN) architecture is proposed. It is suitable for differentiating, within ultrasound images, between the Hepatocellular Carcinoma (HCC), respectively the cirrhotic parenchyma (PAR) on which HCC had evolved. The proposed deep-learning model is compared with other CNN architectures that have been adapted by transfer learning for the ultrasound binary classification task, but also with conventional machine-learning (ML) solutions trained on textural features. The achieved results show that the deep-learning approach overcomes classical machine-learning solutions, by providing a higher classification performance.

Computer-Aided Diagnosis of Congenital Abnormalities of the Kidney and Urinary Tract in Children Using a Multi-Instance Deep Learning Method Based on Ultrasound Imaging Data

Ultrasound images are widely used for diagnosis of congenital abnormalities of the kidney and urinary tract (CAKUT). Since a typical clinical ultrasound image captures 2D information of a specific view plan of the kidney and images of the same kidney on different planes have varied appearances, it is challenging to develop a computer aided diagnosis tool robust to ultrasound images in different views. To overcome this problem, we develop a multi-instance deep learning method for distinguishing children with CAKUT from controls based on their clinical ultrasound images, aiming to automatic diagnose the CAKUT in children based on ultrasound imaging data. Particularly, a multi-instance deep learning method was developed to build a robust pattern classifier to distinguish children with CAKUT from controls based on their ultrasound images in sagittal and transverse views obtained during routine clinical care. The classifier was built on imaging features derived using transfer learning from a pre-trained deep learning model with a mean pooling operator for fusing instance-level classification results. Experimental results have demonstrated that the multi-instance deep learning classifier performed better than classifiers built on either individual sagittal slices or individual transverse slices.

Multi-needle Localization with Attention U-Net in US-guided HDR Prostate Brachytherapy

PURPOSE

Ultrasound (US)-guided high dose rate (HDR) prostate brachytherapy requests the clinicians to place HDR needles (catheters) into the prostate gland under transrectal US (TRUS) guidance in the operating room. The quality of the subsequent radiation treatment plan is largely dictated by the needle placements, which varies upon the experience level of the clinicians and the procedure protocols. Real-time plan dose distribution, if available, could be a vital tool to provide more subjective assessment of the needle placements, hence potentially improving the radiation plan quality and the treatment outcome. However, due to low signal-to-noise ratio (SNR) in US imaging, real-time multi-needle segmentation in 3D TRUS, which is the major obstacle for real-time dose mapping, has not been realized to date. In this study, we propose a deep learning-based method that enables accurate and real-time digitization of the multiple needles in the 3D TRUS images of HDR prostate brachytherapy.

METHODS

A deep learning model based on the U-Net architecture was developed to segment multiple needles in the 3D TRUS images. Attention gates were considered in our model to improve the prediction on the small needle points. Furthermore, the spatial continuity of needles was encoded into our model with total variation (TV) regularization. The combined network was trained on 3D TRUS patches with the deep supervision strategy, where the binary needle annotation images were provided as ground truth. The trained network was then used to localize and segment the HDR needles for a new patient's TRUS images. We evaluated our proposed method based on the needle shaft and tip errors against manually defined ground truth and compared our method with other state-of-art methods (U-Net and deeply supervised attention U-Net).

RESULTS

Our method detected 96% needles of 339 needles from 23 HDR prostate brachytherapy patients with 0.290 ± 0.236 mm at shaft error and 0.442 ± 0.831 mm at tip error. For shaft localization, our method resulted in 96% localizations with less than 0.8 mm error (needle diameter is 1.67 mm), while for tip localization, our method resulted in 75% needles with 0 mm error and 21% needles with 2 mm error (TRUS image slice thickness is 2 mm). No significant difference is observed (P = 0.83) on tip localization between our results with the ground truth. Compared with U-Net and deeply supervised attention U-Net, the proposed method delivers a significant improvement on both shaft error and tip error (P < 0.05).

CONCLUSIONS

We proposed a new segmentation method to precisely localize the tips and shafts of multiple needles in 3D TRUS images of HDR prostate brachytherapy. The 3D rendering of the needles could help clinicians to evaluate the needle placements. It paves the way for the development of real-time plan dose assessment tools that can further elevate the quality and outcome of HDR prostate brachytherapy.

Wavelet-based Computationally-Efficient Computer-Aided Characterization of Liver Steatosis using Conventional B-mode Ultrasound Images

Hepatic steatosis occurs when lipids accumulate in the liver leading to steatohepatitis, which can evolve into cirrhosis and consequently may end with hepatocellular carcinoma. Several automatic classification algorithms have been proposed to detect liver diseases. However, some algorithms are manufacturer-dependent, while others require extensive calculations and consequently prolonged computational time. This may limit the development of real-time and manufacturer-independent computer-aided detection of liver steatosis. This work demonstrates the feasibility of a computationally-efficient and manufacturer-independent wavelet-based computer-aided liver steatosis detection system using conventional B-mode ultrasound (US) imaging. Seven features were extracted from the approximation part of the second-level wavelet packet transform (WPT) of US images. The proposed technique was tested on two datasets of ex-vivo mice livers with and without gelatin embedding, in addition to a third dataset of in-vivo human livers acquired using two different US machines. Using the gelatin-embedded mice liver dataset, the technique exhibited 98.8% accuracy, 97.8% sensitivity, and 100% specificity, and the frame classification time was reduced from 0.4814 s using original US images to 0.1444 s after WPT preprocessing. When the other mice liver dataset was used, the technique showed 85.74% accuracy, 84.4% sensitivity, and 88.5% specificity, and the frame classification time was reduced from 0.5612s to 0.2903 s. Using human liver image data, the best classifier exhibited 92.5% accuracy, 93.0% sensitivity, 91.0% specificity, and the classification time was reduced from 0.660 s to 0.146 s. This technique can be useful for developing computationally-efficient and manufacturer-independent noninvasive CAD systems for fatty liver detection.

A Variance-reduction Approach to Detection of the Thyroid-nodule Boundary on Ultrasound Images

To perform computer-aided diagnosis of the thyroid nodules on ultrasound images, the location and boundary of nodules should be clearly defined. However, the identification of thyroid nodule boundary is a difficult issue due to the biological characteristics of the nodules, the physics and quality of ultrasound imaging, and the subjective factors and operating conditions of the operator. In this study, we propose a novel and semiautomatic method for detecting the boundary of thyroid nodule based on the Variance-Reduction (V-R) statistics without image preprocessing. The region of interest (ROI) is first automatically generated according to the initial inputs of the nodule's major and minor axes. The boundary candidate pixel points are then extracted by using the V-R statistics from the grayscale values of all pixel points in the ROI. Three filtering methods are further applied to eliminate the outlier pixel points to ensure that the remaining candidate pixel points are located on the nodule boundary. Finally, the remaining pixel points are smoothened and linked together to form the final boundary. The proposed method is validated with ultrasound images of 538 thyroid nodules, with manual delineation by experienced radiologist as gold standard. The effectiveness is evaluated and compared with previous publications using boundary error metrics and overlapping area metrics with the same data set. The results show that the normalized average mean boundary error is 1.02%, the true positive overlapping area ratio achieves 93.66% and false positive overlapping area ratio is limited to 7.68%. In conclusion, our proposed method is reliable and effective in detecting thyroid nodule boundary on ultrasound images.

Prediction of Lymph Node Metastasis in Patients With Papillary Thyroid Carcinoma: A Radiomics Method Based on Preoperative Ultrasound Images

BACKGROUND

Papillary thyroid carcinoma is a type of indolent tumor with a dramatically increasing incidence rate and stably high survival rate. Reducing the overdiagnosis and overtreatment of papillary thyroid carcinoma is clinically emergent and important. A radiomics model is proposed in this article to predict lymph node metastasis, the most important risk factor of papillary thyroid carcinoma, based on noninvasive routine preoperative ultrasound images.

METHODS

Four hundred fifty ultrasound manually segmented images of patients with papillary thyroid carcinoma with lymph node status obtained from pathology report were enrolled in our retrospective study. A radiomics evaluation of 614 high-throughput features were calculated, including size, shape, margin, boundary, orientation, position, echo pattern, posterior acoustic pattern, and calcification features. Then, combined feature selection strategy was used to select features with the greatest ability to discriminate lymph node status. A support vector machine classifier was employed to build and validate the prediction model. Another independent testing cohort was used to further evaluate the performance of the radiomics model.

RESULTS

Among 614 radiomics features, 50 selected features most reflecting echo pattern, posterior acoustic pattern, and calcification showed the superior lymph node status distinguishable performance with area under the receiver operating characteristic curve of 0.753, 0.740, and 0.743 separately when using each type of features predicting the lymph node status. The results of model based on all 50 final features predicting the lymph node status shown an area under the receiver operating characteristic curve of 0.782, and accuracy of 0.712. In the independent testing cohort, the proposed approach showed similar results, with area under the receiver operating characteristic curve of 0.727 and accuracy of 0.710.

CONCLUSION

Papillary thyroid carcinoma with lymph node metastasis usually shows a complex echo pattern, posterior region homogeneity, and macrocalcification or multiple calcification. The radiomics model proposed in this article is a promising method for assessing the risk of papillary thyroid carcinoma metastasis noninvasively.

A Dynamic Graph Cuts Method with Integrated Multiple Feature Maps for Segmenting Kidneys in 2D Ultrasound Images

RATIONALE AND OBJECTIVES

Automatic segmentation of kidneys in ultrasound (US) images remains a challenging task because of high speckle noise, low contrast, and large appearance variations of kidneys in US images. Because texture features may improve the US image segmentation performance, we propose a novel graph cuts method to segment kidney in US images by integrating image intensity information and texture feature maps.

MATERIALS AND METHODS

We develop a new graph cuts-based method to segment kidney US images by integrating original image intensity information and texture feature maps extracted using Gabor filters. To handle large appearance variation within kidney images and improve computational efficiency, we build a graph of image pixels close to kidney boundary instead of building a graph of the whole image. To make the kidney segmentation robust to weak boundaries, we adopt localized regional information to measure similarity between image pixels for computing edge weights to build the graph of image pixels. The localized graph is dynamically updated and the graph cuts-based segmentation iteratively progresses until convergence. Our method has been evaluated based on kidney US images of 85 subjects. The imaging data of 20 randomly selected subjects were used as training data to tune parameters of the image segmentation method, and the remaining data were used as testing data for validation.

RESULTS

Experiment results demonstrated that the proposed method obtained promising segmentation results for bilateral kidneys (average Dice index = 0.9446, average mean distance = 2.2551, average specificity = 0.9971, average accuracy = 0.9919), better than other methods under comparison (P < .05, paired Wilcoxon rank sum tests).

CONCLUSIONS

The proposed method achieved promising performance for segmenting kidneys in two-dimensional US images, better than segmentation methods built on any single channel of image information. This method will facilitate extraction of kidney characteristics that may predict important clinical outcomes such as progression of chronic kidney disease.

Classification of breast lesions in ultrasonography using sparse logistic regression and morphology-based texture features

PURPOSE

This work proposes a new reliable computer-aided diagnostic (CAD) system for the diagnosis of breast cancer from breast ultrasound (BUS) images. The system can be useful to reduce the number of biopsies and pathological tests, which are invasive, costly, and often unnecessary.

METHODS

The proposed CAD system classifies breast tumors into benign and malignant classes using morphological and textural features extracted from breast ultrasound (BUS) images. The images are first preprocessed to enhance the edges and filter the speckles. The tumor is then segmented semiautomatically using the watershed method. Having the tumor contour, a set of 855 features including 21 shape-based, 810 contour-based, and 24 textural features are extracted from each tumor. Then, a Bayesian Automatic Relevance Detection (ARD) mechanism is used for computing the discrimination power of different features and dimensionality reduction. Finally, a logistic regression classifier computed the posterior probabilities of malignant vs benign tumors using the reduced set of features.

RESULTS

A dataset of 104 BUS images of breast tumors, including 72 benign and 32 malignant tumors, was used for evaluation using an eightfold cross-validation. The algorithm outperformed six state-of-the-art methods for BUS image classification with large margins by achieving 97.12% accuracy, 93.75% sensitivity, and 98.61% specificity rates.

CONCLUSIONS

Using ARD, the proposed CAD system selects five new features for breast tumor classification and outperforms state-of-the-art, making a reliable and complementary tool to help clinicians diagnose breast cancer.

Enhanced axial and lateral resolution using stabilized pulses

Ultrasound B-scan imaging systems operate under some well-known resolution limits. To improve resolution, the concept of stable pulses, having bounded inverse filters, was previously utilized for the lateral deconvolution. This framework has been extended to the axial direction, enabling a two-dimensional deconvolution. The modeling of the two-way response in the axial direction is discussed, and the deconvolution is performed in the in-phase quadrature data domain. Stable inverse filters are generated and applied for the deconvolution of the image data from Field II simulation, a tissue-mimicking phantom, and in vivo imaging of a carotid artery, where resolution enhancement is observed. Specifically, in simulation results, the resolution is enhanced by as many as 8.75 times laterally and 20.5 times axially considering the [Formula: see text] width of the autocorrelation of the envelope images.

Enhanced axial and lateral resolution using stabilized pulses

Ultrasound B-scan imaging systems operate under some well-known resolution limits. To improve resolution, the concept of stable pulses, having bounded inverse filters, was previously utilized for the lateral deconvolution. This framework has been extended to the axial direction, enabling a two-dimensional deconvolution. The modeling of the two-way response in the axial direction is discussed, and the deconvolution is performed in the in-phase quadrature data domain. Stable inverse filters are generated and applied for the deconvolution of the image data from Field II simulation, a tissue-mimicking phantom, and in vivo imaging of a carotid artery, where resolution enhancement is observed. Specifically, in simulation results, the resolution is enhanced by as many as 8.75 times laterally and 20.5 times axially considering the [Formula: see text] width of the autocorrelation of the envelope images.

Enhanced resolution pulse-echo imaging with stabilized pulses

Many pulse-echo imaging systems use focused beams to improve lateral resolution. The beam width is determined by the choice of source and apodization function, the frequency, and the physics of focusing. Postprocessing strategies to improve lateral resolution can be limited by the need for conditioning the mathematics of inverse filtering, due to instabilities. We present an analysis that defines key constraints on sampled versions of lateral beampatterns. Within these constraints are useful symmetric beampatterns, which, when properly sampled, can have a stable inverse filter. A framework for analysis and processing is described and applied to phantoms and tissues to demonstrate the improvements that can be realized.

Nakagami-based total variation method for speckle reduction in thyroid ultrasound images

A good statistical model is necessary for the reduction in speckle noise. The Nakagami model is more general than the Rayleigh distribution for statistical modeling of speckle in ultrasound images. In this article, the Nakagami-based noise removal method is presented to enhance thyroid ultrasound images and to improve clinical diagnosis. The statistics of log-compressed image are derived from the Nakagami distribution following a maximum a posteriori estimation framework. The minimization problem is solved by optimizing an augmented Lagrange and Chambolle's projection method. The proposed method is evaluated on both artificial speckle-simulated and real ultrasound images. The experimental findings reveal the superiority of the proposed method both quantitatively and qualitatively in comparison with other speckle reduction methods reported in the literature. The proposed method yields an average signal-to-noise ratio gain of more than 2.16 dB over the non-convex regularizer-based speckle noise removal method, 3.83 dB over the Aubert-Aujol model, 1.71 dB over the Shi-Osher model and 3.21 dB over the Rudin-Lions-Osher model on speckle-simulated synthetic images. Furthermore, visual evaluation of the despeckled images shows that the proposed method suppresses speckle noise well while preserving the textures and fine details.

Influence of trunk muscle co-contraction on spinal curvature during sitting

BACKGROUND AND PURPOSE

Slumped sitting is known to increase disc pressure and aggravate chronic low back pain. In addition, it has been recognized that co-contraction of the deep spine-stabilizing muscles enhances lumbar segmental stability and the sacro-iliac joint. The purpose of this study was to compare the electromyographic (EMG) activity of the trunk muscles and the muscle thickness of the transverse abdominis (TrA) during slumped sitting with the same parameters during co-contraction and investigate how co-contraction influences spinal curvature.

SUBJECTS AND METHODS

Nine healthy male volunteers participated in the study. EMG signals were recorded during both sitting postures. In order to measure the muscle thickness of the TrA, ultrasound images were captured. While the subjects performed both sitting postures, spinal curvature was measured using a hand-held device.

RESULTS

Significantly more activity of the trunk muscles, with the exception of the rectus abdominis muscle, and significantly greater muscle thickness of the TrA were observed during co-contraction of the trunk muscles than during slumped sitting. Co-contraction also resulted in significantly increased lumbar lordosis and a greater sacral angle when compared to slumped sitting.

CONCLUSION

In this study, it was demonstrated that the instructions given to the subjects on co-contraction of the trunk muscles during sitting increased muscle activity with the exception of the rectus abdominis muscle, muscle thickness of the TrA, and lumbar lordosis.

Prostate segmentation in HIFU therapy

Prostate segmentation in 3-D transrectal ultrasound images is an important step in the definition of the intra-operative planning of high intensity focused ultrasound (HIFU) therapy. This paper presents two main approaches for the semi-automatic methods based on discrete dynamic contour and optimal surface detection. They operate in 3-D and require a minimal user interaction. They are considered both alone or sequentially combined, with and without postregularization, and applied on anisotropic and isotropic volumes. Their performance, using different metrics, has been evaluated on a set of 28 3-D images by comparison with two expert delineations. For the most efficient algorithm, the symmetric average surface distance was found to be 0.77 mm.

From ultrasound images to block based region motion estimation

By applying a hexagon-diamond search (HDS) method to an ultrasound image, the path of an object is able to be monitored by extracting images into macro-blocks, thereby achieving image redundancy is reduced from one frame to another, and also ascertaining the motion vector within the parameters searched. The HDS algorithm uses six search points to form the six sides of the hexagon pattern, a centre point, and a further four search points to create diamond pattern within the hexagon that clarifies the focus of the subject area.