Multitask Cascade Convolution Neural Networks for Automatic Thyroid Nodule Detection and Recognition

Identification method of thyroid nodule ultrasonography based on self-supervised learning dual-branch attention learning framework

Thyroid ultrasound is a widely used diagnostic technique for thyroid nodules in clinical practice. However, due to the characteristics of ultrasonic imaging, such as low image contrast, high noise levels, and heterogeneous features, detecting and identifying nodules remains challenging. In addition, high-quality labeled medical imaging datasets are rare, and thyroid ultrasound images are no exception, posing a significant challenge for machine learning applications in medical image analysis. In this study, we propose a Dual-branch Attention Learning (DBAL) convolutional neural network framework to enhance thyroid nodule detection by capturing contextual information. Leveraging jigsaw puzzles as a pretext task during network training, we improve the network's generalization ability with limited data. Our framework effectively captures intrinsic features in a global-to-local manner. Experimental results involve self-supervised pre-training on unlabeled ultrasound images and fine-tuning using 1216 clinical ultrasound images from a collaborating hospital. DBAL achieves accurate discrimination of thyroid nodules, with a 88.5% correct diagnosis rate for malignant and benign nodules and a 93.7% area under the ROC curve. This novel approach demonstrates promising potential in clinical applications for its accuracy and efficiency.

Semi-Supervised Thyroid Nodule Detection in Ultrasound Videos

Deep learning techniques have been investigated for the computer-aided diagnosis of thyroid nodules in ultrasound images. However, most existing thyroid nodule detection methods were simply based on static ultrasound images, which cannot well explore spatial and temporal information following the clinical examination process. In this paper, we propose a novel video-based semi-supervised framework for ultrasound thyroid nodule detection. Especially, considering clinical examinations that need to detect thyroid nodules at the ultrasonic probe positions, we first construct an adjacent frame guided detection backbone network by using adjacent supporting reference frames. To further reduce the labour-intensive thyroid nodule annotation in ultrasound videos, we extend the video-based detection in a semi-supervised manner by using both labeled and unlabeled videos. Based on the detection consistency in sequential neighbouring frames, a pseudo label adaptation strategy is proposed for the refinement of unpredicted frames. The proposed framework is validated on 996 transverse viewed and 1088 longitudinal viewed ultrasound videos. Experimental results demonstrated the superior performance of our proposed method in the ultrasound video-based detection of thyroid nodules.

Thyroid Ultrasound Image Database and Marker Mask Inpainting Method for Research and Development

OBJECTIVE

The main objective of this study was to build a rich and high-quality thyroid ultrasound image database (TUD) for computer-aided diagnosis (CAD) systems to support accurate diagnosis and prognostic modeling of thyroid disorders. Because most of the raw thyroid ultrasound images contain artificial markers, which seriously affect the robustness of CAD systems because of their strong prior location information, we propose a marker mask inpainting (MMI) method to erase artificial markers and improve image quality.

METHODS

First, a set of thyroid ultrasound images were collected from the General Hospital of the Northern Theater Command. Then, two modules were designed in MMI, namely, the marker detection (MD) module and marker erasure (ME) module. The MD module detects all markers in the image and stores them in a binary mask. According to the binary mask, the ME module erases the markers and generates an unmarked image. Finally, a new TUD based on the marked images and unmarked images was built. The TUD is carefully annotated and statistically analyzed by professional physicians to ensure accuracy and consistency. Moreover, several normal thyroid gland images and some ancillary information on benign and malignant nodules are provided.

RESULTS

Several typical segmentation models were evaluated on the TUD. The experimental results revealed that our TUD can facilitate the development of more accurate CAD systems for the analysis of thyroid nodule-related lesions in ultrasound images. The effectiveness of our MMI method was determined in quantitative experiments.

CONCLUSION

The rich and high-quality resource TUD promotes the development of more effective diagnostic and treatment methods for thyroid diseases. Furthermore, MMI for erasing artificial markers and generating unmarked images is proposed to improve the quality of thyroid ultrasound images. Our TUD database is available at https://github.com/NEU-LX/TUD-Datebase.

A multitask approach for automated detection and segmentation of thyroid nodules in ultrasound images

An increase in the incidence and diagnosis of thyroid nodules and thyroid cancer underscores the need for a better approach to nodule detection and risk stratification in ultrasound (US) images that can reduce healthcare costs, patient discomfort, and unnecessary invasive procedures. However, variability in ultrasound technique and interpretation makes the diagnostic process partially subjective. Therefore, an automated approach that detects and segments nodules could improve performance on downstream tasks, such as risk stratification. Ultrasound studies were acquired from 280 patients at UCLA Health, totaling 9888 images, and annotated by collaborating radiologists. Current deep learning architectures for segmentation are typically semi-automated because they are evaluated solely on images known to have nodules and do not assess ability to identify suspicious images. However, the proposed multitask approach both detects suspicious images and segments potential nodules; this allows for a clinically translatable model that aptly parallels the workflow for thyroid nodule assessment. The multitask approach is centered on an anomaly detection (AD) module that can be integrated with any UNet architecture variant to improve image-level nodule detection. Of the evaluated multitask models, a UNet with a ImageNet pretrained encoder and AD achieved the highest F1 score of 0.839 and image-wide Dice similarity coefficient of 0.808 on the hold-out test set. Furthermore, models were evaluated on two external validations datasets to demonstrate generalizability and robustness to data variability. Ultimately, the proposed architecture is an automated multitask method that expands on previous methods by successfully both detecting and segmenting nodules in ultrasound.

Shape-margin knowledge augmented network for thyroid nodule segmentation and diagnosis

BACKGROUND AND OBJECTIVE

Thyroid nodule segmentation is a crucial step in the diagnostic procedure of physicians and computer-aided diagnosis systems. However, prevailing studies often treat segmentation and diagnosis as independent tasks, overlooking the intrinsic relationship between these processes. The sequencial steps of these independent tasks in computer-aided diagnosis systems may lead to the accumulation of errors. Therefore, it is worth combining them as a whole by exploring the relationship between thyroid nodule segmentation and diagnosis. According to the diagnostic procedure of thyroid imaging reporting and data system (TI-RADS), the assessment of shape and margin characteristics is the prerequisite for radiologists to discriminate benign and malignant thyroid nodules. Inspired by TI-RADS, this study aims to integrate these tasks into a cohesive process, leveraging the insights from TI-RADS, thereby enhancing the accuracy and interpretability of thyroid nodule analysis.

METHODS

Specifically, this paper proposes a shape-margin knowledge augmented network (SkaNet) for simultaneous thyroid nodule segmentation and diagnosis. Due to the visual feature similarities between segmentation and diagnosis, SkaNet shares visual features in the feature extraction stage and then utilizes a dual-branch architecture to perform thyroid nodule segmentation and diagnosis tasks respectively. In the shared feature extraction, the combination of convolutional feature maps and self-attention maps allows to exploitation of both local information and global patterns in thyroid nodule images. To enhance effective discriminative features, an exponential mixture module is introduced, combining convolutional feature maps and self-attention maps through exponential weighting. Then, SkaNet is jointly optimized by a knowledge augmented multi-task loss function with a constraint penalty term. The constraint penalty term embeds shape and margin characteristics through numerical computations, establishing a vital relationship between thyroid nodule diagnosis results and segmentation masks.

RESULTS

We evaluate the proposed approach on a public thyroid ultrasound dataset (DDTI) and a locally collected thyroid ultrasound dataset. The experimental results reveal the value of our contributions and demonstrate that our approach can yield significant improvements compared with state-of-the-art counterparts.

CONCLUSIONS

SkaNet highlights the potential of combining thyroid nodule segmentation and diagnosis with knowledge augmented learning into a unified framework, which captures the key shape and margin characteristics for discriminating benign and malignant thyroid nodules. Our findings suggest promising insights for advancing computer-aided diagnosis joint with segmentation.

Thyroid Cancer Polygenic Risk Score Improves Classification of Thyroid Nodules as Benign or Malignant

CONTEXT

Thyroid nodule ultrasound-based risk stratification schemas rely on the presence of high-risk sonographic features. However, some malignant thyroid nodules have benign appearance on thyroid ultrasound. New methods for thyroid nodule risk assessment are needed.

OBJECTIVE

We investigated polygenic risk score (PRS) accounting for inherited thyroid cancer risk combined with ultrasound-based analysis for improved thyroid nodule risk assessment.

METHODS

The convolutional neural network classifier was trained on thyroid ultrasound still images and cine clips from 621 thyroid nodules. Phenome-wide association study (PheWAS) and PRS PheWAS were used to optimize PRS for distinguishing benign and malignant nodules. PRS was evaluated in 73 346 participants in the Colorado Center for Personalized Medicine Biobank.

RESULTS

When the deep learning model output was combined with thyroid cancer PRS and genetic ancestry estimates, the area under the receiver operating characteristic curve (AUROC) of the benign vs malignant thyroid nodule classifier increased from 0.83 to 0.89 (DeLong, P value = .007). The combined deep learning and genetic classifier achieved a clinically relevant sensitivity of 0.95, 95% CI [0.88-0.99], specificity of 0.63 [0.55-0.70], and positive and negative predictive values of 0.47 [0.41-0.58] and 0.97 [0.92-0.99], respectively. AUROC improvement was consistent in European ancestry-stratified analysis (0.83 and 0.87 for deep learning and deep learning combined with PRS classifiers, respectively). Elevated PRS was associated with a greater risk of thyroid cancer structural disease recurrence (ordinal logistic regression, P value = .002).

CONCLUSION

Augmenting ultrasound-based risk assessment with PRS improves diagnostic accuracy.

A knowledge-interpretable multi-task learning framework for automated thyroid nodule diagnosis in ultrasound videos

Ultrasound has become the most widely used modality for thyroid nodule diagnosis, due to its portability, real-time feedback, lack of toxicity, and low cost. Recently, the computer-aided diagnosis (CAD) of thyroid nodules has attracted significant attention. However, most existing techniques can only be applied to either static images with prominent features (manually selected from scanning videos) or rely on 'black boxes' that cannot provide interpretable results. In this study, we develop a user-friendly framework for the automated diagnosis of thyroid nodules in ultrasound videos, by simulating the typical diagnostic workflow used by radiologists. This process consists of two orderly part-to-whole tasks. The first interprets the characteristics of each image using prior knowledge, to obtain corresponding frame-wise TI-RADS scores. Associated embedded representations not only provide diagnostic information for radiologists but also reduce computational costs. The second task models temporal contextual information in an embedding vector sequence and selectively enhances important information to distinguish benign and malignant thyroid nodules, thereby improving the efficiency and generalizability of the proposed framework. Experimental results demonstrated this approach outperformed other state-of-the-art video classification methods. In addition to assisting radiologists in understanding model predictions, these CAD results could further ease diagnostic workloads and improve patient care.

Self-supervised enhanced thyroid nodule detection in ultrasound examination video sequences with multi-perspective evaluation

Objective.Ultrasound is the most commonly used examination for the detection and identification of thyroid nodules. Since manual detection is time-consuming and subjective, attempts to introduce machine learning into this process are ongoing. However, the performance of these methods is limited by the low signal-to-noise ratio and tissue contrast of ultrasound images. To address these challenges, we extend thyroid nodule detection from image-based to video-based using the temporal context information in ultrasound videos.Approach.We propose a video-based deep learning model with adjacent frame perception (AFP) for accurate and real-time thyroid nodule detection. Compared to image-based methods, AFP can aggregate semantically similar contextual features in the video. Furthermore, considering the cost of medical image annotation for video-based models, a patch scale self-supervised model (PASS) is proposed. PASS is trained on unlabeled datasets to improve the performance of the AFP model without additional labelling costs.Main results.The PASS model is trained by 92 videos containing 23 773 frames, of which 60 annotated videos containing 16 694 frames were used to train and evaluate the AFP model. The evaluation is performed from the video, frame, nodule, and localization perspectives. In the evaluation of the localization perspective, we used the average precision metric with the intersection-over-union threshold set to 50% (AP@50), which is the area under the smoothed Precision-Recall curve. Our proposed AFP improved AP@50 from 0.256 to 0.390, while the PASS-enhanced AFP further improved the AP@50 to 0.425. AFP and PASS also improve the performance in the valuations of other perspectives based on the localization results.Significance.Our video-based model can mitigate the effects of low signal-to-noise ratio and tissue contrast in ultrasound images and enable the accurate detection of thyroid nodules in real-time. The evaluation from multiple perspectives of the ablation experiments demonstrates the effectiveness of our proposed AFP and PASS models.

The performance of deep learning on thyroid nodule imaging predicts thyroid cancer: A systematic review and meta-analysis of epidemiological studies with independent external test sets

BACKGROUND AND AIMS

It is still controversial whether deep learning (DL) systems add accuracy to thyroid nodule imaging classification based on the recent available evidence. We conducted this study to analyze the current evidence of DL in thyroid nodule imaging diagnosis in both internal and external test sets.

METHODS

Until the end of December 2022, PubMed, IEEE, Embase, Web of Science, and the Cochrane Library were searched. We included primary epidemiological studies using externally validated DL techniques in image-based thyroid nodule appraisal. This systematic review was registered on PROSPERO (CRD42022362892).

RESULTS

We evaluated evidence from 17 primary epidemiological studies using externally validated DL techniques in image-based thyroid nodule appraisal. Fourteen studies were deemed eligible for meta-analysis. The pooled sensitivity, specificity, and area under the curve (AUC) of these DL algorithms were 0.89 (95% confidence interval 0.87-0.90), 0.84 (0.82-0.86), and 0.93 (0.91-0.95), respectively. For the internal validation set, the pooled sensitivity, specificity, and AUC were 0.91 (0.89-0.93), 0.88 (0.85-0.91), and 0.96 (0.93-0.97), respectively. In the external validation set, the pooled sensitivity, specificity, and AUC were 0.87 (0.85-0.89), 0.81 (0.77-0.83), and 0.91 (0.88-0.93), respectively. Notably, in subgroup analyses, DL algorithms still demonstrated exceptional diagnostic validity.

CONCLUSIONS

Current evidence suggests DL-based imaging shows diagnostic performances comparable to clinicians for differentiating thyroid nodules in both the internal and external test sets.

Study on diagnosis of thyroid nodules based on convolutional neural network

OBJECTIVE

An artificial intelligence (AI) algorithm based on convolutional neural networks was used in ultrasound diagnosis in order to evaluate its performance in judging the nature of thyroid nodules and nodule classification.

METHODS

A total of 105 patients with thyroid nodules confirmed by surgery or biopsy were retrospectively analyzed. The properties, characteristics, and classification of thyroid nodules were evaluated by sonographers and by AI to obtain combined diagnoses. Receiver operating characteristic curves were generated to evaluate the performance of AI, the sonographer, and their combined effort in diagnosing the nature of thyroid nodules and classifying their characteristics. In the diagnosis of thyroid nodules with solid components, hypoechoic appearance, indistinct borders, Anteroposterior/transverse diameter ratio > 1(A/T > 1), and calcification performed by sonographers and by AI, the properties exhibited statistically significant differences.

RESULTS

Sonographers had a sensitivity of 80.7%, specificity of 73.7%, accuracy of 79.0%, and area under the curve (AUC) of 0.751 in the diagnosis of benign and malignant thyroid nodules. AI had a sensitivity of 84.5%, specificity of 81.0%, accuracy of 84.7%, and AUC of 0.803. The combined AI and sonographer diagnosis had a sensitivity of 92.1%, specificity of 86.3%, accuracy of 91.7%, and AUC of 0.910.

CONCLUSION

The efficacy of a combined diagnosis for benign and malignant thyroid nodules is higher than that of an AI-based diagnosis alone or a sonographer-based diagnosis alone. The combined diagnosis can reduce unnecessary fine-needle aspiration biopsy procedures and better evaluate the necessity of surgery in clinical practice.

A Deep Learning Framework for the Characterization of Thyroid Nodules from Ultrasound Images Using Improved Inception Network and Multi-Level Transfer Learning

In the past few years, deep learning has gained increasingly widespread attention and has been applied to diagnosing benign and malignant thyroid nodules. It is difficult to acquire sufficient medical images, resulting in insufficient data, which hinders the development of an efficient deep-learning model. In this paper, we developed a deep-learning-based characterization framework to differentiate malignant and benign nodules from the thyroid ultrasound images. This approach improves the recognition accuracy of the inception network by combining squeeze and excitation networks with the inception modules. We have also integrated the concept of multi-level transfer learning using breast ultrasound images as a bridge dataset. This transfer learning approach addresses the issues regarding domain differences between natural images and ultrasound images during transfer learning. This paper aimed to investigate how the entire framework could help radiologists improve diagnostic performance and avoid unnecessary fine-needle aspiration. The proposed approach based on multi-level transfer learning and improved inception blocks achieved higher precision (0.9057 for the benign class and 0.9667 for the malignant class), recall (0.9796 for the benign class and 0.8529 for malignant), and F1-score (0.9412 for benign class and 0.9062 for malignant class). It also obtained an AUC value of 0.9537, which is higher than that of the single-level transfer learning method. The experimental results show that this model can achieve satisfactory classification accuracy comparable to experienced radiologists. Using this model, we can save time and effort as well as deliver potential clinical application value.

Assessment of encoder-decoder-based segmentation models for thyroid ultrasound images

Encoder-decoder-based semantic segmentation models classify image pixels into the corresponding class, such as the ROI (region of interest) or background. In the present study, simple / dilated convolution / series / directed acyclic graph (DAG)-based encoder-decoder semantic segmentation models have been implemented, i.e., SegNet (VGG16), SegNet (VGG19), U-Net, mobileNetv2, ResNet18, ResNet50, Xception and Inception networks for the segment TTUS(Thyroid Tumor Ultrasound) images. Transfer learning has been used to train these segmentation networks using original and despeckled TTUS images. The performance of the networks has been calculated using mIoU and mDC metrics. Based on the exhaustive experiments, it has been observed that ResNet50-based segmentation model obtained the best results objectively with values 0.87 for mIoU, 0.94 for mDC, and also according to radiologist opinion on shape, margin, and echogenicity characteristics of segmented lesions. It is noted that the segmentation model, namely ResNet50, provides better segmentation based on objective and subjective assessment. It may be used in the healthcare system to identify thyroid nodules accurately in real time.

Artificial intelligence in thyroid ultrasound

Artificial intelligence (AI), particularly deep learning (DL) algorithms, has demonstrated remarkable progress in image-recognition tasks, enabling the automatic quantitative assessment of complex medical images with increased accuracy and efficiency. AI is widely used and is becoming increasingly popular in the field of ultrasound. The rising incidence of thyroid cancer and the workload of physicians have driven the need to utilize AI to efficiently process thyroid ultrasound images. Therefore, leveraging AI in thyroid cancer ultrasound screening and diagnosis cannot only help radiologists achieve more accurate and efficient imaging diagnosis but also reduce their workload. In this paper, we aim to present a comprehensive overview of the technical knowledge of AI with a focus on traditional machine learning (ML) algorithms and DL algorithms. We will also discuss their clinical applications in the ultrasound imaging of thyroid diseases, particularly in differentiating between benign and malignant nodules and predicting cervical lymph node metastasis in thyroid cancer. Finally, we will conclude that AI technology holds great promise for improving the accuracy of thyroid disease ultrasound diagnosis and discuss the potential prospects of AI in this field.

A study on the optimal condition of ground truth area for liver tumor detection in ultrasound images using deep learning

PURPOSE

In recent years, efforts to apply artificial intelligence (AI) to the medical field have been growing. In general, a vast amount of high-quality training data is necessary to make great AI. For tumor detection AI, annotation quality is important. In diagnosis and detection of tumors using ultrasound images, humans use not only the tumor area but also the surrounding information, such as the back echo of the tumor. Therefore, we investigated changes in detection accuracy when changing the size of the region of interest (ROI, ground truth area) relative to liver tumors in the training data for the detection AI.

METHODS

We defined D/L as the ratio of the maximum diameter (D) of the liver tumor to the ROI size (L). We created training data by changing the D/L value, and performed learning and testing with YOLOv3.

RESULTS

Our results showed that the detection accuracy was highest when the training data were created with a D/L ratio between 0.8 and 1.0. In other words, it was found that the detection accuracy was improved by setting the ground true bounding box for detection AI training to be in contact with the tumor or slightly larger. We also found that when the D/L ratio was distributed in the training data, the wider the distribution, the lower the detection accuracy.

CONCLUSIONS

Therefore, we recommend that the detector be trained with the D/L value close to a certain value between 0.8 and 1.0 for liver tumor detection from ultrasound images.

A Novel Deep-Learning-Based CADx Architecture for Classification of Thyroid Nodules Using Ultrasound Images

Nodules of thyroid cancer occur in the cells of the thyroid as benign or malign types. Thyroid sonographic images are mostly used for diagnosis of thyroid cancer. The aim of this study is to introduce a computer-aided diagnosis system that can classify the thyroid nodules with high accuracy using the data gathered from ultrasound images. Acquisition and labeling of sub-images were performed by a specialist physician. Then the number of these sub-images were increased using data augmentation methods. Deep features were obtained from the images using a pre-trained deep neural network. The dimensions of the features were reduced and features were improved. The improved features were combined with morphological and texture features. This feature group was rated by a value called similarity coefficient value which was obtained from a similarity coefficient generator module. The nodules were classified as benign or malignant using a multi-layer deep neural network with a pre-weighting layer designed with a novel approach. In this study, a novel multi-layer computer-aided diagnosis system was proposed for thyroid cancer detection. In the first layer of the system, a novel feature extraction method based on the class similarity of images was developed. In the second layer, a novel pre-weighting layer was proposed by modifying the genetic algorithm. The proposed system showed superior performance in different metrics compared to the literature.

Automatic detection and diagnosis of thyroid ultrasound images based on attention mechanism

Incidents of thyroid cancer have dramatically increased in recent years; however, early ultrasound diagnosis can reduce morbidity and mortality. The work in clinical situations relies heavily on the subjective experience of the sonographer. Numerous computer-aided diagnostic techniques exist, but most consider how good the results are, ignoring the pre-image collecting and its usefulness in post-clinical practise. To address these issues, this study proposes a computer-aided diagnosis method based on an attentional mechanism. Due to its lightweight properties, the model can rapidly identify nodules and distinguish between benign and malignant ones without using much hardware. The model uses a bounding box to locate the thyroid nodule and determines whether it is benign or cancerous, and outputs the diagnostic result of the thyroid nodule ultrasound images. The latest attention mechanisms are used to get better results at a fraction of the cost. Additionally, ultrasound images with different features of benign and malignant thyroid nodules were collected following the Thyroid Imaging Reporting and Data System standards. The experimental results showed that the approach identifies and classifies thyroid nodules rapidly and effectively; the mAP value of the results reached 0.89, and the mAP value of malignant nodules reached 0.94, with detection rate of single image reached 7 ms. Young physicians and small hospitals with limited resources can benefit from using this method to assist with thyroid ultrasound examination diagnosis.

Classification for thyroid nodule using ViT with contrastive learning in ultrasound images

The lack of representative features between benign nodules, especially level 3 of Thyroid Imaging Reporting and Data System (TI-RADS), and malignant nodules limits diagnostic accuracy, leading to inconsistent interpretation, overdiagnosis, and unnecessary biopsies. We propose a Vision-Transformer-based (ViT) thyroid nodule classification model using contrast learning, called TC-ViT, to improve accuracy of diagnosis and specificity of biopsy recommendations. ViT can explore the global features of thyroid nodules well. Nodule images are used as ROI to enhance the local features of the ViT. Contrast learning can minimize the representation distance between nodules of the same category, enhance the representation consistency of global and local features, and achieve accurate diagnosis of TI-RADS 3 or malignant nodules. The test results achieve an accuracy of 86.9%. The evaluation metrics show that the network outperforms other classical deep learning-based networks in terms of classification performance. TC-ViT can achieve automatic classification of TI-RADS 3 and malignant nodules on ultrasound images. It can also be used as a key step in computer-aided diagnosis for comprehensive analysis and accurate diagnosis. The code will be available at https://github.com/Jiawei217/TC-ViT.

Robust and accurate pulmonary nodule detection with self-supervised feature learning on domain adaptation

Medical imaging data annotation is expensive and time-consuming. Supervised deep learning approaches may encounter overfitting if trained with limited medical data, and further affect the robustness of computer-aided diagnosis (CAD) on CT scans collected by various scanner vendors. Additionally, the high false-positive rate in automatic lung nodule detection methods prevents their applications in daily clinical routine diagnosis. To tackle these issues, we first introduce a novel self-learning schema to train a pre-trained model by learning rich feature representatives from large-scale unlabeled data without extra annotation, which guarantees a consistent detection performance over novel datasets. Then, a 3D feature pyramid network (3DFPN) is proposed for high-sensitivity nodule detection by extracting multi-scale features, where the weights of the backbone network are initialized by the pre-trained model and then fine-tuned in a supervised manner. Further, a High Sensitivity and Specificity (HS2 ) network is proposed to reduce false positives by tracking the appearance changes among continuous CT slices on Location History Images (LHI) for the detected nodule candidates. The proposed method's performance and robustness are evaluated on several publicly available datasets, including LUNA16, SPIE-AAPM, LungTIME, and HMS. Our proposed detector achieves the state-of-the-art result of 90.6 % sensitivity at 1 / 8 false positive per scan on the LUNA16 dataset. The proposed framework's generalizability has been evaluated on three additional datasets (i.e., SPIE-AAPM, LungTIME, and HMS) captured by different types of CT scanners.

Ultrasound-based deep learning using the VGGNet model for the differentiation of benign and malignant thyroid nodules: A meta-analysis

Objective

The aim of this study was to evaluate the accuracy of deep learning using the convolutional neural network VGGNet model in distinguishing benign and malignant thyroid nodules based on ultrasound images.

Methods

Relevant studies were selected from PubMed, Embase, Cochrane Library, China National Knowledge Infrastructure (CNKI), and Wanfang databases, which used the deep learning-related convolutional neural network VGGNet model to classify benign and malignant thyroid nodules based on ultrasound images. Cytology and pathology were used as gold standards. Furthermore, reported eligibility and risk bias were assessed using the QUADAS-2 tool, and the diagnostic accuracy of deep learning VGGNet was analyzed with pooled sensitivity, pooled specificity, diagnostic odds ratio, and the area under the curve.

Results

A total of 11 studies were included in this meta-analysis. The overall estimates of sensitivity and specificity were 0.87 [95% CI (0.83, 0.91)] and 0.85 [95% CI (0.79, 0.90)], respectively. The diagnostic odds ratio was 38.79 [95% CI (22.49, 66.91)]. The area under the curve was 0.93 [95% CI (0.90, 0.95)]. No obvious publication bias was found.

Conclusion

Deep learning using the convolutional neural network VGGNet model based on ultrasound images performed good diagnostic efficacy in distinguishing benign and malignant thyroid nodules.

Systematic Review Registration

https://www.crd.york.ac.nk/prospero, identifier CRD42022336701.

Local and Context-Attention Adaptive LCA-Net for Thyroid Nodule Segmentation in Ultrasound Images

The thyroid nodule segmentation of ultrasound images is a critical step for the early diagnosis of thyroid cancers in clinics. Due to the weak edge of ultrasound images and the complexity of thyroid tissue structure, it is still challenging to accurately segment the delicate contour of thyroid nodules. A local and context-attention adaptive network (LCA-Net) for thyroid nodule segmentation is proposed to address these shortcomings, which leverages both local feature information from convolution neural networks and global context information from transformers. Firstly, since most existing thyroid nodule segmentation models are skilled at local detail features and lose some context information, we propose a transformers-based context-attention module to capture more global associative information for the network and perceive the edge information of the nodule contour. Secondly, a backbone module with 7×1, 1×7 convolutions and the activation function Mish is designed, which enlarges the receptive field and extracts more feature details. Furthermore, a nodule adaptive convolution (NAC) module is introduced to adaptively deal with thyroid nodules of different sizes and positions, thereby improving the generalization performance of the model. Simultaneously, an optimized loss function is proposed to solve the pixels class imbalance problem in segmentation. The proposed LCA-Net, validated on the public TN-SCUI2020 and TN3K datasets, achieves Dice scores of 90.26% and 82.08% and PA scores of 98.87% and 96.97%, respectively, which outperforms other state-of-the-art thyroid nodule segmentation models. This paper demonstrates the superiority of the proposed LCA-Net for thyroid nodule segmentation, which possesses strong generalization performance and promising segmentation accuracy. Consequently, the proposed model has wide application prospects for thyroid nodule diagnosis in clinics.

A Novel Distant Domain Transfer Learning Framework for Thyroid Image Classification

Medical ultrasound imaging technology is currently the preferred method for early diagnosis of thyroid nodules. Radiologists' analysis of ultrasound images is highly dependent on their clinical experience and is susceptible to intra- and inter-observer variability. Although end-to-end deep learning technique can address these limitations, the difficulty of acquiring annotated medical image makes it very challenging. Transfer learning can alleviate the problems, but the large gap between source and target domain will lead to negative transfer. In this paper, a novel transfer learning method with distant domain high-level feature fusion (DHFF) model is proposed. It reduces the distribution distance between the source domain and the target domain while maintaining the characteristics of respective domains, which can avoid excessive feature fusion while enabling the model to learn more valuable transfer knowledge. The DHFF is validated by multiple public source and private target datasets in experiments. The results show that the classification accuracy of DHFF is up to 88.92% with thyroid ultrasound auxiliary source domains, which is up to 8% higher than existing transfer and distant transfer algorithms.

A lightweight network for automatic thyroid nodules location and recognition with high speed and accuracy in ultrasound images

BACKGROUND

The intelligent diagnosis of thyroid nodules in ultrasound image is an important research issue. Automatically locating the region of interest (ROI) of thyroid nodules and providing pre-diagnosis results can help doctors to diagnose faster and more accurate.

OBJECTIVES

This study aims to propose a model, which can detect multiple nodules stably and accurately in order to avoid missed detection and misjudgment. In addition, the detection speed of the model needs to be fast for real-time diagnosis in ultrasound images.

METHODS

Based on the object detection technology, we propose an accurate, robust and high-speed network with multiscale fusion strategy called Efficient-YOLO, which can realize the localization and recognition of nodules at the same time. Finally, multiple metrics are used to measure the diagnostic ability of the model.

RESULTS

Experimental results conducted on 3,562 ultrasound images show that our new model greatly increases the accuracy and speed of the detection compared with the baseline model. The best mAP is 92.64%, and the fastest detection speed is 45.1 frames per second.

CONCLUSIONS

This study proposed an effective method to diagnosis thyroid nodules automatically, which can meet the real-time requirements, indicating that its effectiveness and feasibility for future clinical application.

A Multi-Scale Densely Connected Convolutional Neural Network for Automated Thyroid Nodule Classification

Automated thyroid nodule classification in ultrasound images is an important way to detect thyroid nodules and to make a more accurate diagnosis. In this paper, we propose a novel deep convolutional neural network (CNN) model, called n-ClsNet, for thyroid nodule classification. Our model consists of a multi-scale classification layer, multiple skip blocks, and a hybrid atrous convolution (HAC) block. The multi-scale classification layer first obtains multi-scale feature maps in order to make full use of image features. After that, each skip-block propagates information at different scales to learn multi-scale features for image classification. Finally, the HAC block is used to replace the downpooling layer so that the spatial information can be fully learned. We have evaluated our n-ClsNet model on the TNUI-2021 dataset. The proposed n-ClsNet achieves an average accuracy (ACC) score of 93.8% in the thyroid nodule classification task, which outperforms several representative state-of-the-art classification methods.

Machine intelligence in non-invasive endocrine cancer diagnostics

Artificial intelligence (AI) has illuminated a clear path towards an evolving health-care system replete with enhanced precision and computing capabilities. Medical imaging analysis can be strengthened by machine learning as the multidimensional data generated by imaging naturally lends itself to hierarchical classification. In this Review, we describe the role of machine intelligence in image-based endocrine cancer diagnostics. We first provide a brief overview of AI and consider its intuitive incorporation into the clinical workflow. We then discuss how AI can be applied for the characterization of adrenal, pancreatic, pituitary and thyroid masses in order to support clinicians in their diagnostic interpretations. This Review also puts forth a number of key evaluation criteria for machine learning in medicine that physicians can use in their appraisals of these algorithms. We identify mitigation strategies to address ongoing challenges around data availability and model interpretability in the context of endocrine cancer diagnosis. Finally, we delve into frontiers in systems integration for AI, discussing automated pipelines and evolving computing platforms that leverage distributed, decentralized and quantum techniques.

An artificial intelligence ultrasound system's ability to distinguish benign from malignant follicular-patterned lesions

OBJECTIVES

To evaluate the application value of a generally trained artificial intelligence (AI) automatic diagnosis system in the malignancy diagnosis of follicular-patterned thyroid lesions (FPTL), including follicular thyroid carcinoma (FTC), adenomatoid hyperplasia nodule (AHN) and follicular thyroid adenoma (FTA) and compare the diagnostic performance with radiologists of different experience levels.

METHODS

We retrospectively reviewed 607 patients with 699 thyroid nodules that included 168 malignant nodules by using postoperative pathology as the gold standard, and compared the diagnostic performances of three radiologists (one junior, two senior) and that of AI automatic diagnosis system in malignancy diagnosis of FPTL in terms of sensitivity, specificity and accuracy, respectively. Pairwise t-test was used to evaluate the statistically significant difference.

RESULTS

The accuracy of the AI system in malignancy diagnosis was 0.71, which was higher than the best radiologist in this study by a margin of 0.09 with a p-value of 2.08×10^-5. Two radiologists had higher sensitivity (0.84 and 0.78) than that of the AI system (0.69) at the cost of having much lower specificity (0.35, 0.57 versus 0.71). One senior radiologist showed balanced sensitivity and specificity (0.62 and 0.54) but both were lower than that of the AI system.

CONCLUSIONS

The generally trained AI automatic diagnosis system can potentially assist radiologists for distinguishing FTC from other FPTL cases that share poorly distinguishable ultrasonographical features.

A deep learning-based method for detecting and classifying the ultrasound images of suspicious thyroid nodules

PURPOSE

The incidence of thyroid cancer has significantly increased in the last few decades. However, diagnosis of the thyroid nodules is labor and time intensive for radiologists and strongly depends on the personal experience of the radiologists. In this pursuit, the present study envisaged to develop a deep learning-based computer-aided diagnosis (CAD) method that enabled the automatic detection and classification of suspicious thyroid nodules in order to reduce the unnecessary fine-needle aspiration biopsy.

METHODS

The CAD method consisted of two main parts: detecting the location of thyroid nodules using a multiscale detection network and classifying the detected thyroid nodules by an attention-based classification network.

RESULTS

The performance of the proposed method was evaluated and compared with that of other state-of-the-art deep learning methods and experienced radiologists. The proposed detection method outperformed three other detection architectures (average precision, 82.1% vs. 78.3%, 77.2%, and 74.8%). Moreover, the classification method showed a superior performance compared with four other state-of-the-art classification networks (accuracy, 94.8% vs. 91.2%, 85.0%, 80.8%, and 72.1%) and that by experienced radiologists (mean value of area under the curve, 0.941 vs. 0.833).

CONCLUSIONS

Our study verified the high efficiency of the proposed detection method. The findings can help improve the diagnostic performance of radiologists. However, the developed CAD system requires more training and evaluation in a large-population study.

Machine Learning Applications for Head and Neck Imaging

The head and neck (HN) consists of a large number of vital anatomic structures within a compact area. Imaging plays a central role in the diagnosis and management of major disorders affecting the HN. This article reviews the recent applications of machine learning (ML) in HN imaging with a focus on deep learning approaches. It categorizes ML applications in HN imaging into deep learning and traditional ML applications and provides examples of each category. It also discusses the main challenges facing the successful deployment of ML-based applications in the clinical setting and provides suggestions for addressing these challenges.

Functional link convolutional neural network for the classification of diabetes mellitus

Diabetes is a faction of metabolic ailments distinguished by hyperglycemia which is the consequence of a defect, in the action of insulin, insulin secretion, or both and producing various abnormalities in the human body. In recent years, the utilization of intelligent systems has been expanded in disease classification and numerous researches have been proposed. In this research article, a variant of Convolutional Neural Network (CNN) that is, Functional Link Convolutional Neural Network (FLCNN) is proposed for the diabetes classification. The main goal of this article is to find the potential of a computationally less complex deep learning network like FLCNN and applied the proposed technique on a real dataset of diabetes for classification. This article also presents the comparative studies where various other machine learning techniques are implemented and outcomes are compared with the proposed FLCNN network. The performance of each classification techniques have been evaluated based on standard measures and also validated with a non-parametric statistical test such as Friedman. Data for modeling diabetes classification is collected from Bombay Medical Hall, Upper Bazar, Ranchi, India. Accuracy achieve by the proposed classifier is more than 90% which is closer to the other state-of-the-art implemented classifiers.

Clinical value of contrast-enhanced ultrasonography in focal hypoechogenic lesions of thyroid

OBJECTIVE

The objectives of this study were to analyze the accuracy of contrast-enhanced ultrasonography (CE-US) in diagnosing focal hypoechogenic lesions of the thyroid (FHLT), and to explore the clinical value of CE-US in the diagnosis of FHLT.

METHODS

Patients undergoing CE-US and ultrasound-guided fine needle aspiration (US-FNA) of FHLT at First Hospital of China Medical University between January 2017 and December 2018 were selected for the study; this included patients with papillary thyroid carcinoma (PTC), subacute thyroiditis (SAT) and focal Hashimoto thyroiditis (FHT). All patients underwent color Doppler ultrasonography (CD-US) after which thyroid image reporting and data system (TI-RADS) grading were done. Then, each patient underwent CE-US and US-FNA. The results of the CE-US were analyzed using descriptive statistics. The cytopathological results from the US-FNAs were the gold standard used to confirm the diagnoses.

RESULTS

A total of 56 patients were selected for the study. In the PTC group (n = 16), grading was as follows: TI-RADS4a, n = 3; TI-RADS4b, n = 12; and TI-RADS4c, n = 1. More patients with PTC showed heterogeneous hypoenhancement (n = 15) than heterogeneous isoenhancement (n = 1) on CE-US. In the SAT group (n = 24), grading was as follows: TI-RADS3, n = 1; TI-RADS4a, n = 18; TI-RADS4b, n = 5. Fewer patients with SAT showed heterogeneous hypoenhancement (n = 2) than heterogeneous isoenhancement (n = 22) on CE-US. In the FHT group (n = 16), grading was as follows: TI-RADS3, n = 1; TI-RADS4a, n = 11; TI-RADS4b, n = 4. Of those in the FHT group, one patient showed heterogeneous isoenhancement, one patient showed heterogeneous hypoenhancement, and 14 showed uniform isoenhancement on CE-US. The diagnostic accuracy of CD-US alone differed significantly from that of CD-US + CE-US (p < 0.05).

CONCLUSION

CE-US has a high diagnostic accuracy for FHLT and can be used to identify PTC, SAT, and FHT.

A Visually Interpretable Deep Learning Framework for Histopathological Image-Based Skin Cancer Diagnosis

Owing to the high incidence rate and the severe impact of skin cancer, the precise diagnosis of malignant skin tumors is a significant goal, especially considering treatment is normally effective if the tumor is detected early. Limited published histopathological image sets and the lack of an intuitive correspondence between the features of lesion areas and a certain type of skin cancer pose a challenge to the establishment of high-quality and interpretable computer-aided diagnostic (CAD) systems. To solve this problem, a light-weight attention mechanism-based deep learning framework, namely, DRANet, is proposed to differentiate 11 types of skin diseases based on a real histopathological image set collected by us during the last 10 years. The CAD system can output not only the name of a certain disease but also a visualized diagnostic report showing possible areas related to the disease. The experimental results demonstrate that the DRANet obtains significantly better performance than baseline models (i.e., InceptionV3, ResNet50, VGG16, and VGG19) with comparable parameter size and competitive accuracy with fewer model parameters. Visualized results produced by the hidden layers of the DRANet actually highlight part of the class-specific regions of diagnostic points and are valuable for decision making in the diagnosis of skin diseases.

Deep Learning Based on ACR TI-RADS Can Improve the Differential Diagnosis of Thyroid Nodules

Objective

The purpose of this study was to improve the differentiation between malignant and benign thyroid nodules using deep learning (DL) in category 4 and 5 based on the Thyroid Imaging Reporting and Data System (TI-RADS, TR) from the American College of Radiology (ACR).

Design and Methods

From June 2, 2017 to April 23, 2019, 2082 thyroid ultrasound images from 1396 consecutive patients with confirmed pathology were retrospectively collected, of which 1289 nodules were category 4 (TR4) and 793 nodules were category 5 (TR5). Ninety percent of the B-mode ultrasound images were applied for training and validation, and the residual 10% and an independent external dataset for testing purpose by three different deep learning algorithms.

Results

In the independent test set, the DL algorithm of best performance got an AUC of 0.904, 0.845, 0.829 in TR4, TR5, and TR4&5, respectively. The sensitivity and specificity of the optimal model was 0.829, 0.831 on TR4, 0.846, 0.778 on TR5, 0.790, 0.779 on TR4&5, versus the radiologists of 0.686 (P=0.108), 0.766 (P=0.101), 0.677 (P=0.211), 0.750 (P=0.128), and 0.680 (P=0.023), 0.761 (P=0.530), respectively.

Conclusions

The study demonstrated that DL could improve the differentiation of malignant from benign thyroid nodules and had significant potential for clinical application on TR4 and TR5.

Artificial intelligence in ultrasound

Ultrasound (US), a flexible green imaging modality, is expanding globally as a first-line imaging technique in various clinical fields following with the continual emergence of advanced ultrasonic technologies and the well-established US-based digital health system. Actually, in US practice, qualified physicians should manually collect and visually evaluate images for the detection, identification and monitoring of diseases. The diagnostic performance is inevitably reduced due to the intrinsic property of high operator-dependence from US. In contrast, artificial intelligence (AI) excels at automatically recognizing complex patterns and providing quantitative assessment for imaging data, showing high potential to assist physicians in acquiring more accurate and reproducible results. In this article, we will provide a general understanding of AI, machine learning (ML) and deep learning (DL) technologies; We then review the rapidly growing applications of AI-especially DL technology in the field of US-based on the following anatomical regions: thyroid, breast, abdomen and pelvis, obstetrics heart and blood vessels, musculoskeletal system and other organs by covering image quality control, anatomy localization, object detection, lesion segmentation, and computer-aided diagnosis and prognosis evaluation; Finally, we offer our perspective on the challenges and opportunities for the clinical practice of biomedical AI systems in US.

On the identification of thyroid nodules using semi-supervised deep learning

Detecting malign cases from thyroid nodule examinations is crucial in healthcare particularly to improve the early detection of such cases. However, malign thyroid nodules can be extremely rare and is hard to find using the traditional rule based or expert-based methods. For this reason, the solutions backed by Machine Learning (ML) algorithms are key to improve the detection rates of such rare cases. In this paper, we investigate the application of ML in the healthcare domain for the detection of rare thyroid nodules. The utilized dataset is collected from 636 distinct patients in 99 unique days in Turkey. In addition to the texture feature data of the Ultrasound (US), we have also included the scores of different assessment methods created by different health institutions (e.g., Korean, American and European thyroid societies) as additional features. For detection of extremely rare malign cases, we use auto-encoder based neural network model. Through numerical results, it is shown that the auto-encoder based model can result in an average Recall score of 0.98 and a Sensitivity score of 1.00 for detecting malign and non-malign cases from the healthcare dataset outperforming the traditional classification algorithms that are trained after Synthetic Minority Oversampling Technique (SMOTE) oversampling.

Thyroid nodule recognition using a joint convolutional neural network with information fusion of ultrasound images and radiofrequency data

OBJECTIVE

To develop a deep learning-based method with information fusion of US images and RF signals for better classification of thyroid nodules (TNs).

METHODS

One hundred sixty-three pairs of US images and RF signals of TNs from a cohort of adult patients were used for analysis. We developed an information fusion-based joint convolutional neural network (IF-JCNN) for the differential diagnosis of malignant and benign TNs. The IF-JCNN contains two branched CNNs for deep feature extraction: one for US images and the other one for RF signals. The extracted features are fused at the backend of IF-JCNN for TN classification.

RESULTS

Across 5-fold cross-validation, the accuracy, sensitivity, specificity, and area under the receiver operating characteristic curve (AUROC) obtained by using the IF-JCNN with both US images and RF signals as inputs for TN classification were respectively 0.896 (95% CI 0.838-0.938), 0.885 (95% CI 0.804-0.941), 0.910 (95% CI 0.815-0.966), and 0.956 (95% CI 0.926-0.987), which were better than those obtained by using only US images: 0.822 (0.755-0.878; p = 0.0044), 0.792 (0.679-0.868, p = 0.0091), 0.866 (0.760-0.937, p = 0.197), and 0.901 (0.855-0.948, p = .0398), or RF signals: 0.767 (0.694-0.829, p < 0.001), 0.781 (0.685-0.859, p = 0.0037), 0.746 (0.625-0.845, p < 0.001), 0.845 (0.786-0.903, p < 0.001).

CONCLUSIONS

The proposed IF-JCNN model filled the gap of just using US images in CNNs to characterize TNs, and it may serve as a promising tool for assisting the diagnosis of thyroid cancer.

KEY POINTS

• Raw radiofrequency signals before ultrasound imaging of thyroid nodules provide useful information that is not carried by ultrasound images. • The information carried by raw radiofrequency signals and ultrasound images for thyroid nodules is complementary. • The performance of deep convolutional neural network for diagnosing thyroid nodules can be significantly improved by fusing US images and RF signals in the model as compared with just using US images.

Segmentation and classification of thyroid follicular neoplasm using cascaded convolutional neural network

In this paper, we present a segmentation and classification method for thyroid follicular neoplasms based on a combination of the prior-based level set method and deep convolutional neural network. The proposed method aims to discriminate thyroid follicular adenoma (TFA) and follicular thyroid carcinoma (FTC) in ultrasound images. In their appearance, these two kinds of tumours have similar shapes, sizes and contrasts. Therefore, it is difficult for even ultrasound specialists to distinguish them. Because of the complex background in thyroid ultrasound images, before distinguishing TFA and FTC, we need to segment the lesions from the whole image for each patient. The main challenge of segmentation is that the images often have weak edges and heterogeneous regions. The main issue of classification is that the accuracy depends on the features extracted from the segmentation results. To solve these problems, we conduct the two tasks, i.e. segmentation and classification, by a cascaded learning architecture. For segmentation, to obtain more accurate results, we exploit the Res-U-net framework and the prior-based level set method to enhance their respective abilities. Then, the classification network is trained by sharing shallow layers of the segmentation network. Testing the proposed method on real patient data shows that it is able to segment the lesion areas in thyroid ultrasound images with a Dice score of 92.65% and to distinguish TFA and FTC with a classification accuracy of 96.00%.

Artificial intelligence to predict the BRAFV600E mutation in patients with thyroid cancer

Purpose

To investigate whether a computer-aided diagnosis (CAD) program developed using the deep learning convolutional neural network (CNN) on neck US images can predict the BRAF^V600E mutation in thyroid cancer.

Methods

469 thyroid cancers in 469 patients were included in this retrospective study. A CAD program recently developed using the deep CNN provided risks of malignancy (0–100%) as well as binary results (cancer or not). Using the CAD program, we calculated the risk of malignancy based on a US image of each thyroid nodule (CAD value). Univariate and multivariate logistic regression analyses were performed including patient demographics, the American College of Radiology (ACR) Thyroid Imaging, Reporting and Data System (TIRADS) categories and risks of malignancy calculated through CAD to identify independent predictive factors for the BRAF^V600E mutation in thyroid cancer. The predictive power of the CAD value and final multivariable model for the BRAF^V600E mutation in thyroid cancer were measured using the area under the receiver operating characteristic (ROC) curves.

Results

In this study, 380 (81%) patients were positive and 89 (19%) patients were negative for the BRAF^V600E mutation. On multivariate analysis, older age (OR = 1.025, p = 0.018), smaller size (OR = 0.963, p = 0.006), and higher CAD value (OR = 1.016, p = 0.004) were significantly associated with the BRAF^V600E mutation. The CAD value yielded an AUC of 0.646 (95% CI: 0.576, 0.716) for predicting the BRAF^V600E mutation, while the multivariable model yielded an AUC of 0.706 (95% CI: 0.576, 0.716). The multivariable model showed significantly better performance than the CAD value alone (p = 0.004).

Conclusion

Deep learning-based CAD for thyroid US can help us predict the BRAF^V600E mutation in thyroid cancer. More multi-center studies with more cases are needed to further validate our study results.

Automated location of thyroid nodules in ultrasound images with improved YOLOV3 network

BACKGROUND

Thyroid ultrasonography is widely used to diagnose thyroid nodules in clinics. Automatic localization of nodules can promote the development of intelligent thyroid diagnosis and reduce workload of radiologists. However, besides the ultrasound image has low contrast and high noise, the thyroid nodules are diverse in shape and vary greatly in size. Thus, thyroid nodule detection in ultrasound images is still a challenging task.

OBJECTIVE

This study proposes an automatic detection algorithm to locate nodules in B ultrasound images and Doppler ultrasound images. This method can be used to screen thyroid nodules and provide a basis for subsequent automatic segmentation and intelligent diagnosis.

METHODS

We develop and optimize an improved YOLOV3 model for detecting thyroid nodules in ultrasound images with B-mode and Doppler mode. Improvements include (1) using the high-resolution network (HRNet) as the basic network for gradually extracting high-level semantic features to reduce the missed detection and misdetection, (2) optimizing the loss function for single target detection like nodules, and (3) obtaining the anchor boxes by clustering the candidate frames of real nodules in the dataset.

RESULTS

The experimental results of applying to 8000 clinical ultrasound images show that the new method developed and tested in this study can effectively detect thyroid nodules. The method achieves 94.53% mean precision and 95.00% mean recall.

CONCLUTIONS

The study demonstrates a new automated method that enables to achieve high detection accuracy and effectively locate thyroid nodules in various ultrasound images without any user interaction, which indicates its potential clinical application value for the thyroid nodule screening.

A Multitask Cascading CNN with MultiScale Infrared Optical Flow Feature Fusion-Based Abnormal Crowd Behavior Monitoring UAV

Visual-based object detection and understanding is an important problem in computer vision and signal processing. Due to their advantages of high mobility and easy deployment, unmanned aerial vehicles (UAV) have become a flexible monitoring platform in recent years. However, visible-light-based methods are often greatly influenced by the environment. As a result, a single type of feature derived from aerial monitoring videos is often insufficient to characterize variations among different abnormal crowd behaviors. To address this, we propose combining two types of features to better represent behavior, namely, multitask cascading CNN (MC-CNN) and multiscale infrared optical flow (MIR-OF), capturing both crowd density and average speed and the appearances of the crowd behaviors, respectively. First, an infrared (IR) camera and Nvidia Jetson TX1 were chosen as an infrared vision system. Since there are no published infrared-based aerial abnormal-behavior datasets, we provide a new infrared aerial dataset named the IR-flying dataset, which includes sample pictures and videos in different scenes of public areas. Second, MC-CNN was used to estimate the crowd density. Third, MIR-OF was designed to characterize the average speed of crowd. Finally, considering two typical abnormal crowd behaviors of crowd aggregating and crowd escaping, the experimental results show that the monitoring UAV system can detect abnormal crowd behaviors in public areas effectively.

Efficient Deep Learning Architecture for Detection and Recognition of Thyroid Nodules

Ultrasonography is widely used in the clinical diagnosis of thyroid nodules. Ultrasound images of thyroid nodules have different appearances, interior features, and blurred borders that are difficult for a physician to diagnose into malignant or benign types merely through visual recognition. The development of artificial intelligence, especially deep learning, has led to great advances in the field of medical image diagnosis. However, there are some challenges to achieve precision and efficiency in the recognition of thyroid nodules. In this work, we propose a deep learning architecture, you only look once v3 dense multireceptive fields convolutional neural network (YOLOv3-DMRF), based on YOLOv3. It comprises a DMRF-CNN and multiscale detection layers. In DMRF-CNN, we integrate dilated convolution with different dilation rates to continue passing the edge and the texture features to deeper layers. Two different scale detection layers are deployed to recognize the different sizes of the thyroid nodules. We used two datasets to train and evaluate the YOLOv3-DMRF during the experiments. One dataset includes 699 original ultrasound images of thyroid nodules collected from a local health physical center. We obtained 10,485 images after data augmentation. Another dataset is an open-access dataset that includes ultrasound images of 111 malignant and 41 benign thyroid nodules. Average precision (AP) and mean average precision (mAP) are used as the metrics for quantitative and qualitative evaluations. We compared the proposed YOLOv3-DMRF with some state-of-the-art deep learning networks. The experimental results show that YOLOv3-DMRF outperforms others on mAP and detection time on both the datasets. Specifically, the values of mAP and detection time were 90.05 and 95.23% and 3.7 and 2.2 s, respectively, on the two test datasets. Experimental results demonstrate that the proposed YOLOv3-DMRF is efficient for detection and recognition of thyroid nodules for ultrasound images.

Computer-Aided Diagnosis Systems in Diagnosing Malignant Thyroid Nodules on Ultrasonography: A Systematic Review and Meta-Analysis

BACKGROUND

Computer-aided diagnosis (CAD) systems are being applied to the ultrasonographic diagnosis of malignant thyroid nodules, but it remains controversial whether the systems add any accuracy for radiologists.

OBJECTIVE

To determine the accuracy of CAD systems in diagnosing malignant thyroid nodules.

METHODS

PubMed, EMBASE, and the Cochrane Library were searched for studies on the diagnostic performance of CAD systems. The diagnostic performance was assessed by pooled sensitivity and specificity, and their accuracy was compared with that of radiologists. The present systematic review was registered in PROSPERO (CRD42019134460).

RESULTS

Nineteen studies with 4,781 thyroid nodules were included. Both the classic machine learning- and the deep learning-based CAD system had good performance in diagnosing malignant thyroid nodules (classic machine learning: sensitivity 0.86 [95% CI 0.79-0.92], specificity 0.85 [95% CI 0.77-0.91], diagnostic odds ratio (DOR) 37.41 [95% CI 24.91-56.20]; deep learning: sensitivity 0.89 [95% CI 0.81-0.93], specificity 0.84 [95% CI 0.75-0.90], DOR 40.87 [95% CI 18.13-92.13]). The diagnostic performance of the deep learning-based CAD system was comparable to that of the radiologists (sensitivity 0.87 [95% CI 0.78-0.93] vs. 0.87 [95% CI 0.85-0.89], specificity 0.85 [95% CI 0.76-0.91] vs. 0.87 [95% CI 0.81-0.91], DOR 40.12 [95% CI 15.58-103.33] vs. DOR 44.88 [95% CI 30.71-65.57]).

CONCLUSIONS

The CAD systems demonstrated good performance in diagnosing malignant thyroid nodules. However, experienced radiologists may still have an advantage over CAD systems during real-time diagnosis.

Ultrasound Image-Based Diagnosis of Malignant Thyroid Nodule Using Artificial Intelligence

Computer-aided diagnosis systems have been developed to assist doctors in diagnosing thyroid nodules to reduce errors made by traditional diagnosis methods, which are mainly based on the experiences of doctors. Therefore, the performance of such systems plays an important role in enhancing the quality of a diagnosing task. Although there have been the state-of-the art studies regarding this problem, which are based on handcrafted features, deep features, or the combination of the two, their performances are still limited. To overcome these problems, we propose an ultrasound image-based diagnosis of the malignant thyroid nodule method using artificial intelligence based on the analysis in both spatial and frequency domains. Additionally, we propose the use of weighted binary cross-entropy loss function for the training of deep convolutional neural networks to reduce the effects of unbalanced training samples of the target classes in the training data. Through our experiments with a popular open dataset, namely the thyroid digital image database (TDID), we confirm the superiority of our method compared to the state-of-the-art methods.

Differentiation of thyroid nodules on US using features learned and extracted from various convolutional neural networks

Thyroid nodules are a common clinical problem. Ultrasonography (US) is the main tool used to sensitively diagnose thyroid cancer. Although US is non-invasive and can accurately differentiate benign and malignant thyroid nodules, it is subjective and its results inevitably lack reproducibility. Therefore, to provide objective and reliable information for US assessment, we developed a CADx system that utilizes convolutional neural networks and the machine learning technique. The diagnostic performances of 6 radiologists and 3 representative results obtained from the proposed CADx system were compared and analyzed.