Thyroid Nodule Classification in Ultrasound Images by Fine-Tuning Deep Convolutional Neural Network

An interpretable two-branch bi-coordinate network based on multi-grained domain knowledge for classification of thyroid nodules in ultrasound images

Computer-aided diagnosis (CAD) for thyroid nodules has been studied for years, yet there are still reliability and interpretability challenges due to the lack of clinically-relevant evidence. To address this issue, inspired by Thyroid Imaging Reporting and Data System (TI-RADS), we propose a novel interpretable two-branch bi-coordinate network based on multi-grained domain knowledge. First, we transform the two types of domain knowledge provided by TI-RADS, namely region-based and boundary-based knowledge, into labels at multi-grained levels: coarse-grained classification labels, and fine-grained region segmentation masks and boundary localization vectors. We combine these two labels to form the Multi-grained Domain Knowledge Representation (MG-DKR) of TI-RADS. Then we design a Two-branch Bi-coordinate network (TB2C-net) which utilizes two branches to predict MG-DKR from both Cartesian and polar images, and uses an attention-based integration module to integrate the features of the two branches for benign-malignant classification. We validated our method on a large cohort containing 3245 patients (with 3558 nodules and 6466 ultrasound images). Results show that our method achieves competitive performance with AUC of 0.93 and ACC of 0.87 compared with other state-of-the-art methods. Ablation experiment results demonstrate the effectiveness of the TB2C-net and MG-DKR, and the knowledge attention map from the integration module provides the interpretability for benign-malignant classification.

Deblurring masked image modeling for ultrasound image analysis

Recently, large pretrained vision foundation models based on masked image modeling (MIM) have attracted unprecedented attention and achieved remarkable performance across various tasks. However, the study of MIM for ultrasound imaging remains relatively unexplored, and most importantly, current MIM approaches fail to account for the gap between natural images and ultrasound, as well as the intrinsic imaging characteristics of the ultrasound modality, such as the high noise-to-signal ratio. In this paper, motivated by the unique high noise-to-signal ratio property in ultrasound, we propose a deblurring MIM approach specialized to ultrasound, which incorporates a deblurring task into the pretraining proxy task. The incorporation of deblurring facilitates the pretraining to better recover the subtle details within ultrasound images that are vital for subsequent downstream analysis. Furthermore, we employ a multi-scale hierarchical encoder to extract both local and global contextual cues for improved performance, especially on pixel-wise tasks such as segmentation. We conduct extensive experiments involving 280,000 ultrasound images for the pretraining and evaluate the downstream transfer performance of the pretrained model on various disease diagnoses (nodule, Hashimoto's thyroiditis) and task types (classification, segmentation). The experimental results demonstrate the efficacy of the proposed deblurring MIM, achieving state-of-the-art performance across a wide range of downstream tasks and datasets. Overall, our work highlights the potential of deblurring MIM for ultrasound image analysis, presenting an ultrasound-specific vision foundation model.

Automatic Classification of Nodules from 2D Ultrasound Images Using Deep Learning Networks

OBJECTIVE

In clinical practice, thyroid nodules are typically visually evaluated by expert physicians using 2D ultrasound images. Based on their assessment, a fine needle aspiration (FNA) may be recommended. However, visually classifying thyroid nodules from ultrasound images may lead to unnecessary fine needle aspirations for patients. The aim of this study is to develop an automatic thyroid ultrasound image classification system to prevent unnecessary FNAs.

METHODS

An automatic computer-aided artificial intelligence system is proposed for classifying thyroid nodules using a fine-tuned deep learning model based on the DenseNet architecture, which incorporates an attention module. The dataset comprises 591 thyroid nodule images categorized based on the Bethesda score. Thyroid nodules are classified as either requiring FNA or not. The challenges encountered in this task include managing variability in image quality, addressing the presence of artifacts in ultrasound image datasets, tackling class imbalance, and ensuring model interpretability. We employed techniques such as data augmentation, class weighting, and gradient-weighted class activation maps (Grad-CAM) to enhance model performance and provide insights into decision making.

RESULTS

Our approach achieved excellent results with an average accuracy of 0.94, F1-score of 0.93, and sensitivity of 0.96. The use of Grad-CAM gives insights on the decision making and then reinforce the reliability of the binary classification for the end-user perspective.

CONCLUSIONS

We propose a deep learning architecture that effectively classifies thyroid nodules as requiring FNA or not from ultrasound images. Despite challenges related to image variability, class imbalance, and interpretability, our method demonstrated a high classification accuracy with minimal false negatives, showing its potential to reduce unnecessary FNAs in clinical settings.

Machine Learning on Ultrasound Texture Analysis Data for Characterizing of Salivary Glandular Tumors: A Feasibility Study

BACKGROUND

Objective quantitative texture characteristics may be helpful in salivary glandular tumor differential diagnosis. This study uses machine learning (ML) to explore and validate the performance of ultrasound (US) texture features in diagnosing salivary glandular tumors.

MATERIAL AND METHODS

122 patients with salivary glandular tumors, including 71 benign and 51 malignant tumors, are enrolled. Representative brightness mode US pictures are selected for further Gray Level Co-occurrence Matrix (GLCM) texture analysis. We use a t-test to test the significance and use the receiver operating characteristic curve method to find the optimal cut-point for these significant features. After splitting 80% of the data into a training set and 20% data into a testing set, we use five machine learning models, k-nearest Neighbors (kNN), Naïve Bayes, Logistic regression, Artificial Neural Networks (ANNs) and supportive vector machine (SVM), to explore and validate the performance of US GLCM texture features in diagnosing salivary glandular tumors.

RESULTS

This study includes 49 female and 73 male patients, with a mean age of 53 years old, ranging from 21 to 93. We find that six GLCM texture features (contrast, inverse difference movement, entropy, dissimilarity, inverse difference and difference entropy) are significantly different between benign and malignant tumors (p < 0.05). In ML, the overall accuracy rates are 74.3% (95%CI: 59.8-88.8%), 94.3% (86.6-100%), 72% (54-89%), 84% (69.5-97.3%) and 73.5% (58.7-88.4%) for kNN, Naïve Bayes, Logistic regression, a one-node ANN and SVM, respectively.

CONCLUSIONS

US texture analysis with ML has potential as an objective and valuable tool to make a differential diagnosis between benign and malignant salivary gland tumors.

Automatic Segmentation and Alignment of Uterine Shapes from 3D Ultrasound Data

BACKGROUND

The uterus is the most important organ in the female reproductive system. Its shape plays a critical role in fertility and pregnancy outcomes. Advances in medical imaging, such as 3D ultrasound, have significantly improved the exploration of the female genital tract, thereby enhancing gynecological healthcare. Despite well-documented data for organs like the liver and heart, large-scale studies on the uterus are lacking. Existing classifications, such as VCUAM and ESHRE/ESGE, provide different definitions for normal uterine shapes but are not based on real-world measurements. Moreover, the lack of comprehensive datasets significantly hinders research in this area. Our research, part of the larger NURSE study, aims to fill this gap by establishing the shape of a normal uterus using real-world 3D vaginal ultrasound scans. This will facilitate research into uterine shape abnormalities associated with infertility and recurrent miscarriages.

METHODS

We developed an automated system for the segmentation and alignment of uterine shapes from 3D ultrasound data, which consists of two steps: automatic segmentation of the uteri in 3D ultrasound scans using deep learning techniques, and alignment of the resulting shapes with standard geometrical approaches, enabling the extraction of the normal shape for future analysis. The system was trained and validated on a comprehensive dataset of 3D ultrasound images from multiple medical centers. Its performance was evaluated by comparing the automated results with manual annotations provided by expert clinicians.

RESULTS

The presented approach demonstrated high accuracy in segmenting and aligning uterine shapes from 3D ultrasound data. The segmentation achieved an average Dice similarity coefficient (DSC) of 0.90. Our method for aligning uterine shapes showed minimal translation and rotation errors compared to traditional methods, with the preliminary average shape exhibiting characteristics consistent with expert findings of a normal uterus.

CONCLUSION

We have presented an approach to automatically segment and align uterine shapes from 3D ultrasound data. We trained a deep learning nnU-Net model that achieved high accuracy and proposed an alignment method using a combination of standard geometrical techniques. Additionally, we have created a publicly available dataset of 3D transvaginal ultrasound volumes with manual annotations of uterine cavities to support further research and development in this field. The dataset and the trained models are available at https://github.com/UL-FRI-LGM/UterUS.

Reinforced Computer-Aided Framework for Diagnosing Thyroid Cancer

Thyroid cancer is the most pervasive disease in the endocrine system and is getting extensive attention. The most prevalent method for an early check is ultrasound examination. Traditional research mainly concentrates on promoting the performance of processing a single ultrasound image using deep learning. However, the complex situation of patients and nodules often makes the model dissatisfactory in terms of accuracy and generalization. Imitating the diagnosis process in reality, a practical diagnosis-oriented computer-aided diagnosis (CAD) framework towards thyroid nodules is proposed, using collaborative deep learning and reinforcement learning. Under the framework, the deep learning model is trained collaboratively with multiparty data; afterward classification results are fused by a reinforcement learning agent to decide the final diagnosis result. Within the architecture, multiparty collaborative learning with privacy-preserving on large-scale medical data brings robustness and generalization, and diagnostic information is modeled as a Markov decision process (MDP) to get final precise diagnosis results. Moreover, the framework is scalable and capable of containing more diagnostic information and multiple sources to pursue a precise diagnosis. A practical dataset of two thousand thyroid ultrasound images is collected and labeled for collaborative training on classification tasks. The simulated experiments have shown the advancement of the framework in promising performance.

From Bench-to-Bedside: How Artificial Intelligence is Changing Thyroid Nodule Diagnostics, a Systematic Review

CONTEXT

Use of artificial intelligence (AI) to predict clinical outcomes in thyroid nodule diagnostics has grown exponentially over the past decade. The greatest challenge is in understanding the best model to apply to one's own patient population, and how to operationalize such a model in practice.

EVIDENCE ACQUISITION

A literature search of PubMed and IEEE Xplore was conducted for English-language publications between January 1, 2015 and January 1, 2023, studying diagnostic tests on suspected thyroid nodules that used AI. We excluded articles without prospective or external validation, nonprimary literature, duplicates, focused on nonnodular thyroid conditions, not using AI, and those incidentally using AI in support of an experimental diagnostic outside standard clinical practice. Quality was graded by Oxford level of evidence.

EVIDENCE SYNTHESIS

A total of 61 studies were identified; all performed external validation, 16 studies were prospective, and 33 compared a model to physician prediction of ground truth. Statistical validation was reported in 50 papers. A diagnostic pipeline was abstracted, yielding 5 high-level outcomes: (1) nodule localization, (2) ultrasound (US) risk score, (3) molecular status, (4) malignancy, and (5) long-term prognosis. Seven prospective studies validated a single commercial AI; strengths included automating nodule feature assessment from US and assisting the physician in predicting malignancy risk, while weaknesses included automated margin prediction and interobserver variability.

CONCLUSION

Models predominantly used US images to predict malignancy. Of 4 Food and Drug Administration-approved products, only S-Detect was extensively validated. Implementing an AI model locally requires data sanitization and revalidation to ensure appropriate clinical performance.

A Deep Learning Approach for Beamforming and Contrast Enhancement of Ultrasound Images in Monostatic Synthetic Aperture Imaging: A Proof-of-Concept

Goal: In this study, we demonstrate that a deep neural network (DNN) can be trained to reconstruct high-contrast images, resembling those produced by the multistatic Synthetic Aperture (SA) method using a 128-element array, leveraging pre-beamforming radiofrequency (RF) signals acquired through the monostatic SA approach. Methods: A U-net was trained using 27200 pairs of RF signals, simulated considering a monostatic SA architecture, with their corresponding delay-and-sum beamformed target images in a multistatic 128-element SA configuration. The contrast was assessed on 500 simulated test images of anechoic/hyperechoic targets. The DNN's performance in reconstructing experimental images of a phantom and different in vivo scenarios was tested too. Results: The DNN, compared to the simple monostatic SA approach used to acquire pre-beamforming signals, generated better-quality images with higher contrast and reduced noise/artifacts. Conclusions: The obtained results suggest the potential for the development of a single-channel setup, simultaneously providing good-quality images and reducing hardware complexity.

Thyroid Ultrasound Image Database and Marker Mask Inpainting Method for Research and Development

OBJECTIVE

The main objective of this study was to build a rich and high-quality thyroid ultrasound image database (TUD) for computer-aided diagnosis (CAD) systems to support accurate diagnosis and prognostic modeling of thyroid disorders. Because most of the raw thyroid ultrasound images contain artificial markers, which seriously affect the robustness of CAD systems because of their strong prior location information, we propose a marker mask inpainting (MMI) method to erase artificial markers and improve image quality.

METHODS

First, a set of thyroid ultrasound images were collected from the General Hospital of the Northern Theater Command. Then, two modules were designed in MMI, namely, the marker detection (MD) module and marker erasure (ME) module. The MD module detects all markers in the image and stores them in a binary mask. According to the binary mask, the ME module erases the markers and generates an unmarked image. Finally, a new TUD based on the marked images and unmarked images was built. The TUD is carefully annotated and statistically analyzed by professional physicians to ensure accuracy and consistency. Moreover, several normal thyroid gland images and some ancillary information on benign and malignant nodules are provided.

RESULTS

Several typical segmentation models were evaluated on the TUD. The experimental results revealed that our TUD can facilitate the development of more accurate CAD systems for the analysis of thyroid nodule-related lesions in ultrasound images. The effectiveness of our MMI method was determined in quantitative experiments.

CONCLUSION

The rich and high-quality resource TUD promotes the development of more effective diagnostic and treatment methods for thyroid diseases. Furthermore, MMI for erasing artificial markers and generating unmarked images is proposed to improve the quality of thyroid ultrasound images. Our TUD database is available at https://github.com/NEU-LX/TUD-Datebase.

Analysis of artificial intelligence in thyroid diagnostics and surgery: A scoping review

BACKGROUND

Artificial Intelligence provides numerous applications in the healthcare sector. The main aim of this study is to evaluate the extent of the current application of artificial intelligence in thyroid diagnostics.

METHODS

Our protocol was based on the Scoping Reviews extension of the Preferred Reporting Items for Systematic Reviews and Meta-analysis (PRISMA-ScR). Information was gathered from PubMed, Cochrane, and EMBASE databases and Google Scholar. Eligible studies were published between 2017 and 2022.

RESULTS

The search identified 133 records, after which 18 articles were included in the scoping review. All the publications were journal articles and discussed various ways that specialists in thyroid diagnostics and surgery have utilized artificial intelligence in their practice.

CONCLUSIONS

The development and incorporation of Artificial Intelligence applications in thyroid diagnostics and surgery has been moderate yet promising. However, applications are currently inconsistent and further research is needed to delineate the true benefit and limitations in this field.

Medical image identification methods: A review

The identification of medical images is an essential task in computer-aided diagnosis, medical image retrieval and mining. Medical image data mainly include electronic health record data and gene information data, etc. Although intelligent imaging provided a good scheme for medical image analysis over traditional methods that rely on the handcrafted features, it remains challenging due to the diversity of imaging modalities and clinical pathologies. Many medical image identification methods provide a good scheme for medical image analysis. The concepts pertinent of methods, such as the machine learning, deep learning, convolutional neural networks, transfer learning, and other image processing technologies for medical image are analyzed and summarized in this paper. We reviewed these recent studies to provide a comprehensive overview of applying these methods in various medical image analysis tasks, such as object detection, image classification, image registration, segmentation, and other tasks. Especially, we emphasized the latest progress and contributions of different methods in medical image analysis, which are summarized base on different application scenarios, including classification, segmentation, detection, and image registration. In addition, the applications of different methods are summarized in different application area, such as pulmonary, brain, digital pathology, brain, skin, lung, renal, breast, neuromyelitis, vertebrae, and musculoskeletal, etc. Critical discussion of open challenges and directions for future research are finally summarized. Especially, excellent algorithms in computer vision, natural language processing, and unmanned driving will be applied to medical image recognition in the future.

Analysis of neural networks for routine classification of sixteen ultrasound upper abdominal cross sections

PURPOSE

Abdominal ultrasound screening requires the capture of multiple standardized plane views as per clinical guidelines. Currently, the extent of adherence to such guidelines is dependent entirely on the skills of the sonographer. The use of neural network classification has the potential to better standardize captured plane views and streamline plane capture reducing the time burden on operators by combatting operator variability.

METHODS

A dataset consisting of 16 routine upper abdominal ultrasound scans from 64 patients was used to test the classification accuracy of 9 neural networks. These networks were tested on both a small, idealised subset of 800 samples as well as full video sweeps of the region of interest using stratified sampling and transfer learning.

RESULTS

The highest validation accuracy attained by both GoogLeNet and InceptionV3 is 83.9% using transfer learning and the large sample set of 26,294 images. A top-2 accuracy of 95.1% was achieved using InceptionV3. Alexnet attained the highest accuracy of 79.5% (top-2 of 91.5%) for the smaller sample set of 800 images. The neural networks evaluated during this study were also successfully able to identify problematic individual cross sections such as between kidneys, with right and left kidney being accurately identified 78.6% and 89.7%, respectively.

CONCLUSION

Dataset size proved a more important factor in determining accuracy than network selection with more complex neural networks providing higher accuracy as dataset size increases and simpler linear neural networks providing better results where the dataset is small.

Patient-level thyroid cancer classification using attention multiple instance learning on fused multi-scale ultrasound image features

For patients with thyroid nodules, the ability to detect and diagnose a malignant nodule is the key to creating an appropriate treatment plan. However, assessments of ultrasound images do not accurately represent malignancy, and often require a biopsy to confirm the diagnosis. Deep learning techniques can classify thyroid nodules from ultrasound images, but current methods depend on manually annotated nodule segmentations. Furthermore, the heterogeneity in the level of magnification across ultrasound images presents a significant obstacle to existing methods. We developed a multi-scale, attention-based multiple-instance learning model which fuses both global and local features of different ultrasound frames to achieve patient-level malignancy classification. Our model demonstrates improved performance with an AUROC of 0.785 (p<0.05) and AUPRC of 0.539, significantly surpassing the baseline model trained on clinical features with an AUROC of 0.667 and AUPRC of 0.444. Improved classification performance better triages the need for biopsy.

Multi-class classification of thyroid nodules from automatic segmented ultrasound images: Hybrid ResNet based UNet convolutional neural network approach

BACKGROUND AND OBJECTIVES

Early detection and diagnosis of thyroid nodule types are important because they can be treated more effectively in their early stages. The types of thyroid nodules are generally stated as atypia of undetermined significance/follicular lesion of undetermined significance (AUS/FLUS), benign follicular, and papillary follicular. The risk of malignancy for AUS/FLUS is typically stated to be between 5 and 15 %, while some studies indicate a risk as high as 25 %. Without complete histology, it is difficult to classify nodules and these diagnostic operations are pricey and risky. To minimize laborious workload and misdiagnosis, recently various AI-based decision support systems have been developed.

METHODS

In this study, a novel AI-based decision support system has been developed for the automated segmentation and classification of the types of thyroid nodules. This system is based on a hybrid deep-learning procedure that makes both an automatic thyroid nodule segmentation and classification tasks, respectively. In this framework, the segmentation is executed with some U-Net architectures such as ResUNet and ResUNet++ integrating with the feature extraction and upsampling with dropout operations to prevent overfitting. The nodule classification task is achieved by various deep nets architecture such as VGG-16, DenseNet121, ResNet-50, and Inception ResNet-v2 considering some accurate classification criteria such as Intersection over Union (IOU), Dice coefficient, accuracy, precision, and recall.

RESULTS

In analysis, a total of 880 patients with ages ranging from 10 to 90 years were included by taking the ultrasound images and demographics. The experimental evaluations showed that ResUNet++ demonstrated excellent segmentation outcomes, attaining remarkable evaluation scores including a dice coefficient of 92.4 % and a mean IOU of 89.7 %. ResNet-50 and Inception ResNet-v2 trained over the images segmented with UNets have shown better performance in terms of achieving high evaluation scores for the classification accuracy such as 96.6 % and 95.0 %, respectively. In addition, ResNet-50 and Inception ResNet-v2 classified AUS/FLUS from the images segmented with UNets with AUC=97.0 % and 96.0 %, respectively.

CONCLUSIONS

The proposed AI-based decision support system improves the automatic segmentation performance of AUS/FLUS and it has shown better performance than available approaches in the literature with respect to ACC, Jaccard and DICE losses. This system has great potential for clinical use by both radiologists and surgeons as well.

The Application of Artificial Intelligence in Thyroid Nodules: A Systematic Review Based on Bibliometric Analysis

BACKGROUND

Thyroid nodules are common lesions in benign and malignant thyroid diseases. More and more studies have been conducted on the feasibility of artificial intelligence (AI) in the detection, diagnosis, and evaluation of thyroid nodules. The aim of this study was to use bibliometric methods to analyze and predict the hot spots and frontiers of AI in thyroid nodules.

METHODS

Articles on the application of artificial intelligence in thyroid nodules were retrieved from the Web of Science core collection database. A website (https://bibliometric.com/), VOSviewer and CiteSpace software were used for bibliometric analyses. The collaboration maps of countries and institutions were analyzed. The cluster and timeline view based on cocitation references and keywords citation bursts visualization map were generated.

RESULTS

The study included 601 papers about AI in thyroid nodules. China contributed to more than half (52.41%) of these publications. The cluster view and timeline view of co-citation references were assembled into 9 clusters, "AI", "deep learning", "papillary thyroid carcinoma", "radiomics", "ultrasound image", "biomarkers", "medical image segmentation", "central lymph node metastasis (CLNM)", and "self-organizing auto-encoder". The "AI", "radiomics", "medical image segmentation", "deep learning", and "CLNM", emerging in the last 10 years and continuing until recent years.

CONCLUSION

An increasing number of scholars were devoted to this field. The potential future research hotspots include risk factor assessment and CLNM prediction of thyroid carcinoma based on radiomics and deep learning, automatic segmentation based on medical images (especially ultrasound images).

A knowledge-interpretable multi-task learning framework for automated thyroid nodule diagnosis in ultrasound videos

Ultrasound has become the most widely used modality for thyroid nodule diagnosis, due to its portability, real-time feedback, lack of toxicity, and low cost. Recently, the computer-aided diagnosis (CAD) of thyroid nodules has attracted significant attention. However, most existing techniques can only be applied to either static images with prominent features (manually selected from scanning videos) or rely on 'black boxes' that cannot provide interpretable results. In this study, we develop a user-friendly framework for the automated diagnosis of thyroid nodules in ultrasound videos, by simulating the typical diagnostic workflow used by radiologists. This process consists of two orderly part-to-whole tasks. The first interprets the characteristics of each image using prior knowledge, to obtain corresponding frame-wise TI-RADS scores. Associated embedded representations not only provide diagnostic information for radiologists but also reduce computational costs. The second task models temporal contextual information in an embedding vector sequence and selectively enhances important information to distinguish benign and malignant thyroid nodules, thereby improving the efficiency and generalizability of the proposed framework. Experimental results demonstrated this approach outperformed other state-of-the-art video classification methods. In addition to assisting radiologists in understanding model predictions, these CAD results could further ease diagnostic workloads and improve patient care.

Automatic detection of thyroid nodules with a real-time artificial intelligence system in a real clinical scenario and the associated influencing factors

BACKGROUND

At present, most articles mainly focused on the diagnosis of thyroid nodules by using artificial intelligence (AI), and there was little research on the detection performance of AI in thyroid nodules.

OBJECTIVE

To explore the value of a real-time AI based on computer-aided diagnosis system in the detection of thyroid nodules and to analyze the factors influencing the detection accuracy.

METHODS

From June 1, 2022 to December 31, 2023, 224 consecutive patients with 587 thyroid nodules were prospective collected. Based on the detection results determined by two experienced radiologists (both with more than 15 years experience in thyroid diagnosis), the detection ability of thyroid nodules of radiologists with different experience levels (junior radiologist with 1 year experience and senior radiologist with 5 years experience in thyroid diagnosis) and real-time AI were compared. According to the logistic regression analysis, the factors influencing the real-time AI detection of thyroid nodules were analyzed.

RESULTS

The detection rate of thyroid nodules by real-time AI was significantly higher than that of junior radiologist (P = 0.013), but lower than that of senior radiologist (P = 0.001). Multivariate logistic regression analysis showed that nodules size, superior pole, outside (near carotid artery), close to vessel, echogenicity (isoechoic, hyperechoic, mixed-echoic), morphology (not very regular, irregular), margin (unclear), ACR TI-RADS category 4 and 5 were significant independent influencing factors (all P < 0.05). With the combination of real-time AI and radiologists, junior and senior radiologist increased the detection rate to 97.4% (P < 0.001) and 99.1% (P = 0.015) respectively.

CONCLUSONS

The real-time AI has good performance in thyroid nodule detection and can be a good auxiliary tool in the clinical work of radiologists.

Comparative performance analysis of binary variants of FOX optimization algorithm with half-quadratic ensemble ranking method for thyroid cancer detection

Thyroid cancer is a life-threatening condition that arises from the cells of the thyroid gland located in the neck's frontal region just below the adam's apple. While it is not as prevalent as other types of cancer, it ranks prominently among the commonly observed cancers affecting the endocrine system. Machine learning has emerged as a valuable medical diagnostics tool specifically for detecting thyroid abnormalities. Feature selection is of vital importance in the field of machine learning as it serves to decrease the data dimensionality and concentrate on the most pertinent features. This process improves model performance, reduces training time, and enhances interpretability. This study examined binary variants of FOX-optimization algorithms for feature selection. The study employed eight transfer functions (S and V shape) to convert the FOX-optimization algorithms into their binary versions. The vision transformer-based pre-trained models (DeiT and Swin Transformer) are used for feature extraction. The extracted features are transformed using locally linear embedding, and binary FOX-optimization algorithms are applied for feature selection in conjunction with the Naïve Bayes classifier. The study utilized two datasets (ultrasound and histopathological) related to thyroid cancer images. The benchmarking is performed using the half-quadratic theory-based ensemble ranking technique. Two TOPSIS-based methods (H-TOPSIS and A-TOPSIS) are employed for initial model ranking, followed by an ensemble technique for final ranking. The problem is treated as multi-objective optimization task with accuracy, F2-score, AUC-ROC and feature space size as optimization goals. The binary FOX-optimization algorithm based on the [Formula: see text] transfer function achieved superior performance compared to other variants using both datasets as well as feature extraction techniques. The proposed framework comprised a Swin transformer to extract features, a Fox optimization algorithm with a V1 transfer function for feature selection, and a Naïve Bayes classifier and obtained the best performance for both datasets. The best model achieved an accuracy of 94.75%, an AUC-ROC value of 0.9848, an F2-Score of 0.9365, an inference time of 0.0353 seconds, and selected 5 features for the ultrasound dataset. For the histopathological dataset, the diagnosis model achieved an overall accuracy of 89.71%, an AUC-ROC score of 0.9329, an F2-Score of 0.8760, an inference time of 0.05141 seconds, and selected 12 features. The proposed model achieved results comparable to existing research with small features space.

Machine and Deep Learning Approaches Applied to Classify Gougerot-Sjögren Syndrome and Jointly Segment Salivary Glands

To diagnose Gougerot-Sjögren syndrome (GSS), ultrasound imaging (US) is a promising tool for helping physicians and experts. Our project focuses on the automatic detection of the presence of GSS using US. Ultrasound imaging suffers from a weak signal-to-noise ratio. Therefore, any classification or segmentation task based on these images becomes a difficult challenge. To address these two tasks, we evaluate different approaches: a classification using a machine learning method along with feature extraction based on a set of measurements following the radiomics guidance and a deep-learning-based classification. We propose, therefore, an innovative method to enhance the training of a deep neural network with a two phases: multiple supervision using joint classification and a segmentation implemented as pretraining. We highlight the fact that our learning methods provide segmentation results similar to those performed by human experts. We obtain proficient segmentation results for salivary glands and promising detection results for Gougerot-Sjögren syndrome; we observe maximal accuracy with the model trained in two phases. Our experimental results corroborate the fact that deep learning and radiomics combined with ultrasound imaging can be a promising tool for the above-mentioned problems.

The performance of deep learning on thyroid nodule imaging predicts thyroid cancer: A systematic review and meta-analysis of epidemiological studies with independent external test sets

BACKGROUND AND AIMS

It is still controversial whether deep learning (DL) systems add accuracy to thyroid nodule imaging classification based on the recent available evidence. We conducted this study to analyze the current evidence of DL in thyroid nodule imaging diagnosis in both internal and external test sets.

METHODS

Until the end of December 2022, PubMed, IEEE, Embase, Web of Science, and the Cochrane Library were searched. We included primary epidemiological studies using externally validated DL techniques in image-based thyroid nodule appraisal. This systematic review was registered on PROSPERO (CRD42022362892).

RESULTS

We evaluated evidence from 17 primary epidemiological studies using externally validated DL techniques in image-based thyroid nodule appraisal. Fourteen studies were deemed eligible for meta-analysis. The pooled sensitivity, specificity, and area under the curve (AUC) of these DL algorithms were 0.89 (95% confidence interval 0.87-0.90), 0.84 (0.82-0.86), and 0.93 (0.91-0.95), respectively. For the internal validation set, the pooled sensitivity, specificity, and AUC were 0.91 (0.89-0.93), 0.88 (0.85-0.91), and 0.96 (0.93-0.97), respectively. In the external validation set, the pooled sensitivity, specificity, and AUC were 0.87 (0.85-0.89), 0.81 (0.77-0.83), and 0.91 (0.88-0.93), respectively. Notably, in subgroup analyses, DL algorithms still demonstrated exceptional diagnostic validity.

CONCLUSIONS

Current evidence suggests DL-based imaging shows diagnostic performances comparable to clinicians for differentiating thyroid nodules in both the internal and external test sets.

Diagnosis of Salivary Gland Tumors Using Transfer Learning with Fine-Tuning and Gradual Unfreezing

Ultrasound is the primary tool for evaluating salivary gland tumors (SGTs); however, tumor diagnosis currently relies on subjective features. This study aimed to establish an objective ultrasound diagnostic method using deep learning. We collected 446 benign and 223 malignant SGT ultrasound images in the training/validation set and 119 benign and 44 malignant SGT ultrasound images in the testing set. We trained convolutional neural network (CNN) models from scratch and employed transfer learning (TL) with fine-tuning and gradual unfreezing to classify malignant and benign SGTs. The diagnostic performances of these models were compared. By utilizing the pretrained ResNet50V2 with fine-tuning and gradual unfreezing, we achieved a 5-fold average validation accuracy of 0.920. The diagnostic performance on the testing set demonstrated an accuracy of 89.0%, a sensitivity of 81.8%, a specificity of 91.6%, a positive predictive value of 78.3%, and a negative predictive value of 93.2%. This performance surpasses that of other models in our study. The corresponding Grad-CAM visualizations were also presented to provide explanations for the diagnosis. This study presents an effective and objective ultrasound method for distinguishing between malignant and benign SGTs, which could assist in preoperative evaluation.

Ultra-fast ultrasound blood flow velocimetry for carotid artery with deep learning

Accurate measurement of blood flow velocity is important for the prevention and early diagnosis of atherosclerosis. However, due to the uncertainty of parameter settings, the autocorrelation velocimetry methods based on clutter filtering are prone to incorrectly filter out the near-wall blood flow signal, resulting in poor velocimetric accuracy. In addition, the Doppler coherent compounding acts as a low-pass filter, which also leads to low values of blood flow velocity estimated by the above methods. Motivated by this status quo, here we propose a deep learning estimator that combines clutter filtering and blood flow velocimetry based on the adaptive property of one-dimensional convolutional neural network (1DCNN). The estimator is operated by first extracting the blood flow signal from the original Doppler echo signal through an affine transformation of the 1D convolution, and then converting the extracted signal into the desired blood flow velocity using a linear transformation function. The effectiveness of the proposed method is verified by simulation as well as in vivo carotid artery data. Compared with typical velocimetry methods such as high-pass filtering (HPF) and singular value decomposition (SVD), the results show that the normalized root means square error (NRMSE) obtained by 1DCNN is reduced by 54.99 % and 53.50 % for forward blood flow velocimetry, and 70.99 % and 69.50 % for reverse blood flow velocimetry, respectively. Consistently, the in vivo measurements demonstrate that the goodness-of-fit of the proposed estimator is improved by 8.72 % and 4.74 % for five subjects. Moreover, the estimation time consumed by 1DCNN is greatly reduced, which costs only 2.91 % of the time of HPF and 12.83 % of the time of SVD. In conclusion, the proposed estimator is a better alternative to the current blood flow velocimetry, and is capable of providing more accurate diagnosis information for vascular diseases in clinical applications.

General Multiscenario Ultrasound Image Tumor Diagnosis Method Based on Unsupervised Domain Adaptation

OBJECTIVE

The utilization of computer-aided diagnosis (CAD) in breast ultrasound image classification has been limited by small sample sizes and domain shift. Current ultrasound classification methods perform inadequately when exposed to cross-domain scenarios, as they struggle with data sets from unobserved domains. In the medical field, there are situations in which all images must share the same networks as they capture the same symptom of the same participant, implying that they share identical structural content. Nevertheless, most domain adaptation methods are not suitable for medical images as they overlook the common features among the images.

METHODS

To overcome these challenges, we propose a novel diverse-domain 2-D feature selection network (FSN), which uses the similarities among medical images and extracts features with a reconstruction network with shared weights. Additionally, it penalizes the feature domain distance through two adversarial learning modules that align the feature space and select common features. Our experiments illustrate that the proposed method is robust and can be applied to ultrasound images of various diseases.

RESULTS

Compared with the latest domain adaptive methods, 2-D FSN markedly enhances the accuracy of classification of breast, thyroid and endoscopic ultrasound images, achieving accuracies of 82.4%, 96.4% and 89.7%, respectively. Furthermore, the model was evaluated on an unsupervised domain adaptation task using ultrasound images from multiple sources and achieved an average accuracy of 77.3% across widely varying domains.

CONCLUSION

In general, 2-D FSN improves the classification ability of the model on multidomain ultrasound data sets through the learning of common features and the combination of multimodule intelligence. The algorithm has good clinical guidance value.

Thyroid Nodule Detection and Region Estimation in Ultrasound Images: A Comparison between Physicians and an Automated Decision Support System Approach

BACKGROUND

Thyroid nodules are very common. In most cases, they are benign, but they can be malignant in a low percentage of cases. The accurate assessment of these nodules is critical to choosing the next diagnostic steps and potential treatment. Ultrasound (US) imaging, the primary modality for assessing these nodules, can lack objectivity due to varying expertise among physicians. This leads to observer variability, potentially affecting patient outcomes.

PURPOSE

This study aims to assess the potential of a Decision Support System (DSS) in reducing these variabilities for thyroid nodule detection and region estimation using US images, particularly in lesser experienced physicians.

METHODS

Three physicians with varying levels of experience evaluated thyroid nodules on US images, focusing on nodule detection and estimating cystic and solid regions. The outcomes were compared to those obtained from a DSS for comparison. Metrics such as classification match percentage and variance percentage were used to quantify differences.

RESULTS

Notable disparities exist between physician evaluations and the DSS assessments: the overall classification match percentage was just 19.2%. Individually, Physicians 1, 2, and 3 had match percentages of 57.6%, 42.3%, and 46.1% with the DSS, respectively. Variances in assessments highlight the subjectivity and observer variability based on physician experience levels.

CONCLUSIONS

The evident variability among physician evaluations underscores the need for supplementary decision-making tools. Given its consistency, the CAD offers potential as a reliable "second opinion" tool, minimizing human-induced variabilities in the critical diagnostic process of thyroid nodules using US images. Future integration of such systems could bolster diagnostic precision and improve patient outcomes.

Ultrasound characteristics of follicular and parafollicular thyroid neoplasms: diagnostic performance of artificial neural network

BACKGROUND

Ultrasound is the first-line imaging modality for detection and classification of thyroid nodules. Certain features observable by ultrasound have recently been equated with potential malignancy. This retrospective cohort study was conducted to test the hypothesis that radiomics of the four categorical divisions (medullary [MTC], papillary [PTC], or follicular [FTC] carcinoma and follicular thyroid adenoma [FTA]) demonstrate distinctive sonographic characteristics. Using an artificial neural network model for proof of concept, these sonographic features served as input.

METHODS

A total of 148 patients were enrolled for study, all with confirmed thyroid pathology in one of the four named categories. Preoperative ultrasound profiles were obtained via standardized protocols. The neural network consisted of seven input neurons; three hidden layers with 50, 250, and 100 neurons, respectively; and one output layer.

RESULTS

Radiomics of contour, structure, and calcifications differed significantly according to nodule type (p = 0.025, p = 0.032, and p = 0.0002, respectively). Levels of accuracy shown by artificial neural network analysis in discriminating among categories ranged from 0.59 to 0.98 (95% confidence interval [CI]: 0.57-0.99), with positive and negative predictive ranges of 0.41-0.99 and 0.78-0.97, respectively.

CONCLUSIONS

Our data indicate that some MTCs, PTCs, FTCs, and FTAs have distinctive sonographic characteristics. However, a significant overlap of these characteristics may impede an explicit classification. Further prospective investigations involving larger patient and nodule numbers and multicenter access should be pursued to determine if neural networks of this sort are beneficial, helping to classify neoplasms of the thyroid gland.

A Framework for Detecting Thyroid Cancer from Ultrasound and Histopathological Images Using Deep Learning, Meta-Heuristics, and MCDM Algorithms

Computer-assisted diagnostic systems have been developed to aid doctors in diagnosing thyroid-related abnormalities. The aim of this research is to improve the diagnosis accuracy of thyroid abnormality detection models that can be utilized to alleviate undue pressure on healthcare professionals. In this research, we proposed deep learning, metaheuristics, and a MCDM algorithms-based framework to detect thyroid-related abnormalities from ultrasound and histopathological images. The proposed method uses three recently developed deep learning techniques (DeiT, Swin Transformer, and Mixer-MLP) to extract features from the thyroid image datasets. The feature extraction techniques are based on the Image Transformer and MLP models. There is a large number of redundant features that can overfit the classifiers and reduce the generalization capabilities of the classifiers. In order to avoid the overfitting problem, six feature transformation techniques (PCA, TSVD, FastICA, ISOMAP, LLE, and UMP) are analyzed to reduce the dimensionality of the data. There are five different classifiers (LR, NB, SVC, KNN, and RF) evaluated using the 5-fold stratified cross-validation technique on the transformed dataset. Both datasets exhibit large class imbalances and hence, the stratified cross-validation technique is used to evaluate the performance. The MEREC-TOPSIS MCDM technique is used for ranking the evaluated models at different analysis stages. In the first stage, the best feature extraction and classification techniques are chosen, whereas, in the second stage, the best dimensionality reduction method is evaluated in wrapper feature selection mode. Two best-ranked models are further selected for the weighted average ensemble learning and features selection using the recently proposed meta-heuristics FOX-optimization algorithm. The PCA+FOX optimization-based feature selection + random forest model achieved the highest TOPSIS score and performed exceptionally well with an accuracy of 99.13%, F2-score of 98.82%, and AUC-ROC score of 99.13% on the ultrasound dataset. Similarly, the model achieved an accuracy score of 90.65%, an F2-score of 92.01%, and an AUC-ROC score of 95.48% on the histopathological dataset. This study exploits the combination novelty of different algorithms in order to improve the thyroid cancer diagnosis capabilities. This proposed framework outperforms the current state-of-the-art diagnostic methods for thyroid-related abnormalities in ultrasound and histopathological datasets and can significantly aid medical professionals by reducing the excessive burden on the medical fraternity.

A Deep Learning Framework for the Characterization of Thyroid Nodules from Ultrasound Images Using Improved Inception Network and Multi-Level Transfer Learning

In the past few years, deep learning has gained increasingly widespread attention and has been applied to diagnosing benign and malignant thyroid nodules. It is difficult to acquire sufficient medical images, resulting in insufficient data, which hinders the development of an efficient deep-learning model. In this paper, we developed a deep-learning-based characterization framework to differentiate malignant and benign nodules from the thyroid ultrasound images. This approach improves the recognition accuracy of the inception network by combining squeeze and excitation networks with the inception modules. We have also integrated the concept of multi-level transfer learning using breast ultrasound images as a bridge dataset. This transfer learning approach addresses the issues regarding domain differences between natural images and ultrasound images during transfer learning. This paper aimed to investigate how the entire framework could help radiologists improve diagnostic performance and avoid unnecessary fine-needle aspiration. The proposed approach based on multi-level transfer learning and improved inception blocks achieved higher precision (0.9057 for the benign class and 0.9667 for the malignant class), recall (0.9796 for the benign class and 0.8529 for malignant), and F1-score (0.9412 for benign class and 0.9062 for malignant class). It also obtained an AUC value of 0.9537, which is higher than that of the single-level transfer learning method. The experimental results show that this model can achieve satisfactory classification accuracy comparable to experienced radiologists. Using this model, we can save time and effort as well as deliver potential clinical application value.

Multistep Automated Data Labelling Procedure (MADLaP) for thyroid nodules on ultrasound: An artificial intelligence approach for automating image annotation

Machine learning (ML) for diagnosis of thyroid nodules on ultrasound is an active area of research. However, ML tools require large, well-labeled datasets, the curation of which is time-consuming and labor-intensive. The purpose of our study was to develop and test a deep-learning-based tool to facilitate and automate the data annotation process for thyroid nodules; we named our tool Multistep Automated Data Labelling Procedure (MADLaP). MADLaP was designed to take multiple inputs including pathology reports, ultrasound images, and radiology reports. Using multiple step-wise 'modules' including rule-based natural language processing, deep-learning-based imaging segmentation, and optical character recognition, MADLaP automatically identified images of a specific thyroid nodule and correctly assigned a pathology label. The model was developed using a training set of 378 patients across our health system and tested on a separate set of 93 patients. Ground truths for both sets were selected by an experienced radiologist. Performance metrics including yield (how many labeled images the model produced) and accuracy (percentage correct) were measured using the test set. MADLaP achieved a yield of 63 % and an accuracy of 83 %. The yield progressively increased as the input data moved through each module, while accuracy peaked part way through. Error analysis showed that inputs from certain examination sites had lower accuracy (40 %) than the other sites (90 %, 100 %). MADLaP successfully created curated datasets of labeled ultrasound images of thyroid nodules. While accurate, the relatively suboptimal yield of MADLaP exposed some challenges when trying to automatically label radiology images from heterogeneous sources. The complex task of image curation and annotation could be automated, allowing for enrichment of larger datasets for use in machine learning development.

Dense Swin Transformer for Classification of Thyroid Nodules

As an early sign of thyroid cancer, thyroid nodules are the most common nodular lesions. As a non-invasive imaging method, ultrasound is widely used in the diagnosis of benign and malignant thyroid nodules. As there is no obvious difference in appearance between the two types of thyroid nodules, and the contrast with the surrounding muscle tissue is too low, it is difficult to distinguish the benign and malignant nodules. Therefore, a dense nodal Swin-Transformer(DST) method for the diagnosis of thyroid nodules is proposed in this paper. Image segmentation is carried out through patch, and feature maps of different sizes are constructed in four stages, which consider different information of each layer of features. In each stage block, a dense connection mechanism is used to make full use of multi-layer features and effectively improve the diagnostic performance. The experimental results of multi-center ultrasound data collected from 17 hospitals show that the accuracy of the proposed method is 87.27%, the sensitivity is 88.63%, and the specific effect is 85.16%, which verifies that the proposed algorithm has the potential to assist clinical practice.

Assessment of encoder-decoder-based segmentation models for thyroid ultrasound images

Encoder-decoder-based semantic segmentation models classify image pixels into the corresponding class, such as the ROI (region of interest) or background. In the present study, simple / dilated convolution / series / directed acyclic graph (DAG)-based encoder-decoder semantic segmentation models have been implemented, i.e., SegNet (VGG16), SegNet (VGG19), U-Net, mobileNetv2, ResNet18, ResNet50, Xception and Inception networks for the segment TTUS(Thyroid Tumor Ultrasound) images. Transfer learning has been used to train these segmentation networks using original and despeckled TTUS images. The performance of the networks has been calculated using mIoU and mDC metrics. Based on the exhaustive experiments, it has been observed that ResNet50-based segmentation model obtained the best results objectively with values 0.87 for mIoU, 0.94 for mDC, and also according to radiologist opinion on shape, margin, and echogenicity characteristics of segmented lesions. It is noted that the segmentation model, namely ResNet50, provides better segmentation based on objective and subjective assessment. It may be used in the healthcare system to identify thyroid nodules accurately in real time.

Breast Tumor Classification in Ultrasound Images by Fusion of Deep Convolutional Neural Network and Shallow LBP Feature

Breast cancer is one of the most dangerous and common cancers in women which leads to a major research topic in medical science. To assist physicians in pre-screening for breast cancer to reduce unnecessary biopsies, breast ultrasound and computer-aided diagnosis (CAD) have been used to distinguish between benign and malignant tumors. In this study, we proposed a CAD system for tumor diagnosis using a multi-channel fusion method and feature extraction structure based on multi-feature fusion on breast ultrasound (BUS) images. In the pre-processing stage, the multi-channel fusion method completed the color conversion of the BUS image to make it contain richer information. In the feature extraction stage, the pre-trained ResNet50 network was selected as the basic network, and three levels of features were combined based on adaptive spatial feature fusion (ASFF), and finally, the shallow local binary pattern (LBP) texture features were fused. Support vector machine (SVM) was used for comparative analysis. A retrospective analysis was carried out, and 1615 breast tumor images (572 benign and 1043 malignant) confirmed by pathological examinations were collected. After data processing and augmentation, for an independent test set consisting of 874 breast ultrasound images (457 benign and 417 malignant), the accuracy, precision, recall, specificity, F1 score, and AUC of our method were 96.91%, 98.75%, 94.72%, 98.91%, 0.97, and 0.991, respectively. The results show that the integration of shallow LBP texture features and multi-level depth features can more effectively improve the comprehensive performance of breast tumor diagnosis, and has strong clinical application value. Compared with the past methods, our proposed method is expected to realize the automatic diagnosis of breast tumors and provide an auxiliary tool for radiologists to accurately diagnose breast diseases.

Artificial intelligence in thyroid ultrasound

Artificial intelligence (AI), particularly deep learning (DL) algorithms, has demonstrated remarkable progress in image-recognition tasks, enabling the automatic quantitative assessment of complex medical images with increased accuracy and efficiency. AI is widely used and is becoming increasingly popular in the field of ultrasound. The rising incidence of thyroid cancer and the workload of physicians have driven the need to utilize AI to efficiently process thyroid ultrasound images. Therefore, leveraging AI in thyroid cancer ultrasound screening and diagnosis cannot only help radiologists achieve more accurate and efficient imaging diagnosis but also reduce their workload. In this paper, we aim to present a comprehensive overview of the technical knowledge of AI with a focus on traditional machine learning (ML) algorithms and DL algorithms. We will also discuss their clinical applications in the ultrasound imaging of thyroid diseases, particularly in differentiating between benign and malignant nodules and predicting cervical lymph node metastasis in thyroid cancer. Finally, we will conclude that AI technology holds great promise for improving the accuracy of thyroid disease ultrasound diagnosis and discuss the potential prospects of AI in this field.

Fine-Needle Aspiration Biopsy Evaluation-Oriented Thyroid Carcinoma Auxiliary Diagnosis

OBJECTIVE

Thyroid carcinoma is one of the most common diseases with an increasing incidence worldwide in recent years. In clinical diagnosis, medical practitioners normally take a preliminary thyroid nodule grading so that highly suspected thyroid nodules can be taken into the fine-needle aspiration (FNA) biopsy to evaluate the malignancy. However, subjective misinterpretations might lead to ambiguous risk stratification of thyroid nodules and unnecessary FNA biopsy.

METHODS

We propose a thyroid carcinoma auxiliary diagnosis method for fine-needle aspiration biopsy evaluation. Through integration of several deep learning models into a multibranch network for thyroid nodule risk stratification in the Thyroid Imaging Reporting and Data System (TIRADS) with pathological features and cascading of a discriminator, our proposed method provides an intelligent auxiliary diagnosis to assist medical practitioners in determining the necessity for further FNA.

DISCUSSION

Experimental results revealed that not only was the rate at which nodules are falsely diagnosed as malignant nodules effectively reduced, which avoids the unnecessary high cost and pain of aspiration biopsy, but also previously missing detected cases were identified with high possibility. By comparing the physicians' diagnosis alone with machine-assisted diagnosis, physicians' diagnostic performance improved with the aid of our proposed method, illustrating that our model can be very helpful in clinical practice.

CONCLUSION

Our proposed method might help medical practitioners avoid subjective interpretations and inter-observer variability. For patients, reliable diagnosis is provided and unnecessary painful diagnostics can be avoided. In other superficial organs such as metastatic lymph nodes and salivary gland tumors, the proposed method might also provide a reliable auxiliary diagnosis for risk stratification.

Generating and Weighting Semantically Consistent Sample Pairs for Ultrasound Contrastive Learning

Well-annotated medical datasets enable deep neural networks (DNNs) to gain strong power in extracting lesion-related features. Building such large and well-designed medical datasets is costly due to the need for high-level expertise. Model pre-training based on ImageNet is a common practice to gain better generalization when the data amount is limited. However, it suffers from the domain gap between natural and medical images. In this work, we pre-train DNNs on ultrasound (US) domains instead of ImageNet to reduce the domain gap in medical US applications. To learn US image representations based on unlabeled US videos, we propose a novel meta-learning-based contrastive learning method, namely Meta Ultrasound Contrastive Learning (Meta-USCL). To tackle the key challenge of obtaining semantically consistent sample pairs for contrastive learning, we present a positive pair generation module along with an automatic sample weighting module based on meta-learning. Experimental results on multiple computer-aided diagnosis (CAD) problems, including pneumonia detection, breast cancer classification, and breast tumor segmentation, show that the proposed self-supervised method reaches state-of-the-art (SOTA). The codes are available at https://github.com/Schuture/Meta-USCL.

The Kidneys Are Not All Normal: Transplanted Kidneys and Their Speckle Distributions

OBJECTIVE

Modelling ultrasound speckle to characterise tissue properties has generated considerable interest. As speckle is dependent on the underlying tissue architecture, modelling it may aid in tasks such as segmentation or disease detection. For the transplanted kidney, where ultrasound is used to investigate dysfunction, it is unknown which statistical distribution best characterises such speckle. This applies to the regions of the transplanted kidney: the cortex, the medulla and the central echogenic complex. Furthermore, it is unclear how these distributions vary by patient variables such as age, sex, body mass index, primary disease or donor type. These traits may influence speckle modelling given their influence on kidney anatomy. We investigate these two aims.

METHODS

B-mode images from n = 821 kidney transplant recipients (one image per recipient) were automatically segmented into the cortex, medulla and central echogenic complex using a neural network. Seven distinct probability distributions were fitted to each region's histogram, and statistical analysis was performed.

DISCUSSION

The Rayleigh and Nakagami distributions had model parameters that differed significantly between the three regions (p ≤ 0.05). Although both had excellent goodness of fit, the Nakagami had higher Kullbeck-Leibler divergence. Recipient age correlated weakly with scale in the cortex (Ω: ρ = 0.11, p = 0.004), while body mass index correlated weakly with shape in the medulla (m: ρ = 0.08, p = 0.04). Neither sex, primary disease nor donor type exhibited any correlation.

CONCLUSION

We propose the Nakagami distribution be used to characterize transplanted kidneys regionally independent of disease etiology and most patient characteristics.

Deep learning for classification of thyroid nodules on ultrasound: validation on an independent dataset

OBJECTIVES

The purpose is to apply a previously validated deep learning algorithm to a new thyroid nodule ultrasound image dataset and compare its performances with radiologists.

METHODS

Prior study presented an algorithm which is able to detect thyroid nodules and then make malignancy classifications with two ultrasound images. A multi-task deep convolutional neural network was trained from 1278 nodules and originally tested with 99 separate nodules. The results were comparable with that of radiologists. The algorithm was further tested with 378 nodules imaged with ultrasound machines from different manufacturers and product types than the training cases. Four experienced radiologists were requested to evaluate the nodules for comparison with deep learning.

RESULTS

The Area Under Curve (AUC) of the deep learning algorithm and four radiologists were calculated with parametric, binormal estimation. For the deep learning algorithm, the AUC was 0.69 (95% CI: 0.64-0.75). The AUC of radiologists were 0.63 (95% CI: 0.59-0.67), 0.66 (95% CI:0.61-0.71), 0.65 (95% CI: 0.60-0.70), and 0.63 (95%CI: 0.58-0.67).

CONCLUSION

In the new testing dataset, the deep learning algorithm achieved similar performances with all four radiologists. The relative performance difference between the algorithm and the radiologists is not significantly affected by the difference of ultrasound scanner.

A Novel Deep-Learning-Based CADx Architecture for Classification of Thyroid Nodules Using Ultrasound Images

Nodules of thyroid cancer occur in the cells of the thyroid as benign or malign types. Thyroid sonographic images are mostly used for diagnosis of thyroid cancer. The aim of this study is to introduce a computer-aided diagnosis system that can classify the thyroid nodules with high accuracy using the data gathered from ultrasound images. Acquisition and labeling of sub-images were performed by a specialist physician. Then the number of these sub-images were increased using data augmentation methods. Deep features were obtained from the images using a pre-trained deep neural network. The dimensions of the features were reduced and features were improved. The improved features were combined with morphological and texture features. This feature group was rated by a value called similarity coefficient value which was obtained from a similarity coefficient generator module. The nodules were classified as benign or malignant using a multi-layer deep neural network with a pre-weighting layer designed with a novel approach. In this study, a novel multi-layer computer-aided diagnosis system was proposed for thyroid cancer detection. In the first layer of the system, a novel feature extraction method based on the class similarity of images was developed. In the second layer, a novel pre-weighting layer was proposed by modifying the genetic algorithm. The proposed system showed superior performance in different metrics compared to the literature.

Multimodal fusion model for classifying placenta ultrasound imaging in pregnancies with hypertension disorders

BACKGROUND

A multimodal fusion model was proposed to assist the traditional visual diagnosis in evaluating the placental features of hypertension disorders of pregnancy (HDP).

OBJECTIVE

The aim of this study was to analyse and compare the placental features between normal and HDP pregnancies and propose a multimodal fusion deep learning model for differentiating and characterizing the placental features from HDP to normal pregnancy.

METHODS

This observational prospective study included 654 pregnant women, including 75 with HDPs. Grayscale ultrasound images (GSIs) and Microflow images (MFIs) of the placentas were collected from all patients during routine obstetric examinations. On the basis of intelligent extraction and features fusion, after quantities of training and optimization, the classification model named GMNet (the intelligent network based on GSIs and MFIs) was introduced for differentiating the placental features of normal and HDP pregnancies. The distributions of placental features extracted by the deep convolutional neural networks (DCNNs) were visualized by Uniform Manifold Approximation and Projection for Dimension Reduction (UMAP). Metrics including sensitivity, specificity, accuracy, and the area under the curve (AUC) were used to score the model. Finally, placental tissue samples were randomly selected for microscopic analyses to prove the interpretability and effectiveness of the GMNet model.

RESULTS

Compared with the Normal group in ultrasonic images, the light spots were rougher and the parts with focal cystic or hypoechogenic lesions were increased in the HDP groups. The overall diagnostic performance of the GMNet model depending on the region of interest (ROI) was excellent (AUC: 97%), with a sensitivity of 90.0%, a specificity of 93.5%, and an accuracy of 93.1%. The fusion features of GSIs and MFIs in the placenta showed a higher discriminative power than single-mode features (fusion features vs GSI features vs MFI features, 97.0% vs 91.2% vs 94.8%). Furthermore, according to the microscopic analysis, unevenly distributed villi, increased syncyte nodules and aggregated intervillous cellulose deposition were particularly frequent in the HDP cases.

CONCLUSIONS

The GMNet model could sensitively identify abnormal changes in the placental microstructure in pregnancies with HDP.

Clinical knowledge embedded method based on multi-task learning for thyroid nodule classification with ultrasound images

Objective. Thyroid nodules are common glandular abnormality that need to be diagnosed as benign or malignant to determine further treatments. Clinically, ultrasonography is the main diagnostic method, but it is highly subjective with severe variability. Recently, many deep-learning-based methods have been proposed to alleviate subjectivity and achieve good results yet, these methods often neglect important guidance from clinical knowledge. Our objective is to utilize such guidance for accurate and reliable thyroid nodule classification.Approach. In this study, a multi-task learning model embedded with clinical knowledge of ACR Thyroid Imaging, Reporting and Data System guideline is proposed. The clinical features defined in the guideline have strong correlations with malignancy and they were modeled as tasks alongside the pathological type. Multi-task learning was utilized to exploit the correlations to improve diagnostic performance. To alleviate the impact of noisy labels on clinical features, a loss-weighting strategy was proposed. Five-fold cross-validation was applied to an internal training set of size 4989, and an external test set of size 243 was used for evaluation.Main results. The proposed multi-task learning model achieved an average AUC of 0.901 and an ensemble AUC of 0.917 on the test set, which significantly outperformed the single-task baseline models.Significance. The results indicated that multi-task learning of clinical features can effectively classify thyroid nodules and reveal the possibility of using clinical indicators as auxiliary tasks to improve performance when diagnosing other diseases.

Classification for thyroid nodule using ViT with contrastive learning in ultrasound images

The lack of representative features between benign nodules, especially level 3 of Thyroid Imaging Reporting and Data System (TI-RADS), and malignant nodules limits diagnostic accuracy, leading to inconsistent interpretation, overdiagnosis, and unnecessary biopsies. We propose a Vision-Transformer-based (ViT) thyroid nodule classification model using contrast learning, called TC-ViT, to improve accuracy of diagnosis and specificity of biopsy recommendations. ViT can explore the global features of thyroid nodules well. Nodule images are used as ROI to enhance the local features of the ViT. Contrast learning can minimize the representation distance between nodules of the same category, enhance the representation consistency of global and local features, and achieve accurate diagnosis of TI-RADS 3 or malignant nodules. The test results achieve an accuracy of 86.9%. The evaluation metrics show that the network outperforms other classical deep learning-based networks in terms of classification performance. TC-ViT can achieve automatic classification of TI-RADS 3 and malignant nodules on ultrasound images. It can also be used as a key step in computer-aided diagnosis for comprehensive analysis and accurate diagnosis. The code will be available at https://github.com/Jiawei217/TC-ViT.

Application of Artificial Intelligence Computer-Assisted Diagnosis Originally Developed for Thyroid Nodules to Breast Lesions on Ultrasound

As thyroid and breast cancer have several US findings in common, we applied an artificial intelligence computer-assisted diagnosis (AI-CAD) software originally developed for thyroid nodules to breast lesions on ultrasound (US) and evaluated its diagnostic performance. From January 2017 to December 2017, 1042 breast lesions (mean size 20.2 ± 11.8 mm) of 1001 patients (mean age 45.9 ± 12.9 years) who underwent US-guided core-needle biopsy were included. An AI-CAD software that was previously trained and validated with thyroid nodules using the convolutional neural network was applied to breast nodules. There were 665 benign breast lesions (63.0%) and 391 breast cancers (37.0%). The area under the receiver operating characteristic curve (AUROC) of AI-CAD to differentiate breast lesions was 0.678 (95% confidence interval: 0.649, 0.707). After fine-tuning AI-CAD with 1084 separate breast lesions, the diagnostic performance of AI-CAD markedly improved (AUC 0.841). This was significantly higher than that of radiologists when the cutoff category was BI-RADS 4a (AUC 0.621, P < 0.001), but lower when the cutoff category was BI-RADS 4b (AUC 0.908, P < 0.001). When applied to breast lesions, the diagnostic performance of an AI-CAD software that had been developed for differentiating malignant and benign thyroid nodules was not bad. However, an organ-specific approach guarantees better diagnostic performance despite the similar US features of thyroid and breast malignancies.

Research on Classification Method of Medical Ultrasound Image Processing Based on Neural Network

In clinical applications, the classification of ultrasound images needs to be processed as an aid to diagnosis. Based on this, a hybrid model of cascaded deep convolutional neural network consisting of two different CNNs and a new classification method are designed and evaluated for its feasibility and effectiveness in ultrasound image classification. A total of 1000 pathological slides of patients with thyroid nodular lesions kept in the Department of Pathology of the First Affiliated Hospital of Lanzhou University, China, were retrospectively collected. After image acquisition, the images were randomly divided into training set, validation set, and test set in the ratio of 4 : 3 : 3. Three convolutional neural network models (VGG 19 model, Inception V3 model, and DenseNet 161 model) with pretraining parameters acquired on the training set were trained, and the models were combined to construct an integrated learning model, and the performance of the models in recognizing pathological images was evaluated based on the test set data. The experimental results show that the VGG 19 model is less effective in classification, with a correct rate of 88.20%, which is lower than that of Inception V3 and DenseNet161 models (92.87% and 92.95%). InceptionV3 and DenseNet161 models have significant advantages in terms of accuracy, number of parameters, and training efficiency, where the DenseNet161 model has faster convergence and better generalization performance, but occupies more video memory in the operation; moreover, the DenseNet161 operation time (1986.48 s) and response time (16 s) are better than the other two models. In addition, the integrated learning of InceptionV3 and DenseNet161 can improve the recognition of pathological images by a single model. Compared with other methods, the performance of the cascaded CNNs proposed in this study is significantly improved, and the multiview strategy can improve the performance of cascaded CNNs. The experimental results demonstrate the potential clinical application of cascaded CNNs, which can provide physicians with an objective second opinion and reduce their heavy workload, in addition to making the diagnosis of thyroid nodules easy and reproducible for people without medical expertise.

Improve follicular thyroid carcinoma diagnosis using computer aided diagnosis system on ultrasound images

OBJECTIVE

We aim to leverage deep learning to develop a computer aided diagnosis (CAD) system toward helping radiologists in the diagnosis of follicular thyroid carcinoma (FTC) on thyroid ultrasonography.

METHODS

A dataset of 1159 images, consisting of 351 images from 138 FTC patients and 808 images from 274 benign follicular-pattern nodule patients, was divided into a balanced and unbalanced dataset, and used to train and test the CAD system based on a transfer learning of a residual network. Six radiologists participated in the experiments to verify whether and how much the proposed CAD system helps to improve their performance.

RESULTS

On the balanced dataset, the CAD system achieved 0.892 of area under the ROC (AUC). The accuracy, recall, precision, and F1-score of the CAD method were 84.66%, 84.66%, 84.77%, 84.65%, while those of the junior and senior radiologists were 56.82%, 56.82%, 56.95%, 56.62% and 64.20%, 64.20%, 64.35%, 64.11% respectively. With the help of CAD, the metrics of the junior and senior radiologists improved to 62.81%, 62.81%, 62.85%, 62.79% and 73.86%, 73.86%, 74.00%, 73.83%. The results almost repeated on the unbalanced dataset. The results show the proposed CAD approach can not only achieve better performance than radiologists, but also significantly improve the radiologists' diagnosis of FTC.

CONCLUSIONS

The performances of the CAD system indicate it is a reliable reference for preoperative diagnosis of FTC, and might assist the development of a fast, accessible screening method for FTC.

Deep learning based on ultrasound to differentiate pathologically proven atypical and typical medullary thyroid carcinoma from follicular thyroid adenoma

OBJECTIVES

To investigate the feasibility and value of deep learning based on grayscale ultrasonography in the differentiation of pathologically proven atypical and typical medullary thyroid carcinoma (MTC) from follicular thyroid adenoma (FTA).

METHODS

The preoperative 770 ultrasound images consisted of 354 MTCs (66% were typical MTCs with a high suspicion sonographic pattern, 34% were atypical MTCs with a suspicion pattern of intermediate or less) and 416 FTAs. All images were delineated manually by a senior sonographer to achieve the regions of interest. Two deep neural networks of ResNet-34 and ResNet-18 were performed on the training set (n = 690). The test data set (n = 80) was subsequently evaluated by the two models and two sonographers, their diagnostic performances and misdiagnosis lesions were compared and analyzed.

RESULTS

The ResNet-34 model shows higher diagnostic ability than the junior sonographer with an area under the receiver operating curve of 0.992 (95% CI: 0.840-0.970)versus 0.754 (95% CI:0.645-0.843). Moreover, 12 of 16 atypical MTCs were successfully identified by the ResNet-34, which is significantly better than the senior and junior sonographer, suggesting that these patients could benefit from timely serological examination and surgical strategy at an earlier stage.

CONCLUSION

Deep learning to differentiate MTC from FTA on grayscale ultrasound may be a useful diagnostic support tool, especially in atypical MTC and FTA. Moreover, the computing time of deep learning is short, which will help to incorporate it into real-time ultrasound diagnosis.

An overview of artificial intelligence in diabetic retinopathy and other ocular diseases

Artificial intelligence (AI), also known as machine intelligence, is a branch of science that empowers machines using human intelligence. AI refers to the technology of rendering human intelligence through computer programs. From healthcare to the precise prevention, diagnosis, and management of diseases, AI is progressing rapidly in various interdisciplinary fields, including ophthalmology. Ophthalmology is at the forefront of AI in medicine because the diagnosis of ocular diseases heavy reliance on imaging. Recently, deep learning-based AI screening and prediction models have been applied to the most common visual impairment and blindness diseases, including glaucoma, cataract, age-related macular degeneration (ARMD), and diabetic retinopathy (DR). The success of AI in medicine is primarily attributed to the development of deep learning algorithms, which are computational models composed of multiple layers of simulated neurons. These models can learn the representations of data at multiple levels of abstraction. The Inception-v3 algorithm and transfer learning concept have been applied in DR and ARMD to reuse fundus image features learned from natural images (non-medical images) to train an AI system with a fraction of the commonly used training data (<1%). The trained AI system achieved performance comparable to that of human experts in classifying ARMD and diabetic macular edema on optical coherence tomography images. In this study, we highlight the fundamental concepts of AI and its application in these four major ocular diseases and further discuss the current challenges, as well as the prospects in ophthalmology.

Development of a Machine Learning-Based Screening Method for Thyroid Nodules Classification by Solving the Imbalance Challenge in Thyroid Nodules Data

BACKGROUND

This study aims to show the impact of imbalanced data and the typical evaluation methods in developing and misleading assessments of machine learning-based models for preoperative thyroid nodules screening.

STUDY DESIGN

A retrospective study.

METHODS

The ultrasonography features for 431 thyroid nodules cases were extracted from medical records of 313 patients in Babol, Iran. Since thyroid nodules are commonly benign, the relevant data are usually unbalanced in classes. It can lead to the bias of learning models toward the majority class. To solve it, a hybrid resampling method called the Smote-was used to creating balance data. Following that, the support vector classification (SVC) algorithm was trained by balance and unbalanced datasets as Models 2 and 3, respectively, in Python language programming. Their performance was then compared with the logistic regression model as Model 1 that fitted traditionally.

RESULTS

The prevalence of malignant nodules was obtained at 14% (n = 61). In addition, 87% of the patients in this study were women. However, there was no difference in the prevalence of malignancy for gender. Furthermore, the accuracy, area under the curve, and geometric mean values were estimated at 92.1%, 93.2%, and 76.8% for Model 1, 91.3%, 93%, and 77.6% for Model 2, and finally, 91%, 92.6% and 84.2% for Model 3, respectively. Similarly, the results identified Micro calcification, Taller than wide shape, as well as lack of ISO and hyperechogenicity features as the most effective malignant variables.

CONCLUSION

Paying attention to data challenges, such as data imbalances, and using proper criteria measures can improve the performance of machine learning models for preoperative thyroid nodules screening.