Use of artificial intelligence and machine learning for estimating malignancy risk of thyroid nodules

Avoidable biopsies? Validating artificial intelligence-based decision support software in indeterminate thyroid nodules

BACKGROUND

Multiple artificial intelligence (AI) systems have been approved to risk-stratify thyroid nodules through sonographic characterization. We sought to validate the ability of one such AI system, Koios DS (Koios Medical, Chicago, IL), to aid in improving risk stratification of indeterminate thyroid nodules.

METHODS

A retrospective single-institution dataset was compiled of 28 cytologically indeterminate thyroid nodules having undergone molecular testing and surgical resection, with surgical pathology categorized as malignant or benign. Nodules were retrospectively evaluated with Koios DS. After nodule selection, automated and AI-adapter-derived Thyroid Imaging Reporting and Data System (TI-RADS) levels were recorded, and agreement with radiologist-derived levels was assessed using Cohen's κ statistic. The performance of malignancy classification was compared between the radiologist and AI-adapter. Biopsy thresholds were re-evaluated using the AI-adapter.

RESULTS

In this cohort, 7 (25%) nodules were malignant on surgical pathology. The median nodule size was 2.4 cm (interquartile range: 1.8-2.9 cm). Median radiologist and automated TI-RADS levels were both 4, with κ 0.25 ("fair agreement"). Malignancy classification by the radiologist provided sensitivity 100%, specificity 33.3%, positive predictive value (PPV) 33.3%, and negative predictive value (NPV) 100%, compared with the AI-adapter's performance with sensitivity 85.7%, specificity 76.2%, PPV 54.5%, and NPV 94.1%. Using the AI-adapter, 14 of 28 biopsies would have been deferred, 13 of which were surgically benign.

CONCLUSION

Koios automated and radiologist-derived TI-RADS levels were in consistent agreement for indeterminate thyroid nodules. Malignancy reclassification with the AI-adapter improved PPV at minimal cost to NPV. Risk stratification with the addition of the AI-adapter may allow for more accurate patient counseling and the avoidance of biopsies in select cases that would otherwise be cytologically indeterminate.

The Risks and Challenges of Artificial Intelligence in Endocrinology

Artificial intelligence (AI) holds the promise of addressing many of the numerous challenges healthcare faces, which include a growing burden of illness, an increase in chronic health conditions and disabilities due to aging and epidemiological changes, higher demand for health services, overworked and burned-out clinicians, greater societal expectations, and rising health expenditures. While technological advancements in processing power, memory, storage, and the abundance of data have empowered computers to handle increasingly complex tasks with remarkable success, AI introduces a variety of meaningful risks and challenges. Among these are issues related to accuracy and reliability, bias and equity, errors and accountability, transparency, misuse, and privacy of data. As AI systems continue to rapidly integrate into healthcare settings, it is crucial to recognize the inherent risks they bring. These risks demand careful consideration to ensure the responsible and safe deployment of AI in healthcare.

Enhancing the fairness of AI prediction models by Quasi-Pareto improvement among heterogeneous thyroid nodule population

Artificial Intelligence (AI) models for medical diagnosis often face challenges of generalizability and fairness. We highlighted the algorithmic unfairness in a large thyroid ultrasound dataset with significant diagnostic performance disparities across subgroups linked causally to sample size imbalances. To address this, we introduced the Quasi-Pareto Improvement (QPI) approach and a deep learning implementation (QP-Net) combining multi-task learning and domain adaptation to improve model performance among disadvantaged subgroups without compromising overall population performance. On the thyroid ultrasound dataset, our method significantly mitigated the area under curve (AUC) disparity for three less-prevalent subgroups by 0.213, 0.112, and 0.173 while maintaining the AUC for dominant subgroups; we also further confirmed the generalizability of our approach on two public datasets: the ISIC2019 skin disease dataset and the CheXpert chest radiograph dataset. Here we show the QPI approach to be widely applicable in promoting AI for equitable healthcare outcomes.

Visual fatigue a comprehensive review of mechanisms of occurrence, animal model design and nutritional intervention strategies

When the eyes work intensively, it is easy to have eye discomfort such as blurred vision, soreness, dryness, and tearing, that is, visual fatigue. Visual fatigue not only affects work and study efficiency, but long-term visual fatigue can also easily affect physical and mental health. In recent years, with the popularization of electronic products, although it has brought convenience to the office and study, it has also caused more frequent visual fatigue among people who use electronic devices. Moreover, studies have reported that the number of people with visual fatigue is showing a trend of increasing year by year. The range of people involved is also extensive, especially students, people who have been engaged in computer work and fine instruments (such as microscopes) for a long time, and older adults with aging eye function. More and more studies have proposed that supplementation with the proper nutrients can effectively relieve visual fatigue and promote eye health. This review discusses the physiological mechanisms of visual fatigue and the design ideas of animal experiments from the perspective of modern nutritional science. Functional food ingredients with the ability to alleviate visual fatigue are discussed in detail.

Machine Learning Model as a Useful Tool for Prediction of Thyroid Nodules Histology, Aggressiveness and Treatment-Related Complications

Thyroid nodules are very common, 5-15% of which are malignant. Despite the low mortality rate of well-differentiated thyroid cancer, some variants may behave aggressively, making nodule differentiation mandatory. Ultrasound and fine-needle aspiration biopsy are simple, safe, cost-effective and accurate diagnostic tools, but have some potential limits. Recently, machine learning (ML) approaches have been successfully applied to healthcare datasets to predict the outcomes of surgical procedures. The aim of this work is the application of ML to predict tumor histology (HIS), aggressiveness and post-surgical complications in thyroid patients. This retrospective study was conducted at the ENT Division of Eastern Piedmont University, Novara (Italy), and reported data about 1218 patients who underwent surgery between January 2006 and December 2018. For each patient, general information, HIS and outcomes are reported. For each prediction task, we trained ML models on pre-surgery features alone as well as on both pre- and post-surgery data. The ML pipeline included data cleaning, oversampling to deal with unbalanced datasets and exploration of hyper-parameter space for random forest models, testing their stability and ranking feature importance. The main results are (i) the construction of a rich, hand-curated, open dataset including pre- and post-surgery features (ii) the development of accurate yet explainable ML models. Results highlight pre-screening as the most important feature to predict HIS and aggressiveness, and that, in our population, having an out-of-range (Low) fT3 dosage at pre-operative examination is strongly associated with a higher aggressiveness of the disease. Our work shows how ML models can find patterns in thyroid patient data and could support clinicians to refine diagnostic tools and improve their accuracy.

Gender Equality Trends of First Authors in Publications of Artificial Intelligence and Thyroid

Thyroid diseases are diverse, ranging from benign conditions to potentially life-threatening disorders. Recently, the application of artificial intelligence (AI) in evaluating thyroid disease has significantly enhanced medical research, diagnosis prediction, and algorithm development. Coupled with this advancement is the rising focus on the importance of gender equality in scientific publications. This study delves into the gender trends of first authors in papers related to "Artificial Intelligence and Thyroid" sourced from PubMed from 2003 to 2022, scrutinizing these trends based on both country and year. A bibliometric analysis was conducted on PubMed to retrieve relevant articles over this 19-year time span. Following this, the names and affiliated countries of the first authors were determined. The Namsor app, a tool for classifying personal names by gender, origin, or ethnicity, was then used to segregate the data based on gender. Statistical analyses were performed using the R software- ARIMA model and Fisher's exact test was applied to examine the correlations between gender and country of origin. From the 254 analyzed articles, 43.5% of the first authors were female, while 56.69% were male. The year 2022 saw the most significant number of female first-author publications. Intriguingly, the European Journal of Radiology was prominent due to its favorable gender ratio. Moreover, the association between gender and country was significant, with China being a standout. Limitations included focusing only on PubMed journals and using a third party for gender identification. Nevertheless, the study reveals a move toward gender parity in AI and thyroid research over the past 18 years, emphasizing the importance of sustained efforts for academic inclusivity.

Development of a machine learning-based fine-grained risk stratification system for thyroid nodules using predefined clinicoradiological features

OBJECTIVE

We constructed and validated a machine learning-based malignancy risk estimation model using predefined clinicoradiological features, and evaluated its clinical utility for the management of thyroid nodules.

METHODS

In total, 5708 benign (n = 4597) and malignant (n = 1111) thyroid nodules were collected from 5081 consecutive patients treated in 26 institutions. Seventeen experienced radiologists evaluated nodule characteristics on ultrasonographic images. Eight predictive models were used to stratify the thyroid nodules according to malignancy risk; model performance was assessed via nested 10-fold cross-validation. The best-performing algorithm was externally validated using data for 454 thyroid nodules from a tertiary hospital, then compared to the Thyroid Imaging Reporting and Data System (TIRADS)-based interpretations of radiologists (American College of Radiology, European and Korean TIRADS, and AACE/ACE/AME guidelines).

RESULTS

The area under the receiver operating characteristic (AUROC) curves of the algorithms ranged from 0.773 to 0.862. The sensitivities, specificities, positive predictive values, and negative predictive values of the best-performing models were 74.1-76.6%, 80.9-83.4%, 49.2-51.9%, and 93.0-93.5%, respectively. For the external validation set, the ElasticNet values were 83.2%, 89.2%, 81.8%, and 90.1%, respectively. The corresponding TIRADS values were 66.5-85.0%, 61.3-80.8%, 45.9-72.1%, and 81.5-90.3%, respectively. The new model exhibited a significantly higher AUROC and specificity than did the TIRADS risk stratification, although its sensitivity was similar.

CONCLUSION

We developed a reliable machine learning-based predictive model that demonstrated enhanced specificity when stratifying thyroid nodules according to malignancy risk. This system will contribute to improved personalized management of thyroid nodules.

KEY POINTS

• The area under the receiver operating characteristic (AUROC) curve, sensitivity, and specificity of our model were 0.914, 83.2%, and 89.2%, respectively (derived using the validation dataset). • Compared to the TIRADS values, the AUROC and specificity are significantly higher, while the sensitivity is similar. • An interactive version of our AI algorithm is at http://tirads.cdss.co.kr .

Beyond high hopes: A scoping review of the 2019-2021 scientific discourse on machine learning in medical imaging

Machine learning has become a key driver of the digital health revolution. That comes with a fair share of high hopes and hype. We conducted a scoping review on machine learning in medical imaging, providing a comprehensive outlook of the field's potential, limitations, and future directions. Most reported strengths and promises included: improved (a) analytic power, (b) efficiency (c) decision making, and (d) equity. Most reported challenges included: (a) structural barriers and imaging heterogeneity, (b) scarcity of well-annotated, representative and interconnected imaging datasets (c) validity and performance limitations, including bias and equity issues, and (d) the still missing clinical integration. The boundaries between strengths and challenges, with cross-cutting ethical and regulatory implications, remain blurred. The literature emphasizes explainability and trustworthiness, with a largely missing discussion about the specific technical and regulatory challenges surrounding these concepts. Future trends are expected to shift towards multi-source models, combining imaging with an array of other data, in a more open access, and explainable manner.

The progress of radiomics in thyroid nodules

Due to the development of Artificial Intelligence (AI), Machine Learning (ML), and the improvement of medical imaging equipment, radiomics has become a popular research in recent years. Radiomics can obtain various quantitative features from medical images, highlighting the invisible image traits and significantly enhancing the ability of medical imaging identification and prediction. The literature indicates that radiomics has a high potential in identifying and predicting thyroid nodules. So in this article, we explain the development, definition, and workflow of radiomics. And then, we summarize the applications of various imaging techniques in identifying benign and malignant thyroid nodules, predicting invasiveness and metastasis of thyroid lymph nodes, forecasting the prognosis of thyroid malignancies, and some new advances in molecular level and deep learning. The shortcomings of this technique are also summarized, and future development prospects are provided.

Artificial Intelligence for Thyroid Nodule Characterization: Where Are We Standing?

Simple Summary

In the present review, an up-to-date summary of the state of the art of artificial intelligence (AI) implementation for thyroid nodule characterization and cancer is provided. The opinion on the real effectiveness of AI systems remains controversial. Taking into consideration the largest and most scientifically valid studies, it is possible to state that AI provides results that are comparable or inferior to expert ultrasound specialists and radiologists. Promising data approve AI as a support tool and simultaneously highlight the need for a radiologist supervisory framework for AI provided results. Therefore, current solutions might be more suitable for educational purposes.

Abstract

Machine learning (ML) is an interdisciplinary sector in the subset of artificial intelligence (AI) that creates systems to set up logical connections using algorithms, and thus offers predictions for complex data analysis. In the present review, an up-to-date summary of the current state of the art regarding ML and AI implementation for thyroid nodule ultrasound characterization and cancer is provided, highlighting controversies over AI application as well as possible benefits of ML, such as, for example, training purposes. There is evidence that AI increases diagnostic accuracy and significantly limits inter-observer variability by using standardized mathematical algorithms. It could also be of aid in practice settings with limited sub-specialty expertise, offering a second opinion by means of radiomics and computer-assisted diagnosis. The introduction of AI represents a revolutionary event in thyroid nodule evaluation, but key issues for further implementation include integration with radiologist expertise, impact on workflow and efficiency, and performance monitoring.

Artificial intelligence and thyroid disease management: considerations for thyroid function tests

Artificial intelligence (AI) is transforming healthcare and offers new tools in clinical research, personalized medicine, and medical diagnostics. Thyroid function tests represent an important asset for physicians in the diagnosis and monitoring of pathologies. Artificial intelligence tools can clearly assist physicians and specialists in laboratory medicine to optimize test prescription, tests interpretation, decision making, process optimization, and assay design. Our article is reviewing several of these aspects. As thyroid AI models rely on large data sets, which often requires distributed learning from multi-center contributions, this article also briefly discusses this issue.

The Future of Thyroid Nodule Risk Stratification

Clinical evidence supports the association of ultrasound features with benign or malignant thyroid nodules and serves as the basis for sonographic stratification of thyroid nodules, according to an estimated thyroid cancer risk. Contemporary guidelines recommend management strategies according to thyroid cancer risk, thyroid nodule size, and the clinical scenario. Yet, reproducible and accurate thyroid nodule risk stratification requires expertise, time, and understanding of the weight different ultrasound features have on thyroid cancer risk. The application of artificial intelligence to overcome these limitations is promising and has the potential to improve the care of patients with thyroid nodules.

External validation of AIBx, an artificial intelligence model for risk stratification, in thyroid nodules

BACKGROUND

Artificial intelligence algorithms could be used to risk-stratify thyroid nodules and may reduce the subjectivity of ultrasonography. One such algorithm is AIBx which has shown good performance. However, external validation is crucial prior to clinical implementation.

MATERIALS AND METHODS

Patients harboring thyroid nodules 1-4 cm in size, undergoing thyroid surgery from 2014 to 2016 in a single institution, were included. A histological diagnosis was obtained in all cases. Medullary thyroid cancer, metastasis from other cancers, thyroid lymphomas, and purely cystic nodules were excluded. Retrospectively, transverse ultrasound images of the nodules were analyzed by AIBx, and the results were compared with histopathology and Thyroid Imaging Reporting and Data System (TIRADS), calculated by experienced physicians.

RESULTS

Out of 329 patients, 257 nodules from 209 individuals met the eligibility criteria. Fifty-one nodules (20%) were malignant. AIBx had a negative predictive value (NPV) of 89.2%. Sensitivity, specificity, and positive predictive values (PPV) were 78.4, 44.2, and 25.8%, respectively. Considering both TIRADS 4 and TIRADS 5 nodules as malignant lesions resulted in an NPV of 93.0%, while PPV and specificity were only 22.4 and 19.4%, respectively. By combining AIBx with TIRADS, no malignant nodules were overlooked.

CONCLUSION

When applied to ultrasound images obtained in a different setting than used for training, AIBx had comparable NPVs to TIRADS. AIBx performed even better when combined with TIRADS, thus reducing false negative assessments. These data support the concept of AIBx for thyroid nodules, and this tool may help less experienced operators by reducing the subjectivity inherent to thyroid ultrasound interpretation.

Comparative Analysis of Computer-Aided Diagnosis and Computer-Assisted Subjective Assessment in Thyroid Ultrasound

The value of computer-aided diagnosis (CAD) and computer-assisted techniques equipped with different TIRADS remains ambiguous. Parallel diagnosis performances of computer-assisted subjective assessments and CAD were compared based on AACE, ATA, EU, and KSThR TIRADS. CAD software computed the diagnosis of 162 thyroid nodule sonograms. Two raters (R₁ and R₂) independently rated the sonographic features of the nodules using an online risk calculator while blinded to pathology results. Diagnostic efficiency measures were calculated based on the final pathology results. R₁ had higher diagnostic performance outcomes than CAD with similarities between KSThR (SEN: 90.3% vs. 83.9%, p = 0.57; SPEC: 46% vs. 51%, p = 0.21; AUROC: 0.76 vs. 0.67, p = 0.02), and EU (SEN: 85.5% vs. 79%, p = 0.82; SPEC: 62% vs. 55%, p = 0.27; AUROC: 0.74 vs. 0.67, p = 0.06). Similarly, R₂ had higher AUROC and specificity but lower sensitivity than CAD (KSThR-AUROC: 0.74 vs. 0.67, p = 0.13; SPEC: 61% vs. 46%, p = 0.02 and SEN: 75.8% vs. 83.9%, p = 0.31, and EU-AUROC: 0.69 vs. 0.67, p = 0.57, SPEC: 64% vs. 55%, p = 0.19, and SEN: 71% vs. 79%, p = 0.51, respectively). CAD had higher sensitivity but lower specificity than both R₁ and R₂ with AACE for 114 specified nodules (SEN: 92.5% vs. 88.7%, p = 0.50; 92.5% vs. 79.3%, p = 0.02, and SPEC: 26.2% vs. 54.1%, p = 0.001; 26.2% vs. 62.3%, p < 0.001, respectively). All diagnostic performance outcomes were comparable for ATA with 96 specified nodules. Computer-assisted subjective interpretation using KSThR is more ideal for ruling out papillary thyroid carcinomas than CAD. Future larger multi-center and multi-rater prospective studies with a diverse representation of thyroid cancers are necessary to validate these findings.

An integrated AI model to improve diagnostic accuracy of ultrasound and output known risk features in suspicious thyroid nodules

OBJECTIVES

From the viewpoint of ultrasound (US) physicians, an ideal thyroid US computer-assisted diagnostic (CAD) system for thyroid cancer should perform well in suspicious thyroid nodules with atypical risk features and be able to output explainable results. This study aims to develop an explainable US CAD model for suspicious thyroid nodules.

METHODS

A total of 2992 solid or almost-solid thyroid nodules were analyzed retrospectively. All nodules had pathological results (1070 malignancies and 1992 benignities) confirmed by ultrasound-guided fine-needle aspiration cytology and histopathology after thyroidectomy. A deep learning model (ResNet50) and a multiple risk features learning ensemble model (XGBoost) were used to train the US images of 2794 thyroid nodules. Then, an integrated AI model was generated by combining both models. The diagnostic accuracies of the three AI models (ResNet50, XGBoost, and the integrated model) were predicted in a testing set including 198 thyroid nodules and compared to the diagnostic efficacy of five ultrasonographers.

RESULTS

The accuracy of the integrated model was 76.77%, while the mean accuracy of the ultrasonographers was 68.38%. Of the risk features, microcalcifications showed the highest contribution to the diagnosis of malignant nodules.

CONCLUSIONS

The integrated AI model in our study can improve the diagnostic accuracy of suspicious thyroid nodules and output the known risk features simultaneously, thus aiding in training young ultrasonographers by linking the explainable results to their clinical experience and advancing the acceptance of AI diagnosis for thyroid cancer in clinical practice.

KEY POINTS

• We developed an artificial intelligence (AI) diagnosis model based on both deep learning and multiple risk feature ensemble learning methods. • The AI diagnosis model showed higher diagnostic accuracy for suspicious thyroid nodules than ultrasonographers. • The AI diagnosis model showed partial explainability by outputting the known risk features, thus aiding young ultrasonic doctors in increasing the diagnostic level for thyroid cancer.