1
|
Rakhshani Rad S, Mohammadi ZH, Zadeh MJ, Mosleh-Shirazi MA, Dehesh T. Identification of important symptoms and diagnostic hypothyroidism patients using machine learning algorithms. Ann Med Surg (Lond) 2024; 86:3233-3241. [PMID: 38846869 PMCID: PMC11152820 DOI: 10.1097/ms9.0000000000002068] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/02/2023] [Accepted: 03/30/2024] [Indexed: 06/09/2024] Open
Abstract
Background Hypothyroidism is one of the most common endocrine diseases. It is, however, usually challenging for physicians to diagnose due to nonspecific symptoms. The usual procedure for diagnosis of Hypothyroidism is a blood test. In recent years, machine learning algorithms have proved to be powerful tools in medicine due to their diagnostic accuracy. In this study, the authors aim to predict and identify the most important symptoms of Hypothyroidism using machine learning algorithms. Method In this cross-sectional, single-center study, 1296 individuals who visited an endocrinologist for the first time with symptoms of Hypothyroidism were studied, 676 of whom were identified as patients through thyroid-stimulating hormone testing. The outcome was binary (with Hypothyroidism /without Hypothyroidism). In a comparative analysis, random forest, decision tree, and logistic regression methods were used to diagnose primary Hypothyroidism. Results Symptoms such as tiredness, unusual cold feeling, yellow skin (jaundice), cold hands and feet, numbness of hands, loss of appetite, and weight Hypothyroidism gain were recognized as the most important symptoms in identifying Hypothyroidism. Among the studied algorithms, random forest had the best performance in identifying these symptoms (accuracy=0.83, kappa=0.46, sensitivity=0.88, specificity=0.88). Conclusions The findings suggest that machine learning methods can identify Hypothyroidism patients who show relatively simple symptoms with acceptable accuracy without the need for a blood test. Greater familiarity and utilization of such methods by physicians may, therefore, reduce the expense and stress burden of clinical testing.
Collapse
Affiliation(s)
| | - Zahra H. Mohammadi
- Endocrinology and Metabolism Research Center, Institute of Basic and Clinical Physiology Sciences
| | | | - Mohammad A. Mosleh-Shirazi
- Department of Radio-Oncology
- Ionizing and Non-Ionizing Radiation Protection Research Center (INIRPRC), School of Paramedical Sciences, Shiraz University of Medical Sciences, Shiraz, Iran
| | - Tania Dehesh
- Modeling in Health Research Center, Institute for Futures Studies in Health, Kerman University of Medical Sciences, Kerman
| |
Collapse
|
2
|
Ma X, Han X, Zhang L. An Improved k-Nearest Neighbor Algorithm for Recognition and Classification of Thyroid Nodules. JOURNAL OF ULTRASOUND IN MEDICINE : OFFICIAL JOURNAL OF THE AMERICAN INSTITUTE OF ULTRASOUND IN MEDICINE 2024; 43:1025-1036. [PMID: 38400537 DOI: 10.1002/jum.16429] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/22/2023] [Revised: 01/17/2024] [Accepted: 01/28/2024] [Indexed: 02/25/2024]
Abstract
OBJECTIVES To complete the task of automatic recognition and classification of thyroid nodules and solve the problem of high classification error rates when the samples are imbalanced. METHODS An improved k-nearest neighbor (KNN) algorithm is proposed and a method for automatic thyroid nodule classification based on the improved KNN algorithm is established. In the improved KNN algorithm, we consider not only the number of class labels for various classes of data in KNNs, but also the corresponding weights. And we use the Minkowski distance measure instead of the Euclidean distance measure. RESULTS A total of 508 ultrasound images of thyroid nodules, including 415 benign nodules and 93 malignant nodules, were used in the paper. Experimental results show the improved KNN has 0.872549 accuracy, 0.867347 precision, 1 recall, and 0.928962 F1-score. At the same time, we also considered the influence of different distance weights, the value of k, different distance measures on the classification results. CONCLUSIONS A comparison result shows that our method has a better performance than the traditional KNN and other classical machine learning methods.
Collapse
Affiliation(s)
- Xuesi Ma
- School of Mathematics and Information Science, Henan Polytechnic University, Jiaozuo, China
| | - Xiang Han
- School of Mathematics and Information Science, Henan Polytechnic University, Jiaozuo, China
| | - Lina Zhang
- School of Computer Science and Technology, Henan Polytechnic University, Jiaozuo, China
| |
Collapse
|
3
|
Akter S, Mustafa HA. Analysis and interpretability of machine learning models to classify thyroid disease. PLoS One 2024; 19:e0300670. [PMID: 38820460 PMCID: PMC11142566 DOI: 10.1371/journal.pone.0300670] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/01/2023] [Accepted: 03/01/2024] [Indexed: 06/02/2024] Open
Abstract
Thyroid disease classification plays a crucial role in early diagnosis and effective treatment of thyroid disorders. Machine learning (ML) techniques have demonstrated remarkable potential in this domain, offering accurate and efficient diagnostic tools. Most of the real-life datasets have imbalanced characteristics that hamper the overall performance of the classifiers. Existing data balancing techniques process the whole dataset at a time that sometimes causes overfitting and underfitting. However, the complexity of some ML models, often referred to as "black boxes," raises concerns about their interpretability and clinical applicability. This paper presents a comprehensive study focused on the analysis and interpretability of various ML models for classifying thyroid diseases. In our work, we first applied a new data-balancing mechanism using a clustering technique and then analyzed the performance of different ML algorithms. To address the interpretability challenge, we explored techniques for model explanation and feature importance analysis using eXplainable Artificial Intelligence (XAI) tools globally as well as locally. Finally, the XAI results are validated with the domain experts. Experimental results have shown that our proposed mechanism is efficient in diagnosing thyroid disease and can explain the models effectively. The findings can contribute to bridging the gap between adopting advanced ML techniques and the clinical requirements of transparency and accountability in diagnostic decision-making.
Collapse
Affiliation(s)
- Sumya Akter
- Institute of Information and Communication Engineering, Bangladesh University of Engineering and Technology, Dhaka, Bangladesh
- Department of Computer Science and Engineering, Hajee Mohammad Danesh Science and Technology University, Dinajpur, Bangladesh
| | - Hossen A. Mustafa
- Institute of Information and Communication Engineering, Bangladesh University of Engineering and Technology, Dhaka, Bangladesh
| |
Collapse
|
4
|
Parveen HS, Karthik S, M S K. Neural harmony: revolutionizing thyroid nodule diagnosis with hybrid networks and genetic algorithms. Comput Methods Biomech Biomed Engin 2024:1-18. [PMID: 38647355 DOI: 10.1080/10255842.2024.2341969] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/13/2023] [Accepted: 04/06/2024] [Indexed: 04/25/2024]
Abstract
In the contemporary world, thyroid disease poses a prevalent health issue, particularly affecting women's well-being. Recognizing the significance of maternal thyroid (MT) hormones in fetal neurodevelopment during the first half of pregnancy, this study introduces the HNN-GSO model. This groundbreaking hybrid approach, utilizing the MT dataset, integrates ResNet-50 and Artificial Neural Network (ANN) within a Glow-worm Swarm Optimization (GSO) framework for optimal parameter tuning. With a comprehensive methodology involving dataset preprocessing and Genetic Algorithm (GA) for feature selection, our model leverages ResNet-50 for feature extraction and ANN for classification tasks. Implemented in Python, the HNN-GSO model outperforms existing models, including K-Nearest Neighbors (KNN), Support Vector Machine (SVM), Logistic Regression (LR), Random Forest (RF), ResNet, GoogleNet, and ANN, achieving an impressive accuracy rate of 98%. This success underscores the effectiveness of our approach in complex classification tasks within machine learning (ML) and pattern recognition, specifically tailored to the Thyroid Ultrasound Images (TUI) Dataset. To provide a comprehensive understanding of performance, additional statistical measures such as precision, recall, and F1 score were considered. The HNN-GSO model consistently outperformed competitors across these metrics, showcasing its superiority in MT classification. The HNN-GSO model seamlessly combines ResNet-50's feature extraction, ANN's classification robustness, and GSO's optimization for unparalleled performance. This research offers a promising framework for advancing ML methodologies, enhancing accuracy, and efficiency in classification tasks related to MT health.
Collapse
Affiliation(s)
- H Summia Parveen
- Department of Computer Science and Engineering, Sri Eshwar College of Engineering, Coimbatore-641202
| | - S Karthik
- Department of Computer Science & Engineering, SNS College of Technology, Coimbatore, India
| | - Kavitha M S
- Department of Computer Science & Engineering, SNS College of Technology, Coimbatore, India
| |
Collapse
|
5
|
Ji S. SSC: The novel self-stack ensemble model for thyroid disease prediction. PLoS One 2024; 19:e0295501. [PMID: 38170718 PMCID: PMC10763970 DOI: 10.1371/journal.pone.0295501] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/30/2022] [Accepted: 11/22/2023] [Indexed: 01/05/2024] Open
Abstract
Thyroid disease presents a significant health risk, lowering the quality of life and increasing treatment costs. The diagnosis of thyroid disease can be challenging, especially for inexperienced practitioners. Machine learning has been established as one of the methods for disease diagnosis based on previous studies. This research introduces a novel and more effective technique for predicting thyroid disease by utilizing machine learning methodologies, surpassing the performance of previous studies in this field. This study utilizes the UCI thyroid disease dataset, which consists of 9172 samples and 30 features, and exhibits a highly imbalanced target class distribution. However, machine learning algorithms trained on imbalanced thyroid disease data face challenges in reliably detecting minority data and disease. To address this issue, re-sampling is employed, which modifies the ratio between target classes to balance the data. In this study, the down-sampling approach is utilized to achieve a balanced distribution of target classes. A novel RF-based self-stacking classifier is presented in this research for efficient thyroid disease detection. The proposed approach demonstrates the ability to diagnose primary hypothyroidism, increased binding protein, compensated hypothyroidism, and concurrent non-thyroidal illness with an accuracy of 99.5%. The recommended model exhibits state-of-the-art performance, achieving 100% macro precision, 100% macro recall, and 100% macro F1-score. A thorough comparative assessment is conducted to demonstrate the viability of the proposed approach, including several machine learning classifiers, deep neural networks, and ensemble voting classifiers. The results of K-fold cross-validation provide further support for the efficacy of the proposed self-stacking classifier.
Collapse
Affiliation(s)
- Shengjun Ji
- School of information, Xi’an University of Finance and Economics, Xi’an, China
| |
Collapse
|
6
|
Chen Z, Ying TC, Chen J, Wang Y, Wu C, Su Z. Assessment of Renal Fibrosis in Patients With Chronic Kidney Disease Using Shear Wave Elastography and Clinical Features: A Random Forest Approach. ULTRASOUND IN MEDICINE & BIOLOGY 2023; 49:1665-1671. [PMID: 37105772 DOI: 10.1016/j.ultrasmedbio.2023.03.024] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/05/2023] [Revised: 03/27/2023] [Accepted: 03/30/2023] [Indexed: 05/17/2023]
Abstract
OBJECTIVE Renal fibrosis is the common pathological hallmark of chronic kidney disease (CKD) progression. In this study, a random forest (RF) classifier based on 2-D shear wave elastography (SWE) and clinical features for the differential severity of renal fibrosis in patients with CKD is proposed. METHODS A total of 162 patients diagnosed with CKD who underwent 2-D SWE and renal biopsy were prospectively enrolled from April 2019 to December 2021 and then randomized into training (n = 114) and validation (n = 48) cohorts at a ratio of 7:3. The least absolute shrinkage and selection operator (LASSO) regression and recursive feature elimination for support vector machines (SVM-RFE) algorithm were employed to select renal fibrosis-related features from clinical information and elastosonographic findings. An RF model was subsequently constructed using the aforementioned informative parameters in the training cohort and evaluated in terms of discrimination, calibration and clinical utility in both cohorts. RESULTS The LASSO and SVM-RFE analyses revealed that age, sex, blood urea nitrogen, renal resistive index, hypertension and the 2D-SWE value were independent risk variables associated with renal fibrosis severity. The established RF model incorporating these six variables exhibited fine discrimination in both the derivation (area under the curve [AUC]: 0.84, 95% confidence interval [CI]: 0.76-0.91) and validation (AUC: 0.88, 95% CI: 0.77-0.98) cohorts. Moreover, the calibration curve revealed satisfactory predictive accuracy, and the decision curve analysis revealed a significant clinical net benefit. CONCLUSION The developed RF model, via a combination of the 2-D SWE value and clinical information, indicated satisfactory diagnostic performance and clinical practicality toward differentiating moderate-severe from mild renal fibrosis, which may provide critical insight into risk stratification for patients with CKD.
Collapse
Affiliation(s)
- Ziman Chen
- Department of Health Technology and Informatics, Hong Kong Polytechnic University, Kowloon, Hong Kong, China
| | - Tin Cheung Ying
- Department of Health Technology and Informatics, Hong Kong Polytechnic University, Kowloon, Hong Kong, China
| | - Jiaxin Chen
- Department of Ultrasound, Fifth Affiliated Hospital of Sun Yat-sen University, Zhuhai, China
| | - Yingli Wang
- Ultrasound Department, EDAN Instruments, Inc., Shenzhen, China
| | - Chaoqun Wu
- Department of Ultrasound, Fifth Affiliated Hospital of Sun Yat-sen University, Zhuhai, China
| | - Zhongzhen Su
- Department of Ultrasound, Fifth Affiliated Hospital of Sun Yat-sen University, Zhuhai, China.
| |
Collapse
|
7
|
Alyoubi EH, Moria KM, Alghamdi JS, Tayeb HO. An Optimized Deep Learning Model for Predicting Mild Cognitive Impairment Using Structural MRI. SENSORS (BASEL, SWITZERLAND) 2023; 23:5648. [PMID: 37420812 DOI: 10.3390/s23125648] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/04/2023] [Revised: 06/05/2023] [Accepted: 06/10/2023] [Indexed: 07/09/2023]
Abstract
Early diagnosis of mild cognitive impairment (MCI) with magnetic resonance imaging (MRI) has been shown to positively affect patients' lives. To save time and costs associated with clinical investigation, deep learning approaches have been used widely to predict MCI. This study proposes optimized deep learning models for differentiating between MCI and normal control samples. In previous studies, the hippocampus region located in the brain is used extensively to diagnose MCI. The entorhinal cortex is a promising area for diagnosing MCI since severe atrophy is observed when diagnosing the disease before the shrinkage of the hippocampus. Due to the small size of the entorhinal cortex area relative to the hippocampus, limited research has been conducted on the entorhinal cortex brain region for predicting MCI. This study involves the construction of a dataset containing only the entorhinal cortex area to implement the classification system. To extract the features of the entorhinal cortex area, three different neural network architectures are optimized independently: VGG16, Inception-V3, and ResNet50. The best outcomes were achieved utilizing the convolution neural network classifier and the Inception-V3 architecture for feature extraction, with accuracy, sensitivity, specificity, and area under the curve scores of 70%, 90%, 54%, and 69%, respectively. Furthermore, the model has an acceptable balance between precision and recall, achieving an F1 score of 73%. The results of this study validate the effectiveness of our approach in predicting MCI and may contribute to diagnosing MCI through MRI.
Collapse
Affiliation(s)
- Esraa H Alyoubi
- Department of Computer Science, College of Computing and Information Technology, King Abdulaziz University, Jeddah 21589, Saudi Arabia
| | - Kawthar M Moria
- Department of Computer Science, College of Computing and Information Technology, King Abdulaziz University, Jeddah 21589, Saudi Arabia
| | - Jamaan S Alghamdi
- Department of Diagnostic Radiology, Faculty of Applied Medical Sciences, King Abdulaziz University, Jeddah 21589, Saudi Arabia
| | - Haythum O Tayeb
- The Neuroscience Research Unit, Faculty of Medicine, King Abdulaziz University, Jeddah 21589, Saudi Arabia
| |
Collapse
|
8
|
Guetari R, Ayari H, Sakly H. Computer-aided diagnosis systems: a comparative study of classical machine learning versus deep learning-based approaches. Knowl Inf Syst 2023; 65:1-41. [PMID: 37361377 PMCID: PMC10205571 DOI: 10.1007/s10115-023-01894-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/23/2022] [Revised: 04/23/2023] [Accepted: 04/25/2023] [Indexed: 06/28/2023]
Abstract
The diagnostic phase of the treatment process is essential for patient guidance and follow-up. The accuracy and effectiveness of this phase can determine the life or death of a patient. For the same symptoms, different doctors may come up with different diagnoses whose treatments may, instead of curing a patient, be fatal. Machine learning (ML) brings new solutions to healthcare professionals to save time and optimize the appropriate diagnosis. ML is a data analysis method that automates the creation of analytical models and promotes predictive data. There are several ML models and algorithms that rely on features extracted from, for example, a patient's medical images to indicate whether a tumor is benign or malignant. The models differ in the way they operate and the method used to extract the discriminative features of the tumor. In this article, we review different ML models for tumor classification and COVID-19 infection to evaluate the different works. The computer-aided diagnosis (CAD) systems, which we referred to as classical, are based on accurate feature identification, usually performed manually or with other ML techniques that are not involved in classification. The deep learning-based CAD systems automatically perform the identification and extraction of discriminative features. The results show that the two types of DAC have quite close performances but the use of one or the other type depends on the datasets. Indeed, manual feature extraction is necessary when the size of the dataset is small; otherwise, deep learning is used.
Collapse
Affiliation(s)
- Ramzi Guetari
- SERCOM Laboratory, Polytechnic School of Tunisia, University of Carthage, PO Box 743, La Marsa, 2078 Tunisia
| | - Helmi Ayari
- SERCOM Laboratory, Polytechnic School of Tunisia, University of Carthage, PO Box 743, La Marsa, 2078 Tunisia
| | - Houneida Sakly
- RIADI Laboratory, National School of Computer Sciences, University of Manouba, Manouba, 2010 Tunisia
| |
Collapse
|
9
|
Liao B, Liang J, Guo B, Jia X, Lu J, Zhang T, Sun R. ILSHIP: An interpretable and predictive model for hypothyroidism. Comput Biol Med 2023; 154:106578. [PMID: 36738707 DOI: 10.1016/j.compbiomed.2023.106578] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/23/2022] [Revised: 01/08/2023] [Accepted: 01/22/2023] [Indexed: 02/01/2023]
Abstract
Hypothyroidism is one of the common endocrine diseases, and its incidence is increasing year by year. Due to the insidious nature of this disease, it often leads to delayed treatment and even misdiagnosis. This paper proposes ILSHIP, an interpretable predictive model for hypothyroidism, to reduce its diagnostic complexity as well as improve the predictive performance and interpretability of existing models. First, the ILSHIP prediction model was built based on label encoding, missing value processing, feature selection, and data enhancement of the dataset. Second, the comprehensive performance of ILSHIP was compared with twelve existing related study models and eleven mainstream models, such as XGBoost and MLP. The experimental results showed that, based on the optimal hyperparameters the ILSHIP model can achieve 99.392%, 99.437%, 99.348%, 99.381%, and 99.960% in accuracy, recall, specificity, F1, and AUC, respectively. The accuracy of the ILSHIP model was about 0.7%-15.4% higher than the existing models. By introducing the SHAP framework into the ILSHIP model, important features affecting hypothyroidism such as thyroid stimulating hormone (TSH) and free thyroxine index (FTI) were also identified, and the influencing factors for different individuals were finally analyzed to provide a basis for medical personnel to monitor the condition.
Collapse
Affiliation(s)
- Bin Liao
- College of Big Data Statistics, Guizhou University of Finance and Economics, Guiyang, 550025, PR China
| | - Jinming Liang
- College of Statistics and Data Science, Xinjiang University of Finance and Economics, Urumqi, 830012, PR China.
| | - Binglei Guo
- School of Computer Engineering, Hubei University of Arts and Science, Xiangyang, 441053, PR China
| | - Xiaoyao Jia
- College of Statistics and Data Science, Xinjiang University of Finance and Economics, Urumqi, 830012, PR China
| | - Jiarong Lu
- College of Statistics and Data Science, Xinjiang University of Finance and Economics, Urumqi, 830012, PR China
| | - Tao Zhang
- Department of Medical Engineering and Technology, Xinjiang Medical University, Urumqi, 830011, PR China
| | - Ruina Sun
- College of Statistics and Data Science, Xinjiang University of Finance and Economics, Urumqi, 830012, PR China; Institute of Information Engineering, Chinese Academy of Sciences, Beijing, 100093, PR China; School of Networks Security, University of Chinese Academy of Sciences, Beijing, 100049, PR China
| |
Collapse
|
10
|
Chaganti R, Rustam F, De La Torre Díez I, Mazón JLV, Rodríguez CL, Ashraf I. Thyroid Disease Prediction Using Selective Features and Machine Learning Techniques. Cancers (Basel) 2022; 14:cancers14163914. [PMID: 36010907 PMCID: PMC9405591 DOI: 10.3390/cancers14163914] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/14/2022] [Revised: 08/09/2022] [Accepted: 08/11/2022] [Indexed: 11/21/2022] Open
Abstract
Simple Summary The study presents a thyroid disease prediction approach which utilizes random forest-based features to obtain high accuracy. The approach can obtain a 0.99 accuracy to predict ten thyroid diseases. Abstract Thyroid disease prediction has emerged as an important task recently. Despite existing approaches for its diagnosis, often the target is binary classification, the used datasets are small-sized and results are not validated either. Predominantly, existing approaches focus on model optimization and the feature engineering part is less investigated. To overcome these limitations, this study presents an approach that investigates feature engineering for machine learning and deep learning models. Forward feature selection, backward feature elimination, bidirectional feature elimination, and machine learning-based feature selection using extra tree classifiers are adopted. The proposed approach can predict Hashimoto’s thyroiditis (primary hypothyroid), binding protein (increased binding protein), autoimmune thyroiditis (compensated hypothyroid), and non-thyroidal syndrome (NTIS) (concurrent non-thyroidal illness). Extensive experiments show that the extra tree classifier-based selected feature yields the best results with 0.99 accuracy and an F1 score when used with the random forest classifier. Results suggest that the machine learning models are a better choice for thyroid disease detection regarding the provided accuracy and the computational complexity. K-fold cross-validation and performance comparison with existing studies corroborate the superior performance of the proposed approach.
Collapse
Affiliation(s)
| | - Furqan Rustam
- Department of Software Engineering, School of System Sciences, University of Management and Technology, Lahore 54770, Pakistan
| | - Isabel De La Torre Díez
- Department of Signal Theory and Communications and Telematic Engineering, University of Valladolid, Paseo de Belén 15, 47011 Valladolid, Spain
- Correspondence: (I.D.L.T.D.); (I.A.)
| | - Juan Luis Vidal Mazón
- Higher Polytechnic School, Universidad Europea del Atlántico, Parque Científico y Tecnológico de Cantabria, Isabel Torres 21, 39011 Santander, Spain
- Project Department, Universidade Internacional do Cuanza, Cuito EN250, Bié, Angola
- Department of Project Management, Universidad Internacional Iberoamericana, Arecibo, PR 00613, USA
| | - Carmen Lili Rodríguez
- Higher Polytechnic School, Universidad Europea del Atlántico, Parque Científico y Tecnológico de Cantabria, Isabel Torres 21, 39011 Santander, Spain
- Department of Project Management, Universidad Internacional Iberoamericana, Campeche 24560, Mexico
| | - Imran Ashraf
- Department of Information and Communication Engineering, Yeungnam University, Gyeongsan 38541, Korea
- Correspondence: (I.D.L.T.D.); (I.A.)
| |
Collapse
|