1
|
Teuho J, Schultz J, Klén R, Juarez-Orozco LE, Knuuti J, Saraste A, Ono N, Kanaya S. Explainable deep-learning-based ischemia detection using hybrid O-15 H 2O perfusion positron emission tomography and computed tomography imaging with clinical data. J Nucl Cardiol 2024; 38:101889. [PMID: 38852900 DOI: 10.1016/j.nuclcard.2024.101889] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/08/2024] [Revised: 04/30/2024] [Accepted: 05/25/2024] [Indexed: 06/11/2024]
Abstract
BACKGROUND We developed an explainable deep-learning (DL)-based classifier to identify flow-limiting coronary artery disease (CAD) by O-15 H2O perfusion positron emission tomography computed tomography (PET/CT) and coronary CT angiography (CTA) imaging. The classifier uses polar map images with numerical data and visualizes data findings. METHODS A DLmodel was implemented and evaluated on 138 individuals, consisting of a combined image-and data-based classifier considering 35 clinical, CTA, and PET variables. Data from invasive coronary angiography were used as reference. Performance was evaluated with clinical classification using accuracy (ACC), area under the receiver operating characteristic curve (AUC), F1 score (F1S), sensitivity (SEN), specificity (SPE), precision (PRE), net benefit, and Cohen's Kappa. Statistical testing was conducted using McNemar's test. RESULTS The DL model had a median ACC = 0.8478, AUC = 0.8481, F1S = 0.8293, SEN = 0.8500, SPE = 0.8846, and PRE = 0.8500. Improved detection of true-positive and false-negative cases, increased net benefit in thresholds up to 34%, and comparable Cohen's kappa was seen, reaching similar performance to clinical reading. Statistical testing revealed no significant differences between DL model and clinical reading. CONCLUSIONS The combined DL model is a feasible and an effective method in detection of CAD, allowing to highlight important data findings individually in interpretable manner.
Collapse
Affiliation(s)
- Jarmo Teuho
- Data Science Center, Nara Institute of Science and Technology, Nara, Japan; Turku PET Centre, University of Turku, Turku, Finland; Turku PET Centre, Turku University Hospital, Turku, Finland.
| | - Jussi Schultz
- Turku PET Centre, University of Turku, Turku, Finland; Turku PET Centre, Turku University Hospital, Turku, Finland
| | - Riku Klén
- Turku PET Centre, University of Turku, Turku, Finland; Turku PET Centre, Turku University Hospital, Turku, Finland
| | - Luis Eduardo Juarez-Orozco
- Department of Cardiology, Division Heart and Lungs, University Medical Center Utrecht, Utrecht University, Utrecht, the Netherlands; Department of Cardiology, Meader Medical Center, Amersfoort, the Netherlands
| | - Juhani Knuuti
- Turku PET Centre, University of Turku, Turku, Finland; Turku PET Centre, Turku University Hospital, Turku, Finland
| | - Antti Saraste
- Turku PET Centre, Turku University Hospital, Turku, Finland; Heart Centre, Turku University Hospital and University of Turku, Turku, Finland
| | - Naoaki Ono
- Data Science Center, Nara Institute of Science and Technology, Nara, Japan; Department of Science and Technology, Nara Institute of Science and Technology, Nara, Japan
| | - Shigehiko Kanaya
- Data Science Center, Nara Institute of Science and Technology, Nara, Japan; Department of Science and Technology, Nara Institute of Science and Technology, Nara, Japan
| |
Collapse
|
2
|
Nurmirinta TAT, Turunen MJ, Korhonen RK, Tohka J, Liukkonen MK, Mononen ME. Two-Stage Classification of Future Knee Osteoarthritis Severity After 8 Years Using MRI: Data from the Osteoarthritis Initiative. Ann Biomed Eng 2024:10.1007/s10439-024-03578-x. [PMID: 38980544 DOI: 10.1007/s10439-024-03578-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/04/2023] [Accepted: 06/28/2024] [Indexed: 07/10/2024]
Abstract
Currently, there are no methods or tools available in clinical practice for classifying future knee osteoarthritis (KOA). In this study, we aimed to fill this gap by classifying future KOA into three severity grades: KL01 (healthy), KL2 (moderate), and KL34 (severe) based on the Kellgren-Lawrance scale. Due to the complex nature of multiclass classification, we used a two-stage method, which separates the classification task into two binary classifications (KL01 vs. KL234 in the first stage and KL2 vs. KL34 in the second stage). Our machine learning (ML) model used two Balanced Random Forest algorithms and was trained with gender, age, height, weight, and quantitative knee morphology obtained from magnetic resonance imaging. Our training dataset comprised longitudinal 8-year follow-up data of 1213 knees from the Osteoarthritis Initiative. Through extensive experimentation with various feature combinations, we identified KL baseline and weight as the most essential features, while gender surprisingly proved to be one of the least influential feature. Our best classification model generated a weighted F1 score of 79.0% and a balanced accuracy of 65.9%. The area under the receiver operating characteristic curve was 83.0% for healthy (KL01) versus moderate (KL2) or severe (KL34) KOA patients and 86.6% for moderate (KL2) versus severe (KL34) KOA patients. We found a statistically significant difference in performance between our two-stage classification model and the traditional single-stage classification model. These findings demonstrate the encouraging results of our two-stage classification model for multiclass KOA severity classification, suggesting its potential application in clinical settings in future.
Collapse
Affiliation(s)
- Teemu A T Nurmirinta
- Department of Technical Physics, University of Eastern Finland, POB 1627, FI-70211, Kuopio, Finland.
- Diagnostic Imaging Centre, Kuopio University Hospital, The Wellbeing Services County of North Savo, Kuopio, Finland.
| | - Mikael J Turunen
- Department of Technical Physics, University of Eastern Finland, POB 1627, FI-70211, Kuopio, Finland
- Science Service Center, Kuopio University Hospital, The Wellbeing Services County of North Savo, Kuopio, Finland
| | - Rami K Korhonen
- Department of Technical Physics, University of Eastern Finland, POB 1627, FI-70211, Kuopio, Finland
| | - Jussi Tohka
- AI Virtanen Institute for Molecular Sciences, University of Eastern Finland, Kuopio, Finland
| | - Mimmi K Liukkonen
- Diagnostic Imaging Centre, Kuopio University Hospital, The Wellbeing Services County of North Savo, Kuopio, Finland
| | - Mika E Mononen
- Department of Technical Physics, University of Eastern Finland, POB 1627, FI-70211, Kuopio, Finland
| |
Collapse
|
3
|
Hase H, Mine Y, Okazaki S, Yoshimi Y, Ito S, Peng TY, Sano M, Koizumi Y, Kakimoto N, Tanimoto K, Murayama T. Sex estimation from maxillofacial radiographs using a deep learning approach. Dent Mater J 2024; 43:394-399. [PMID: 38599831 DOI: 10.4012/dmj.2023-253] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/12/2024]
Abstract
The purpose of this study was to construct deep learning models for more efficient and reliable sex estimation. Two deep learning models, VGG16 and DenseNet-121, were used in this retrospective study. In total, 600 lateral cephalograms were analyzed. A saliency map was generated by gradient-weighted class activation mapping for each output. The two deep learning models achieved high values in each performance metric according to accuracy, sensitivity (recall), precision, F1 score, and areas under the receiver operating characteristic curve. Both models showed substantial differences in the positions indicated in saliency maps for male and female images. The positions in saliency maps also differed between VGG16 and DenseNet-121, regardless of sex. This analysis of our proposed system suggested that sex estimation from lateral cephalograms can be achieved with high accuracy using deep learning.
Collapse
Affiliation(s)
- Hiroki Hase
- Department of Medical Systems Engineering, Graduate School of Biomedical and Health Sciences, Hiroshima University
| | - Yuichi Mine
- Department of Medical Systems Engineering, Graduate School of Biomedical and Health Sciences, Hiroshima University
- Project Research Center for Integrating Digital Dentistry, Hiroshima University
| | - Shota Okazaki
- Department of Medical Systems Engineering, Graduate School of Biomedical and Health Sciences, Hiroshima University
- Project Research Center for Integrating Digital Dentistry, Hiroshima University
| | - Yuki Yoshimi
- Department of Orthodontics and Craniofacial Developmental Biology, Graduate School of Biomedical and Health Sciences, Hiroshima University
| | - Shota Ito
- Department of Orthodontics and Craniofacial Developmental Biology, Graduate School of Biomedical and Health Sciences, Hiroshima University
| | - Tzu-Yu Peng
- School of Dentistry, College of Oral Medicine, Taipei Medical University
| | - Mizuho Sano
- Department of Medical Systems Engineering, Graduate School of Biomedical and Health Sciences, Hiroshima University
| | - Yuma Koizumi
- Department of Orthodontics and Craniofacial Developmental Biology, Graduate School of Biomedical and Health Sciences, Hiroshima University
| | - Naoya Kakimoto
- School of Dentistry, College of Oral Medicine, Taipei Medical University
| | - Kotaro Tanimoto
- Department of Oral and Maxillofacial Radiology, Graduate School of Biomedical and Health Sciences, Hiroshima University
| | - Takeshi Murayama
- Department of Medical Systems Engineering, Graduate School of Biomedical and Health Sciences, Hiroshima University
- Project Research Center for Integrating Digital Dentistry, Hiroshima University
| |
Collapse
|
4
|
Lu W, Wang Q, Liu L, Luo W. Exploring the mystery of colon cancer from the perspective of molecular subtypes and treatment. Sci Rep 2024; 14:10883. [PMID: 38740818 DOI: 10.1038/s41598-024-60495-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/30/2023] [Accepted: 04/23/2024] [Indexed: 05/16/2024] Open
Abstract
The molecular categorization of colon cancer patients remains elusive. Gene set enrichment analysis (GSEA), which investigates the dysregulated genes among tumor and normal samples, has revealed the pivotal role of epithelial-to-mesenchymal transition (EMT) in colon cancer pathogenesis. In this study, we employed multi-clustering method for grouping data, resulting in the identification of two clusters characterized by varying prognostic outcomes. These two subgroups not only displayed disparities in overall survival (OS) but also manifested variations in clinical variables, genetic mutation, and gene expression profiles. Using the nearest template prediction (NTP) method, we were able to replicate the molecular classification effectively within the original dataset and validated it across multiple independent datasets, underscoring its robust repeatability. Furthermore, we constructed two prognostic signatures tailored to each of these subgroups. Our molecular classification, centered on EMT, hold promise in offering fresh insights into the therapy strategies and prognosis assessment for colon cancer.
Collapse
Affiliation(s)
- Wenhong Lu
- The Second Affiliated Hospital of Hunan University of Chinese Medicine, Changsha, 410005, Hunan, People's Republic of China
| | - Qiwei Wang
- Hunan Provincial Rehabilitation Hospital, Changsha, 410007, Hunan, People's Republic of China
| | - Lifang Liu
- The First Affiliated Hospital of Hunan University of Chinese Medicine, Changsha, 410007, Hunan, People's Republic of China
| | - Wenpeng Luo
- The Second Affiliated Hospital of Hunan University of Chinese Medicine, Changsha, 410005, Hunan, People's Republic of China.
| |
Collapse
|
5
|
Rainio O, Teuho J, Klén R. Evaluation metrics and statistical tests for machine learning. Sci Rep 2024; 14:6086. [PMID: 38480847 PMCID: PMC10937649 DOI: 10.1038/s41598-024-56706-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/13/2023] [Accepted: 03/09/2024] [Indexed: 03/17/2024] Open
Abstract
Research on different machine learning (ML) has become incredibly popular during the past few decades. However, for some researchers not familiar with statistics, it might be difficult to understand how to evaluate the performance of ML models and compare them with each other. Here, we introduce the most common evaluation metrics used for the typical supervised ML tasks including binary, multi-class, and multi-label classification, regression, image segmentation, object detection, and information retrieval. We explain how to choose a suitable statistical test for comparing models, how to obtain enough values of the metric for testing, and how to perform the test and interpret its results. We also present a few practical examples about comparing convolutional neural networks used to classify X-rays with different lung infections and detect cancer tumors in positron emission tomography images.
Collapse
Affiliation(s)
- Oona Rainio
- Turku PET Centre, University of Turku and Turku University Hospital, Turku, Finland.
| | - Jarmo Teuho
- Turku PET Centre, University of Turku and Turku University Hospital, Turku, Finland
| | - Riku Klén
- Turku PET Centre, University of Turku and Turku University Hospital, Turku, Finland
| |
Collapse
|
6
|
Massago M, Massago M, Iora PH, Tavares Gurgel SJ, Conegero CI, Carolino IDR, Mushi MM, Chaves Forato GA, de Souza JVP, Hernandes Rocha TA, Bonfim S, Staton CA, Nihei OK, Vissoci JRN, de Andrade L. Applicability of machine learning algorithm to predict the therapeutic intervention success in Brazilian smokers. PLoS One 2024; 19:e0295970. [PMID: 38437221 PMCID: PMC10911606 DOI: 10.1371/journal.pone.0295970] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/09/2023] [Accepted: 12/02/2023] [Indexed: 03/06/2024] Open
Abstract
Smoking cessation is an important public health policy worldwide. However, as far as we know, there is a lack of screening of variables related to the success of therapeutic intervention (STI) in Brazilian smokers by machine learning (ML) algorithms. To address this gap in the literature, we evaluated the ability of eight ML algorithms to correctly predict the STI in Brazilian smokers who were treated at a smoking cessation program in Brazil between 2006 and 2017. The dataset was composed of 12 variables and the efficacies of the algorithms were measured by accuracy, sensitivity, specificity, positive predictive value (PPV) and area under the receiver operating characteristic curve. We plotted a decision tree flowchart and also measured the odds ratio (OR) between each independent variable and the outcome, and the importance of the variable for the best model based on PPV. The mean global values for the metrics described above were, respectively, 0.675±0.028, 0.803±0.078, 0.485±0.146, 0.705±0.035 and 0.680±0.033. Supporting vector machines performed the best algorithm with a PPV of 0.726±0.031. Smoking cessation drug use was the roof of decision tree with OR of 4.42 and importance of variable of 100.00. Increase in the number of relapses also promoted a positive outcome, while higher consumption of cigarettes resulted in the opposite. In summary, the best model predicted 72.6% of positive outcomes correctly. Smoking cessation drug use and higher number of relapses contributed to quit smoking, while higher consumption of cigarettes showed the opposite effect. There are important strategies to reduce the number of smokers and increase STI by increasing services and drug treatment for smokers.
Collapse
Affiliation(s)
- Miyoko Massago
- PhD Student in the Postgraduate Program in Health Sciences, State University of Maringa, Maringa, Parana, Brazil
| | - Mamoru Massago
- Master in Computer Sciences, State University of Maringa, Maringa, Parana, Brazil
| | - Pedro Henrique Iora
- Professor in the Morphological Sciences Department, State University of Maringa, Maringa, Parana, Brazil
| | | | - Celso Ivam Conegero
- Professor in the Department of Medicine, State University of Maringa, Maringa, Parana, Brazil
| | | | - Maria Muzanila Mushi
- Global Emergency Medicine Innovation and Implementation Research Center, Duke University School of Medicine, Duke Global Health Institute, Durham, North Carolina, United States of America
| | | | - João Vitor Perez de Souza
- Assistant Professor of Emergency Medicine and Global Health, Duke Global Health Institute, Department of Emergency Medicine, Duke University School of Medicine, Durham, North Carolina, United States of America
| | - Thiago Augusto Hernandes Rocha
- Assistant Professor of Emergency Medicine and Global Health, Duke Global Health Institute, Department of Emergency Medicine, Duke University School of Medicine, Durham, North Carolina, United States of America
| | - Samile Bonfim
- PhD Student in the Postgraduate Program in Health Sciences, State University of Maringa, Maringa, Parana, Brazil
| | - Catherine Ann Staton
- Assistant Professor of Emergency Medicine and Global Health, Duke Global Health Institute, Department of Emergency Medicine, Duke University School of Medicine, Durham, North Carolina, United States of America
| | - Oscar Kenji Nihei
- Professor in the Center of Education, Literature and Health, Western Parana State University, Foz do Iguaçu, Parana, Brazil
| | - João Ricardo Nickenig Vissoci
- Assistant Professor of Emergency Medicine and Global Health, Duke Global Health Institute, Department of Emergency Medicine, Duke University School of Medicine, Durham, North Carolina, United States of America
| | - Luciano de Andrade
- Professor in the Postgraduate Program in Health Sciences, State University of Maringa, Maringa, Parana, Brazil
| |
Collapse
|
7
|
Xu J. On the bias in the AUC variance estimate. Pattern Recognit Lett 2024; 178:62-68. [PMID: 38186922 PMCID: PMC10768968 DOI: 10.1016/j.patrec.2023.12.012] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/09/2024]
Abstract
The area under the Receiver Operating Characteristic (ROC) curve (AUC) is a standard metric for quantifying and comparing binary classifiers. A popular approach to estimating the AUCs and the associated variabilities - the variance of the AUC or the full covariance matrix of multiple correlated AUCs - is the one proposed by DeLong et al [1], which is based on the Mann Whitney two-sample U-statistics. The bias of a variance estimator is an important factor in applications such as hypothesis testing and construction of confidence intervals - a negatively biased variance estimator may lead to incorrect conclusions, and a positive bias is conservative hence preferable. In this work, we show that the (co-)variance estimate in DeLong's approach is always positively biased. More specifically, the difference matrix between the expectation of the estimated covariance and the true covariance is a positive semi-definite matrix. This bias is non-negligible when the sample size is small, and quickly diminishes as the sample size increases. Our method relies on constructing, from the AUC kernel, a random variable whose (co-)variance matrix coincides with the bias, thereby establishing the claim. We also discuss alternative approaches to AUC variance estimation that may potentially reduce the bias.
Collapse
Affiliation(s)
- Jingyan Xu
- Department of Radiology, Johns Hopkins University, MD, USA
| |
Collapse
|
8
|
Moses JC, Adibi S, Angelova M, Islam SMS. Time-domain heart rate variability features for automatic congestive heart failure prediction. ESC Heart Fail 2024; 11:378-389. [PMID: 38009405 PMCID: PMC10804149 DOI: 10.1002/ehf2.14593] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/20/2023] [Revised: 10/10/2023] [Accepted: 11/07/2023] [Indexed: 11/28/2023] Open
Abstract
AIMS Heart failure is a serious condition that often goes undiagnosed in primary care due to the lack of reliable diagnostic tools and the similarity of its symptoms with other diseases. Non-invasive monitoring of heart rate variability (HRV), which reflects the activity of the autonomic nervous system, could offer a novel and accurate way to detect and manage heart failure patients. This study aimed to assess the feasibility of using machine learning techniques on HRV data as a non-invasive biomarker to classify healthy adults and those with heart failure. METHODS AND RESULTS We used digitized electrocardiogram recordings from 54 adults with normal sinus rhythm and 44 adults categorized into New York Heart Association classes 1, 2, and 3, suffering from congestive heart failure. All recordings were sourced from the PhysioNet database. Following data pre-processing, we performed time-domain HRV analysis on all individual recordings, including root mean square of the successive difference in adjacent RR interval (RRi) (RMSSD), the standard deviation of RRi (SDNN, the NN stands for natural or sinus intervals), the standard deviation of the successive differences between successive RRi (SDSD), the number or percentage of RRi longer than 50 ms (NN50 and pNN50), and the average value of RRi [mean RR interval (mRRi)]. In our experimental classification performance evaluation, on the computed HRV parameters, we optimized hyperparameters and performed five-fold cross-validation using four machine learning classification algorithms: support vector machine, k-nearest neighbour (KNN), naïve Bayes, and decision tree (DT). We evaluated the prediction accuracy of these models using performance criteria, namely, precision, recall, specificity, F1 score, and overall accuracy. For added insight, we also presented receiver operating characteristic (ROC) plots and area under the ROC curve (AUC) values. The overall best performance accuracy of 77% was achieved when KNN and DT were trained on computed HRV parameters with a 5 min time window. KNN obtained an AUC of 0.77, while DT attained 0.78. Additionally, in the classification of severe congestive heart failure, KNN and DT had the best accuracy of 91%, with KNN achieving an AUC of 0.88 and DT obtaining 0.92. CONCLUSIONS The results show that HRV can accurately predict severe congestive heart failure. The findings of this study could inform the use of machine learning approaches on non-invasive HRV, to screen congestive heart failure individuals in primary care.
Collapse
Affiliation(s)
| | - Sasan Adibi
- School of Information TechnologyDeakin UniversityBurwoodVIC3125Australia
| | - Maia Angelova
- School of Information TechnologyDeakin UniversityBurwoodVIC3125Australia
- Aston Digital Futures Institute, College of Physical Sciences and EngineeringAston UniversityBirminghamUK
| | | |
Collapse
|
9
|
Sourkatti H, Pettersson K, van der Sanden B, Lindholm M, Plomp J, Määttänen I, Henttonen P, Närväinen J. Investigation of different ML approaches in classification of emotions induced by acute stress. Heliyon 2024; 10:e23611. [PMID: 38173518 PMCID: PMC10761802 DOI: 10.1016/j.heliyon.2023.e23611] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/09/2023] [Revised: 11/02/2023] [Accepted: 12/07/2023] [Indexed: 01/05/2024] Open
Abstract
Background Machine learning is becoming a common tool in monitoring emotion. However, methodological studies of the processing pipeline are scarce, especially ones using subjective appraisals as ground truth. New method A novel protocol was used to induce cognitive load and physical discomfort, and emotional dimensions (arousal, valence, and dominance) were reported after each task. The performance of five common ML models with a versatile set of features (physiological features, task performance data, and personality trait) was compared in binary classification of subjectively assessed emotions. Results The psychophysiological responses proved the protocol was successful in changing the mental state from baseline, also the cognitive and physical tasks were different. The optimization and performance of ML models used for emotion detection were evaluated. Additionally, methods to account for imbalanced classes were applied and shown to improve the classification performance. Comparison with existing methods Classification of human emotional states often assumes the states are determined by the stimuli. However, individual appraisals vary. None of the past studies have classified subjective emotional dimensions with a set of features including biosignals, personality and behavior. Conclusion Our data represent a typical setup in affective computing utilizing psychophysiological monitoring: N is low compared to number of features, inter-individual variability is high, and class imbalance cannot be avoided. Our observations are a) if possible, include features representing physiology, behavior and personality, b) use simple models and limited number of features to improve interpretability, c) address the possible imbalance, d) if the data size allows, use nested cross-validation.
Collapse
Affiliation(s)
- Heba Sourkatti
- VTT Technical Research Center of Finland, Tekniikantie 1, 02150 Espoo, Finland
| | - Kati Pettersson
- VTT Technical Research Center of Finland, Tekniikantie 1, 02150 Espoo, Finland
| | | | - Mikko Lindholm
- VTT Technical Research Center of Finland, Tekniikantie 1, 02150 Espoo, Finland
| | - Johan Plomp
- VTT Technical Research Center of Finland, Tekniikantie 1, 02150 Espoo, Finland
| | - Ilmari Määttänen
- University of Helsinki, Department of Psychology and Logopedics, Faculty of Medicine, P.O. Box 63, 00014 University of Helsinki, Finland
| | - Pentti Henttonen
- University of Helsinki, Department of Psychology and Logopedics, Faculty of Medicine, P.O. Box 63, 00014 University of Helsinki, Finland
| | - Johanna Närväinen
- VTT Technical Research Center of Finland, Tekniikantie 1, 02150 Espoo, Finland
| |
Collapse
|
10
|
Wilimitis D, Walsh CG. Practical Considerations and Applied Examples of Cross-Validation for Model Development and Evaluation in Health Care: Tutorial. JMIR AI 2023; 2:e49023. [PMID: 38875530 PMCID: PMC11041453 DOI: 10.2196/49023] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/15/2023] [Revised: 09/19/2023] [Accepted: 09/28/2023] [Indexed: 06/16/2024]
Abstract
Cross-validation remains a popular means of developing and validating artificial intelligence for health care. Numerous subtypes of cross-validation exist. Although tutorials on this validation strategy have been published and some with applied examples, we present here a practical tutorial comparing multiple forms of cross-validation using a widely accessible, real-world electronic health care data set: Medical Information Mart for Intensive Care-III (MIMIC-III). This tutorial explored methods such as K-fold cross-validation and nested cross-validation, highlighting their advantages and disadvantages across 2 common predictive modeling use cases: classification (mortality) and regression (length of stay). We aimed to provide readers with reproducible notebooks and best practices for modeling with electronic health care data. We also described sets of useful recommendations as we demonstrated that nested cross-validation reduces optimistic bias but comes with additional computational challenges. This tutorial might improve the community's understanding of these important methods while catalyzing the modeling community to apply these guides directly in their work using the published code.
Collapse
Affiliation(s)
- Drew Wilimitis
- Vanderbilt University Medical Center, Vanderbilt University, Nashville, TN, United States
| | - Colin G Walsh
- Vanderbilt University Medical Center, Vanderbilt University, Nashville, TN, United States
| |
Collapse
|
11
|
Saini HK, Chouhan SS, Kathuria A, Sarkar AK. Evaluating overtaking and filtering maneuver of motorcyclists and car drivers using advanced trajectory data analysis. Int J Inj Contr Saf Promot 2023; 30:530-546. [PMID: 37343167 DOI: 10.1080/17457300.2023.2225162] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/27/2022] [Accepted: 06/11/2023] [Indexed: 06/23/2023]
Abstract
The present paper compares motorized two-wheeler (MTW) and passenger car's interactions with the rest of the traffic in urban roads while performing overtaking and filtering maneuvers. To better understand filtering maneuvers of motorcyclists and car drivers, an attempt was made to propose a new measure, i.e. pore size ratio. Additionally, the factors affecting lateral width acceptance for motorcyclists and car drivers while overtaking and filtering were studied using advanced trajectory data. A regression model was developed to predict the significant factors affecting motorcyclist's and car driver's decisions to accept lateral width with the adjacent vehicle while performing overtaking and filtering maneuvers. Finally, a comparative analysis between machine learning and the probit model revealed that, in the present case, machine learning models perform better than the probit model in terms of the model's discernment power. The findings of this study will help ameliorate the power of existing microsimulation tools.
Collapse
Affiliation(s)
- Harish Kumar Saini
- Department of Civil Engineering, Indian Institute of Technology, Jammu, Jammu and Kashmir, India
| | - Shivam Singh Chouhan
- Department of Civil Engineering, Indian Institute of Technology, Jammu, Jammu and Kashmir, India
| | - Ankit Kathuria
- Department of Civil Engineering, Indian Institute of Technology, Jammu, Jammu and Kashmir, India
| | - Ashoke Kumar Sarkar
- Department of Civil Engineering, Indian Institute of Technology, Jammu, Jammu and Kashmir, India
| |
Collapse
|
12
|
Linus A, Tanska P, Sarin JK, Nippolainen E, Tiitu V, Mäkelä JTA, Töyräs J, Korhonen RK, Mononen ME, Afara IO. Visible and Near-Infrared Spectroscopy Enables Differentiation of Normal and Early Osteoarthritic Human Knee Joint Articular Cartilage. Ann Biomed Eng 2023; 51:2245-2257. [PMID: 37332006 PMCID: PMC10518273 DOI: 10.1007/s10439-023-03261-7] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/23/2022] [Accepted: 05/27/2023] [Indexed: 06/20/2023]
Abstract
Osteoarthritis degenerates cartilage and impairs joint function. Early intervention opportunities are missed as current diagnostic methods are insensitive to early tissue degeneration. We investigated the capability of visible light-near-infrared spectroscopy (Vis-NIRS) to differentiate normal human cartilage from early osteoarthritic one. Vis-NIRS spectra, biomechanical properties and the state of osteoarthritis (OARSI grade) were quantified from osteochondral samples harvested from different anatomical sites of human cadaver knees. Two support vector machines (SVM) classifiers were developed based on the Vis-NIRS spectra and OARSI scores. The first classifier was designed to distinguish normal (OARSI: 0-1) from general osteoarthritic cartilage (OARSI: 2-5) to check the general suitability of the approach yielding an average accuracy of 75% (AUC = 0.77). Then, the second classifier was designed to distinguish normal from early osteoarthritic cartilage (OARSI: 2-3) yielding an average accuracy of 71% (AUC = 0.73). Important wavelength regions for differentiating normal from early osteoarthritic cartilage were related to collagen organization (wavelength region: 400-600 nm), collagen content (1000-1300 nm) and proteoglycan content (1600-1850 nm). The findings suggest that Vis-NIRS allows objective differentiation of normal and early osteoarthritic tissue, e.g., during arthroscopic repair surgeries.
Collapse
Affiliation(s)
- Awuniji Linus
- Department of Technical Physics, University of Eastern Finland, 70211, Kuopio, Finland.
| | - Petri Tanska
- Department of Technical Physics, University of Eastern Finland, 70211, Kuopio, Finland
| | - Jaakko K Sarin
- Department of Medical Physics, Medical Imaging Center, Pirkanmaa Hospital District, Tampere, Finland
| | - Ervin Nippolainen
- Department of Technical Physics, University of Eastern Finland, 70211, Kuopio, Finland
| | - Virpi Tiitu
- Institute of Biomedicine, University of Eastern Finland, Kuopio, Finland
| | - Janne T A Mäkelä
- Department of Technical Physics, University of Eastern Finland, 70211, Kuopio, Finland
- Science Service Center, Kuopio University Hospital, Kuopio, Finland
| | - Juha Töyräs
- Department of Technical Physics, University of Eastern Finland, 70211, Kuopio, Finland
- Science Service Center, Kuopio University Hospital, Kuopio, Finland
- School of Information Technology and Electrical Engineering, The University of Queensland, Brisbane, Australia
| | - Rami K Korhonen
- Department of Technical Physics, University of Eastern Finland, 70211, Kuopio, Finland
| | - Mika E Mononen
- Department of Technical Physics, University of Eastern Finland, 70211, Kuopio, Finland
| | - Isaac O Afara
- Department of Technical Physics, University of Eastern Finland, 70211, Kuopio, Finland
- School of Information Technology and Electrical Engineering, The University of Queensland, Brisbane, Australia
| |
Collapse
|
13
|
Zhou AX, Aczon MD, Laksana E, Ledbetter DR, Wetzel RC. Narrowing the gap: expected versus deployment performance. J Am Med Inform Assoc 2023; 30:1474-1485. [PMID: 37311708 PMCID: PMC10436142 DOI: 10.1093/jamia/ocad100] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/23/2022] [Revised: 04/25/2023] [Accepted: 05/31/2023] [Indexed: 06/15/2023] Open
Abstract
OBJECTIVES Successful model development requires both an accurate a priori understanding of future performance and high performance on deployment. Optimistic estimations of model performance that are unrealized in real-world clinical settings can contribute to nonuse of predictive models. This study used 2 tasks, predicting ICU mortality and Bi-Level Positive Airway Pressure failure, to quantify: (1) how well internal test performances derived from different methods of partitioning data into development and test sets estimate future deployment performance of Recurrent Neural Network models and (2) the effects of including older data in the training set on models' performance. MATERIALS AND METHODS The cohort consisted of patients admitted between 2010 and 2020 to the Pediatric Intensive Care Unit of a large quaternary children's hospital. 2010-2018 data were partitioned into different development and test sets to measure internal test performance. Deployable models were trained on 2010-2018 data and assessed on 2019-2020 data, which was conceptualized to represent a real-world deployment scenario. Optimism, defined as the overestimation of the deployed performance by internal test performance, was measured. Performances of deployable models were also compared with each other to quantify the effect of including older data during training. RESULTS, DISCUSSION, AND CONCLUSION Longitudinal partitioning methods, where models are tested on newer data than the development set, yielded the least optimism. Including older years in the training dataset did not degrade deployable model performance. Using all available data for model development fully leveraged longitudinal partitioning by measuring year-to-year performance.
Collapse
Affiliation(s)
- Alice X Zhou
- Department of Anesthesiology and Critical Care Medicine, Children’s Hospital Los Angeles, Los Angeles, California, USA
- Laura P. and Leland K. Whittier Virtual Pediatric Intensive Care Unit, Children’s Hospital Los Angeles, Los Angeles, California, USA
| | - Melissa D Aczon
- Department of Anesthesiology and Critical Care Medicine, Children’s Hospital Los Angeles, Los Angeles, California, USA
- Laura P. and Leland K. Whittier Virtual Pediatric Intensive Care Unit, Children’s Hospital Los Angeles, Los Angeles, California, USA
| | - Eugene Laksana
- Department of Anesthesiology and Critical Care Medicine, Children’s Hospital Los Angeles, Los Angeles, California, USA
- Laura P. and Leland K. Whittier Virtual Pediatric Intensive Care Unit, Children’s Hospital Los Angeles, Los Angeles, California, USA
| | - David R Ledbetter
- Advanced Analytics for Healthcare, KPMG International Limited, Dallas, Texas, USA
| | - Randall C Wetzel
- Department of Anesthesiology and Critical Care Medicine, Children’s Hospital Los Angeles, Los Angeles, California, USA
- Laura P. and Leland K. Whittier Virtual Pediatric Intensive Care Unit, Children’s Hospital Los Angeles, Los Angeles, California, USA
- Department of Pediatrics and Anesthesiology, University of Southern California Keck School of Medicine, Los Angeles, California, USA
| |
Collapse
|
14
|
Oh SS, Kuang I, Jeong H, Song JY, Ren B, Moon JY, Park EC, Kawachi I. Predicting Fetal Alcohol Spectrum Disorders Using Machine Learning Techniques: Multisite Retrospective Cohort Study. J Med Internet Res 2023; 25:e45041. [PMID: 37463016 PMCID: PMC10394506 DOI: 10.2196/45041] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/13/2022] [Revised: 05/22/2023] [Accepted: 06/18/2023] [Indexed: 07/21/2023] Open
Abstract
BACKGROUND Fetal alcohol syndrome (FAS) is a lifelong developmental disability that occurs among individuals with prenatal alcohol exposure (PAE). With improved prediction models, FAS can be diagnosed or treated early, if not completely prevented. OBJECTIVE In this study, we sought to compare different machine learning algorithms and their FAS predictive performance among women who consumed alcohol during pregnancy. We also aimed to identify which variables (eg, timing of exposure to alcohol during pregnancy and type of alcohol consumed) were most influential in generating an accurate model. METHODS Data from the collaborative initiative on fetal alcohol spectrum disorders from 2007 to 2017 were used to gather information about 595 women who consumed alcohol during pregnancy at 5 hospital sites around the United States. To obtain information about PAE, questionnaires or in-person interviews, as well as reviews of medical, legal, or social service records were used to gather information about alcohol consumption. Four different machine learning algorithms (logistic regression, XGBoost, light gradient-boosting machine, and CatBoost) were trained to predict the prevalence of FAS at birth, and model performance was measured by analyzing the area under the receiver operating characteristics curve (AUROC). Of the total cases, 80% were randomly selected for training, while 20% remained as test data sets for predicting FAS. Feature importance was also analyzed using Shapley values for the best-performing algorithm. RESULTS Overall, there were 20 cases of FAS within a total population of 595 individuals with PAE. Most of the drinking occurred in the first trimester only (n=491) or throughout all 3 trimesters (n=95); however, there were also reports of drinking in the first and second trimesters only (n=8), and 1 case of drinking in the third trimester only (n=1). The CatBoost method delivered the best performance in terms of AUROC (0.92) and area under the precision-recall curve (AUPRC 0.51), followed by the logistic regression method (AUROC 0.90; AUPRC 0.59), the light gradient-boosting machine (AUROC 0.89; AUPRC 0.52), and XGBoost (AUROC 0.86; AURPC 0.45). Shapley values in the CatBoost model revealed that 12 variables were considered important in FAS prediction, with drinking throughout all 3 trimesters of pregnancy, maternal age, race, and type of alcoholic beverage consumed (eg, beer, wine, or liquor) scoring highly in overall feature importance. For most predictive measures, the best performance was obtained by the CatBoost algorithm, with an AUROC of 0.92, precision of 0.50, specificity of 0.29, F1 score of 0.29, and accuracy of 0.96. CONCLUSIONS Machine learning algorithms were able to identify FAS risk with a prediction performance higher than that of previous models among pregnant drinkers. For small training sets, which are common with FAS, boosting mechanisms like CatBoost may help alleviate certain problems associated with data imbalances and difficulties in optimization or generalization.
Collapse
Affiliation(s)
- Sarah Soyeon Oh
- Department of Social and Behavioral Sciences, Harvard TH Chan School of Public Health, Boston, MA, United States
- Institute of Health Services Research, Yonsei University College of Medicine, Seoul, Republic of Korea
| | - Irene Kuang
- Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Cambridge, MA, United States
| | - Hyewon Jeong
- Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Cambridge, MA, United States
| | - Jin-Yeop Song
- Department of Physics, Massachusetts Institute of Technology, Cambridge, MA, United States
| | - Boyu Ren
- Department of Psychiatry, Harvard Medical School, Boston, MA, United States
| | - Jong Youn Moon
- Artificial Intelligence and Big-Data Convergence Center, Gil Medical Center, Gachon University College of Medicine, Incheon, Republic of Korea
| | - Eun-Cheol Park
- Institute of Health Services Research, Yonsei University College of Medicine, Seoul, Republic of Korea
| | - Ichiro Kawachi
- Department of Social and Behavioral Sciences, Harvard TH Chan School of Public Health, Boston, MA, United States
| |
Collapse
|
15
|
Tilwani D, Bradshaw J, Sheth A, O'Reilly C. ECG Recordings as Predictors of Very Early Autism Likelihood: A Machine Learning Approach. Bioengineering (Basel) 2023; 10:827. [PMID: 37508854 PMCID: PMC10376813 DOI: 10.3390/bioengineering10070827] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/07/2023] [Revised: 06/22/2023] [Accepted: 07/05/2023] [Indexed: 07/30/2023] Open
Abstract
In recent years, there has been a rise in the prevalence of autism spectrum disorder (ASD). The diagnosis of ASD requires behavioral observation and standardized testing completed by highly trained experts. Early intervention for ASD can begin as early as 1-2 years of age, but ASD diagnoses are not typically made until ages 2-5 years, thus delaying the start of intervention. There is an urgent need for non-invasive biomarkers to detect ASD in infancy. While previous research using physiological recordings has focused on brain-based biomarkers of ASD, this study investigated the potential of electrocardiogram (ECG) recordings as an ASD biomarker in 3-6-month-old infants. We recorded the heart activity of infants at typical and elevated familial likelihood for ASD during naturalistic interactions with objects and caregivers. After obtaining the ECG signals, features such as heart rate variability (HRV) and sympathetic and parasympathetic activities were extracted. Then we evaluated the effectiveness of multiple machine learning classifiers for classifying ASD likelihood. Our findings support our hypothesis that infant ECG signals contain important information about ASD familial likelihood. Amongthe various machine learning algorithms tested, KNN performed best according to sensitivity (0.70 ± 0.117), F1-score (0.689 ± 0.124), precision (0.717 ± 0.128), accuracy (0.70 ± 0.117, p-value = 0.02), and ROC (0.686 ± 0.122, p-value = 0.06). These results suggest that ECG signals contain relevant information about the likelihood of an infant developing ASD. Future studies should consider the potential of information contained in ECG, and other indices of autonomic control, for the development of biomarkers of ASD in infancy.
Collapse
Affiliation(s)
- Deepa Tilwani
- Artificial Intelligence Institute, University of South Carolina, Columbia, SC 29208, USA
- Department of Computer Science and Engineering, University of South Carolina, Columbia, SC 29208, USA
- Carolina Autism and Neurodevelopment Research Center, University of South Carolina, Columbia, SC 29208, USA
- Institute for Mind and Brain, University of South Carolina, Columbia, SC 29208, USA
| | - Jessica Bradshaw
- Carolina Autism and Neurodevelopment Research Center, University of South Carolina, Columbia, SC 29208, USA
- Institute for Mind and Brain, University of South Carolina, Columbia, SC 29208, USA
- Department of Psychology, University of South Carolina, Columbia, SC 29208, USA
| | - Amit Sheth
- Artificial Intelligence Institute, University of South Carolina, Columbia, SC 29208, USA
- Department of Computer Science and Engineering, University of South Carolina, Columbia, SC 29208, USA
| | - Christian O'Reilly
- Artificial Intelligence Institute, University of South Carolina, Columbia, SC 29208, USA
- Department of Computer Science and Engineering, University of South Carolina, Columbia, SC 29208, USA
- Carolina Autism and Neurodevelopment Research Center, University of South Carolina, Columbia, SC 29208, USA
- Institute for Mind and Brain, University of South Carolina, Columbia, SC 29208, USA
| |
Collapse
|
16
|
Kaba Ş, Haci H, Isin A, Ilhan A, Conkbayir C. The Application of Deep Learning for the Segmentation and Classification of Coronary Arteries. Diagnostics (Basel) 2023; 13:2274. [PMID: 37443668 DOI: 10.3390/diagnostics13132274] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/16/2023] [Revised: 06/30/2023] [Accepted: 07/03/2023] [Indexed: 07/15/2023] Open
Abstract
In recent years, the prevalence of coronary artery disease (CAD) has become one of the leading causes of death around the world. Accurate stenosis detection of coronary arteries is crucial for timely treatment. Cardiologists use visual estimations when reading coronary angiography images to diagnose stenosis. As a result, they face various challenges which include high workloads, long processing times and human error. Computer-aided segmentation and classification of coronary arteries, as to whether stenosis is present or not, significantly reduces the workload of cardiologists and human errors caused by manual processes. Moreover, deep learning techniques have been shown to aid medical experts in diagnosing diseases using biomedical imaging. Thus, this study proposes the use of automatic segmentation of coronary arteries using U-Net, ResUNet-a, UNet++, models and classification using DenseNet201, EfficientNet-B0, Mobilenet-v2, ResNet101 and Xception models. In the case of segmentation, the comparative analysis of the three models has shown that U-Net achieved the highest score with a 0.8467 Dice score and 0.7454 Jaccard Index in comparison with UNet++ and ResUnet-a. Evaluation of the classification model's performances has shown that DenseNet201 performed better than other pretrained models with 0.9000 accuracy, 0.9833 specificity, 0.9556 PPV, 0.7746 Cohen's Kappa and 0.9694 Area Under the Curve (AUC).
Collapse
Affiliation(s)
- Şerife Kaba
- Department of Biomedical Engineering, Near East University, TRNC Mersin 10, Nicosia 99138, Turkey
| | - Huseyin Haci
- Department of Electrical-Electronic Engineering, Near East University, TRNC Mersin 10, Nicosia 99138, Turkey
| | - Ali Isin
- Department of Biomedical Engineering, Cyprus International University, TRNC Mersin 10, Nicosia 99138, Turkey
| | - Ahmet Ilhan
- Department of Computer Engineering, Near East University, TRNC Mersin 10, Nicosia 99138, Turkey
| | - Cenk Conkbayir
- Department of Cardiology, Near East University, TRNC Mersin 10, Nicosia 99138, Turkey
| |
Collapse
|
17
|
Antikainen E, Linnosmaa J, Umer A, Oksala N, Eskola M, van Gils M, Hernesniemi J, Gabbouj M. Transformers for cardiac patient mortality risk prediction from heterogeneous electronic health records. Sci Rep 2023; 13:3517. [PMID: 36864069 PMCID: PMC9978282 DOI: 10.1038/s41598-023-30657-1] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/07/2022] [Accepted: 02/27/2023] [Indexed: 03/04/2023] Open
Abstract
With over 17 million annual deaths, cardiovascular diseases (CVDs) dominate the cause of death statistics. CVDs can deteriorate the quality of life drastically and even cause sudden death, all the while inducing massive healthcare costs. This work studied state-of-the-art deep learning techniques to predict increased risk of death in CVD patients, building on the electronic health records (EHR) of over 23,000 cardiac patients. Taking into account the usefulness of the prediction for chronic disease patients, a prediction period of six months was selected. Two major transformer models that rely on learning bidirectional dependencies in sequential data, BERT and XLNet, were trained and compared. To our knowledge, the presented work is the first to apply XLNet on EHR data to predict mortality. The patient histories were formulated as time series consisting of varying types of clinical events, thus enabling the model to learn increasingly complex temporal dependencies. BERT and XLNet achieved an average area under the receiver operating characteristic curve (AUC) of 75.5% and 76.0%, respectively. XLNet surpassed BERT in recall by 9.8%, suggesting that it captures more positive cases than BERT, which is the main focus of recent research on EHRs and transformers.
Collapse
Affiliation(s)
- Emmi Antikainen
- VTT Technical Research Centre of Finland Ltd., 33101, Tampere, Finland.
| | - Joonas Linnosmaa
- grid.6324.30000 0004 0400 1852VTT Technical Research Centre of Finland Ltd., 33101 Tampere, Finland
| | - Adil Umer
- grid.6324.30000 0004 0400 1852VTT Technical Research Centre of Finland Ltd., 33101 Tampere, Finland
| | - Niku Oksala
- grid.502801.e0000 0001 2314 6254Faculty of Medicine and Health Technology, Tampere University, 33720 Tampere, Finland ,Finnish Cardiovascular Research Center Tampere, 33520 Tampere, Finland ,grid.412330.70000 0004 0628 2985Vascular Centre, Tampere University Hospital, 33520 Tampere, Finland
| | - Markku Eskola
- Finnish Cardiovascular Research Center Tampere, 33520 Tampere, Finland ,grid.412330.70000 0004 0628 2985Tays Heart Hospital, Tampere University Hospital, 33521 Tampere, Finland
| | - Mark van Gils
- grid.502801.e0000 0001 2314 6254Faculty of Medicine and Health Technology, Tampere University, 33720 Tampere, Finland
| | - Jussi Hernesniemi
- grid.502801.e0000 0001 2314 6254Faculty of Medicine and Health Technology, Tampere University, 33720 Tampere, Finland ,Finnish Cardiovascular Research Center Tampere, 33520 Tampere, Finland ,grid.412330.70000 0004 0628 2985Tays Heart Hospital, Tampere University Hospital, 33521 Tampere, Finland
| | - Moncef Gabbouj
- grid.502801.e0000 0001 2314 6254Faculty of Information Technology and Communication Sciences, Tampere University, 33720 Tampere, Finland
| |
Collapse
|
18
|
Heiskanen M, Das Gupta S, Mills JD, van Vliet EA, Manninen E, Ciszek R, Andrade P, Puhakka N, Aronica E, Pitkänen A. Discovery and Validation of Circulating microRNAs as Biomarkers for Epileptogenesis after Experimental Traumatic Brain Injury-The EPITARGET Cohort. Int J Mol Sci 2023; 24:ijms24032823. [PMID: 36769143 PMCID: PMC9918096 DOI: 10.3390/ijms24032823] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/27/2022] [Revised: 01/18/2023] [Accepted: 01/26/2023] [Indexed: 02/05/2023] Open
Abstract
Traumatic brain injury (TBI) causes 10-20% of structural epilepsies and 5% of all epilepsies. The lack of prognostic biomarkers for post-traumatic epilepsy (PTE) is a major obstacle to the development of anti-epileptogenic treatments. Previous studies revealed TBI-induced alterations in blood microRNA (miRNA) levels, and patients with epilepsy exhibit dysregulation of blood miRNAs. We hypothesized that acutely altered plasma miRNAs could serve as prognostic biomarkers for brain damage severity and the development of PTE. To investigate this, epileptogenesis was induced in adult male Sprague Dawley rats by lateral fluid-percussion-induced TBI. Epilepsy was defined as the occurrence of at least one unprovoked seizure during continuous 1-month video-electroencephalography monitoring in the sixth post-TBI month. Cortical pathology was analyzed by magnetic resonance imaging on day 2 (D2), D7, and D21, and by histology 6 months post-TBI. Small RNA sequencing was performed from tail-vein plasma samples on D2 and D9 after TBI (n = 16, 7 with and 9 without epilepsy) or sham operation (n = 4). The most promising miRNA biomarker candidates were validated by droplet digital polymerase chain reaction in a validation cohort of 115 rats (8 naïve, 17 sham, and 90 TBI rats [21 with epilepsy]). These included 7 brain-enriched plasma miRNAs (miR-434-3p, miR-9a-3p, miR-136-3p, miR-323-3p, miR-124-3p, miR-212-3p, and miR-132-3p) that were upregulated on D2 post-TBI (p < 0.001 for all compared with naïve rats). The acute post-TBI plasma miRNA profile did not predict the subsequent development of PTE or PTE severity. Plasma miRNA levels, however, predicted the cortical pathology severity on D2 (Spearman ρ = 0.345-0.582, p < 0.001), D9 (ρ = 0.287-0.522, p < 0.001-0.01), D21 (ρ = 0.269-0.581, p < 0.001-0.05) and at 6 months post-TBI (ρ = 0.230-0.433, p < 0.001-0.05). We found that the levels of 6 of 7 miRNAs also reflected mild brain injury caused by the craniotomy during sham operation (ROC AUC 0.76-0.96, p < 0.001-0.05). In conclusion, our findings revealed that increased levels of neuronally enriched miRNAs in the blood circulation after TBI reflect the extent of cortical injury in the brain but do not predict PTE development.
Collapse
Affiliation(s)
- Mette Heiskanen
- A.I. Virtanen Institute for Molecular Sciences, University of Eastern Finland, 70211 Kuopio, Finland
| | - Shalini Das Gupta
- A.I. Virtanen Institute for Molecular Sciences, University of Eastern Finland, 70211 Kuopio, Finland
| | - James D. Mills
- Department of (Neuro)Pathology, Amsterdam UMC, University of Amsterdam, 1105 AZ Amsterdam, The Netherlands
- Department of Clinical and Experimental Epilepsy, UCL Queen Square Institute of Neurology, London WC1N 3BG, UK
- Chalfont Centre for Epilepsy, Buckinghamshire SL9 0RJ, UK
| | - Erwin A. van Vliet
- Department of (Neuro)Pathology, Amsterdam UMC, University of Amsterdam, 1105 AZ Amsterdam, The Netherlands
- Swammerdam Institute for Life Sciences, Center for Neuroscience, University of Amsterdam, 1098 XH Amsterdam, The Netherlands
| | - Eppu Manninen
- A.I. Virtanen Institute for Molecular Sciences, University of Eastern Finland, 70211 Kuopio, Finland
| | - Robert Ciszek
- A.I. Virtanen Institute for Molecular Sciences, University of Eastern Finland, 70211 Kuopio, Finland
| | - Pedro Andrade
- A.I. Virtanen Institute for Molecular Sciences, University of Eastern Finland, 70211 Kuopio, Finland
| | - Noora Puhakka
- A.I. Virtanen Institute for Molecular Sciences, University of Eastern Finland, 70211 Kuopio, Finland
| | - Eleonora Aronica
- Department of (Neuro)Pathology, Amsterdam UMC, University of Amsterdam, 1105 AZ Amsterdam, The Netherlands
- Stichting Epilepsie Instellingen Nederland, 2103 SW Heemstede, The Netherlands
| | - Asla Pitkänen
- A.I. Virtanen Institute for Molecular Sciences, University of Eastern Finland, 70211 Kuopio, Finland
- Correspondence:
| |
Collapse
|
19
|
Gomes SRBS, von Schantz M, Leocadio-Miguel M. Predicting depressive symptoms in middle-aged and elderly adults using sleep data and clinical health markers: A machine learning approach. Sleep Med 2023; 102:123-131. [PMID: 36641929 DOI: 10.1016/j.sleep.2023.01.002] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/25/2022] [Revised: 12/13/2022] [Accepted: 01/02/2023] [Indexed: 01/05/2023]
Abstract
OBJECTIVES Comorbid depression is a highly prevalent and debilitating condition in middle-aged and elderly adults, particularly when associated with obesity, diabetes, and sleep disturbances. In this context, there is a growing need to develop efficient screening methods for cases based on clinical health markers for these comorbidities and sleep data. Thus, our objective was to detect depressive symptoms in these subjects, considering general biomarkers of obesity and diabetes and variables related to sleep and physical exercise through a machine learning approach. METHODS We used the National Health and Nutrition Examination Survey (NHANES) 2015-2016 data. Eighteen variables on self-reported physical activity, self-reported sleep habits, sleep disturbance indicative, anthropometric measurements, sociodemographic characteristics and plasma biomarkers of obesity and diabetes were selected as predictors. A total of 2907 middle-aged and elderly subjects were eligible for the study. Supervised learning algorithms such as Lasso penalized Logistic Regression (LR), Random Forest (RF) and Extreme Gradient Boosting (XGBoost) were implemented. RESULTS XGBoost provided greater accuracy and precision (87%), with a proportion of hits in cases with depressive symptoms above 80%. In addition, daytime sleepiness was the most significant predictor variable for predicting depressive symptoms. CONCLUSIONS Sleep and physical activity variables, in addition to obesity and diabetes biomarkers, together assume significant importance to predict, with accuracy and precision of 87%, the occurrence of depressive symptoms in middle-aged and elderly individuals.
Collapse
Affiliation(s)
| | | | - Mario Leocadio-Miguel
- Department of Physiology and Behavior, Federal University of Rio Grande Do Norte, Natal, Rio Grande do Norte, Brazil.
| |
Collapse
|
20
|
Atek S, Bianchini F, De Vito C, Cardinale V, Novelli S, Pesaresi C, Eugeni M, Mecella M, Rescio A, Petronzio L, Vincenzi A, Pistillo P, Giusto G, Pasquali G, Alvaro D, Villari P, Mancini M, Gaudenzi P. A predictive decision support system for coronavirus disease 2019 response management and medical logistic planning. Digit Health 2023; 9:20552076231185475. [PMID: 37545633 PMCID: PMC10399258 DOI: 10.1177/20552076231185475] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/21/2022] [Accepted: 06/14/2023] [Indexed: 08/08/2023] Open
Abstract
Objective Coronavirus disease 2019 demonstrated the inconsistencies in adequately responding to biological threats on a global scale due to a lack of powerful tools for assessing various factors in the formation of the epidemic situation and its forecasting. Decision support systems have a role in overcoming the challenges in health monitoring systems in light of current or future epidemic outbreaks. This paper focuses on some applied examples of logistic planning, a key service of the Earth Cognitive System for Coronavirus Disease 2019 project, here presented, evidencing the added value of artificial intelligence algorithms towards predictive hypotheses in tackling health emergencies. Methods Earth Cognitive System for Coronavirus Disease 2019 is a decision support system designed to support healthcare institutions in monitoring, management and forecasting activities through artificial intelligence, social media analytics, geospatial analysis and satellite imaging. The monitoring, management and prediction of medical equipment logistic needs rely on machine learning to predict the regional risk classification colour codes, the emergency rooms attendances, and the forecast of regional medical supplies, synergically enhancing geospatial and temporal dimensions. Results The overall performance of the regional risk colour code classifier yielded a high value of the macro-average F1-score (0.82) and an accuracy of 85%. The prediction of the emergency rooms attendances for the Lazio region yielded a very low root mean square error (<11 patients) and a high positive correlation with the actual values for the major hospitals of the Lazio region which admit about 90% of the region's patients. The prediction of the medicinal purchases for the regions of Lazio and Piemonte has yielded a low root mean squared percentage error of 16%. Conclusions Accurate forecasting of the evolution of new cases and drug utilisation enables the resulting excess demand throughout the supply chain to be managed more effectively. Forecasting during a pandemic becomes essential for effective government decision-making, managing supply chain resources, and for informing tough policy decisions.
Collapse
Affiliation(s)
- Sofiane Atek
- Department of Aerospace and Mechanical Engineering, Sapienza University of Rome, Rome, Italy
| | | | - Corrado De Vito
- Department of Public Health and Infectious Diseases, Sapienza University of Rome, Rome, Italy
| | - Vincenzo Cardinale
- Department of Medico-Surgical Sciences and Biotechnologies, Sapienza University of Rome, Umberto I Policlinico of Rome, Rome, Italy
| | - Simone Novelli
- Department of Aerospace and Mechanical Engineering, Sapienza University of Rome, Rome, Italy
| | - Cristiano Pesaresi
- Department of Letters and Modern Cultures, Sapienza University of Rome, Rome, Italy
| | - Marco Eugeni
- Department of Aerospace and Mechanical Engineering, Sapienza University of Rome, Rome, Italy
| | - Massimo Mecella
- Department of Computer, Control, and Management Engineering Antonio Ruberti, Sapienza University of Rome, Rome, Italy
| | | | | | | | | | | | | | - Domenico Alvaro
- Sapienza Information-Based Technology InnovaTion Center for Health (STITCH), Sapienza University of Rome, Rome, Italy
| | - Paolo Villari
- Department of Public Health and Infectious Diseases, Sapienza University of Rome, Rome, Italy
| | - Marco Mancini
- Department of Letters and Modern Cultures, Sapienza University of Rome, Rome, Italy
| | - Paolo Gaudenzi
- Department of Aerospace and Mechanical Engineering, Sapienza University of Rome, Rome, Italy
| |
Collapse
|
21
|
Jamshidi E, Asgary A, Kharrazi AY, Tavakoli N, Zali A, Mehrazi M, Jamshidi M, Farrokhi B, Maher A, von Garnier C, Rahi SJ, Mansouri N. Personalized predictions of adverse side effects of the COVID-19 vaccines. Heliyon 2023; 9:e12753. [PMID: 36597482 PMCID: PMC9800018 DOI: 10.1016/j.heliyon.2022.e12753] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/01/2021] [Revised: 12/27/2022] [Accepted: 12/27/2022] [Indexed: 12/31/2022] Open
Abstract
Background Misconceptions about adverse side effects are thought to influence public acceptance of the Coronavirus disease 2019 (COVID-19) vaccines negatively. To address such perceived disadvantages of vaccines, a novel machine learning (ML) approach was designed to generate personalized predictions of the most common adverse side effects following injection of six different COVID-19 vaccines based on personal and health-related characteristics. Methods Prospective data of adverse side effects following COVID-19 vaccination in 19943 participants from Iran and Switzerland was utilized. Six vaccines were studied: The AZD1222, Sputnik V, BBIBP-CorV, COVAXIN, BNT162b2, and the mRNA-1273 vaccine. The eight side effects were considered as the model output: fever, fatigue, headache, nausea, chills, joint pain, muscle pain, and injection site reactions. The total input parameters for the first and second dose predictions were 46 and 54 features, respectively, including age, gender, lifestyle variables, and medical history. The performances of multiple ML models were compared using Area Under the Receiver Operating Characteristic Curve (ROC-AUC). Results The total number of people receiving the first dose of the AZD1222, Sputnik V, BBIBP-CorV, COVAXIN, BNT162b2, and mRNA-1273 were 6022, 7290, 5279, 802, 277, and 273, respectively. For the second dose, the numbers were 2851, 5587, 3841, 599, 242 and 228. The Logistic Regression model for predicting different side effects of the first dose achieved ROC-AUCs of 0.620-0.686, 0.685-0.716, 0.632-0.727, 0.527-0.598, 0.548-0.655, 0.545-0.712 for the AZD1222, Sputnik V, BBIBP-CorV, COVAXIN, BNT162b2 and mRNA-1273 vaccines, respectively. The second dose models yielded ROC-AUCs of 0.777-0.867, 0.795-0.848, 0.857-0.906, 0.788-0.875, 0.683-0.850, and 0.486-0.680, respectively. Conclusions Using a large cohort of recipients vaccinated with COVID-19 vaccines, a novel and personalized strategy was established to predict the occurrence of the most common adverse side effects with high accuracy. This technique can serve as a tool to inform COVID-19 vaccine selection and generate personalized factsheets to curb concerns about adverse side effects.
Collapse
Affiliation(s)
- Elham Jamshidi
- Functional Neurosurgery Research Center, Shohada Tajrish Comprehensive Neurosurgical Center of Excellence, Shahid Beheshti University of Medical Sciences, Tehran, Iran
| | - Amirhossein Asgary
- Department of Biotechnology, College of Sciences, University of Tehran, Tehran, Iran
| | | | - Nader Tavakoli
- Trauma and Injury Research Center, Iran University of Medical Sciences, Tehran, Iran
| | - Alireza Zali
- Functional Neurosurgery Research Center, Shohada Tajrish Comprehensive Neurosurgical Center of Excellence, Shahid Beheshti University of Medical Sciences, Tehran, Iran
| | - Maryam Mehrazi
- Trauma and Injury Research Center, Iran University of Medical Sciences, Tehran, Iran
| | - Masoud Jamshidi
- Department of Exercise Physiology, Tehran University, Tehran, Iran
| | - Babak Farrokhi
- Health Network Administration Center, Undersecretary for Health Affairs, Ministry of Health and Medical Education, Tehran, Iran
| | - Ali Maher
- School of Management and Medical Education, Shahid Beheshti University of Medical Sciences, Tehran, Iran
| | - Christophe von Garnier
- Division of Pulmonary Medicine, Department of Medicine, Lausanne University Hospital (CHUV), University of Lausanne (UNIL), Lausanne, Switzerland
| | - Sahand Jamal Rahi
- Laboratory of the Physics of Biological Systems, Institute of Physics, École polytechnique fédérale de Lausanne (EPFL), Lausanne, Switzerland
| | - Nahal Mansouri
- Division of Pulmonary Medicine, Department of Medicine, Lausanne University Hospital (CHUV), University of Lausanne (UNIL), Lausanne, Switzerland
- Swiss Institute for Experimental Cancer Research (ISREC), School of Life Sciences, École polytechnique fédérale de Lausanne (EPFL), Lausanne, Switzerland
- Research Group on Artificial Intelligence in Pulmonary Medicine, Division of Pulmonary Medicine, Lausanne University Hospital (CHUV), Lausanne, Switzerland
| |
Collapse
|
22
|
Joslyn S, Alexander K. Evaluating artificial intelligence algorithms for use in veterinary radiology. Vet Radiol Ultrasound 2022; 63 Suppl 1:871-879. [PMID: 36514228 DOI: 10.1111/vru.13159] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/23/2021] [Revised: 02/16/2022] [Accepted: 03/30/2022] [Indexed: 12/15/2022] Open
Abstract
Artificial intelligence is increasingly being used for applications in veterinary radiology, including detection of abnormalities and automated measurements. Unlike human radiology, there is no formal regulation or validation of AI algorithms for veterinary medicine and both general practitioner and specialist veterinarians must rely on their own judgment when deciding whether or not to incorporate AI algorithms to aid their clinical decision-making. The benefits and challenges to developing clinically useful and diagnostically accurate AI algorithms are discussed. Considerations for the development of AI research projects are also addressed. A framework is suggested to help veterinarians, in both research and clinical practice contexts, assess AI algorithms for veterinary radiology.
Collapse
Affiliation(s)
- Steve Joslyn
- ACVR/ECVDI AI Education and Development Committee, Vedi, Perth, Western Australia, Australia
| | - Kate Alexander
- ACVR/ECVDI AI Education and Development Committee, DMV Veterinary Center, Lachine, Quebec, Canada
| |
Collapse
|
23
|
Eye movement behavior in a real-world virtual reality task reveals ADHD in children. Sci Rep 2022; 12:20308. [PMID: 36434040 PMCID: PMC9700686 DOI: 10.1038/s41598-022-24552-4] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/12/2022] [Accepted: 11/16/2022] [Indexed: 11/26/2022] Open
Abstract
Eye movements and other rich data obtained in virtual reality (VR) environments resembling situations where symptoms are manifested could help in the objective detection of various symptoms in clinical conditions. In the present study, 37 children with attention deficit hyperactivity disorder and 36 typically developing controls (9-13 y.o) played a lifelike prospective memory game using head-mounted display with inbuilt 90 Hz eye tracker. Eye movement patterns had prominent group differences, but they were dispersed across the full performance time rather than associated with specific events or stimulus features. A support vector machine classifier trained on eye movement data showed excellent discrimination ability with 0.92 area under curve, which was significantly higher than for task performance measures or for eye movements obtained in a visual search task. We demonstrated that a naturalistic VR task combined with eye tracking allows accurate prediction of attention deficits, paving the way for precision diagnostics.
Collapse
|
24
|
Ghorashi SM, Fazeli A, Hedayat B, Mokhtari H, Jalali A, Ahmadi P, Chalian H, Bragazzi NL, Shirani S, Omidi N. Comparison of conventional scoring systems to machine learning models for the prediction of major adverse cardiovascular events in patients undergoing coronary computed tomography angiography. Front Cardiovasc Med 2022; 9:994483. [PMID: 36386332 PMCID: PMC9643500 DOI: 10.3389/fcvm.2022.994483] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/14/2022] [Accepted: 10/05/2022] [Indexed: 08/04/2023] Open
Abstract
BACKGROUND The study aims to compare the prognostic performance of conventional scoring systems to a machine learning (ML) model on coronary computed tomography angiography (CCTA) to discriminate between the patients with and without major adverse cardiovascular events (MACEs) and to find the most important contributing factor of MACE. MATERIALS AND METHODS From November to December 2019, 500 of 1586 CCTA scans were included and analyzed, then six conventional scores were calculated for each participant, and seven ML models were designed. Our study endpoints were all-cause mortality, non-fatal myocardial infarction, late coronary revascularization, and hospitalization for unstable angina or heart failure. Score performance was assessed by area under the curve (AUC) analysis. RESULTS Of 500 patients (mean age: 60 ± 10; 53.8% male subjects) referred for CCTA, 416 patients have met inclusion criteria, 46 patients with early (<90 days) cardiac evaluation (due to the inability to clarify the reason for the assessment, deterioration of the symptoms vs. the CCTA result), and 38 patients because of missed follow-up were not enrolled in the final analysis. Forty-six patients (11.0%) developed MACE within 20.5 ± 7.9 months of follow-up. Compared to conventional scores, ML models showed better performance, except only one model which is eXtreme Gradient Boosting had lower performance than conventional scoring systems (AUC:0.824, 95% confidence interval (CI): 0.701-0.947). Between ML models, random forest, ensemble with generalized linear, and ensemble with naive Bayes were shown to have higher prognostic performance (AUC: 0.92, 95% CI: 0.85-0.99, AUC: 0.90, 95% CI: 0.81-0.98, and AUC: 0.89, 95% CI: 0.82-0.97), respectively. Coronary artery calcium score (CACS) had the highest correlation with MACE. CONCLUSION Compared to the conventional scoring system, ML models using CCTA scans show improved prognostic prediction for MACE. Anatomical features were more important than clinical characteristics.
Collapse
Affiliation(s)
| | - Amir Fazeli
- Tehran Heart Center, Tehran University of Medical Science, Tehran, Iran
| | - Behnam Hedayat
- Tehran Heart Center, Tehran University of Medical Science, Tehran, Iran
| | - Hamid Mokhtari
- Biomedical Engineering and Physics Department, School of Medicine, Shahid Beheshti University of Medical Sciences, Tehran, Iran
| | - Arash Jalali
- Tehran Heart Center, Tehran University of Medical Science, Tehran, Iran
| | - Pooria Ahmadi
- Tehran Heart Center, Tehran University of Medical Science, Tehran, Iran
| | - Hamid Chalian
- Division of Cardiothoracic Imaging, Department of Radiology, University of Washington, Seattle, WA, United States
| | - Nicola Luigi Bragazzi
- Laboratory for Industrial and Applied Mathematics (LIAM), Department of Mathematics and Statistics, York University, Toronto, ON, Canada
| | - Shapour Shirani
- Department of Cardiovascular Imaging, Tehran Heart Center, Tehran University of Medical Sciences, Tehran, Iran
| | - Negar Omidi
- Department of Cardiovascular Imaging, Tehran Heart Center, Tehran University of Medical Sciences, Tehran, Iran
| |
Collapse
|
25
|
Zhou B, Yu J, Cai X, Wu S. Constructing a molecular subtype model of colon cancer using machine learning. Front Pharmacol 2022; 13:1008207. [PMID: 36188575 PMCID: PMC9523145 DOI: 10.3389/fphar.2022.1008207] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/31/2022] [Accepted: 08/26/2022] [Indexed: 11/21/2022] Open
Abstract
Background: Colon cancer (CRC) is one of the malignant tumors with a high incidence in the world. Many previous studies on CRC have focused on clinical research. With the in-depth study of CRC, the role of molecular mechanisms in CRC has become increasingly important. Currently, machine learning is widely used in medicine. By combining machine learning with molecular mechanisms, we can better understand CRC’s pathogenesis and develop new treatments for it. Methods and materials: We used the R language to construct molecular subtypes of colon cancer and subsequently explored prognostic genes with GEPIA2. Enrichment analysis is used by WebGestalt to obtain differential genes. Protein–protein interaction networks of differential genes were constructed using the STRING database and the Cytoscape tool. TIMER2.0 and TISIDB databases were used to investigate the correlation of these genes with immune-infiltrating cells and immune targets. The cBioportal database was used to explore genomic alterations. Results: In our study, the molecular prognostic model of CRC was constructed to study the prognostic factors of CRC, and finally, it was found that Charcot–Leyden crystal galectin (CLC), zymogen granule protein 16 (ZG16), leucine-rich repeat-containing protein 26 (LRRC26), intelectin 1 (ITLN1), UDP-GlcNAc: betaGal beta-1,3-N-acetylglucosaminyltransferase 6 (B3GNT6), chloride channel accessory 1 (CLCA1), growth factor independent 1 transcriptional repressor (GFI1), aquaporin 8 (AQP8), HEPACAM family member 2 (HEPACAM2), and UDP glucuronosyltransferase family 2 member B15 (UGT2B15) were correlated with the subtype model of CRC prognosis. Enrichment analysis shows that differential genes were mainly associated with immune-inflammatory pathways. GFI1 and CLC were associated with immune cells, immunoinhibitors, and immunostimulator. Genomic analysis shows that there were no significant changes in differential genes. Conclusion: By constructing molecular subtypes of colon cancer, we discovered new colon cancer prognostic markers, which can provide direction for new treatments in the future.
Collapse
Affiliation(s)
- Bo Zhou
- Department of General Surgery, Ningbo Medical Center Lihuili Hospital, Ningbo University, Ningbo, China
| | - Jiazi Yu
- Department of General Surgery, Ningbo Medical Center Lihuili Hospital, Ningbo University, Ningbo, China
| | - Xingchen Cai
- Medical School, Ningbo University, Ningbo, China
| | - Shugeng Wu
- Medical School, Ningbo University, Ningbo, China
- *Correspondence: Shugeng Wu,
| |
Collapse
|
26
|
Zrubka Z, Csabai I, Hermann Z, Golicki D, Prevolnik-Rupel V, Ogorevc M, Gulácsi L, Péntek M. Predicting Patient-Level 3-Level Version of EQ-5D Index Scores From a Large International Database Using Machine Learning and Regression Methods. VALUE IN HEALTH : THE JOURNAL OF THE INTERNATIONAL SOCIETY FOR PHARMACOECONOMICS AND OUTCOMES RESEARCH 2022; 25:1590-1601. [PMID: 35300933 DOI: 10.1016/j.jval.2022.01.024] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/28/2021] [Revised: 11/30/2021] [Accepted: 01/18/2022] [Indexed: 06/14/2023]
Abstract
OBJECTIVES This study aimed to evaluate the performance of machine learning and regression methods in the prediction of 3-level version of EQ-5D (EQ-5D-3L) index scores from a large diverse data set. METHODS A total of 30 studies from 3 countries were combined. Predictions were performed via eXtreme Gradient Boosting classification (XGBC), eXtreme Gradient Boosting regression (XGBR) and ordinary least squares (OLS) regression using 10-fold cross-validation and 80%/20% partition for training and testing. We evaluated 6 prediction scenarios using 3 samples (general population, patients, total) and 2 predictor sets: demographic and disease-related variables with/without patient-reported outcomes. Model performance was evaluated by mean absolute error and percent of predictions within clinically irrelevant error range and within correct health severity group (EQ-5D-3L index <0.45, 0.45-0.926, >0.926). RESULTS The data set involved 26 318 individuals (clinical settings n = 6214, general population n = 20 104) and 26 predictor variables plus diagnoses. Using all predictors and the total sample, mean absolute error values were 0.153, 0.126, and 0.131, percent of predictions within clinically irrelevant error range were 47.6%, 39.5%, and 37.4%, and within the correct health severity group were 56.3%, 64.9%, and 63.3% by XGBC, XGBR, and OLS, respectively. The performance of models depended on the applied evaluation criteria, the target population, the included predictors, and the EQ-5D-3L index score range. CONCLUSIONS Regression models (XGBR and OLS) outperformed XGBC, yet prediction errors were outside the clinically irrelevant error range for most respondents. Our results highlight the importance of systematic patient-reported outcome (EQ-5D) data collection. Dialogs between artificial intelligence and outcomes research experts are encouraged to enhance the value of accumulating data in health systems.
Collapse
Affiliation(s)
- Zsombor Zrubka
- Health Economics Research Center, University Research and Innovation Center, Óbuda University, Budapest, Hungary; Corvinus Institue for Advanced Studies, Corvinus University of Budapest, Budapest, Hungary.
| | - István Csabai
- Department of Physics of Complex Systems, ELTE Eötvös Loránd University, Budapest, Hungary
| | - Zoltán Hermann
- Institute of Economics, Centre for Economic and Regional Studies, Budapest, Hungary; Institute of Economics, Corvinus University of Budapest, Budapest, Hungary
| | - Dominik Golicki
- Department of Experimental and Clinical Pharmacology, Medical University of Warsaw, Warsaw, Poland
| | | | - Marko Ogorevc
- Institute for Economic Research, Ljubljana, Slovenia
| | - László Gulácsi
- Health Economics Research Center, University Research and Innovation Center, Óbuda University, Budapest, Hungary; Corvinus Institue for Advanced Studies, Corvinus University of Budapest, Budapest, Hungary
| | - Márta Péntek
- Health Economics Research Center, University Research and Innovation Center, Óbuda University, Budapest, Hungary
| |
Collapse
|
27
|
Okazaki S, Mine Y, Iwamoto Y, Urabe S, Mitsuhata C, Nomura R, Kakimoto N, Murayama T. Analysis of the feasibility of using deep learning for multiclass classification of dental anomalies on panoramic radiographs. Dent Mater J 2022; 41:889-895. [PMID: 36002296 DOI: 10.4012/dmj.2022-098] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/23/2022]
Abstract
The aim of the feasibility study was to construct deep learning models for the classification of multiple dental anomalies in panoramic radiographs. Panoramic radiographs with single supernumerary teeth and/or odontomas were considered the "case" group; panoramic radiographs with no dental anomalies were considered the "control" group. The dataset comprised 150 panoramic radiographs: 50 each of no dental anomalies, single supernumerary teeth, and odontomas. To classify the panoramic radiographs into case and control categories, we employed AlexNet, which is a convolutional neural network model. AlexNet was able to classify whole panoramic radiographs into two or three classes, according to the presence or absence of supernumerary teeth or odontomas. The performance metrics of the three-class classification were 70%, 70.8%, 70%, and 69.7% for accuracy, precision, sensitivity, and F1 score, respectively, in the macro average. These results support the feasibility of using deep learning to detect multiple dental anomalies in panoramic radiographs.
Collapse
Affiliation(s)
- Shota Okazaki
- Department of Medical System Engineering, Graduate School of Biomedical and Health Sciences, Hiroshima University
| | - Yuichi Mine
- Department of Medical System Engineering, Graduate School of Biomedical and Health Sciences, Hiroshima University
| | - Yuko Iwamoto
- Department of Pediatric Dentistry, Graduate School of Biomedical and Health Sciences, Hiroshima University
| | - Shiho Urabe
- Department of Medical System Engineering, Graduate School of Biomedical and Health Sciences, Hiroshima University
| | - Chieko Mitsuhata
- Department of Pediatric Dentistry, Graduate School of Biomedical and Health Sciences, Hiroshima University
| | - Ryota Nomura
- Department of Pediatric Dentistry, Graduate School of Biomedical and Health Sciences, Hiroshima University
| | - Naoya Kakimoto
- Department of Oral and Maxillofacial Radiology, Graduate School of Biomedical and Health Sciences, Hiroshima University
| | - Takeshi Murayama
- Department of Medical System Engineering, Graduate School of Biomedical and Health Sciences, Hiroshima University
| |
Collapse
|
28
|
Sandham MH, Hedgecock EA, Siegert RJ, Narayanan A, Hocaoglu MB, Higginson IJ. Intelligent Palliative Care Based on Patient-Reported Outcome Measures. J Pain Symptom Manage 2022; 63:747-757. [PMID: 35026384 DOI: 10.1016/j.jpainsymman.2021.11.008] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/03/2021] [Revised: 11/11/2021] [Accepted: 11/16/2021] [Indexed: 12/31/2022]
Abstract
CONTEXT The growth of patient reported outcome measures data in palliative care provides an opportunity for machine learning to identify patterns in patient responses signifying different phases of illness. OBJECTIVES The study will explore if machine learning and network analysis can identify phases in patient palliative status through symptoms reported on the Integrated Palliative Care Outcome Scale (IPOS). METHODS A partly cross-sectional and partially longitudinal observational study was undertaken using the Australasian Karnofsky Performance Scale (AKPS); Integrated Palliative Care Outcome Scale (IPOS); Phase of Illness (POI). Patient palliative records (n = 1507, 65% stable, 20% unstable, 9% deteriorating, 2% terminal) from 804 adult patients enrolled in a New Zealand palliative care service were analysed using a combination of statistical, machine learning and network analysis techniques. RESULTS Data from IPOS showed considerable variation with phase. Also, network analysis showed clear associations between items by phase. Six machine learning techniques identified the most important variables for predicting possible transition between phases of illness. Network analysis for all patients showed that Poor Appetite and Loss of Energy were central IPOS items, with Loss of Energy linked to Drowsiness, Shortness of Breath and Lack of Mobility on the one hand, and Poor Appetite linked to Nausea, Vomiting, Constipation and Sore and Dry Mouth on the other. CONCLUSION These preliminary results, when coupled with the latest technological developments in mobile apps and wearable technology, could point the way to increased use of digital therapeutics in continuous palliative care monitoring.
Collapse
Affiliation(s)
- Margaret H Sandham
- School of Clinical Sciences (M.S., R.S.), Auckland University of Technology, Auckland, New Zealand.
| | - Emma A Hedgecock
- Specialty Medicine and Health of Older People, Waitemata District Health Board, Private Bag (E.A.H.), Takapuna, New Zealand
| | - Richard J Siegert
- School of Clinical Sciences (M.S., R.S.), Auckland University of Technology, Auckland, New Zealand
| | - Ajit Narayanan
- School of Engineering, Computer and Mathematical Sciences (A.N.), Auckland University of Technology, Auckland, New Zealand
| | - Mevhibe B Hocaoglu
- Cicely Saunders Institute of Palliative Care, Florence Nightingale Faculty of Nursing, Midwifery and Palliative Care (M.B.H., I.J.H.), King's College London, London, UK
| | - Irene J Higginson
- Cicely Saunders Institute of Palliative Care, Florence Nightingale Faculty of Nursing, Midwifery and Palliative Care (M.B.H., I.J.H.), King's College London, London, UK
| |
Collapse
|
29
|
Siy Van VT, Antonio VA, Siguin CP, Gordoncillo NP, Sescon JT, Go CC, Miro EP. Predicting undernutrition among elementary schoolchildren in the Philippines using machine learning algorithms. Nutrition 2022; 96:111571. [DOI: 10.1016/j.nut.2021.111571] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/26/2021] [Revised: 11/23/2021] [Accepted: 12/05/2021] [Indexed: 11/30/2022]
|
30
|
Cui F, Li S, Zhang Z, Sui M, Cao C, El-Latif Hesham A, Zou Q. DeepMC-iNABP: Deep learning for multiclass identification and classification of nucleic acid-binding proteins. Comput Struct Biotechnol J 2022; 20:2020-2028. [PMID: 35521556 PMCID: PMC9065708 DOI: 10.1016/j.csbj.2022.04.029] [Citation(s) in RCA: 11] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/08/2022] [Revised: 04/06/2022] [Accepted: 04/20/2022] [Indexed: 11/29/2022] Open
Abstract
Nucleic acid-binding proteins (NABPs), including DNA-binding proteins (DBPs) and RNA-binding proteins (RBPs), play vital roles in gene expression. Accurate identification of these proteins is crucial. However, there are two existing challenges: one is the problem of ignoring DNA- and RNA-binding proteins (DRBPs), and the other is a cross-predicting problem referring to DBP predictors predicting DBPs as RBPs, and vice versa. In this study, we proposed a computational predictor, called DeepMC-iNABP, with the goal of solving these difficulties by utilizing a multiclass classification strategy and deep learning approaches. DBPs, RBPs, DRBPs and non-NABPs as separate classes of data were used for training the DeepMC-iNABP model. The results on test data collected in this study and two independent test datasets showed that DeepMC-iNABP has a strong advantage in identifying the DRBPs and has the ability to alleviate the cross-prediction problem to a certain extent. The web-server of DeepMC-iNABP is freely available at http://www.deepmc-inabp.net/. The datasets used in this research can also be downloaded from the website.
Collapse
Affiliation(s)
- Feifei Cui
- School of Computer Science and Technology, Hainan University, Haikou 570228, China
- Institute of Fundamental and Frontier Sciences, University of Electronic Science and Technology of China, Chengdu 610054, China
- Yangtze Delta Region Institute (Quzhou), University of Electronic Science and Technology of China, Quzhou 324000, China
| | - Shuang Li
- Beidahuang Industry Group General Hospital, Harbin 150001, China
| | - Zilong Zhang
- School of Computer Science and Technology, Hainan University, Haikou 570228, China
- Institute of Fundamental and Frontier Sciences, University of Electronic Science and Technology of China, Chengdu 610054, China
- Yangtze Delta Region Institute (Quzhou), University of Electronic Science and Technology of China, Quzhou 324000, China
| | - Miaomiao Sui
- Graduate School Agricultural and Life Science, The University of Tokyo, Tokyo 1138657, Japan
| | - Chen Cao
- School of Biomedical Engineering and Informatics, Nanjing Medical University, Nanjing 211166, China
| | - Abd El-Latif Hesham
- Genetics Department, Faculty of Agriculture, Beni-Suef University, Beni-Suef 62511, Egypt
| | - Quan Zou
- Institute of Fundamental and Frontier Sciences, University of Electronic Science and Technology of China, Chengdu 610054, China
- Yangtze Delta Region Institute (Quzhou), University of Electronic Science and Technology of China, Quzhou 324000, China
- Corresponding author at: Institute of Fundamental and Frontier Sciences, University of Electronic Science and Technology of China, Chengdu 610054, China.
| |
Collapse
|
31
|
Supporting peace negotiations in the Yemen war through machine learning. DATA & POLICY 2022. [DOI: 10.1017/dap.2022.19] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/05/2022] Open
Abstract
Abstract
Today’s conflicts are becoming increasingly complex, fluid, and fragmented, often involving a host of national and international actors with multiple and often divergent interests. This development poses significant challenges for conflict mediation, as mediators struggle to make sense of conflict dynamics, such as the range of conflict parties and the evolution of their political positions, the distinction between relevant and less relevant actors in peace-making, or the identification of key conflict issues and their interdependence. International peace efforts appear ill-equipped to successfully address these challenges. While technology is already being experimented with and used in a range of conflict related fields, such as conflict predicting or information gathering, less attention has been given to how technology can contribute to conflict mediation. This case study contributes to emerging research on the use of state-of-the-art machine learning technologies and techniques in conflict mediation processes. Using dialogue transcripts from peace negotiations in Yemen, this study shows how machine-learning can effectively support mediating teams by providing them with tools for knowledge management, extraction and conflict analysis. Apart from illustrating the potential of machine learning tools in conflict mediation, the article also emphasizes the importance of interdisciplinary and participatory, cocreation methodology for the development of context-sensitive and targeted tools and to ensure meaningful and responsible implementation.
Collapse
|
32
|
Sajjadian M, Lam RW, Milev R, Rotzinger S, Frey BN, Soares CN, Parikh SV, Foster JA, Turecki G, Müller DJ, Strother SC, Farzan F, Kennedy SH, Uher R. Machine learning in the prediction of depression treatment outcomes: a systematic review and meta-analysis. Psychol Med 2021; 51:2742-2751. [PMID: 35575607 DOI: 10.1017/s0033291721003871] [Citation(s) in RCA: 41] [Impact Index Per Article: 13.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/12/2022]
Abstract
BACKGROUND Multiple treatments are effective for major depressive disorder (MDD), but the outcomes of each treatment vary broadly among individuals. Accurate prediction of outcomes is needed to help select a treatment that is likely to work for a given person. We aim to examine the performance of machine learning methods in delivering replicable predictions of treatment outcomes. METHODS Of 7732 non-duplicate records identified through literature search, we retained 59 eligible reports and extracted data on sample, treatment, predictors, machine learning method, and treatment outcome prediction. A minimum sample size of 100 and an adequate validation method were used to identify adequate-quality studies. The effects of study features on prediction accuracy were tested with mixed-effects models. Fifty-four of the studies provided accuracy estimates or other estimates that allowed calculation of balanced accuracy of predicting outcomes of treatment. RESULTS Eight adequate-quality studies reported a mean accuracy of 0.63 [95% confidence interval (CI) 0.56-0.71], which was significantly lower than a mean accuracy of 0.75 (95% CI 0.72-0.78) in the other 46 studies. Among the adequate-quality studies, accuracies were higher when predicting treatment resistance (0.69) and lower when predicting remission (0.60) or response (0.56). The choice of machine learning method, feature selection, and the ratio of features to individuals were not associated with reported accuracy. CONCLUSIONS The negative relationship between study quality and prediction accuracy, combined with a lack of independent replication, invites caution when evaluating the potential of machine learning applications for personalizing the treatment of depression.
Collapse
Affiliation(s)
- Mehri Sajjadian
- Department of Psychiatry, Dalhousie University, Halifax, NS, Canada
| | - Raymond W Lam
- Department of Psychiatry, University of British Columbia, Vancouver, BC, Canada
| | - Roumen Milev
- Department of Psychiatry and Psychology, Queen's University, Providence Care Hospital, Kingston, ON, Canada
| | - Susan Rotzinger
- Department of Psychiatry, University of Toronto, Toronto, ON, Canada
- Department of Psychiatry, St. Michael's Hospital, University of Toronto, Toronto, Ontario, Canada
| | - Benicio N Frey
- Department of Psychiatry and Behavioural Neurosciences, McMaster University, Hamilton, ON, Canada
- Mood Disorders Program and Women's Health Concerns Clinic, St. Joseph's Healthcare Hamilton, Hamilton, ON, Canada
| | - Claudio N Soares
- Department of Psychiatry, Queen's University School of Medicine, Kingston, ON, Canada
| | - Sagar V Parikh
- Department of Psychiatry, University of Michigan, Ann Arbor, MI, USA
| | - Jane A Foster
- Department of Psychiatry & Behavioural Neurosciences, St. Joseph's Healthcare, Hamilton, ON, Canada
| | - Gustavo Turecki
- Department of Psychiatry, Douglas Institute, McGill University, Montreal, QC, Canada
| | - Daniel J Müller
- Campbell Family Mental Health Research Institute, Center for Addiction and Mental Health, Toronto, ON, Canada
- Department of Psychiatry, University of Toronto, Toronto, ON, Canada
| | - Stephen C Strother
- Baycrest and Department of Medical Biophysics, Rotman Research Center, University of Toronto, Toronto, ON, Canada
| | - Faranak Farzan
- eBrain Lab, School of Mechatronic Systems Engineering, Simon Fraser University, Surrey, BC, Canada
| | - Sidney H Kennedy
- Department of Psychiatry, University of Toronto, Toronto, ON, Canada
- Department of Psychiatry, St. Michael's Hospital, University of Toronto, Toronto, Ontario, Canada
- Department of Psychiatry, University Health Network, Toronto, ON, Canada
- Krembil Research Centre, University Health Network, University of Toronto, Toronto, ON, Canada
| | - Rudolf Uher
- Department of Psychiatry, Dalhousie University, Halifax, NS, Canada
| |
Collapse
|
33
|
Coppola F, Faggioni L, Gabelloni M, De Vietro F, Mendola V, Cattabriga A, Cocozza MA, Vara G, Piccinino A, Lo Monaco S, Pastore LV, Mottola M, Malavasi S, Bevilacqua A, Neri E, Golfieri R. Human, All Too Human? An All-Around Appraisal of the "Artificial Intelligence Revolution" in Medical Imaging. Front Psychol 2021; 12:710982. [PMID: 34650476 PMCID: PMC8505993 DOI: 10.3389/fpsyg.2021.710982] [Citation(s) in RCA: 48] [Impact Index Per Article: 16.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/18/2021] [Accepted: 09/02/2021] [Indexed: 12/22/2022] Open
Abstract
Artificial intelligence (AI) has seen dramatic growth over the past decade, evolving from a niche super specialty computer application into a powerful tool which has revolutionized many areas of our professional and daily lives, and the potential of which seems to be still largely untapped. The field of medicine and medical imaging, as one of its various specialties, has gained considerable benefit from AI, including improved diagnostic accuracy and the possibility of predicting individual patient outcomes and options of more personalized treatment. It should be noted that this process can actively support the ongoing development of advanced, highly specific treatment strategies (e.g., target therapies for cancer patients) while enabling faster workflow and more efficient use of healthcare resources. The potential advantages of AI over conventional methods have made it attractive for physicians and other healthcare stakeholders, raising much interest in both the research and the industry communities. However, the fast development of AI has unveiled its potential for disrupting the work of healthcare professionals, spawning concerns among radiologists that, in the future, AI may outperform them, thus damaging their reputations or putting their jobs at risk. Furthermore, this development has raised relevant psychological, ethical, and medico-legal issues which need to be addressed for AI to be considered fully capable of patient management. The aim of this review is to provide a brief, hopefully exhaustive, overview of the state of the art of AI systems regarding medical imaging, with a special focus on how AI and the entire healthcare environment should be prepared to accomplish the goal of a more advanced human-centered world.
Collapse
Affiliation(s)
- Francesca Coppola
- Department of Radiology, IRCCS Azienda Ospedaliero Universitaria di Bologna, Bologna, Italy
- SIRM Foundation, Italian Society of Medical and Interventional Radiology, Milan, Italy
| | - Lorenzo Faggioni
- Academic Radiology, Department of Translational Research, University of Pisa, Pisa, Italy
| | - Michela Gabelloni
- Academic Radiology, Department of Translational Research, University of Pisa, Pisa, Italy
| | - Fabrizio De Vietro
- Academic Radiology, Department of Translational Research, University of Pisa, Pisa, Italy
| | - Vincenzo Mendola
- Academic Radiology, Department of Translational Research, University of Pisa, Pisa, Italy
| | - Arrigo Cattabriga
- Department of Radiology, IRCCS Azienda Ospedaliero Universitaria di Bologna, Bologna, Italy
| | - Maria Adriana Cocozza
- Department of Radiology, IRCCS Azienda Ospedaliero Universitaria di Bologna, Bologna, Italy
| | - Giulio Vara
- Department of Radiology, IRCCS Azienda Ospedaliero Universitaria di Bologna, Bologna, Italy
| | - Alberto Piccinino
- Department of Radiology, IRCCS Azienda Ospedaliero Universitaria di Bologna, Bologna, Italy
| | - Silvia Lo Monaco
- Department of Radiology, IRCCS Azienda Ospedaliero Universitaria di Bologna, Bologna, Italy
| | - Luigi Vincenzo Pastore
- Department of Radiology, IRCCS Azienda Ospedaliero Universitaria di Bologna, Bologna, Italy
| | - Margherita Mottola
- Department of Computer Science and Engineering, University of Bologna, Bologna, Italy
| | - Silvia Malavasi
- Department of Computer Science and Engineering, University of Bologna, Bologna, Italy
| | - Alessandro Bevilacqua
- Department of Computer Science and Engineering, University of Bologna, Bologna, Italy
| | - Emanuele Neri
- SIRM Foundation, Italian Society of Medical and Interventional Radiology, Milan, Italy
- Academic Radiology, Department of Translational Research, University of Pisa, Pisa, Italy
| | - Rita Golfieri
- Department of Radiology, IRCCS Azienda Ospedaliero Universitaria di Bologna, Bologna, Italy
| |
Collapse
|