1
|
Campanioni S, Veiga C, Prieto-González JM, González-Nóvoa JA, Busto L, Martinez C, Alberte-Woodward M, García de Soto J, Pouso-Diz J, Fernández Ceballos MDLÁ, Agis-Balboa RC. Explainable machine learning on baseline MRI predicts multiple sclerosis trajectory descriptors. PLoS One 2024; 19:e0306999. [PMID: 39012871 PMCID: PMC11251627 DOI: 10.1371/journal.pone.0306999] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/08/2024] [Accepted: 06/26/2024] [Indexed: 07/18/2024] Open
Abstract
Multiple sclerosis (MS) is a multifaceted neurological condition characterized by challenges in timely diagnosis and personalized patient management. The application of Artificial Intelligence (AI) to MS holds promises for early detection, accurate diagnosis, and predictive modeling. The objectives of this study are: 1) to propose new MS trajectory descriptors that could be employed in Machine Learning (ML) regressors and classifiers to predict patient evolution; 2) to explore the contribution of ML models in discerning MS trajectory descriptors using only baseline Magnetic Resonance Imaging (MRI) studies. This study involved 446 MS patients who had a baseline MRI, at least two measurements of Expanded Disability Status Scale (EDSS), and a 1-year follow-up. Patients were divided into two groups: 1) for model development and 2) for evaluation. Three descriptors: β1, β2, and EDSS(t), were related to baseline MRI parameters using regression and classification XGBoost models. Shapley Additive Explanations (SHAP) analysis enhanced model transparency by identifying influential features. The results of this study demonstrate the potential of AI in predicting MS progression using the proposed patient trajectories and baseline MRI scans, outperforming classic Multiple Linear Regression (MLR) methods. In conclusion, MS trajectory descriptors are crucial; incorporating AI analysis into MRI assessments presents promising opportunities to advance predictive capabilities. SHAP analysis enhances model interpretation, revealing feature importance for clinical decisions.
Collapse
Affiliation(s)
- Silvia Campanioni
- Galicia Sur Health Research Institute (IIS Galicia Sur), Cardiovascular Research Group, Vigo, Spain
| | - César Veiga
- Galicia Sur Health Research Institute (IIS Galicia Sur), Cardiovascular Research Group, Vigo, Spain
| | - José María Prieto-González
- Health Research Institute of Santiago de Compostela (IDIS), Translational Research in Neurological Diseases Group, Santiago University Hospital Complex, SERGAS-USC, Santiago de Compostela, Spain
- Neuro Epigenetics Lab, Health Research Institute of Santiago de Compostela (IDIS), Santiago University Hospital Complex, Santiago de Compostela, Spain
- Neurology Service, Santiago University Hospital Complex, Santiago de Compostela, Spain
| | - José A. González-Nóvoa
- Galicia Sur Health Research Institute (IIS Galicia Sur), Cardiovascular Research Group, Vigo, Spain
| | - Laura Busto
- Galicia Sur Health Research Institute (IIS Galicia Sur), Cardiovascular Research Group, Vigo, Spain
| | - Carlos Martinez
- Galicia Sur Health Research Institute (IIS Galicia Sur), Cardiovascular Research Group, Vigo, Spain
| | - Miguel Alberte-Woodward
- Health Research Institute of Santiago de Compostela (IDIS), Translational Research in Neurological Diseases Group, Santiago University Hospital Complex, SERGAS-USC, Santiago de Compostela, Spain
- Neuro Epigenetics Lab, Health Research Institute of Santiago de Compostela (IDIS), Santiago University Hospital Complex, Santiago de Compostela, Spain
- Neurology Service, Santiago University Hospital Complex, Santiago de Compostela, Spain
| | - Jesús García de Soto
- Health Research Institute of Santiago de Compostela (IDIS), Translational Research in Neurological Diseases Group, Santiago University Hospital Complex, SERGAS-USC, Santiago de Compostela, Spain
- Neuro Epigenetics Lab, Health Research Institute of Santiago de Compostela (IDIS), Santiago University Hospital Complex, Santiago de Compostela, Spain
- Neurology Service, Santiago University Hospital Complex, Santiago de Compostela, Spain
| | - Jessica Pouso-Diz
- Health Research Institute of Santiago de Compostela (IDIS), Translational Research in Neurological Diseases Group, Santiago University Hospital Complex, SERGAS-USC, Santiago de Compostela, Spain
- Neuro Epigenetics Lab, Health Research Institute of Santiago de Compostela (IDIS), Santiago University Hospital Complex, Santiago de Compostela, Spain
- Neurology Service, Santiago University Hospital Complex, Santiago de Compostela, Spain
| | - María de los Ángeles Fernández Ceballos
- Health Research Institute of Santiago de Compostela (IDIS), Translational Research in Neurological Diseases Group, Santiago University Hospital Complex, SERGAS-USC, Santiago de Compostela, Spain
- Neuro Epigenetics Lab, Health Research Institute of Santiago de Compostela (IDIS), Santiago University Hospital Complex, Santiago de Compostela, Spain
- Neurology Service, Santiago University Hospital Complex, Santiago de Compostela, Spain
| | - Roberto Carlos Agis-Balboa
- Health Research Institute of Santiago de Compostela (IDIS), Translational Research in Neurological Diseases Group, Santiago University Hospital Complex, SERGAS-USC, Santiago de Compostela, Spain
- Neuro Epigenetics Lab, Health Research Institute of Santiago de Compostela (IDIS), Santiago University Hospital Complex, Santiago de Compostela, Spain
- Neurology Service, Santiago University Hospital Complex, Santiago de Compostela, Spain
| |
Collapse
|
2
|
El Morr C, Kundi B, Mobeen F, Taleghani S, El-Lahib Y, Gorman R. AI and disability: A systematic scoping review. Health Informatics J 2024; 30:14604582241285743. [PMID: 39287175 DOI: 10.1177/14604582241285743] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 09/19/2024]
Abstract
Background: Artificial intelligence (AI) can enhance life experiences and present challenges for people with disabilities. Objectives: This study aims to investigate the relationship between AI and disability, exploring the potential benefits and challenges of using AI for people with disabilities. Methods: A systematic scoping review was conducted using eight online databases; 45 scholarly articles from the last 5 years were identified and selected for thematic analysis. Results: The review's findings revealed AI's potential to enhance healthcare; however, it showed a high prevalence of a narrow medical model of disability and an ableist perspective in AI research. This raises concerns about the perpetuation of biases and discrimination against individuals with disabilities in the development and deployment of AI technologies. Conclusion: We recommend shifting towards a social model of disability, promoting interdisciplinary collaboration, addressing AI bias and discrimination, prioritizing privacy and security in AI development, focusing on accessibility and usability, investing in education and training, and advocating for robust policy and regulatory frameworks. The review emphasizes the urgent need for further research to ensure that AI benefits all members of society equitably and that future AI systems are designed with inclusivity and accessibility as core principles.
Collapse
Affiliation(s)
- Christo El Morr
- School of Health Policy and Management, York University, Toronto, ON, Canada
| | - Bushra Kundi
- Master of Science in eHealth, McMaster University, Hamilton, ON, Canada
| | - Fariah Mobeen
- School of Health Policy and Management, York University, Toronto, ON, Canada
| | - Sarah Taleghani
- School of Health Policy and Management, York University, Toronto, ON, Canada
| | - Yahya El-Lahib
- Faculty of Social Work, University of Calgary, Calgary, AB, Canada
| | - Rachel Gorman
- School of Health Policy and Management, York University, Toronto, ON, Canada
| |
Collapse
|
3
|
De Brouwer E, Becker T, Werthen-Brabants L, Dewulf P, Iliadis D, Dekeyser C, Laureys G, Van Wijmeersch B, Popescu V, Dhaene T, Deschrijver D, Waegeman W, De Baets B, Stock M, Horakova D, Patti F, Izquierdo G, Eichau S, Girard M, Prat A, Lugaresi A, Grammond P, Kalincik T, Alroughani R, Grand’Maison F, Skibina O, Terzi M, Lechner-Scott J, Gerlach O, Khoury SJ, Cartechini E, Van Pesch V, Sà MJ, Weinstock-Guttman B, Blanco Y, Ampapa R, Spitaleri D, Solaro C, Maimone D, Soysal A, Iuliano G, Gouider R, Castillo-Triviño T, Sánchez-Menoyo JL, Laureys G, van der Walt A, Oh J, Aguera-Morales E, Altintas A, Al-Asmi A, de Gans K, Fragoso Y, Csepany T, Hodgkinson S, Deri N, Al-Harbi T, Taylor B, Gray O, Lalive P, Rozsa C, McGuigan C, Kermode A, Sempere AP, Mihaela S, Simo M, Hardy T, Decoo D, Hughes S, Grigoriadis N, Sas A, Vella N, Moreau Y, Peeters L. Machine-learning-based prediction of disability progression in multiple sclerosis: An observational, international, multi-center study. PLOS DIGITAL HEALTH 2024; 3:e0000533. [PMID: 39052668 PMCID: PMC11271865 DOI: 10.1371/journal.pdig.0000533] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/29/2023] [Accepted: 05/14/2024] [Indexed: 07/27/2024]
Abstract
BACKGROUND Disability progression is a key milestone in the disease evolution of people with multiple sclerosis (PwMS). Prediction models of the probability of disability progression have not yet reached the level of trust needed to be adopted in the clinic. A common benchmark to assess model development in multiple sclerosis is also currently lacking. METHODS Data of adult PwMS with a follow-up of at least three years from 146 MS centers, spread over 40 countries and collected by the MSBase consortium was used. With basic inclusion criteria for quality requirements, it represents a total of 15, 240 PwMS. External validation was performed and repeated five times to assess the significance of the results. Transparent Reporting for Individual Prognosis Or Diagnosis (TRIPOD) guidelines were followed. Confirmed disability progression after two years was predicted, with a confirmation window of six months. Only routinely collected variables were used such as the expanded disability status scale, treatment, relapse information, and MS course. To learn the probability of disability progression, state-of-the-art machine learning models were investigated. The discrimination performance of the models is evaluated with the area under the receiver operator curve (ROC-AUC) and under the precision recall curve (AUC-PR), and their calibration via the Brier score and the expected calibration error. All our preprocessing and model code are available at https://gitlab.com/edebrouwer/ms_benchmark, making this task an ideal benchmark for predicting disability progression in MS. FINDINGS Machine learning models achieved a ROC-AUC of 0⋅71 ± 0⋅01, an AUC-PR of 0⋅26 ± 0⋅02, a Brier score of 0⋅1 ± 0⋅01 and an expected calibration error of 0⋅07 ± 0⋅04. The history of disability progression was identified as being more predictive for future disability progression than the treatment or relapses history. CONCLUSIONS Good discrimination and calibration performance on an external validation set is achieved, using only routinely collected variables. This suggests machine-learning models can reliably inform clinicians about the future occurrence of progression and are mature for a clinical impact study.
Collapse
Affiliation(s)
| | - Thijs Becker
- I-Biostat, Hasselt University, Belgium
- Data Science Institute, Hasselt University, Belgium
| | | | - Pieter Dewulf
- KERMIT, Department of Data Analysis and Mathematical Modelling, Ghent University, Belgium
| | - Dimitrios Iliadis
- KERMIT, Department of Data Analysis and Mathematical Modelling, Ghent University, Belgium
| | - Cathérine Dekeyser
- Department of Neurology, Ghent University, Belgium
- 4 Brain, Ghent University, Belgium
- Biomedical Research Institute, Hasselt University, Belgium
| | - Guy Laureys
- Department of Neurology, Ghent University, Belgium
- 4 Brain, Ghent University, Belgium
| | - Bart Van Wijmeersch
- Noorderhart ziekenhuizen Pelt, Belgium
- Universitair MS Centrum Hasselt-Pelt, Belgium
| | - Veronica Popescu
- Noorderhart ziekenhuizen Pelt, Belgium
- Universitair MS Centrum Hasselt-Pelt, Belgium
| | - Tom Dhaene
- SUMO, IDLAB, Ghent University - imec, Belgium
| | | | - Willem Waegeman
- KERMIT, Department of Data Analysis and Mathematical Modelling, Ghent University, Belgium
| | - Bernard De Baets
- KERMIT, Department of Data Analysis and Mathematical Modelling, Ghent University, Belgium
| | - Michiel Stock
- KERMIT, Department of Data Analysis and Mathematical Modelling, Ghent University, Belgium
- Biobix, Department of Data Analysis and Mathematical Modelling, Ghent University, Belgium
| | - Dana Horakova
- Charles University in Prague and General University Hospital, Prague, Czech Republic
| | - Francesco Patti
- Department of Medical and Surgical Sciences and Advanced Technologies, GF Ingrassia, Catania, Italy
| | | | - Sara Eichau
- Hospital Universitario Virgen Macarena, Sevilla, Spain
| | - Marc Girard
- CHUM and Université de Montreal, Montreal, Canada
| | | | - Alessandra Lugaresi
- IRCCS Istituto delle Scienze Neurologiche di Bologna, Bologna, Italia and Dipartimento di Scienze Biomediche e Neuromotorie, Università di Bologna, Bologna, Italia
| | | | - Tomas Kalincik
- Melbourne MS Centre, Department of Neurology, Royal Melbourne Hospital, Melbourne, Australia
- CORe, Department of Medicine, University of Melbourne, Melbourne, Australia
| | | | | | | | | | | | - Oliver Gerlach
- Academic MS Center Zuyderland, Department of Neurology, Zuyderland Medical Center, Sittard-Geleen, The Netherlands
- School for Mental Health and Neuroscience, Maastricht University, Maastricht, The Netherlands
| | - Samia J. Khoury
- American University of Beirut Medical Center, Beirut, Lebanon
| | | | | | - Maria José Sà
- Centro Hospitalar Universitario de Sao Joao, Porto, Portugal
| | | | | | | | - Daniele Spitaleri
- Azienda Ospedaliera di Rilievo Nazionale San Giuseppe Moscati Avellino, Avellino, Italy
| | - Claudio Solaro
- Dept. of Rehabilitation, CRFF Mons. Luigi Novarese, Moncrivello, Italy
| | - Davide Maimone
- MS center, UOC Neurologia, ARNAS Garibaldi, Catania, Italy
| | - Aysun Soysal
- Bakirkoy Education and Research Hospital for Psychiatric and Neurological Diseases, Istanbul, Turkey
| | | | | | | | | | | | | | - Jiwon Oh
- St. Michael’s Hospital, Toronto, Canada
| | | | - Ayse Altintas
- Koc University, School of Medicine, Istanbul, Turkey
| | - Abdullah Al-Asmi
- College of Medicine & Health Sciences and Sultan Qaboos University Hospital, SQU, Oman
| | | | - Yara Fragoso
- Universidade Metropolitana de Santos, Santos, Brazil
| | | | | | - Norma Deri
- Hospital Fernandez, Capital Federal, Argentina
| | - Talal Al-Harbi
- King Fahad Specialist Hospital-Dammam, Khobar, Saudi Arabia
| | | | - Orla Gray
- South Eastern HSC Trust, Belfast, United Kingdom
| | | | - Csilla Rozsa
- Jahn Ferenc Teaching Hospital, Budapest, Hungary
| | | | - Allan Kermode
- University of Western Australia, Nedlands, Australia
| | | | - Simu Mihaela
- Emergency Clinical County Hospital Pius Brinzeu, Timisoara, Romania and University of Medicine and Pharmacy Victor Babes, Timisoara, Romania
| | | | - Todd Hardy
- Concord Repatriation General Hospital, Sydney, Australia
| | - Danny Decoo
- AZ Alma Ziekenhuis, Sijsele - Damme, Belgium
| | | | | | | | | | | | - Liesbet Peeters
- Data Science Institute, Hasselt University, Belgium
- Universitair MS Centrum Hasselt-Pelt, Belgium
| |
Collapse
|
4
|
Wu Y, Xiang C, Wang Z, Fang Y. Interpretable prediction models for disability in older adults with hypertension: the Chinese Longitudinal Healthy Longevity and Happy Family Study. Psychogeriatrics 2024; 24:645-654. [PMID: 38514389 DOI: 10.1111/psyg.13112] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/01/2023] [Revised: 02/14/2024] [Accepted: 03/05/2024] [Indexed: 03/23/2024]
Abstract
BACKGROUND Older adults with hypertension have a high risk of disability, while an accurate risk prediction model is still lacking. This study aimed to construct interpretable disability prediction models for older Chinese with hypertension based on multiple time intervals. METHODS Data were collected from the Chinese Longitudinal Healthy Longevity and Happy Family Study for 2008-2018. A total of 1602, 1108, and 537 older adults were included for the periods of 2008-2012, 2008-2014, and 2008-2018, respectively. Disability was measured by basic activities of daily living. Least absolute shrinkage and selection operator (LASSO) was applied for feature selection. Five machine learning algorithms combined with LASSO set and full-variable set were used to predict 4-, 6-, and 10-year disability risk, respectively. Area under the receiver operating characteristic curve was used as the main metric for selection of the optimal model. SHapley Additive exPlanations (SHAP) was used to explore important predictors of the optimal model. RESULTS Random forest in full-variable set and XGBoost in LASSO set were the optimal models for 4-year prediction. Support vector machine was the optimal model for 6-year prediction on both sets. For 10-year prediction, deep neural network in full variable set and logistic regression in LASSO set were optimal models. Age ranked the most important predictor. Marital status, body mass index, score of Mini-Mental State Examination, and psychological well-being score were also important predictors. CONCLUSIONS Machine learning shows promise in screening out older adults at high risk of disability. Disability prevention strategies should specifically focus on older patients with unfortunate marriage, high BMI, and poor cognitive and psychological conditions.
Collapse
Affiliation(s)
- Yafei Wu
- School of Public Health, Xiamen University, Xiamen, China
- Key Laboratory of Health Technology Assessment of Fujian Province, Xiamen, China
| | - Chaoyi Xiang
- School of Public Health, Xiamen University, Xiamen, China
- Key Laboratory of Health Technology Assessment of Fujian Province, Xiamen, China
| | - Zongjie Wang
- School of Public Health, Xiamen University, Xiamen, China
- Key Laboratory of Health Technology Assessment of Fujian Province, Xiamen, China
| | - Ya Fang
- School of Public Health, Xiamen University, Xiamen, China
- Key Laboratory of Health Technology Assessment of Fujian Province, Xiamen, China
- National Institute for Data Science in Health and Medicine, Xiamen University, Xiamen, China
| |
Collapse
|
5
|
Kalincik T, Roos I, Sharmin S. Observational studies of treatment effectiveness in neurology. Brain 2023; 146:4799-4808. [PMID: 37587541 PMCID: PMC10690012 DOI: 10.1093/brain/awad278] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/24/2023] [Revised: 07/07/2023] [Accepted: 07/13/2023] [Indexed: 08/18/2023] Open
Abstract
The capacity and power of data from cohorts, registries and randomized trials to provide answers to contemporary clinical questions in neurology has increased considerably over the past two decades. Novel sophisticated statistical methods are enabling us to harness these data to guide treatment decisions, but their complexity is making appraisal of clinical evidence increasingly demanding. In this review, we discuss several methodological aspects of contemporary research of treatment effectiveness in observational data in neurology, aimed at academic neurologists and analysts specializing in outcomes research. The review discusses specifics of the sources of observational data and their key features. It focuses on the limitations of observational data and study design, as well as statistical approaches aimed to overcome these limitations. Among the examples of leading clinical themes typically studied with analyses of observational data, the review discusses methodological approaches to comparative treatment effectiveness, development of diagnostic criteria and definitions of clinical outcomes. Finally, this review provides a brief summary of key points that will help clinical audience critically evaluate design and analytical aspects of studies of disease outcomes using observational data.
Collapse
Affiliation(s)
- Tomas Kalincik
- CORe, Department of Medicine, University of Melbourne, Melbourne, 3050, VIC, Australia
- Neuroimmunology Centre, Department of Neurology, Royal Melbourne Hospital, Melbourne, 3000, VIC, Australia
| | - Izanne Roos
- CORe, Department of Medicine, University of Melbourne, Melbourne, 3050, VIC, Australia
- Neuroimmunology Centre, Department of Neurology, Royal Melbourne Hospital, Melbourne, 3000, VIC, Australia
| | - Sifat Sharmin
- CORe, Department of Medicine, University of Melbourne, Melbourne, 3050, VIC, Australia
- Neuroimmunology Centre, Department of Neurology, Royal Melbourne Hospital, Melbourne, 3000, VIC, Australia
| |
Collapse
|
6
|
Reeve K, On BI, Havla J, Burns J, Gosteli-Peter MA, Alabsawi A, Alayash Z, Götschi A, Seibold H, Mansmann U, Held U. Prognostic models for predicting clinical disease progression, worsening and activity in people with multiple sclerosis. Cochrane Database Syst Rev 2023; 9:CD013606. [PMID: 37681561 PMCID: PMC10486189 DOI: 10.1002/14651858.cd013606.pub2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 09/09/2023]
Abstract
BACKGROUND Multiple sclerosis (MS) is a chronic inflammatory disease of the central nervous system that affects millions of people worldwide. The disease course varies greatly across individuals and many disease-modifying treatments with different safety and efficacy profiles have been developed recently. Prognostic models evaluated and shown to be valid in different settings have the potential to support people with MS and their physicians during the decision-making process for treatment or disease/life management, allow stratified and more precise interpretation of interventional trials, and provide insights into disease mechanisms. Many researchers have turned to prognostic models to help predict clinical outcomes in people with MS; however, to our knowledge, no widely accepted prognostic model for MS is being used in clinical practice yet. OBJECTIVES To identify and summarise multivariable prognostic models, and their validation studies for quantifying the risk of clinical disease progression, worsening, and activity in adults with MS. SEARCH METHODS We searched MEDLINE, Embase, and the Cochrane Database of Systematic Reviews from January 1996 until July 2021. We also screened the reference lists of included studies and relevant reviews, and references citing the included studies. SELECTION CRITERIA We included all statistically developed multivariable prognostic models aiming to predict clinical disease progression, worsening, and activity, as measured by disability, relapse, conversion to definite MS, conversion to progressive MS, or a composite of these in adult individuals with MS. We also included any studies evaluating the performance of (i.e. validating) these models. There were no restrictions based on language, data source, timing of prognostication, or timing of outcome. DATA COLLECTION AND ANALYSIS Pairs of review authors independently screened titles/abstracts and full texts, extracted data using a piloted form based on the Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS), assessed risk of bias using the Prediction Model Risk Of Bias Assessment Tool (PROBAST), and assessed reporting deficiencies based on the checklist items in Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD). The characteristics of the included models and their validations are described narratively. We planned to meta-analyse the discrimination and calibration of models with at least three external validations outside the model development study but no model met this criterion. We summarised between-study heterogeneity narratively but again could not perform the planned meta-regression. MAIN RESULTS We included 57 studies, from which we identified 75 model developments, 15 external validations corresponding to only 12 (16%) of the models, and six author-reported validations. Only two models were externally validated multiple times. None of the identified external validations were performed by researchers independent of those that developed the model. The outcome was related to disease progression in 39 (41%), relapses in 8 (8%), conversion to definite MS in 17 (18%), and conversion to progressive MS in 27 (28%) of the 96 models or validations. The disease and treatment-related characteristics of included participants, and definitions of considered predictors and outcome, were highly heterogeneous amongst the studies. Based on the publication year, we observed an increase in the percent of participants on treatment, diversification of the diagnostic criteria used, an increase in consideration of biomarkers or treatment as predictors, and increased use of machine learning methods over time. Usability and reproducibility All identified models contained at least one predictor requiring the skills of a medical specialist for measurement or assessment. Most of the models (44; 59%) contained predictors that require specialist equipment likely to be absent from primary care or standard hospital settings. Over half (52%) of the developed models were not accompanied by model coefficients, tools, or instructions, which hinders their application, independent validation or reproduction. The data used in model developments were made publicly available or reported to be available on request only in a few studies (two and six, respectively). Risk of bias We rated all but one of the model developments or validations as having high overall risk of bias. The main reason for this was the statistical methods used for the development or evaluation of prognostic models; we rated all but two of the included model developments or validations as having high risk of bias in the analysis domain. None of the model developments that were externally validated or these models' external validations had low risk of bias. There were concerns related to applicability of the models to our research question in over one-third (38%) of the models or their validations. Reporting deficiencies Reporting was poor overall and there was no observable increase in the quality of reporting over time. The items that were unclearly reported or not reported at all for most of the included models or validations were related to sample size justification, blinding of outcome assessors, details of the full model or how to obtain predictions from it, amount of missing data, and treatments received by the participants. Reporting of preferred model performance measures of discrimination and calibration was suboptimal. AUTHORS' CONCLUSIONS The current evidence is not sufficient for recommending the use of any of the published prognostic prediction models for people with MS in clinical routine today due to lack of independent external validations. The MS prognostic research community should adhere to the current reporting and methodological guidelines and conduct many more state-of-the-art external validation studies for the existing or newly developed models.
Collapse
Affiliation(s)
- Kelly Reeve
- Epidemiology, Biostatistics and Prevention Institute, University of Zürich, Zurich, Switzerland
| | - Begum Irmak On
- Institute for Medical Information Processing, Biometry and Epidemiology, Ludwig-Maximilians-Universität München, Munich, Germany
| | - Joachim Havla
- lnstitute of Clinical Neuroimmunology, Ludwig-Maximilians-Universität München, Munich, Germany
| | - Jacob Burns
- Institute for Medical Information Processing, Biometry and Epidemiology, Ludwig-Maximilians-Universität München, Munich, Germany
- Pettenkofer School of Public Health, Munich, Germany
| | | | - Albraa Alabsawi
- Institute for Medical Information Processing, Biometry and Epidemiology, Ludwig-Maximilians-Universität München, Munich, Germany
| | - Zoheir Alayash
- Institute for Medical Information Processing, Biometry and Epidemiology, Ludwig-Maximilians-Universität München, Munich, Germany
- Institute of Health Services Research in Dentistry, University of Münster, Muenster, Germany
| | - Andrea Götschi
- Epidemiology, Biostatistics and Prevention Institute, University of Zürich, Zurich, Switzerland
| | | | - Ulrich Mansmann
- Institute for Medical Information Processing, Biometry and Epidemiology, Ludwig-Maximilians-Universität München, Munich, Germany
- Pettenkofer School of Public Health, Munich, Germany
| | - Ulrike Held
- Epidemiology, Biostatistics and Prevention Institute, University of Zürich, Zurich, Switzerland
| |
Collapse
|
7
|
Toh EMS, Ng KJ, Motani M, Lim MJR. Letter: Deep Neural Networks Can Accurately Detect Blood Loss and Hemorrhage Control Task Success from Video. Neurosurgery 2023; 93:e79-e80. [PMID: 37462368 DOI: 10.1227/neu.0000000000002590] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/07/2023] [Accepted: 05/09/2023] [Indexed: 08/16/2023] Open
Affiliation(s)
- Emma Min Shuen Toh
- Yong Loo Lin School of Medicine, National University of Singapore, Singapore , Singapore
| | - Kai Jie Ng
- Yong Loo Lin School of Medicine, National University of Singapore, Singapore , Singapore
| | - Mehul Motani
- Department of Electrical & Computer Engineering, College of Design and Engineering, National University of Singapore, Singapore
- N.1 Institute for Health, National University of Singapore, Singapore , Singapore
- Institute of Data Science, National University of Singapore, Singapore , Singapore
- Institute for Digital Medicine (WisDM), Yong Loo Lin School of Medicine, National University of Singapore, Singapore
| | - Mervyn Jun Rui Lim
- Division of Neurosurgery, University Surgical Cluster, National University Hospital, Singapore , Singapore
| |
Collapse
|
8
|
Pungitore S, Subbian V. Assessment of Prediction Tasks and Time Window Selection in Temporal Modeling of Electronic Health Record Data: a Systematic Review. JOURNAL OF HEALTHCARE INFORMATICS RESEARCH 2023; 7:313-331. [PMID: 37637723 PMCID: PMC10449760 DOI: 10.1007/s41666-023-00143-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/08/2022] [Revised: 04/12/2023] [Accepted: 07/28/2023] [Indexed: 08/29/2023]
Abstract
Temporal electronic health record (EHR) data are often preferred for clinical prediction tasks because they offer more complete representations of a patient's pathophysiology than static data. A challenge when working with temporal EHR data is problem formulation, which includes defining the time windows of interest and the prediction task. Our objective was to conduct a systematic review that assessed the definition and reporting of concepts relevant to temporal clinical prediction tasks. We searched PubMed® and IEEE Xplore® databases for studies from January 1, 2010 applying machine learning models to EHR data for patient outcome prediction. Publications applying time-series methods were selected for further review. We identified 92 studies and summarized them by clinical context and definition and reporting of the prediction problem. For the time windows of interest, 12 studies did not discuss window lengths, 57 used a single set of window lengths, and 23 evaluated the relationship between window length and model performance. We also found that 72 studies had appropriate reporting of the prediction task. However, evaluation of prediction problem formulation for temporal EHR data was complicated by heterogeneity in assessing and reporting of these concepts. Even among studies modeling similar clinical outcomes, there were variations in terminology used to describe the prediction problem, rationale for window lengths, and determination of the outcome of interest. As temporal modeling using EHR data expands, minimal reporting standards should include time-series specific concerns to promote rigor and reproducibility in future studies and facilitate model implementation in clinical settings. Supplementary Information The online version contains supplementary material available at 10.1007/s41666-023-00143-4.
Collapse
Affiliation(s)
- Sarah Pungitore
- Program in Applied Mathematics, Department of Mathematics, 617 N Santa Rita Ave, Tucson, AZ 85721 USA
| | - Vignesh Subbian
- Department of Biomedical Engineering, The University of Arizona, Tucson, AZ 85721-0020 USA
- Department of Systems and Industrial Engineering, The University of Arizona, Tucson, AZ 85721-0020 USA
| |
Collapse
|
9
|
Eskofier BM, Klucken J. Predictive Models for Health Deterioration: Understanding Disease Pathways for Personalized Medicine. Annu Rev Biomed Eng 2023; 25:131-156. [PMID: 36854259 DOI: 10.1146/annurev-bioeng-110220-030247] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/02/2023]
Abstract
Artificial intelligence (AI) and machine learning (ML) methods are currently widely employed in medicine and healthcare. A PubMed search returns more than 100,000 articles on these topics published between 2018 and 2022 alone. Notwithstanding several recent reviews in various subfields of AI and ML in medicine, we have yet to see a comprehensive review around the methods' use in longitudinal analysis and prediction of an individual patient's health status within a personalized disease pathway. This review seeks to fill that gap. After an overview of the AI and ML methods employed in this field and of specific medical applications of models of this type, the review discusses the strengths and limitations of current studies and looks ahead to future strands of research in this field. We aim to enable interested readers to gain a detailed impression of the research currently available and accordingly plan future work around predictive models for deterioration in health status.
Collapse
Affiliation(s)
- Bjoern M Eskofier
- Machine Learning and Data Analytics Lab, Department of Artificial Intelligence in Biomedical Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany;
| | - Jochen Klucken
- Digital Medicine Group, Luxembourg Centre for Systems Biomedicine, Université du Luxembourg, Belvaux, Luxembourg
- Digital Medicine Group, Department of Precision Health, Luxembourg Institute of Health, Strassen, Luxembourg
- Centre Hospitalier de Luxembourg, Luxembourg City, Luxembourg
| |
Collapse
|
10
|
Sergooris A, Verbrugghe J, Matheve T, Van Den Houte M, Bonnechère B, Corten K, Bogaerts K, Timmermans A. Clinical phenotypes and prognostic factors in persons with hip osteoarthritis undergoing total hip arthroplasty: protocol for a longitudinal prospective cohort study (HIPPROCLIPS). BMC Musculoskelet Disord 2023; 24:224. [PMID: 36964541 PMCID: PMC10039547 DOI: 10.1186/s12891-023-06326-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/04/2022] [Accepted: 03/15/2023] [Indexed: 03/26/2023] Open
Abstract
BACKGROUND Large heterogeneity exists in the clinical manifestation of hip osteoarthritis (OA). It is therefore not surprising that pain and disability in individuals with hip OA and after total hip arthroplasty (THA) cannot be explained by biomedical variables alone. Indeed, also maladaptive pain-related cognitions and emotions can contribute to pain and disability, and can lead to poor treatment outcomes. Traumatic experiences, mental disorders, self-efficacy and social support can influence stress appraisal and strategies to cope with pain, but their influence on pain and disability has not yet been established in individuals with hip OA undergoing THA. This study aims (1) to determine the influence of traumatic experiences and mental disorders on pain processing before and shortly after THA (2) to identify preoperative clinical phenotypes in individuals with hip OA eligible for THA, (3) to identify pre- and early postoperative prognostic factors for outcomes in pain and disability after THA, and (4) to identify postoperative clinical phenotypes in individuals after THA. METHODS This prospective longitudinal cohort study will investigate 200 individuals undergoing THA for hip OA. Phenotyping variables and candidate prognostic factors include pain-related fear-avoidance behaviour, perceived injustice, mental disorders, traumatic experiences, self-efficacy, and social support. Peripheral and central pain mechanisms will be assessed with thermal quantitative sensory testing. The primary outcome measure is the hip disability and osteoarthritis outcome score. Other outcome measures include performance-based measures, hip muscle strength, the patient-specific functional scale, pain intensity, global perceived effect, and outcome satisfaction. All these measurements will be performed before surgery, as well as 6 weeks, 3 months, and 12 months after surgery. Pain-related cognitions and emotions will additionally be assessed in the early postoperative phase, on the first, third, fifth, and seventh day after THA. Main statistical methods that will be used to answer the respective research questions include: LASSO regression, decision tree learning, gradient boosting algorithms, and recurrent neural networks. DISCUSSION The identification of clinical phenotypes and prognostic factors for outcomes in pain and disability will be a first step towards pre- and postoperative precision medicine for individuals with hip OA undergoing THA. TRIAL REGISTRATION ClinicalTrials.gov: NCT05265858. Registered on 04/03/2022.
Collapse
Affiliation(s)
- Abner Sergooris
- REVAL Rehabilitation Research, Faculty of Rehabilitation Sciences, Hasselt University, Agoralaan Building A - B-3590, Diepenbeek, Belgium.
| | - Jonas Verbrugghe
- REVAL Rehabilitation Research, Faculty of Rehabilitation Sciences, Hasselt University, Agoralaan Building A - B-3590, Diepenbeek, Belgium
| | - Thomas Matheve
- REVAL Rehabilitation Research, Faculty of Rehabilitation Sciences, Hasselt University, Agoralaan Building A - B-3590, Diepenbeek, Belgium
- Spine, Head and Pain Research Unit Ghent, Department of Rehabilitation Sciences, Ghent University, Ghent, Belgium
| | - Maaike Van Den Houte
- REVAL Rehabilitation Research, Faculty of Rehabilitation Sciences, Hasselt University, Agoralaan Building A - B-3590, Diepenbeek, Belgium
- Laboratory for Brain-Gut Axis Studies (LABGAS), Translational Research Center for Gastrointestinal Disorders (TARGID), Department of Chronic Diseases and Metabolism, University of Leuven, Leuven, Belgium
| | - Bruno Bonnechère
- REVAL Rehabilitation Research, Faculty of Rehabilitation Sciences, Hasselt University, Agoralaan Building A - B-3590, Diepenbeek, Belgium
| | - Kristoff Corten
- Department of Orthopaedics - Hip Unit, Ziekenhuis Oost-Limburg, Genk, Belgium
- Centre for Translational Psychological Research (TRACE), Ziekenhuis Oost-Limburg, Genk, Belgium
| | - Katleen Bogaerts
- REVAL Rehabilitation Research, Faculty of Rehabilitation Sciences, Hasselt University, Agoralaan Building A - B-3590, Diepenbeek, Belgium
- Health Psychology, Faculty of Psychology and Educational Sciences, KU Leuven, Leuven, Belgium
| | - Annick Timmermans
- REVAL Rehabilitation Research, Faculty of Rehabilitation Sciences, Hasselt University, Agoralaan Building A - B-3590, Diepenbeek, Belgium
| |
Collapse
|
11
|
Xiang C, Wu Y, Jia M, Fang Y. Machine learning-based prediction of disability risk in geriatric patients with hypertension for different time intervals. Arch Gerontol Geriatr 2023; 105:104835. [PMID: 36335673 DOI: 10.1016/j.archger.2022.104835] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/03/2022] [Revised: 10/06/2022] [Accepted: 10/09/2022] [Indexed: 12/13/2022]
Abstract
BACKGROUND The risk of disability in older adults with hypertension is substantially high, and prediction of disability risk is crucial for subsequent management. This study aimed to construct prediction models of disability risk for geriatric patients with hypertension at different time intervals, as well as to assess the important predictors and influencing factors of disability. METHODS This study collected data from the Chinese Longitudinal Healthy Longevity and Happy Family Study. There were 1576, 1083 and 506 hypertension patients aged 65+ in 2008 who were free of disability at baseline and had completed outcome information in follow-up of 2008-2012, 2008-2014, 2008-2018. We built five machine learning (ML) models to predict the disability risk. The classic statistical logistic regression (classic-LR) and shapley additive explanations (SHAP) was further introduced to explore possible causal factors and interpret the optimal models' decisions. RESULTS Among the five ML models, logistic regression, extreme gradient boosting, and deep neural network were the optimal models for detecting 4-, 6-, and 10-year disability risk with their AUC-ROCs reached 0.759, 0.728, 0.694 respectively. The classic-LR revealed potential casual factors for disability and the results of SHAP demonstrated important features for risk prediction, reinforcing the trust of decision makers towards black-box models. CONCLUSION The optimal models hold promise for screening out hypertensive old adults at high risk of disability to implement further targeted intervention and the identified key factors may be of additional value in analyzing the causal mechanisms of disability, thereby providing basis to practical application.
Collapse
Affiliation(s)
- Chaoyi Xiang
- The State Key Laboratory of Molecular Vaccine and Molecular Diagnostics, School of Public Health, Xiamen University, Xiang'an Nan Road, Xiang'an District, Xiamen, Fujian 361102, China; Key Laboratory of Health Technology Assessment of Fujian Province, School of Public Health, Xiamen University, China
| | - Yafei Wu
- The State Key Laboratory of Molecular Vaccine and Molecular Diagnostics, School of Public Health, Xiamen University, Xiang'an Nan Road, Xiang'an District, Xiamen, Fujian 361102, China; National Institute for Data Science in Health and Medicine, Xiamen University, China; Key Laboratory of Health Technology Assessment of Fujian Province, School of Public Health, Xiamen University, China
| | - Maoni Jia
- The State Key Laboratory of Molecular Vaccine and Molecular Diagnostics, School of Public Health, Xiamen University, Xiang'an Nan Road, Xiang'an District, Xiamen, Fujian 361102, China; Key Laboratory of Health Technology Assessment of Fujian Province, School of Public Health, Xiamen University, China
| | - Ya Fang
- The State Key Laboratory of Molecular Vaccine and Molecular Diagnostics, School of Public Health, Xiamen University, Xiang'an Nan Road, Xiang'an District, Xiamen, Fujian 361102, China; National Institute for Data Science in Health and Medicine, Xiamen University, China; Key Laboratory of Health Technology Assessment of Fujian Province, School of Public Health, Xiamen University, China.
| |
Collapse
|
12
|
Ferrè L, Clarelli F, Pignolet B, Mascia E, Frasca M, Santoro S, Sorosina M, Bucciarelli F, Moiola L, Martinelli V, Comi G, Liblau R, Filippi M, Valentini G, Esposito F. Combining Clinical and Genetic Data to Predict Response to Fingolimod Treatment in Relapsing Remitting Multiple Sclerosis Patients: A Precision Medicine Approach. J Pers Med 2023; 13:jpm13010122. [PMID: 36675783 PMCID: PMC9861774 DOI: 10.3390/jpm13010122] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/14/2022] [Revised: 12/30/2022] [Accepted: 12/30/2022] [Indexed: 01/11/2023] Open
Abstract
A personalized approach is strongly advocated for treatment selection in Multiple Sclerosis patients due to the high number of available drugs. Machine learning methods proved to be valuable tools in the context of precision medicine. In the present work, we applied machine learning methods to identify a combined clinical and genetic signature of response to fingolimod that could support the prediction of drug response. Two cohorts of fingolimod-treated patients from Italy and France were enrolled and divided into training, validation, and test set. Random forest training and robust feature selection were performed in the first two sets respectively, and the independent test set was used to evaluate model performance. A genetic-only model and a combined clinical-genetic model were obtained. Overall, 381 patients were classified according to the NEDA-3 criterion at 2 years; we identified a genetic model, including 123 SNPs, that was able to predict fingolimod response with an AUROC= 0.65 in the independent test set. When combining clinical data, the model accuracy increased to an AUROC= 0.71. Integrating clinical and genetic data by means of machine learning methods can help in the prediction of response to fingolimod, even though further studies are required to definitely extend this approach to clinical applications.
Collapse
Affiliation(s)
- Laura Ferrè
- Neurology and Neurorehabilitation Unit, IRCCS San Raffaele Hospital, 20132 Milan, Italy
- Laboratory of Human Genetics of Neurological Disorders, IRCCS San Raffaele Hospital, 20132 Milan, Italy
- Vita-Salute San Raffaele University, 20132 Milan, Italy
| | - Ferdinando Clarelli
- Laboratory of Human Genetics of Neurological Disorders, IRCCS San Raffaele Hospital, 20132 Milan, Italy
| | - Beatrice Pignolet
- Centre Hospitalier Universitaire de Toulouse, CEDEX 9, 31059 Toulouse, France
- Institut Toulousain des Maladies Infectieuses et Inflammatoires (Infinity), INSERM UMR1291–CNRS UMR5051—Université Toulouse III, CEDEX 3, 31024 Toulouse, France
| | - Elisabetta Mascia
- Laboratory of Human Genetics of Neurological Disorders, IRCCS San Raffaele Hospital, 20132 Milan, Italy
| | - Marco Frasca
- AnacletoLab, Dipartimento di Informatica, Università degli Studi di Milano, 20133 Milan, Italy
- Data Science Research Center, Università degli Studi di Milano, 20133 Milan, Italy
- Infolife National Lab, CINI, 00185 Rome, Italy
| | - Silvia Santoro
- Laboratory of Human Genetics of Neurological Disorders, IRCCS San Raffaele Hospital, 20132 Milan, Italy
| | - Melissa Sorosina
- Laboratory of Human Genetics of Neurological Disorders, IRCCS San Raffaele Hospital, 20132 Milan, Italy
| | - Florence Bucciarelli
- Centre Hospitalier Universitaire de Toulouse, CEDEX 9, 31059 Toulouse, France
- Institut Toulousain des Maladies Infectieuses et Inflammatoires (Infinity), INSERM UMR1291–CNRS UMR5051—Université Toulouse III, CEDEX 3, 31024 Toulouse, France
| | - Lucia Moiola
- Neurology and Neurorehabilitation Unit, IRCCS San Raffaele Hospital, 20132 Milan, Italy
| | - Vittorio Martinelli
- Neurology and Neurorehabilitation Unit, IRCCS San Raffaele Hospital, 20132 Milan, Italy
| | | | - Roland Liblau
- Institut Toulousain des Maladies Infectieuses et Inflammatoires (Infinity), INSERM UMR1291–CNRS UMR5051—Université Toulouse III, CEDEX 3, 31024 Toulouse, France
- Department of Immunology, Toulouse University Hospitals, CEDEX 3, 31024 Toulouse, France
| | - Massimo Filippi
- Neurology and Neurorehabilitation Unit, IRCCS San Raffaele Hospital, 20132 Milan, Italy
- Vita-Salute San Raffaele University, 20132 Milan, Italy
- Neuroimaging Research Unit, IRCCS San Raffaele Hospital, 20132 Milan, Italy
- Neurophisiology Unit, IRCCS San Raffaele Hospital, 20132 Milan, Italy
| | - Giorgio Valentini
- AnacletoLab, Dipartimento di Informatica, Università degli Studi di Milano, 20133 Milan, Italy
- Data Science Research Center, Università degli Studi di Milano, 20133 Milan, Italy
- Infolife National Lab, CINI, 00185 Rome, Italy
| | - Federica Esposito
- Neurology and Neurorehabilitation Unit, IRCCS San Raffaele Hospital, 20132 Milan, Italy
- Laboratory of Human Genetics of Neurological Disorders, IRCCS San Raffaele Hospital, 20132 Milan, Italy
- Correspondence:
| |
Collapse
|
13
|
Fuh-Ngwa V, Zhou Y, Melton PE, van der Mei I, Charlesworth JC, Lin X, Zarghami A, Broadley SA, Ponsonby AL, Simpson-Yap S, Lechner-Scott J, Taylor BV. Ensemble machine learning identifies genetic loci associated with future worsening of disability in people with multiple sclerosis. Sci Rep 2022; 12:19291. [PMID: 36369345 PMCID: PMC9652373 DOI: 10.1038/s41598-022-23685-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2022] [Accepted: 11/03/2022] [Indexed: 11/13/2022] Open
Abstract
Limited studies have been conducted to identify and validate multiple sclerosis (MS) genetic loci associated with disability progression. We aimed to identify MS genetic loci associated with worsening of disability over time, and to develop and validate ensemble genetic learning model(s) to identify people with MS (PwMS) at risk of future worsening. We examined associations of 208 previously established MS genetic loci with the risk of worsening of disability; we learned ensemble genetic decision rules and validated the predictions in an external dataset. We found 7 genetic loci (rs7731626: HR 0.92, P = 2.4 × 10-5; rs12211604: HR 1.16, P = 3.2 × 10-7; rs55858457: HR 0.93, P = 3.7 × 10-7; rs10271373: HR 0.90, P = 1.1 × 10-7; rs11256593: HR 1.13, P = 5.1 × 10-57; rs12588969: HR = 1.10, P = 2.1 × 10-10; rs1465697: HR 1.09, P = 1.7 × 10-128) associated with risk worsening of disability; most of which were located near or tagged to 13 genomic regions enriched in peptide hormones and steroids biosynthesis pathways by positional and eQTL mapping. The derived ensembles produced a set of genetic decision rules that can be translated to provide additional prognostic values to existing clinical predictions, with the additional benefit of incorporating relevant genetic information into clinical decision making for PwMS. The present study extends our knowledge of MS progression genetics and provides the basis of future studies regarding the functional significance of the identified loci.
Collapse
Affiliation(s)
- Valery Fuh-Ngwa
- grid.1009.80000 0004 1936 826XMenzies Institute for Medical Research, University of Tasmania, 17 Liverpool St, Hobart, TAS 7000 Australia
| | - Yuan Zhou
- grid.1009.80000 0004 1936 826XMenzies Institute for Medical Research, University of Tasmania, 17 Liverpool St, Hobart, TAS 7000 Australia
| | - Phillip E. Melton
- grid.1009.80000 0004 1936 826XMenzies Institute for Medical Research, University of Tasmania, 17 Liverpool St, Hobart, TAS 7000 Australia
| | - Ingrid van der Mei
- grid.1009.80000 0004 1936 826XMenzies Institute for Medical Research, University of Tasmania, 17 Liverpool St, Hobart, TAS 7000 Australia
| | - Jac C. Charlesworth
- grid.1009.80000 0004 1936 826XMenzies Institute for Medical Research, University of Tasmania, 17 Liverpool St, Hobart, TAS 7000 Australia
| | - Xin Lin
- grid.1009.80000 0004 1936 826XMenzies Institute for Medical Research, University of Tasmania, 17 Liverpool St, Hobart, TAS 7000 Australia
| | - Amin Zarghami
- grid.1009.80000 0004 1936 826XMenzies Institute for Medical Research, University of Tasmania, 17 Liverpool St, Hobart, TAS 7000 Australia
| | - Simon A. Broadley
- grid.1022.10000 0004 0437 5432Menzies Health Institute Queensland and School of Medicine, Griffith University Gold Coast, G40 Griffith Health Centre, QLD 4222, Australia
| | - Anne-Louise Ponsonby
- grid.1058.c0000 0000 9442 535XDeveloping Brain Division, The Florey Institute for Neuroscience and Mental Health, Royal Children’s Hospital, University of Melbourne Murdoch Children’s Research Institute, Parkville, VIC 3052 Australia
| | - Steve Simpson-Yap
- grid.1008.90000 0001 2179 088XNeuroepidemiology Unit, Melbourne School of Population & Global Health, The University of Melbourne, Melbourne, VIC 3053 Australia
| | - Jeannette Lechner-Scott
- grid.266842.c0000 0000 8831 109XDepartment of Neurology, Hunter Medical Research Institute, Hunter New England Health, University of Newcastle, Callaghan, NSW 2310 Australia
| | - Bruce V. Taylor
- grid.1009.80000 0004 1936 826XMenzies Institute for Medical Research, University of Tasmania, 17 Liverpool St, Hobart, TAS 7000 Australia
| |
Collapse
|
14
|
Taloni A, Farrelly FA, Pontillo G, Petsas N, Giannì C, Ruggieri S, Petracca M, Brunetti A, Pozzilli C, Pantano P, Tommasin S. Evaluation of Disability Progression in Multiple Sclerosis via Magnetic-Resonance-Based Deep Learning Techniques. Int J Mol Sci 2022; 23:ijms231810651. [PMID: 36142563 PMCID: PMC9505100 DOI: 10.3390/ijms231810651] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/06/2022] [Revised: 09/02/2022] [Accepted: 09/05/2022] [Indexed: 11/16/2022] Open
Abstract
Short-term disability progression was predicted from a baseline evaluation in patients with multiple sclerosis (MS) using their three-dimensional T1-weighted (3DT1) magnetic resonance images (MRI). One-hundred-and-eighty-one subjects diagnosed with MS underwent 3T-MRI and were followed up for two to six years at two sites, with disability progression defined according to the expanded-disability-status-scale (EDSS) increment at the follow-up. The patients’ 3DT1 images were bias-corrected, brain-extracted, registered onto MNI space, and divided into slices along coronal, sagittal, and axial projections. Deep learning image classification models were applied on slices and devised as ResNet50 fine-tuned adaptations at first on a large independent dataset and secondly on the study sample. The final classifiers’ performance was evaluated via the area under the curve (AUC) of the false versus true positive diagram. Each model was also tested against its null model, obtained by reshuffling patients’ labels in the training set. Informative areas were found by intersecting slices corresponding to models fulfilling the disability progression prediction criteria. At follow-up, 34% of patients had disability progression. Five coronal and five sagittal slices had one classifier surviving the AUC evaluation and null test and predicted disability progression (AUC > 0.72 and AUC > 0.81, respectively). Likewise, fifteen combinations of classifiers and axial slices predicted disability progression in patients (AUC > 0.69). Informative areas were the frontal areas, mainly within the grey matter. Briefly, 3DT1 images may give hints on disability progression in MS patients, exploiting the information hidden in the MRI of specific areas of the brain.
Collapse
Affiliation(s)
- Alessandro Taloni
- Institute for Complex Systems, National Research Council (ISC-CNR), 00185 Rome, Italy
| | | | - Giuseppe Pontillo
- Department of Advanced Biomedical Sciences, Federico II University of Naples, 80131 Naples, Italy
- Department of Electrical Engineering and Information Technology, Federico II University of Naples, 80125 Naples, Italy
| | - Nikolaos Petsas
- Department of Radiology, IRCCS NEUROMED, 86077 Pozzilli, Italy
| | - Costanza Giannì
- Department of Human Neurosciences, Sapienza University of Rome, 00185 Rome, Italy
| | - Serena Ruggieri
- Department of Human Neurosciences, Sapienza University of Rome, 00185 Rome, Italy
- Neuroimmunology Unit, IRCSS Fondazione Santa Lucia, 00179 Rome, Italy
| | - Maria Petracca
- Department of Human Neurosciences, Sapienza University of Rome, 00185 Rome, Italy
- Department of Neuroscience, Reproductive Sciences and Odontostomatology, Federico II University of Naples, 80131 Naples, Italy
| | - Arturo Brunetti
- Department of Advanced Biomedical Sciences, Federico II University of Naples, 80131 Naples, Italy
| | - Carlo Pozzilli
- Department of Human Neurosciences, Sapienza University of Rome, 00185 Rome, Italy
| | - Patrizia Pantano
- Department of Radiology, IRCCS NEUROMED, 86077 Pozzilli, Italy
- Department of Human Neurosciences, Sapienza University of Rome, 00185 Rome, Italy
| | - Silvia Tommasin
- Department of Human Neurosciences, Sapienza University of Rome, 00185 Rome, Italy
- Correspondence:
| |
Collapse
|
15
|
Wu Y, Xiang C, Jia M, Fang Y. Interpretable classifiers for prediction of disability trajectories using a nationwide longitudinal database. BMC Geriatr 2022; 22:627. [PMID: 35902789 PMCID: PMC9336105 DOI: 10.1186/s12877-022-03295-x] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/23/2021] [Accepted: 07/12/2022] [Indexed: 11/10/2022] Open
Abstract
OBJECTIVES To explore the heterogeneous disability trajectories and construct explainable machine learning models for effective prediction of long-term disability trajectories and understanding the mechanisms of predictions among the elderly Chinese at community level. METHODS This study retrospectively collected data from the Chinese Longitudinal Healthy Longevity and Happy Family Study between 2002 and 2018. A total of 4149 subjects aged 65 + in 2002 with completed activities of daily living (ADL) information for at least three waves were included. The mixed growth model was used to identify disability trajectories, and five machine learning models were further established to predict disability trajectories using epidemiological variables. An explainable approach was deployed to understand the model's decisions. RESULTS Three distinct disability trajectories, including normal class (77.3%), progressive class (15.5%), and high-onset class (7.2%), were identified for three-class prediction. The latter two were further merged into abnormal class, accompanied by normal class for two-class prediction. Machine learning, especially random forest and extreme gradient boosting achieved good performance in both two tasks. ADL, age, leisure activity, cognitive function, and blood pressure were key predictors. CONCLUSION The findings suggest that machine learning showed good performance and maybe of additional value in analyzing quality indicators in predicting disability trajectories, thereby providing basis to personalize intervention measures.
Collapse
Affiliation(s)
- Yafei Wu
- The State Key Laboratory of Molecular Vaccine and Molecular Diagnostics, School of Public Health, Xiamen University, Xiamen, 361102, Fujian, China.,National Institute for Data Science in Health and Medicine, Xiamen University, Xiamen, 361102, Fujian, China.,Key Laboratory of Health Technology Assessment of Fujian Province, School of Public Health, Xiamen University, Xiamen, 361102, Fujian, China.,School of Public Health, Xiamen University, Xiang'an Nan Road, Xiang'an District, Xiamen, 361102, Fujian, China
| | - Chaoyi Xiang
- The State Key Laboratory of Molecular Vaccine and Molecular Diagnostics, School of Public Health, Xiamen University, Xiamen, 361102, Fujian, China.,Key Laboratory of Health Technology Assessment of Fujian Province, School of Public Health, Xiamen University, Xiamen, 361102, Fujian, China.,School of Public Health, Xiamen University, Xiang'an Nan Road, Xiang'an District, Xiamen, 361102, Fujian, China
| | - Maoni Jia
- The State Key Laboratory of Molecular Vaccine and Molecular Diagnostics, School of Public Health, Xiamen University, Xiamen, 361102, Fujian, China.,Key Laboratory of Health Technology Assessment of Fujian Province, School of Public Health, Xiamen University, Xiamen, 361102, Fujian, China.,School of Public Health, Xiamen University, Xiang'an Nan Road, Xiang'an District, Xiamen, 361102, Fujian, China
| | - Ya Fang
- The State Key Laboratory of Molecular Vaccine and Molecular Diagnostics, School of Public Health, Xiamen University, Xiamen, 361102, Fujian, China. .,National Institute for Data Science in Health and Medicine, Xiamen University, Xiamen, 361102, Fujian, China. .,Key Laboratory of Health Technology Assessment of Fujian Province, School of Public Health, Xiamen University, Xiamen, 361102, Fujian, China. .,School of Public Health, Xiamen University, Xiang'an Nan Road, Xiang'an District, Xiamen, 361102, Fujian, China.
| |
Collapse
|
16
|
Motor evoked potentials for multiple sclerosis, a multiyear follow-up dataset. Sci Data 2022; 9:207. [PMID: 35577808 PMCID: PMC9110383 DOI: 10.1038/s41597-022-01335-0] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/15/2021] [Accepted: 04/14/2022] [Indexed: 12/03/2022] Open
Abstract
Multiple sclerosis (MS) is a chronic disease affecting millions of people worldwide. Through the demyelinating and axonal pathology of MS, the signal conduction in the central nervous system is affected. Evoked potential measurements allow clinicians to monitor this process and can be used for decision support. We share a dataset that contains motor evoked potential (MEP) measurements, in which the brain is stimulated and the resulting signal is measured in the hands and feet. This results in time series of 100 milliseconds long. Typically, both hands and feet are measured in one hospital visit. The dataset contains 5586 visits of 963 patients, performed in day-to-day clinical care over a period of 6 years. The dataset consists of approximately 100,000 MEP. Clinical metadata such as the expanded disability status scale, sex, and age is also available. This dataset can be used to explore the role of evoked potentials in MS research and patient care. It may also be used as a benchmark for time series analysis and predictive modelling. Measurement(s) | Abnormal upper-limb motor evoked potentials • Abnormal lower-limb motor evoked potentials • Kurtzke Expanded Disability Status Scale Clinical Classification | Technology Type(s) | Transcranial Magnetic Stimulation • performing a clinical assessment | Factor Type(s) | Date of birth • Sex • Measurement date | Sample Characteristic - Organism | Homo sapiens | Sample Characteristic - Location | Belgium |
Collapse
|
17
|
Alves P, Green E, Leavy M, Friedler H, Curhan G, Marci C, Boussios C. Validation of a machine learning approach to estimate expanded disability status scale scores for multiple sclerosis. Mult Scler J Exp Transl Clin 2022; 8:20552173221108635. [PMID: 35755008 PMCID: PMC9228644 DOI: 10.1177/20552173221108635] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/03/2022] [Accepted: 06/06/2022] [Indexed: 11/17/2022] Open
Abstract
Background Disability assessment using the Expanded Disability Status Scale (EDSS) is important to inform treatment decisions and monitor the progression of multiple sclerosis. Yet, EDSS scores are documented infrequently in electronic medical records. Objective To validate a machine learning model to estimate EDSS scores for multiple sclerosis patients using clinical notes from neurologists. Methods A machine learning model was developed to estimate EDSS scores on specific encounter dates using clinical notes from neurologist visits. The OM1 MS Registry data were used to create a training cohort of 2632 encounters and a separate validation cohort of 857 encounters, all with clinician-recorded EDSS scores. Model performance was assessed using the area under the receiver-operating-characteristic curve (AUC), positive predictive value (PPV), and negative predictive value (NPV), calculated using a binarized version of the outcome. The Spearman R and Pearson R values were calculated. The model was then applied to encounters without clinician-recorded EDSS scores in the MS Registry. Results The model had a PPV of 0.85, NPV of 0.85, and AUC of 0.91. The model had a Spearman R value of 0.75 and Pearson R value of 0.74 when evaluating performance using the continuous estimated EDSS and clinician-recorded EDSS scores. Application of the model to eligible encounters resulted in the generation of eEDSS scores for an additional 190,282 encounters from 13,249 patients. Conclusion EDSS scores can be estimated with very good performance using a machine learning model applied to clinical notes, thus increasing the utility of real-world data sources for research purposes.
Collapse
Affiliation(s)
| | - Eric Green
- Data Science, OM1, Inc., Boston, MA, USA
| | | | | | | | - Carl Marci
- Mental Health and Neuroscience, OM1, Inc., Boston, MA, USA
| | | |
Collapse
|
18
|
Swetlik C, Bove R, McGinley M. Clinical and Research Applications of the Electronic Medical Record in Multiple Sclerosis: A Narrative Review of Current Uses and Future Applications. Int J MS Care 2022; 24:287-294. [PMID: 36545651 PMCID: PMC9749832 DOI: 10.7224/1537-2073.2022-066] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
BACKGROUND The electronic medical record (EMR) has revolutionized health care workflow and delivery. It has evolved from a clinical adjunct to a multifaceted tool, with uses relevant to patient care and research. METHODS A MEDLINE literature review was conducted to identify data regarding the use of EMR for multiple sclerosis (MS) clinical care and research. RESULTS Of 282 relevant articles identified, 29 were included. A variety of EMR integrated platforms with features specific to MS have been designed, with options for documenting disease course, disability status, and treatment. Research efforts have focused on early diagnosis identification, relapse prediction, and surrogates for disability status. CONCLUSIONS The available platforms and associated research support the utility of harnessing EMR for MS care. The adoption of a core set of discrete EMR elements should be considered to support future research efforts and the ability to harmonize data across institutions.
Collapse
Affiliation(s)
- Carol Swetlik
- From the Mellen Center for Multiple Sclerosis, Cleveland Clinic, Cleveland, OH, USA (CS, MM)
| | - Riley Bove
- The UCSF Weill Institute for Neurosciences, University of California San Francisco, San Francisco, CA, USA (RB)
| | - Marisa McGinley
- From the Mellen Center for Multiple Sclerosis, Cleveland Clinic, Cleveland, OH, USA (CS, MM)
| |
Collapse
|
19
|
López-Dorado A, Ortiz M, Satue M, Rodrigo MJ, Barea R, Sánchez-Morla EM, Cavaliere C, Rodríguez-Ascariz JM, Orduna-Hospital E, Boquete L, Garcia-Martin E. Early Diagnosis of Multiple Sclerosis Using Swept-Source Optical Coherence Tomography and Convolutional Neural Networks Trained with Data Augmentation. SENSORS (BASEL, SWITZERLAND) 2021; 22:167. [PMID: 35009710 PMCID: PMC8747672 DOI: 10.3390/s22010167] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/03/2021] [Revised: 12/21/2021] [Accepted: 12/22/2021] [Indexed: 05/07/2023]
Abstract
BACKGROUND The aim of this paper is to implement a system to facilitate the diagnosis of multiple sclerosis (MS) in its initial stages. It does so using a convolutional neural network (CNN) to classify images captured with swept-source optical coherence tomography (SS-OCT). METHODS SS-OCT images from 48 control subjects and 48 recently diagnosed MS patients have been used. These images show the thicknesses (45 × 60 points) of the following structures: complete retina, retinal nerve fiber layer, two ganglion cell layers (GCL+, GCL++) and choroid. The Cohen distance is used to identify the structures and the regions within them with greatest discriminant capacity. The original database of OCT images is augmented by a deep convolutional generative adversarial network to expand the CNN's training set. RESULTS The retinal structures with greatest discriminant capacity are the GCL++ (44.99% of image points), complete retina (26.71%) and GCL+ (22.93%). Thresholding these images and using them as inputs to a CNN comprising two convolution modules and one classification module obtains sensitivity = specificity = 1.0. CONCLUSIONS Feature pre-selection and the use of a convolutional neural network may be a promising, nonharmful, low-cost, easy-to-perform and effective means of assisting the early diagnosis of MS based on SS-OCT thickness data.
Collapse
Affiliation(s)
- Almudena López-Dorado
- Biomedical Engineering Group, Department of Electronics, University of Alcalá, 28801 Alcalá de Henares, Spain; (A.L.-D.); (R.B.); (C.C.); (J.M.R.-A.)
| | - Miguel Ortiz
- Computer Vision, Imaging and Machine Intelligence Research Group, Interdisciplinary Center for Security, Reliability and Trust (SnT), University of Luxembourg, 4365 Luxembourg, Luxembourg;
| | - María Satue
- Miguel Servet Ophthalmology Innovation and Research Group (GIMSO), Department of Ophthalmology, Aragon Institute for Health Research (IIS Aragon), Miguel Servet University Hospital, University of Zaragoza, 50018 Zaragoza, Spain; (M.S.); (M.J.R.); (E.O.-H.)
| | - María J. Rodrigo
- Miguel Servet Ophthalmology Innovation and Research Group (GIMSO), Department of Ophthalmology, Aragon Institute for Health Research (IIS Aragon), Miguel Servet University Hospital, University of Zaragoza, 50018 Zaragoza, Spain; (M.S.); (M.J.R.); (E.O.-H.)
| | - Rafael Barea
- Biomedical Engineering Group, Department of Electronics, University of Alcalá, 28801 Alcalá de Henares, Spain; (A.L.-D.); (R.B.); (C.C.); (J.M.R.-A.)
| | - Eva M. Sánchez-Morla
- Department of Psychiatry, Hospital 12 de Octubre Research Institute (i+12), 28041 Madrid, Spain;
- Faculty of Medicine, Complutense University of Madrid, 28040 Madrid, Spain
- Biomedical Research Networking Centre in Mental Health (CIBERSAM), 28029 Madrid, Spain
| | - Carlo Cavaliere
- Biomedical Engineering Group, Department of Electronics, University of Alcalá, 28801 Alcalá de Henares, Spain; (A.L.-D.); (R.B.); (C.C.); (J.M.R.-A.)
| | - José M. Rodríguez-Ascariz
- Biomedical Engineering Group, Department of Electronics, University of Alcalá, 28801 Alcalá de Henares, Spain; (A.L.-D.); (R.B.); (C.C.); (J.M.R.-A.)
| | - Elvira Orduna-Hospital
- Miguel Servet Ophthalmology Innovation and Research Group (GIMSO), Department of Ophthalmology, Aragon Institute for Health Research (IIS Aragon), Miguel Servet University Hospital, University of Zaragoza, 50018 Zaragoza, Spain; (M.S.); (M.J.R.); (E.O.-H.)
| | - Luciano Boquete
- Biomedical Engineering Group, Department of Electronics, University of Alcalá, 28801 Alcalá de Henares, Spain; (A.L.-D.); (R.B.); (C.C.); (J.M.R.-A.)
| | - Elena Garcia-Martin
- Miguel Servet Ophthalmology Innovation and Research Group (GIMSO), Department of Ophthalmology, Aragon Institute for Health Research (IIS Aragon), Miguel Servet University Hospital, University of Zaragoza, 50018 Zaragoza, Spain; (M.S.); (M.J.R.); (E.O.-H.)
| |
Collapse
|