1
|
Din Abdul Jabbar MA, Guo L, Nag S, Guo Y, Simmons Z, Pioro EP, Ramasamy S, Yeo CJJ. Predicting amyotrophic lateral sclerosis (ALS) progression with machine learning. Amyotroph Lateral Scler Frontotemporal Degener 2024; 25:242-255. [PMID: 38052485 DOI: 10.1080/21678421.2023.2285443] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/24/2023] [Accepted: 11/14/2023] [Indexed: 12/07/2023]
Abstract
OBJECTIVE To predict ALS progression with varying observation and prediction window lengths, using machine learning (ML). METHODS We used demographic, clinical, and laboratory parameters from 5030 patients in the Pooled Resource Open-Access ALS Clinical Trials (PRO-ACT) database to model ALS disease progression as fast (at least 1.5 points decline in ALS Functional Rating Scale-Revised (ALSFRS-R) per month) or non-fast, using Extreme Gradient Boosting (XGBoost) and Bayesian Long Short Term Memory (BLSTM). XGBoost identified predictors of progression while BLSTM provided a confidence level for each prediction. RESULTS ML models achieved area under receiver-operating-characteristics curve (AUROC) of 0.570-0.748 and were non-inferior to clinician assessments. Performance was similar with observation lengths of a single visit, 3, 6, or 12 months and on a holdout validation dataset, but was better for longer prediction lengths. 21 important predictors were identified, with the top 3 being days since disease onset, past ALSFRS-R and forced vital capacity. Nonstandard predictors included phosphorus, chloride and albumin. BLSTM demonstrated higher performance for the samples about which it was most confident. Patient screening by models may reduce hypothetical Phase II/III clinical trial sizes by 18.3%. CONCLUSION Similar accuracies across ML models using different observation lengths suggest that a clinical trial observation period could be shortened to a single visit and clinical trial sizes reduced. Confidence levels provided by BLSTM gave additional information on the trustworthiness of predictions, which could aid decision-making. The identified predictors of ALS progression are potential biomarkers and therapeutic targets for further research.
Collapse
Affiliation(s)
- Muzammil Arif Din Abdul Jabbar
- Department of Physiology, Development and Neuroscience, University of Cambridge, Cambridge, UK
- Agency for Science, Technology and Research (A*STAR), Singapore, Singapore
| | - Ling Guo
- Institute for Infocomm Research (I2R), Agency for Science, Technology and Research (A*STAR), Singapore, Singapore
| | - Sonakshi Nag
- Institute for Infocomm Research (I2R), Agency for Science, Technology and Research (A*STAR), Singapore, Singapore
| | - Yang Guo
- Institute for Infocomm Research (I2R), Agency for Science, Technology and Research (A*STAR), Singapore, Singapore
| | - Zachary Simmons
- Department of Neurology, Pennsylvania State University College of Medicine, State College, PA, USA
| | - Erik P Pioro
- Department of Neurology, Feinberg School of Medicine, Northwestern University, Chicago, IL, USA
| | - Savitha Ramasamy
- Institute for Infocomm Research (I2R), Agency for Science, Technology and Research (A*STAR), Singapore, Singapore
| | - Crystal Jing Jing Yeo
- Agency for Science, Technology and Research (A*STAR), Singapore, Singapore
- Department of Neurology, Feinberg School of Medicine, Northwestern University, Chicago, IL, USA
- Lee Kong Chien School of Medicine, Imperial College London and Nanyang Technological University Singapore, Singapore, Singapore
- School of Medicine, Medical Sciences and Nutrition, University of Aberdeen, Aberdeen, UK
- National Neuroscience Institute, Singapore, Singapore
- Duke-NUS Medical School, Singapore, Singapore
| |
Collapse
|
2
|
Lawley A, Hampson R, Worrall K, Dobie G. Prescriptive Method for Optimizing Cost of Data Collection and Annotation in Machine Learning of Clinical Ultrasound. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2023; 2023:1-4. [PMID: 38082737 DOI: 10.1109/embc40787.2023.10340858] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/18/2023]
Abstract
Machine learning in medical ultrasound faces a major challenge: the prohibitive costs of producing and annotating clinical data. Optimizing the data collection and annotation will improve model training efficiency, reducing project cost and times. This paper prescribes a 2-phase method for cost optimization based on iterative accuracy/sample size predictions, and active learning for annotation optimization. METHODS Using public breast, fetal, and lung ultrasound datasets we can: Optimize data collection by statistically predicting accuracy for a desired dataset size; and optimize labeling efficiency using Active Learning, where predictions with lowest certainty were labelled manually using feedback. A practical case study on BUSI data was used to demonstrate the method prescribed in this work. RESULTS With small data subsets, ~10%, dataset size vs. final accuracy relations can be predicted with diminishing results after 50% usage. Manual annotation was reduced by ~10% using active learning to focus the annotation. CONCLUSION This led to cost reductions of 50%-66%, depending on requirements and initial cost model, on BUSI dataset with a negligible accuracy drop of 3.75% from theoretical maximums.Clinical Relevance- This work provides methodology to optimize dataset size and manual data labelling, this allows generation of cost-effective datasets, of interest to all, but particularly for financially limited trials and feasibility studies, Reducing the time burden on annotating clinicians.
Collapse
|
3
|
Huang B, Geng X, Yu Z, Zhang C, Chen Z. Dynamic effects of prognostic factors and individual survival prediction for amyotrophic lateral sclerosis disease. Ann Clin Transl Neurol 2023; 10:892-903. [PMID: 37014017 PMCID: PMC10270250 DOI: 10.1002/acn3.51771] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/27/2023] [Revised: 03/22/2023] [Accepted: 03/23/2023] [Indexed: 04/05/2023] Open
Abstract
OBJECTIVE Amyotrophic lateral sclerosis (ALS) is a neurodegenerative disease affecting motor neurons, with broad heterogeneity in disease progression and survival in different patients. Therefore, an accurate prediction model will be crucial to implement timely interventions and prolong patient survival time. METHODS A total of 1260 ALS patients from the PRO-ACT database were included in the analysis. Their demographics, clinical variables, and death reports were included. We constructed an ALS dynamic Cox model through the landmarking approach. The predictive performance of the model at different landmark time points was evaluated by calculating the area under the curve (AUC) and Brier score. RESULTS Three baseline covariates and seven time-dependent covariates were selected to construct the ALS dynamic Cox model. For better prognostic analysis, this model identified dynamic effects of treatment, albumin, creatinine, calcium, hematocrit, and hemoglobin. Its prediction performance (at all landmark time points, AUC ≥ 0.70 and Brier score ≤ 0.12) was better than that of the traditional Cox model, and it predicted the dynamic 6-month survival probability according to the longitudinal information of individual patients. INTERPRETATION We developed an ALS dynamic Cox model with ALS longitudinal clinical trial datasets as the inputs. This model can not only capture the dynamic prognostic effect of both baseline and longitudinal covariates but also make individual survival predictions in real time, which are valuable for improving the prognosis of ALS patients and providing a reference for clinicians to make clinical decisions.
Collapse
Affiliation(s)
- Baoyi Huang
- Department of Biostatistics, School of Public Health (Guangdong Provincial Key Laboratory of Tropical Disease Research)Southern Medical UniversityGuangzhouChina
| | - Xiang Geng
- Department of Biostatistics, School of Public Health (Guangdong Provincial Key Laboratory of Tropical Disease Research)Southern Medical UniversityGuangzhouChina
| | - Zhiyin Yu
- Department of Biostatistics, School of Public Health (Guangdong Provincial Key Laboratory of Tropical Disease Research)Southern Medical UniversityGuangzhouChina
| | - Chengfeng Zhang
- Department of Biostatistics, School of Public Health (Guangdong Provincial Key Laboratory of Tropical Disease Research)Southern Medical UniversityGuangzhouChina
| | - Zheng Chen
- Department of Biostatistics, School of Public Health (Guangdong Provincial Key Laboratory of Tropical Disease Research)Southern Medical UniversityGuangzhouChina
| |
Collapse
|
4
|
Artificial Intelligence Applied to clinical trials: opportunities and challenges. HEALTH AND TECHNOLOGY 2023; 13:203-213. [PMID: 36923325 PMCID: PMC9974218 DOI: 10.1007/s12553-023-00738-2] [Citation(s) in RCA: 30] [Impact Index Per Article: 30.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/05/2022] [Accepted: 02/08/2023] [Indexed: 03/06/2023]
Abstract
Background Clinical Trials (CTs) remain the foundation of safe and effective drug development. Given the evolving data-driven and personalized medicine approach in healthcare, it is imperative for companies and regulators to utilize tailored Artificial Intelligence (AI) solutions that enable expeditious and streamlined clinical research. In this paper, we identified opportunities, challenges, and potential implications of AI in CTs. Methods Following an extensive search in relevant databases and websites, we gathered publications tackling the use of AI and Machine Learning (ML) in CTs from the past 5 years in the US and Europe, including Regulatory Authorities' documents. Results Documented applications of AI commonly concern the oncology field and are mostly being applied in the area of recruitment. Main opportunities discussed aim to create efficiencies across CT activities, including the ability to reduce sample sizes, improve enrollment and conduct faster, more optimized adaptive CTs. While AI is an area of enthusiastic development, the identified challenges are ethical in nature and relate to data availability, standards, and most importantly, lack of regulatory guidance hindering the acceptance of AI tools in drug development. However, future implications are significant and are anticipated to improve the probability of success, reduce trial burden and overall, speed up research and regulatory approval. Conclusion The use of AI in CTs is in its relative infancy; however, it is a fast-evolving field. As regulators provide more guidance on the acceptability of AI in specific areas, we anticipate the scope of use to broaden and the volume of implementation to increase rapidly.
Collapse
|
5
|
Kobayakawa Y, Todaka K, Hashimoto Y, Ko S, Shiraishi W, Kishimoto J, Kira JI, Yamasaki R, Isobe N. A novel quantitative indicator for disease progression rate in amyotrophic lateral sclerosis. J Neurol Sci 2022; 442:120389. [PMID: 36041329 DOI: 10.1016/j.jns.2022.120389] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/14/2022] [Revised: 08/16/2022] [Accepted: 08/20/2022] [Indexed: 10/31/2022]
Abstract
OBJECTIVE The current study sought to develop a new indicator for disease progression rate in amyotrophic lateral sclerosis (ALS). METHODS We used a nonparametric method to score diverse patterns of decline in the percentage of predicted forced vital capacity (%FVC) in patients with ALS. This involved 6317 longitudinal %FVC data sets from 920 patients in the Pooled Resource Open-Access ALS Clinical Trials (PRO-ACT) database volunteered by PRO-ACT Consortium members. To assess the utility of the derived scores as a disease indicator, we examined changes over time, the association with prognosis, and correlation with the Risk Profile of the Treatment Research Initiative to Cure ALS (TRICALS). Our local cohort (n = 92) was used for external validation. RESULTS We derived scores ranging from 35 to 106 points to construct the FVC Decline Pattern scale (FVC-DiP). Individuals' FVC-DiP scores were determined from a single measurement of %FVC and disease duration at assessment. Although the %FVC declined over the disease course (p < 0.0001), the FVC-DiP remained relatively stable. Low FVC-DiP scores were associated with rapid disease progression. Using our cohort, we demonstrated an association between FVC-DiP and the survival prognosis, the stability of the FVC-DiP per individual, and a correlation between FVC-DiP scores and the TRICALS Risk Profile (r2 = 0.904, p < 0.0001). CONCLUSIONS FVC-DiP scores reflected patterns of declining %FVC over the natural course of ALS and indicated the disease progression rate. The FVC-DiP may enable easy assessment of disease progression patterns and could be used for assessing treatment efficacy.
Collapse
Affiliation(s)
- Yuko Kobayakawa
- Department of Neurology, Neurological Institute, Graduate School of Medical Sciences, Kyushu University, Fukuoka 812-8582, Japan; Center for Clinical and Translational Research, Kyushu University Hospital, Fukuoka 812-8582, Japan
| | - Koji Todaka
- Center for Clinical and Translational Research, Kyushu University Hospital, Fukuoka 812-8582, Japan
| | - Yu Hashimoto
- Department of Neurology, Neurological Institute, Graduate School of Medical Sciences, Kyushu University, Fukuoka 812-8582, Japan
| | - Senri Ko
- Department of Neurology, Neurological Institute, Graduate School of Medical Sciences, Kyushu University, Fukuoka 812-8582, Japan
| | - Wataru Shiraishi
- Department of Neurology, Neurological Institute, Graduate School of Medical Sciences, Kyushu University, Fukuoka 812-8582, Japan
| | - Junji Kishimoto
- Center for Clinical and Translational Research, Kyushu University Hospital, Fukuoka 812-8582, Japan
| | - Jun-Ichi Kira
- Department of Neurology, Neurological Institute, Graduate School of Medical Sciences, Kyushu University, Fukuoka 812-8582, Japan; Translational Neuroscience Center, Graduate School of Medicine, School of Pharmacy at Fukuoka, International University of Health and Welfare, Okawa, Fukuoka 831-8501, Japan; Department of Neurology, Brain and Nerve Center, Fukuoka Central Hospital, International University of Health and Welfare, Fukuoka 810-0022, Japan
| | - Ryo Yamasaki
- Department of Neurology, Neurological Institute, Graduate School of Medical Sciences, Kyushu University, Fukuoka 812-8582, Japan
| | - Noriko Isobe
- Department of Neurology, Neurological Institute, Graduate School of Medical Sciences, Kyushu University, Fukuoka 812-8582, Japan.
| | | |
Collapse
|
6
|
Fournier CN. Considerations for Amyotrophic Lateral Sclerosis (ALS) Clinical Trial Design. Neurotherapeutics 2022; 19:1180-1192. [PMID: 35819713 PMCID: PMC9275386 DOI: 10.1007/s13311-022-01271-2] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 06/28/2022] [Indexed: 11/20/2022] Open
Abstract
Thoughtful clinical trial design is critical for efficient therapeutic development, particularly in the field of amyotrophic lateral sclerosis (ALS), where trials often aim to detect modest treatment effects among a population with heterogeneous disease progression. Appropriate outcome measure selection is necessary for trials to provide decisive and informative results. Investigators must consider the outcome measure's reliability, responsiveness to detect change when change has actually occurred, clinical relevance, and psychometric performance. ALS clinical trials can also be performed more efficiently by utilizing statistical enrichment techniques. Innovations in ALS prediction models allow for selection of participants with less heterogeneity in disease progression rates without requiring a lead-in period, or participants can be stratified according to predicted progression. Statistical enrichment can reduce the needed sample size and improve study power, but investigators must find a balance between optimizing statistical efficiency and retaining generalizability of study findings to the broader ALS population. Additional progress is still needed for biomarker development and validation to confirm target engagement in ALS treatment trials. Selection of an appropriate biofluid biomarker depends on the treatment mechanism of interest, and biomarker studies should be incorporated into early phase trials. Inclusion of patients with ALS as advisors and advocates can strengthen clinical trial design and study retention, but more engagement efforts are needed to improve diversity and equity in ALS research studies. Another challenge for ALS therapeutic development is identifying ways to respect patient autonomy and improve access to experimental treatment, something that is strongly desired by many patients with ALS and ALS advocacy organizations. Expanded access programs that run concurrently to well-designed and adequately powered randomized controlled trials may provide an opportunity to broaden access to promising therapeutics without compromising scientific integrity or rushing regulatory approval of therapies without adequate proof of efficacy.
Collapse
Affiliation(s)
- Christina N Fournier
- Department of Neurology, Emory University, Atlanta, GA, USA.
- Department of Veterans Affairs, Atlanta, GA, USA.
| |
Collapse
|
7
|
Bede P, Murad A, Lope J, Li Hi Shing S, Finegan E, Chipika RH, Hardiman O, Chang KM. Phenotypic categorisation of individual subjects with motor neuron disease based on radiological disease burden patterns: A machine-learning approach. J Neurol Sci 2022; 432:120079. [PMID: 34875472 DOI: 10.1016/j.jns.2021.120079] [Citation(s) in RCA: 21] [Impact Index Per Article: 10.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/16/2021] [Revised: 11/25/2021] [Accepted: 11/29/2021] [Indexed: 12/20/2022]
Abstract
Motor neuron disease is an umbrella term encompassing a multitude of clinically heterogeneous phenotypes. The early and accurate categorisation of patients is hugely important, as MND phenotypes are associated with markedly different prognoses, progression rates, care needs and benefit from divergent management strategies. The categorisation of patients shortly after symptom onset is challenging, and often lengthy clinical monitoring is needed to assign patients to the appropriate phenotypic subgroup. In this study, a multi-class machine-learning strategy was implemented to classify 300 patients based on their radiological profile into diagnostic labels along the UMN-LMN spectrum. A comprehensive panel of cortical thickness measures, subcortical grey matter variables, and white matter integrity metrics were evaluated in a multilayer perceptron (MLP) model. Additional exploratory analyses were also carried out using discriminant function analyses (DFA). Excellent classification accuracy was achieved for amyotrophic lateral sclerosis in the testing cohort (93.7%) using the MLP model, but poor diagnostic accuracy was detected for primary lateral sclerosis (43.8%) and poliomyelitis survivors (60%). Feature importance analyses highlighted the relevance of white matter diffusivity metrics and the evaluation of cerebellar indices, cingulate measures and thalamic radiation variables to discriminate MND phenotypes. Our data suggest that radiological data from single patients may be meaningfully interpreted if large training data sets are available and the provision of diagnostic probability outcomes may be clinically useful in patients with short symptom duration. The computational interpretation of multimodal radiology datasets herald viable diagnostic, prognostic and clinical trial applications.
Collapse
Affiliation(s)
- Peter Bede
- Computational Neuroimaging Group, Biomedical Sciences Institute, Trinity College Dublin, Ireland; Pitié-Salpêtrière University Hospital, Sorbonne University, Paris, France.
| | - Aizuri Murad
- Computational Neuroimaging Group, Biomedical Sciences Institute, Trinity College Dublin, Ireland
| | - Jasmin Lope
- Computational Neuroimaging Group, Biomedical Sciences Institute, Trinity College Dublin, Ireland
| | - Stacey Li Hi Shing
- Computational Neuroimaging Group, Biomedical Sciences Institute, Trinity College Dublin, Ireland
| | - Eoin Finegan
- Computational Neuroimaging Group, Biomedical Sciences Institute, Trinity College Dublin, Ireland
| | - Rangariroyashe H Chipika
- Computational Neuroimaging Group, Biomedical Sciences Institute, Trinity College Dublin, Ireland
| | - Orla Hardiman
- Computational Neuroimaging Group, Biomedical Sciences Institute, Trinity College Dublin, Ireland
| | - Kai Ming Chang
- Computational Neuroimaging Group, Biomedical Sciences Institute, Trinity College Dublin, Ireland; Department of Electronics and Computer Science, University of Southampton, UK
| |
Collapse
|