1
|
Hossain MZ, Daskalaki E, Brüstle A, Desborough J, Lueck CJ, Suominen H. The role of machine learning in developing non-magnetic resonance imaging based biomarkers for multiple sclerosis: a systematic review. BMC Med Inform Decis Mak 2022; 22:242. [PMID: 36109726 PMCID: PMC9476596 DOI: 10.1186/s12911-022-01985-5] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/15/2021] [Accepted: 09/02/2022] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND Multiple sclerosis (MS) is a neurological condition whose symptoms, severity, and progression over time vary enormously among individuals. Ideally, each person living with MS should be provided with an accurate prognosis at the time of diagnosis, precision in initial and subsequent treatment decisions, and improved timeliness in detecting the need to reassess treatment regimens. To manage these three components, discovering an accurate, objective measure of overall disease severity is essential. Machine learning (ML) algorithms can contribute to finding such a clinically useful biomarker of MS through their ability to search and analyze datasets about potential biomarkers at scale. Our aim was to conduct a systematic review to determine how, and in what way, ML has been applied to the study of MS biomarkers on data from sources other than magnetic resonance imaging. METHODS Systematic searches through eight databases were conducted for literature published in 2014-2020 on MS and specified ML algorithms. RESULTS Of the 1, 052 returned papers, 66 met the inclusion criteria. All included papers addressed developing classifiers for MS identification or measuring its progression, typically, using hold-out evaluation on subsets of fewer than 200 participants with MS. These classifiers focused on biomarkers of MS, ranging from those derived from omics and phenotypical data (34.5% clinical, 33.3% biological, 23.0% physiological, and 9.2% drug response). Algorithmic choices were dependent on both the amount of data available for supervised ML (91.5%; 49.2% classification and 42.3% regression) and the requirement to be able to justify the resulting decision-making principles in healthcare settings. Therefore, algorithms based on decision trees and support vector machines were commonly used, and the maximum average performance of 89.9% AUC was found in random forests comparing with other ML algorithms. CONCLUSIONS ML is applicable to determining how candidate biomarkers perform in the assessment of disease severity. However, applying ML research to develop decision aids to help clinicians optimize treatment strategies and analyze treatment responses in individual patients calls for creating appropriate data resources and shared experimental protocols. They should target proceeding from segregated classification of signals or natural language to both holistic analyses across data modalities and clinically-meaningful differentiation of disease.
Collapse
Affiliation(s)
- Md Zakir Hossain
- School of Computing, College of Engineering and Computer Science, Australian National University, Canberra, ACT Australia
| | - Elena Daskalaki
- School of Computing, College of Engineering and Computer Science, Australian National University, Canberra, ACT Australia
| | - Anne Brüstle
- The John Curtin School of Medical Research, College of Health and Medicine, Australian National University, Canberra, ACT Australia
| | - Jane Desborough
- Department of Health Services Research and Policy, Research School of Population Health, College of Health and Medicine, Australian National University, Canberra, ACT Australia
| | - Christian J. Lueck
- Department of Neurology, Canberra Hospital, Canberra, ACT Australia
- ANU Medical School, College of Health and Medicine, Australian National University, Canberra, ACT Australia
| | - Hanna Suominen
- School of Computing, College of Engineering and Computer Science, Australian National University, Canberra, ACT Australia
- Department of Computing, University of Turku, Turku, Finland
| |
Collapse
|
2
|
Harrison CJ, Sidey-Gibbons CJ, Klassen AF, Wong Riff KWY, Furniss D, Swan MC, Rodrigues JN. Recursive Partitioning vs Computerized Adaptive Testing to Reduce the Burden of Health Assessments in Cleft Lip and/or Palate: Comparative Simulation Study. J Med Internet Res 2021; 23:e26412. [PMID: 34328443 PMCID: PMC8367147 DOI: 10.2196/26412] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/10/2020] [Revised: 02/25/2021] [Accepted: 05/24/2021] [Indexed: 11/13/2022] Open
Abstract
BACKGROUND Computerized adaptive testing (CAT) has been shown to deliver short, accurate, and personalized versions of the CLEFT-Q patient-reported outcome measure for children and young adults born with a cleft lip and/or palate. Decision trees may integrate clinician-reported data (eg, age, gender, cleft type, and planned treatments) to make these assessments even shorter and more accurate. OBJECTIVE We aimed to create decision tree models incorporating clinician-reported data into adaptive CLEFT-Q assessments and compare their accuracy to traditional CAT models. METHODS We used relevant clinician-reported data and patient-reported item responses from the CLEFT-Q field test to train and test decision tree models using recursive partitioning. We compared the prediction accuracy of decision trees to CAT assessments of similar length. Participant scores from the full-length questionnaire were used as ground truth. Accuracy was assessed through Pearson's correlation coefficient of predicted and ground truth scores, mean absolute error, root mean squared error, and a two-tailed Wilcoxon signed-rank test comparing squared error. RESULTS Decision trees demonstrated poorer accuracy than CAT comparators and generally made data splits based on item responses rather than clinician-reported data. CONCLUSIONS When predicting CLEFT-Q scores, individual item responses are generally more informative than clinician-reported data. Decision trees that make binary splits are at risk of underfitting polytomous patient-reported outcome measure data and demonstrated poorer performance than CATs in this study.
Collapse
Affiliation(s)
- Conrad J Harrison
- Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, United Kingdom
| | - Chris J Sidey-Gibbons
- MD Anderson Center for INSPiRED Cancer Care, University of Texas, Houston, TX, United States
| | - Anne F Klassen
- Department of Pediatrics, McMaster University, Hamilton, ON, Canada
| | - Karen W Y Wong Riff
- Department of Plastic and Reconstructive Surgery, Hospital for Sick Children, Toronto, ON, Canada
| | - Dominic Furniss
- Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, United Kingdom
| | - Marc C Swan
- Spires Cleft Centre, John Radcliffe Hospital, Oxford, United Kingdom
| | - Jeremy N Rodrigues
- Warwick Clinical Trials Unit, University of Warwick, Coventry, United Kingdom.,Department of Plastic Surgery, Stoke Mandeville Hospital, Buckinghamshire Healthcare NHS Trust, Aylesbury, United Kingdom
| |
Collapse
|
3
|
Fernandes S, Fond G, Zendjidjian X, Michel P, Baumstarck K, Lancon C, Berna F, Schurhoff F, Aouizerate B, Henry C, Etain B, Samalin L, Leboyer M, Llorca PM, Coldefy M, Auquier P, Boyer L. The Patient-Reported Experience Measure for Improving qUality of care in Mental health (PREMIUM) project in France: study protocol for the development and implementation strategy. Patient Prefer Adherence 2019; 13:165-177. [PMID: 30718945 PMCID: PMC6345324 DOI: 10.2147/ppa.s172100] [Citation(s) in RCA: 16] [Impact Index Per Article: 3.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/25/2022] Open
Abstract
BACKGROUND Measuring the quality and performance of health care is a major challenge in improving the efficiency of a health system. Patient experience is one important measure of the quality of health care, and the use of patient-reported experience measures (PREMs) is recommended. The aims of this project are 1) to develop item banks of PREMs that assess the quality of health care for adult patients with psychiatric disorders (schizophrenia, bipolar disorder, and depression) and to validate computerized adaptive testing (CAT) to support the routine use of PREMs; and 2) to analyze the implementation and acceptability of the CAT among patients, professionals, and health authorities. METHODS This multicenter and cross-sectional study is based on a mixed method approach, integrating qualitative and quantitative methodologies in two main phases: 1) item bank and CAT development based on a standardized procedure, including conceptual work and definition of the domain mapping, item selection, calibration of the item bank and CAT simulations to elaborate the administration algorithm, and CAT validation; and 2) a qualitative study exploring the implementation and acceptability of the CAT among patients, professionals, and health authorities. DISCUSSION The development of a set of PREMs on quality of care in mental health that overcomes the limitations of previous works (ie, allowing national comparisons regardless of the characteristics of patients and care and based on modern testing using item banks and CAT) could help health care professionals and health system policymakers to identify strategies to improve the quality and efficiency of mental health care. TRIAL REGISTRATION NCT02491866.
Collapse
Affiliation(s)
- Sara Fernandes
- Aix-Marseille University, School of Medicine, CEReSS - Health Service Research and Quality of Life Center - EA 3279 Research Unit, Marseille, France, Email
| | - Guillaume Fond
- Aix-Marseille University, School of Medicine, CEReSS - Health Service Research and Quality of Life Center - EA 3279 Research Unit, Marseille, France, Email
| | - Xavier Zendjidjian
- Aix-Marseille University, School of Medicine, CEReSS - Health Service Research and Quality of Life Center - EA 3279 Research Unit, Marseille, France, Email
| | - Pierre Michel
- Aix-Marseille University, School of Medicine, CEReSS - Health Service Research and Quality of Life Center - EA 3279 Research Unit, Marseille, France, Email
| | - Karine Baumstarck
- Aix-Marseille University, School of Medicine, CEReSS - Health Service Research and Quality of Life Center - EA 3279 Research Unit, Marseille, France, Email
| | - Christophe Lancon
- Aix-Marseille University, School of Medicine, CEReSS - Health Service Research and Quality of Life Center - EA 3279 Research Unit, Marseille, France, Email
| | | | | | | | | | | | | | | | | | - Magali Coldefy
- Institute for Research and Information in Health Economics (IRDES), Paris, France
| | - Pascal Auquier
- Aix-Marseille University, School of Medicine, CEReSS - Health Service Research and Quality of Life Center - EA 3279 Research Unit, Marseille, France, Email
| | - Laurent Boyer
- Aix-Marseille University, School of Medicine, CEReSS - Health Service Research and Quality of Life Center - EA 3279 Research Unit, Marseille, France, Email
| |
Collapse
|