1
|
Speiser JL, Kerr WT, Ziegler A. Common Critiques and Recommendations for Studies in Neurology Using Machine Learning Methods. Neurology 2024; 103:e209861. [PMID: 39236270 PMCID: PMC11379123 DOI: 10.1212/wnl.0000000000209861] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 09/07/2024] Open
Abstract
Machine learning (ML) methods are becoming more prevalent in the neurology literature as alternatives to traditional statistical methods to address challenges in the analysis of modern data sets. Despite the increase in the popularity of ML methods in neurology studies, some authors do not fully address all items recommended in reporting guidelines. The authors of this Research Methods article are members of the Neurology® editorial board and have reviewed many studies using ML methods. In their review reports, several critiques often appear, which could be avoided if guidance were available. In this article, we detail common critiques found in ML research studies and make recommendations for how to avoid them. The first critique involves misalignment of the study goals and the analysis conducted. The second critique focuses on ML terminology being appropriately used. Critiques 3-6 are related to the study design: justifying sample sizes and the suitability of the data set for the study goals, describing the ML analysis pipeline sufficiently, quantifying the amount of missing data and providing information about missing data handling, and including uncertainty estimates for key metrics. The seventh critique focuses on fairly describing both strengths and limitations of the ML study, including the analysis methodology and results. We provide examples in neurology for each critique and guidance on how to avoid the critique. Overall, we recommend that authors use ML-specific checklists developed by research consortia for designing and reporting studies using ML. We also recommend that authors involve both a statistician and an ML expert in work that uses ML. Although our list of critiques is not exhaustive, our recommendations should help improve the quality and rigor of ML studies. ML has great potential to revolutionize neurology, but investigators need to conduct and report the results in a way that allows readers to fully evaluate the benefits and limitations of ML approaches.
Collapse
Affiliation(s)
- Jaime L Speiser
- From the Department of Biostatistics and Data Science (J.L.S.), Wake Forest University School of Medicine, Winston-Salem, NC; Department of Neurology and Biomedical Informatics (W.T.K.), University of Pittsburgh, PA; Cardio-CARE (A.Z.), Medizincampus Davos, Switzerland; Department of Cardiology and Population Health Innovation (A.Z.), University Medical Center Hamburg-Eppendorf, Germany; and Department of Mathematics, Statistics and Computer Science (A.Z.), University of KwaZulu-Natal, Berea, Durban, South Africa
| | - Wesley T Kerr
- From the Department of Biostatistics and Data Science (J.L.S.), Wake Forest University School of Medicine, Winston-Salem, NC; Department of Neurology and Biomedical Informatics (W.T.K.), University of Pittsburgh, PA; Cardio-CARE (A.Z.), Medizincampus Davos, Switzerland; Department of Cardiology and Population Health Innovation (A.Z.), University Medical Center Hamburg-Eppendorf, Germany; and Department of Mathematics, Statistics and Computer Science (A.Z.), University of KwaZulu-Natal, Berea, Durban, South Africa
| | - Andreas Ziegler
- From the Department of Biostatistics and Data Science (J.L.S.), Wake Forest University School of Medicine, Winston-Salem, NC; Department of Neurology and Biomedical Informatics (W.T.K.), University of Pittsburgh, PA; Cardio-CARE (A.Z.), Medizincampus Davos, Switzerland; Department of Cardiology and Population Health Innovation (A.Z.), University Medical Center Hamburg-Eppendorf, Germany; and Department of Mathematics, Statistics and Computer Science (A.Z.), University of KwaZulu-Natal, Berea, Durban, South Africa
| |
Collapse
|
2
|
Kerr WT, McFarlane KN, Figueiredo Pucci G. The present and future of seizure detection, prediction, and forecasting with machine learning, including the future impact on clinical trials. Front Neurol 2024; 15:1425490. [PMID: 39055320 PMCID: PMC11269262 DOI: 10.3389/fneur.2024.1425490] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2024] [Accepted: 06/03/2024] [Indexed: 07/27/2024] Open
Abstract
Seizures have a profound impact on quality of life and mortality, in part because they can be challenging both to detect and forecast. Seizure detection relies upon accurately differentiating transient neurological symptoms caused by abnormal epileptiform activity from similar symptoms with different causes. Seizure forecasting aims to identify when a person has a high or low likelihood of seizure, which is related to seizure prediction. Machine learning and artificial intelligence are data-driven techniques integrated with neurodiagnostic monitoring technologies that attempt to accomplish both of those tasks. In this narrative review, we describe both the existing software and hardware approaches for seizure detection and forecasting, as well as the concepts for how to evaluate the performance of new technologies for future application in clinical practice. These technologies include long-term monitoring both with and without electroencephalography (EEG) that report very high sensitivity as well as reduced false positive detections. In addition, we describe the implications of seizure detection and forecasting upon the evaluation of novel treatments for seizures within clinical trials. Based on these existing data, long-term seizure detection and forecasting with machine learning and artificial intelligence could fundamentally change the clinical care of people with seizures, but there are multiple validation steps necessary to rigorously demonstrate their benefits and costs, relative to the current standard.
Collapse
Affiliation(s)
- Wesley T. Kerr
- Department of Neurology, University of Pittsburgh, Pittsburgh, PA, United States
| | | | | |
Collapse
|
3
|
Dlamini Z, Molefi T, Khanyile R, Mkhabele M, Damane B, Kokoua A, Bida M, Saini KS, Chauke-Malinga N, Luvhengo TE, Hull R. From Incidence to Intervention: A Comprehensive Look at Breast Cancer in South Africa. Oncol Ther 2024; 12:1-11. [PMID: 37910378 PMCID: PMC10881925 DOI: 10.1007/s40487-023-00248-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/20/2023] [Accepted: 09/27/2023] [Indexed: 11/03/2023] Open
Abstract
The formidable impact of breast cancer extends globally, with South Africa facing pronounced challenges, including significant disparities in breast cancer screening, treatment and survival along ethnic and socioeconomic lines. Over the last two decades, breast cancer incidence has increased and now accounts for a substantial portion of cancers in women. Ethnic disparities in terms of screening, incidence and survival exacerbate the issue, leading to delayed diagnosis among Black patients and highlighting healthcare inequities. These concerning trends underscore the urgency of enhancing breast cancer screening while mitigating treatment delays, although obstacles within the healthcare system impede progress. The intersection of breast cancer and human immunodeficiency virus (HIV) further complicates matters and particularly affects the Black population. Tackling the aforementioned disparities in breast cancer in South Africa mandates a multifaceted strategy. Robust screening efforts, particularly those targeting marginalised communities, are crucial for early detection. Concurrently, expedited treatment initiation is imperative. Addressing HIV-related complexities requires tailored interventions to ensure effective care. These multifaceted disparities require pan African research and cooperation as well as tailored interventions to enhance breast cancer care within the African region.
Collapse
Affiliation(s)
- Zodwa Dlamini
- SAMRC Precision Oncology Research Unit (PORU), DSI/NRF SARChI Chair in Precision Oncology and Cancer Prevention (POCP), Pan African Cancer Research Institute (PACRI), University of Pretoria, Pretoria, 0001, South Africa.
| | - Thulo Molefi
- SAMRC Precision Oncology Research Unit (PORU), DSI/NRF SARChI Chair in Precision Oncology and Cancer Prevention (POCP), Pan African Cancer Research Institute (PACRI), University of Pretoria, Pretoria, 0001, South Africa
- Department of Medical Oncology, Faculty of Health Sciences, Steve Biko Academic Hospital, University of Pretoria, Pretoria, 0001, South Africa
| | - Richard Khanyile
- SAMRC Precision Oncology Research Unit (PORU), DSI/NRF SARChI Chair in Precision Oncology and Cancer Prevention (POCP), Pan African Cancer Research Institute (PACRI), University of Pretoria, Pretoria, 0001, South Africa
- Department of Medical Oncology, Faculty of Health Sciences, Steve Biko Academic Hospital, University of Pretoria, Pretoria, 0001, South Africa
| | - Mahlori Mkhabele
- SAMRC Precision Oncology Research Unit (PORU), DSI/NRF SARChI Chair in Precision Oncology and Cancer Prevention (POCP), Pan African Cancer Research Institute (PACRI), University of Pretoria, Pretoria, 0001, South Africa
| | - Botle Damane
- SAMRC Precision Oncology Research Unit (PORU), DSI/NRF SARChI Chair in Precision Oncology and Cancer Prevention (POCP), Pan African Cancer Research Institute (PACRI), University of Pretoria, Pretoria, 0001, South Africa
- Department of Surgery, Steve Biko Academic Hospital, University of Pretoria, Pretoria, 0001, South Africa
| | - Alexandre Kokoua
- Laboratory of Anatomy, Experimental Surgery and Biomechanics (LANCEB), University of Félix Houphouët-Boigny, 01 BP V 166 Abidjan 01, Abidjan, Ivory Coast
| | - Meshack Bida
- SAMRC Precision Oncology Research Unit (PORU), DSI/NRF SARChI Chair in Precision Oncology and Cancer Prevention (POCP), Pan African Cancer Research Institute (PACRI), University of Pretoria, Pretoria, 0001, South Africa
- Department of Anatomical Pathology, National Health Laboratory Service (NHLS), University of Pretoria, Hatfield, 0028, South Africa
| | - Kamal S Saini
- Fortrea Inc, Durham, NC, USA
- Addenbrooke's Hospital, Cambridge University Hospitals NHS Foundation Trust, Cambridge, UK
| | - Nkhensani Chauke-Malinga
- SAMRC Precision Oncology Research Unit (PORU), DSI/NRF SARChI Chair in Precision Oncology and Cancer Prevention (POCP), Pan African Cancer Research Institute (PACRI), University of Pretoria, Pretoria, 0001, South Africa
- Department of Plastic, Reconstructive and Aesthetic Surgery, Steve Biko Academic Hospital University of Pretoria, Hatfield, 0028, South Africa
| | - Thifhelimbilu Emmanuel Luvhengo
- Department of Surgery, Charlotte Maxeke Johannesburg Academic Hospital, University of the Witwatersrand, Parktown, Johannesburg, 2193, South Africa
| | - Rodney Hull
- SAMRC Precision Oncology Research Unit (PORU), DSI/NRF SARChI Chair in Precision Oncology and Cancer Prevention (POCP), Pan African Cancer Research Institute (PACRI), University of Pretoria, Pretoria, 0001, South Africa
| |
Collapse
|
4
|
Kerr WT, McFarlane KN. Machine Learning and Artificial Intelligence Applications to Epilepsy: a Review for the Practicing Epileptologist. Curr Neurol Neurosci Rep 2023; 23:869-879. [PMID: 38060133 DOI: 10.1007/s11910-023-01318-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 10/24/2023] [Indexed: 12/08/2023]
Abstract
PURPOSE OF REVIEW Machine Learning (ML) and Artificial Intelligence (AI) are data-driven techniques to translate raw data into applicable and interpretable insights that can assist in clinical decision making. Some of these tools have extremely promising initial results, earning both great excitement and creating hype. This non-technical article reviews recent developments in ML/AI in epilepsy to assist the current practicing epileptologist in understanding both the benefits and limitations of integrating ML/AI tools into their clinical practice. RECENT FINDINGS ML/AI tools have been developed to assist clinicians in almost every clinical decision including (1) predicting future epilepsy in people at risk, (2) detecting and monitoring for seizures, (3) differentiating epilepsy from mimics, (4) using data to improve neuroanatomic localization and lateralization, and (5) tracking and predicting response to medical and surgical treatments. We also discuss practical, ethical, and equity considerations in the development and application of ML/AI tools including chatbots based on Large Language Models (e.g., ChatGPT). ML/AI tools will change how clinical medicine is practiced, but, with rare exceptions, the transferability to other centers, effectiveness, and safety of these approaches have not yet been established rigorously. In the future, ML/AI will not replace epileptologists, but epileptologists with ML/AI will replace epileptologists without ML/AI.
Collapse
Affiliation(s)
- Wesley T Kerr
- Department of Neurology, University of Pittsburgh, 3471 Fifth Ave, Kaufmann 811.22, Pittsburgh, PA, 15213, USA.
- Department of Biomedical Informatics, University of Pittsburgh, 3471 Fifth Ave, Kaufmann 811.22, Pittsburgh, PA, 15213, USA.
- Department of Neurology, Michigan Medicine, University of Michigan, Ann Arbor, MI, USA.
| | - Katherine N McFarlane
- Department of Neurology, University of Pittsburgh, 3471 Fifth Ave, Kaufmann 811.22, Pittsburgh, PA, 15213, USA
| |
Collapse
|