1
|
Kerr WT, Kok N, Reddy AS, McFarlane KN, Stern JM, Pennell PB, Stacey W, French J. Demonstration of Group-Level and Individual-Level Efficacy Using Time-to-Event Designs for Clinical Trials of Antiseizure Medications. Neurology 2024; 103:e209713. [PMID: 39052963 PMCID: PMC11271390 DOI: 10.1212/wnl.0000000000209713] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/30/2023] [Accepted: 05/24/2024] [Indexed: 07/27/2024] Open
Abstract
BACKGROUND AND OBJECTIVES Participants with treatment-resistant epilepsy who are randomized to add-on placebo and remain in a trial for the typical 3 to 5-month maintenance period may be at increased risk of adverse outcomes. A novel trial design has been suggested, time to prerandomization monthly seizure count (T-PSC), which would limit participants' time on ineffective therapy. We reanalyzed 11 completed trials to determine whether the primary efficacy conclusions at T-PSC matched each of the original, longer trials. METHODS A total of 11 double-blind, placebo-controlled trials of levetiracetam, brivaracetam, lacosamide, topiramate, and lamotrigine for either focal-onset or generalized-onset epilepsy were selected. We evaluated the group-level and individual-level efficacy of treatments including the median percent reduction (MPR) in seizure frequency and 50% responder rate (50RR) at T-PSC, time to second seizure, and time to first seizure compared with the full-length trial. RESULTS The primary efficacy conclusions of 10 of the 11 trials would have been the same with a T-PSC design compared with the traditional design (the exception of lamotrigine had a very high initial placebo response). As a proportion of the full-length effect size, 90% of the MPR and 85% of the 50RR were seen at T-PSC (95% CI 73%-113% and 65%-110%, respectively). Using the T-PSC design, the time on blinded treatment was at least 312 participant-years shorter (40% of total duration) and 142,000 seizures occurred during this time (60% of total seizures). By contrast, the time to first or second seizure designs reproduced group-level effect size, but the primary efficacy conclusions of each trial and individual-level efficacy correspondence were fair to poor. DISCUSSION These results support the use of this trial design for new epilepsy medication trials because this reanalysis of 11 randomized controlled trials demonstrated that observation until T-PSC was sufficient to demonstrate efficacy while potentially improving participant safety by reducing the time of exposure to placebo and inadequate treatment. Despite analysis of 11 trials including 3,619 participants, we did not observe a significant reduction in the group-level effect size, which is directly related to statistical power. The next step is to evaluate whether T-PSC is sufficient to evaluate safety as measured by adverse events.
Collapse
Affiliation(s)
- Wesley T Kerr
- From the Department of Neurology (W.T.K., N.K., A.S.R., K.N.M., P.B.P.), and Department of Biomedical Informatics (W.T.K.), University of Pittsburgh, PA; Department of Neurology (W.T.K., N.K., A.S.R., W.S.), University of Michigan, Ann Arbor; Department of Neurology (W.T.K., J.M.S.), University of California, Los Angeles; Department of Biomedical Engineering (W.S.), University of Michigan, Ann Arbor; and Comprehensive Epilepsy Center (J.F.), New York University Grossman School of Medicine, New York
| | - Neo Kok
- From the Department of Neurology (W.T.K., N.K., A.S.R., K.N.M., P.B.P.), and Department of Biomedical Informatics (W.T.K.), University of Pittsburgh, PA; Department of Neurology (W.T.K., N.K., A.S.R., W.S.), University of Michigan, Ann Arbor; Department of Neurology (W.T.K., J.M.S.), University of California, Los Angeles; Department of Biomedical Engineering (W.S.), University of Michigan, Ann Arbor; and Comprehensive Epilepsy Center (J.F.), New York University Grossman School of Medicine, New York
| | - Advith S Reddy
- From the Department of Neurology (W.T.K., N.K., A.S.R., K.N.M., P.B.P.), and Department of Biomedical Informatics (W.T.K.), University of Pittsburgh, PA; Department of Neurology (W.T.K., N.K., A.S.R., W.S.), University of Michigan, Ann Arbor; Department of Neurology (W.T.K., J.M.S.), University of California, Los Angeles; Department of Biomedical Engineering (W.S.), University of Michigan, Ann Arbor; and Comprehensive Epilepsy Center (J.F.), New York University Grossman School of Medicine, New York
| | - Katherine N McFarlane
- From the Department of Neurology (W.T.K., N.K., A.S.R., K.N.M., P.B.P.), and Department of Biomedical Informatics (W.T.K.), University of Pittsburgh, PA; Department of Neurology (W.T.K., N.K., A.S.R., W.S.), University of Michigan, Ann Arbor; Department of Neurology (W.T.K., J.M.S.), University of California, Los Angeles; Department of Biomedical Engineering (W.S.), University of Michigan, Ann Arbor; and Comprehensive Epilepsy Center (J.F.), New York University Grossman School of Medicine, New York
| | - John M Stern
- From the Department of Neurology (W.T.K., N.K., A.S.R., K.N.M., P.B.P.), and Department of Biomedical Informatics (W.T.K.), University of Pittsburgh, PA; Department of Neurology (W.T.K., N.K., A.S.R., W.S.), University of Michigan, Ann Arbor; Department of Neurology (W.T.K., J.M.S.), University of California, Los Angeles; Department of Biomedical Engineering (W.S.), University of Michigan, Ann Arbor; and Comprehensive Epilepsy Center (J.F.), New York University Grossman School of Medicine, New York
| | - Page B Pennell
- From the Department of Neurology (W.T.K., N.K., A.S.R., K.N.M., P.B.P.), and Department of Biomedical Informatics (W.T.K.), University of Pittsburgh, PA; Department of Neurology (W.T.K., N.K., A.S.R., W.S.), University of Michigan, Ann Arbor; Department of Neurology (W.T.K., J.M.S.), University of California, Los Angeles; Department of Biomedical Engineering (W.S.), University of Michigan, Ann Arbor; and Comprehensive Epilepsy Center (J.F.), New York University Grossman School of Medicine, New York
| | - William Stacey
- From the Department of Neurology (W.T.K., N.K., A.S.R., K.N.M., P.B.P.), and Department of Biomedical Informatics (W.T.K.), University of Pittsburgh, PA; Department of Neurology (W.T.K., N.K., A.S.R., W.S.), University of Michigan, Ann Arbor; Department of Neurology (W.T.K., J.M.S.), University of California, Los Angeles; Department of Biomedical Engineering (W.S.), University of Michigan, Ann Arbor; and Comprehensive Epilepsy Center (J.F.), New York University Grossman School of Medicine, New York
| | - Jacqueline French
- From the Department of Neurology (W.T.K., N.K., A.S.R., K.N.M., P.B.P.), and Department of Biomedical Informatics (W.T.K.), University of Pittsburgh, PA; Department of Neurology (W.T.K., N.K., A.S.R., W.S.), University of Michigan, Ann Arbor; Department of Neurology (W.T.K., J.M.S.), University of California, Los Angeles; Department of Biomedical Engineering (W.S.), University of Michigan, Ann Arbor; and Comprehensive Epilepsy Center (J.F.), New York University Grossman School of Medicine, New York
| |
Collapse
|
2
|
Kerr WT, Ngo LY, Zhu L, Patten A, Cheng JY, Reddy AS, French JA. Time to prerandomization seizure count design sufficiently assessed the safety and tolerability of perampanel for the treatment of primary generalized tonic-clonic seizures. Epilepsia 2024; 65:2412-2422. [PMID: 38864472 PMCID: PMC11325753 DOI: 10.1111/epi.18023] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/22/2024] [Revised: 05/08/2024] [Accepted: 05/08/2024] [Indexed: 06/13/2024]
Abstract
OBJECTIVE Static assignment of participants in randomized clinical trials to placebo or ineffective treatment confers risk from continued seizures. An alternative trial design of time to exceed prerandomization monthly seizure count (T-PSC) has replicated the efficacy conclusions of traditionally designed trials, with shorter exposure to placebo and ineffective treatment. Trials aim to evaluate efficacy as well as safety and tolerability; therefore, we evaluated whether this T-PSC design also could replicate the trial's safety and tolerability conclusions. METHODS We retrospectively applied the T-PSC design to analyze treatment-emergent adverse events (TEAEs) from a blinded, placebo-controlled trial of perampanel for primary generalized tonic-clonic seizures (NCT01393743). The safety analysis set consisted of 81 and 82 participants randomized to perampanel and placebo arms, respectively. We evaluated the incidences of TEAEs, treatment-related TEAEs, serious TEAEs, and TEAEs of special interest that occurred before T-PSC relative to those observed during the full-length trial. RESULTS Of the 67 and 59 participants who experienced TEAEs in the perampanel and placebo arms during full-length trial, 66 (99%) and 54 (92%) participants experienced TEAEs with onset occurring before T-PSC, respectively. When limited to treatment-related TEAEs, 55 of 56 (98%) and 32 of 37 (86%) participants reported treatment-related TEAEs that occurred before T-PSC in the perampanel and placebo arms, respectively. There were more TEAEs after T-PSC with placebo as compared to perampanel (Fisher exact odds ratio = 8.6, p = .035), which resulted in overestimation of the difference in TEAE rate. There was a numerical reduction in serious TEAEs (3/13 occurred after T-PSC, one in placebo and two in perampanel). SIGNIFICANCE Almost all TEAEs occurred before T-PSC. More treatment-related TEAEs occurred after T-PSC for participants randomized to placebo than perampanel, which may be due to either a shorter T-PSC or delayed time to TEAE for placebo.
Collapse
Affiliation(s)
- Wesley T Kerr
- Department of Neurology, University of Pittsburgh, Pittsburgh, Pennsylvania, USA
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, Pennsylvania, USA
| | | | | | | | | | - Advith S Reddy
- Department of Neurology, University of Pittsburgh, Pittsburgh, Pennsylvania, USA
| | - Jacqueline A French
- Comprehensive Epilepsy Center, New York University Grossman School of Medicine, New York, New York, USA
| |
Collapse
|
3
|
Terman SW, Kirkpatrick L, Kerr WT, Akiyama LF, Baajour W, Atilgan D, Dorotan MKC, Choi HW, French JA. Challenges and directions in epilepsy diagnostics and therapeutics: Proceedings of the 17th Epilepsy Therapies and Diagnostics Development conference. Epilepsia 2024; 65:846-860. [PMID: 38135921 PMCID: PMC11018495 DOI: 10.1111/epi.17875] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/10/2023] [Revised: 12/19/2023] [Accepted: 12/20/2023] [Indexed: 12/24/2023]
Abstract
Substantial efforts are underway toward optimizing the diagnosis, monitoring, and treatment of seizures and epilepsy. We describe preclinical programs in place for screening investigational therapeutic candidates in animal models, with particular attention to identifying and eliminating drugs that might paradoxically aggravate seizure burden. After preclinical development, we discuss challenges and solutions in the design and regulatory logistics of clinical trial execution, and efforts to develop disease biomarkers and interventions that may be not only seizure-suppressing, but also disease-modifying. As disease-modifying treatments are designed, there is clear recognition that, although seizures represent one critical therapeutic target, targeting nonseizure outcomes like cognitive development or functional outcomes requires changes to traditional designs. This reflects our increasing understanding that epilepsy is a disease with profound impact on quality of life for the patient and caregivers due to both seizures themselves and other nonseizure factors. This review examines selected key challenges and future directions in epilepsy diagnostics and therapeutics, from drug discovery to translational application.
Collapse
Affiliation(s)
- Samuel W Terman
- University of Michigan Department of Neurology, Ann Arbor, MI 48109, USA
| | - Laura Kirkpatrick
- University of Pittsburgh Department of Neurology, Pittsburgh, PA 15213, USA
- University of Pittsburgh Department of Pediatrics, Pittsburgh, PA 15213, USA
| | - Wesley T Kerr
- University of Michigan Department of Neurology, Ann Arbor, MI 48109, USA
- University of Pittsburgh Department of Neurology, Pittsburgh, PA 15213, USA
- University of Pittsburgh Department of Biomedical Informatics, Pittsburgh, PA 15213, USA
| | - Lisa F Akiyama
- University of Washington Department of Neurology, Seattle, WA 98105, USA
| | - Wadih Baajour
- University of Texas Health Science Center at Houston, Department of Neurology, Houston, TX 77030, USA
| | - Deniz Atilgan
- University of Texas Health Science Center at Houston, Department of Neurology, Houston, TX 77030, USA
| | | | - Hyoung Won Choi
- Emory University Department of Pediatrics, Division of Neurology, Atlanta, GA 30322
| | - Jacqueline A French
- NYU Grossman School of Medicine and NYU Langone Health, New York, NY 10016, USA
| |
Collapse
|
4
|
Goldenholz DM, Karoly PJ, Viana PF, Nurse E, Loddenkemper T, Schulze-Bonhage A, Vieluf S, Bruno E, Nasseri M, Richardson MP, Brinkmann BH, Westover MB. Minimum clinical utility standards for wearable seizure detectors: A simulation study. Epilepsia 2024; 65:1017-1028. [PMID: 38366862 PMCID: PMC11018505 DOI: 10.1111/epi.17917] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/12/2023] [Revised: 01/11/2024] [Accepted: 02/01/2024] [Indexed: 02/18/2024]
Abstract
OBJECTIVE Epilepsy management employs self-reported seizure diaries, despite evidence of seizure underreporting. Wearable and implantable seizure detection devices are now becoming more widely available. There are no clear guidelines about what levels of accuracy are sufficient. This study aimed to simulate clinical use cases and identify the necessary level of accuracy for each. METHODS Using a realistic seizure simulator (CHOCOLATES), a ground truth was produced, which was then sampled to generate signals from simulated seizure detectors of various capabilities. Five use cases were evaluated: (1) randomized clinical trials (RCTs), (2) medication adjustment in clinic, (3) injury prevention, (4) sudden unexpected death in epilepsy (SUDEP) prevention, and (5) treatment of seizure clusters. We considered sensitivity (0%-100%), false alarm rate (FAR; 0-2/day), and device type (external wearable vs. implant) in each scenario. RESULTS The RCT case was efficient for a wide range of wearable parameters, though implantable devices were preferred. Lower accuracy wearables resulted in subtle changes in the distribution of patients enrolled in RCTs, and therefore higher sensitivity and lower FAR values were preferred. In the clinic case, a wide range of sensitivity, FAR, and device type yielded similar results. For injury prevention, SUDEP prevention, and seizure cluster treatment, each scenario required high sensitivity and yet was minimally influenced by FAR. SIGNIFICANCE The choice of use case is paramount in determining acceptable accuracy levels for a wearable seizure detection device. We offer simulation results for determining and verifying utility for specific use case and specific wearable parameters.
Collapse
Affiliation(s)
- Daniel M Goldenholz
- Department of Neurology, Harvard Medical School, Boston, Massachusetts, USA
- Department of Neurology, Beth Israel Deaconess Medical Center, Boston, Massachusetts, USA
| | - Philippa J Karoly
- Department of Neurology, University of Melbourne, Melbourne, Victoria, Australia
| | - Pedro F Viana
- School of Neuroscience, Institute of Psychiatry, Psychology, and Neuroscience, King's College London, London, UK
- Faculty of Medicine, University of Lisbon, Lisbon, Portugal
| | - Ewan Nurse
- Seer Medical, Melbourne, Victoria, Australia
| | - Tobias Loddenkemper
- Department of Neurology, Harvard Medical School, Boston, Massachusetts, USA
- Department of Neurology, Boston Children's Hospital, Boston, Massachusetts, USA
| | - Andreas Schulze-Bonhage
- Epilepsy Center, University Medical Center Freiburg-University of Freiburg, Freiburg, Germany
| | - Solveig Vieluf
- Department of Neurology, Harvard Medical School, Boston, Massachusetts, USA
- Department of Neurology, Boston Children's Hospital, Boston, Massachusetts, USA
| | - Elisa Bruno
- School of Neuroscience, Institute of Psychiatry, Psychology, and Neuroscience, King's College London, London, UK
| | - Mona Nasseri
- Department of Neurology, Mayo Clinic, Rochester, Minnesota, USA
| | - Mark P Richardson
- School of Neuroscience, Institute of Psychiatry, Psychology, and Neuroscience, King's College London, London, UK
| | | | - M Brandon Westover
- Department of Neurology, Harvard Medical School, Boston, Massachusetts, USA
- Department of Neurology, Beth Israel Deaconess Medical Center, Boston, Massachusetts, USA
- Department of Neurology, Massachusetts General Hospital, Boston, Massachusetts, USA
- McCace Center, Boston, Massachusetts, USA
| |
Collapse
|
5
|
Goldenholz DM, Goldenholz SR, Habib S, Westover MB. Inductive reasoning with large language models: a simulated randomized controlled trial for epilepsy. MEDRXIV : THE PREPRINT SERVER FOR HEALTH SCIENCES 2024:2024.03.18.24304493. [PMID: 38562831 PMCID: PMC10984041 DOI: 10.1101/2024.03.18.24304493] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 04/04/2024]
Abstract
Importance The analysis of electronic medical records at scale to learn from clinical experience is currently very challenging. The integration of artificial intelligence (AI), specifically foundational large language models (LLMs), into an analysis pipeline may overcome some of the current limitations of modest input sizes, inaccuracies, biases, and incomplete knowledge bases. Objective To explore the effectiveness of using an LLM for generating realistic clinical data and other LLMs for summarizing and synthesizing information in a model system, simulating a randomized clinical trial (RCT) in epilepsy to demonstrate the potential of inductive reasoning via medical chart review. Design An LLM-generated simulated RCT based on a RCT for treatment with an antiseizure medication, cenobamate, including a placebo arm and a full-strength drug arm, evaluated by an LLM-based pipeline versus a human reader. Setting Simulation based on realistic seizure diaries, treatment effects, reported symptoms and clinical notes generated by LLMs with multiple different neurologist writing styles. Participants Simulated cohort of 240 patients, divided 1:1 into placebo and drug arms. Intervention Utilization of LLMs for the generation of clinical notes and for the synthesis of data from these notes, aiming to evaluate the efficacy and safety of cenobamate in seizure control either with a human evaluator or AI-pipeline. Measures The AI and human analysis focused on identifying the number of seizures, symptom reports, and treatment efficacy, with statistical analysis comparing the 50%-responder rate and median percentage change between the placebo and drug arms, as well as side effect rates in each arm. Results AI closely mirrored human analysis, demonstrating the drug's efficacy with marginal differences (<3%) in identifying both drug efficacy and reported symptoms. Conclusions and Relevance This study showcases the potential of LLMs accurately simulate and analyze clinical trials. Significantly, it highlights the ability of LLMs to reconstruct essential trial elements, identify treatment effects, and recognize reported symptoms, within a realistic clinical framework. The findings underscore the relevance of LLMs in future clinical research, offering a scalable, efficient alternative to traditional data mining methods without the need for specialized medical language training.
Collapse
Affiliation(s)
- Daniel M Goldenholz
- Department of Neurology, Harvard Medical School, Boston USA
- Department of Neurology, Beth Israel Deaconess Medical Center, Boston USA
| | - Shira R Goldenholz
- Department of Neurology, Beth Israel Deaconess Medical Center, Boston USA
| | - Sara Habib
- Department of Neurology, Harvard Medical School, Boston USA
- Department of Neurology, Beth Israel Deaconess Medical Center, Boston USA
| | - M Brandon Westover
- Department of Neurology, Harvard Medical School, Boston USA
- Department of Neurology, Beth Israel Deaconess Medical Center, Boston USA
| |
Collapse
|
6
|
Kerr WT, McFarlane KN. Machine Learning and Artificial Intelligence Applications to Epilepsy: a Review for the Practicing Epileptologist. Curr Neurol Neurosci Rep 2023; 23:869-879. [PMID: 38060133 DOI: 10.1007/s11910-023-01318-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 10/24/2023] [Indexed: 12/08/2023]
Abstract
PURPOSE OF REVIEW Machine Learning (ML) and Artificial Intelligence (AI) are data-driven techniques to translate raw data into applicable and interpretable insights that can assist in clinical decision making. Some of these tools have extremely promising initial results, earning both great excitement and creating hype. This non-technical article reviews recent developments in ML/AI in epilepsy to assist the current practicing epileptologist in understanding both the benefits and limitations of integrating ML/AI tools into their clinical practice. RECENT FINDINGS ML/AI tools have been developed to assist clinicians in almost every clinical decision including (1) predicting future epilepsy in people at risk, (2) detecting and monitoring for seizures, (3) differentiating epilepsy from mimics, (4) using data to improve neuroanatomic localization and lateralization, and (5) tracking and predicting response to medical and surgical treatments. We also discuss practical, ethical, and equity considerations in the development and application of ML/AI tools including chatbots based on Large Language Models (e.g., ChatGPT). ML/AI tools will change how clinical medicine is practiced, but, with rare exceptions, the transferability to other centers, effectiveness, and safety of these approaches have not yet been established rigorously. In the future, ML/AI will not replace epileptologists, but epileptologists with ML/AI will replace epileptologists without ML/AI.
Collapse
Affiliation(s)
- Wesley T Kerr
- Department of Neurology, University of Pittsburgh, 3471 Fifth Ave, Kaufmann 811.22, Pittsburgh, PA, 15213, USA.
- Department of Biomedical Informatics, University of Pittsburgh, 3471 Fifth Ave, Kaufmann 811.22, Pittsburgh, PA, 15213, USA.
- Department of Neurology, Michigan Medicine, University of Michigan, Ann Arbor, MI, USA.
| | - Katherine N McFarlane
- Department of Neurology, University of Pittsburgh, 3471 Fifth Ave, Kaufmann 811.22, Pittsburgh, PA, 15213, USA
| |
Collapse
|
7
|
Goldenholz DM, Goldenholz EB, Kaptchuk TJ. Quantifying and controlling the impact of regression to the mean on randomized controlled trials in epilepsy. Epilepsia 2023; 64:2635-2643. [PMID: 37505116 PMCID: PMC10592227 DOI: 10.1111/epi.17730] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/17/2023] [Revised: 07/24/2023] [Accepted: 07/25/2023] [Indexed: 07/29/2023]
Abstract
OBJECTIVE Randomized controlled trials (RCTs) in epilepsy for drug treatments are plagued by high costs. One potential remedy is to reduce placebo response via better control over regression to the mean (RTM). Here, RTM represents an initial observed seizure rate higher than the long-term average, which gradually settles closer to the average, resulting in apparent response to treatment. This study used simulation to clarify the relationship between eligibility criteria and RTM. METHODS Using a statistically realistic seizure diary simulator, the impact of RTM on placebo response and trial efficacy was explored by varying eligibility criteria for a traditional treatment phase II/III RCT for drug-resistant epilepsy. RESULTS When the baseline period was included in the eligibility criteria, increasingly larger fractions of RTM were observed (25%-47% vs. 23%-25%). Higher fractions of RTM corresponded with higher expected placebo responses (50% responder rate [RR50]: 2%-9% vs. 0%-8%) and lower statistical efficacy (RR50: 47%-67% vs. 47%-81%). The exclusion of baseline from eligibility criteria was shown to decrease the number of patients needed by roughly 30%. SIGNIFICANCE The manipulation of eligibility criteria for RCTs has a predictable and important impact on RTM, and therefore on placebo response; the difference between drug and placebo was more easily detected. This in turn impacts trial efficacy and therefore cost. This study found dramatic improvements in efficacy and cost when baseline was not included in eligibility.
Collapse
Affiliation(s)
| | | | - Ted J Kaptchuk
- Beth Israel Deaconess Medical Center, Boston, Massachusetts, USA
| |
Collapse
|
8
|
Kerr WT, Reddy AS, Seo SH, Kok N, Stacey WC, Stern JM, Pennell PB, French JA. Increasing challenges to trial recruitment and conduct over time. Epilepsia 2023; 64:2625-2634. [PMID: 37440282 PMCID: PMC10592378 DOI: 10.1111/epi.17716] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/05/2023] [Revised: 07/10/2023] [Accepted: 07/11/2023] [Indexed: 07/14/2023]
Abstract
OBJECTIVE This study was undertaken to evaluate how the challenges in the recruitment and retention of participants in clinical trials for focal onset epilepsy have changed over time. METHODS In this systematic analysis of randomized clinical trials of adjunct antiseizure medications for medication-resistant focal onset epilepsy, we evaluated how the numbers of participants, sites, and countries have changed since the first such trial in 1990. We also evaluated the proportion of participants who completed each trial phase and their reasons for early trial exit. We analyzed these trends using mixed effects generalized linear models accounting for the influence of the number of trial sites and trial-specific variability. RESULTS The number of participants per site has steadily decreased over decades, with recent trials recruiting fewer than five participants per site (reduction by .16 participants/site/year, p < .0001). Fewer participants also progressed from recruitment to randomization over time (odds ratio = .94/year, p = .014). Concurrently, there has been an increase in the placebo response over time (increase in median percent reduction of .4%/year, p = .02; odds ratio of increase in 50% responder rate of 1.03/year, p = .02), which was not directly associated with the number of sites per trial (p > .20). SIGNIFICANCE This historical analysis highlights the increasing challenges with participant recruitment and retention, as well as increasing placebo response. It serves as a call to action to change clinical trial design to address these challenges.
Collapse
Affiliation(s)
- Wesley T. Kerr
- Department of Neurology, University of Michigan, Ann Arbor, Michigan, USA
- Department of Neurology, University of Pittsburgh, Pittsburgh, Pennsylvania, USA
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, Pennsylvania, USA
| | - Advith S. Reddy
- Department of Neurology, University of Michigan, Ann Arbor, Michigan, USA
| | - Sung Hyun Seo
- Department of Neurology, University of Michigan, Ann Arbor, Michigan, USA
| | - Neo Kok
- Department of Neurology, University of Michigan, Ann Arbor, Michigan, USA
| | - William C. Stacey
- Department of Neurology, University of Michigan, Ann Arbor, Michigan, USA
- Department of Biomedical Engineering, University of Michigan, Ann Arbor, Michigan, USA
| | - John M. Stern
- Department of Neurology, David Geffen School of Medicine at the University of California, Los Angeles, Los Angeles, California, USA
| | - Page B. Pennell
- Department of Neurology, University of Pittsburgh, Pittsburgh, Pennsylvania, USA
| | | |
Collapse
|
9
|
Kerr WT, Auvin S, Van der Geyten S, Kenney C, Novak G, Fountain NB, Grzeskowiak C, French JA. Time-to-event clinical trial designs: Existing evidence and remaining concerns. Epilepsia 2023; 64:1699-1708. [PMID: 37073881 PMCID: PMC10524279 DOI: 10.1111/epi.17621] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/21/2023] [Revised: 04/17/2023] [Accepted: 04/18/2023] [Indexed: 04/20/2023]
Abstract
Well-designed placebo-controlled clinical trials are critical to the development of novel treatments for epilepsy, but their design has not changed for decades. Patients, clinicians, regulators, and innovators all have concerns that recruiting for trials is challenging, in part, due to the static design of maintaining participants for long periods on add-on placebo when there are an increasing number of options for therapy. A traditional trial maintains participants on blinded treatment for a static period (e.g., 12 weeks of maintenance), during which participants on placebo have an elevated risk of sudden unexpected death in epilepsy compared to patients on an active treatment. Time-to-event trials observe participants on blinded treatment until a key event occurs (e.g., post-randomization seizure count matches pre-randomization monthly seizure count). In this article, we review the evidence for these designs based on re-analysis of prior trials, one published trial that used a time-to-second seizure design, and experience from an ongoing blinded trial. We also discuss remaining concerns regarding time-to-event trials. We conclude that, despite potential limitations, time-to-event trials are a potential promising mechanism to make trials more patient friendly and reduce placebo exposure, which are urgent needs to improve safety and increase recruitment to trials.
Collapse
Affiliation(s)
- Wesley T. Kerr
- Department of Neurology, Michigan Medicine, University of Michigan, Ann Arbor, Michigan, USA
| | - Stéphane Auvin
- Assistance Publique-Hôpitaux de Paris, Service de Neurologie Pédiatrique, Hôpital Robert Debré, Paris, France
- Université Paris-Cité, Institut National de la Santé et de la Recherche Médicale NeuroDiderot, Paris, France
- Institut Universitaire de France, Paris, France
| | - Serge Van der Geyten
- Janssen Research & Development, a Division of Janssen Pharmaceutica NV, Beerse, Belgium
| | | | - Gerald Novak
- Janssen Research & Development, a Division of Janssen Pharmaceutica NV, Beerse, Belgium
| | - Nathan B. Fountain
- Department of Neurology, University of Virginia, Charlottesville, Virginia, USA
| | - Caitlin Grzeskowiak
- Research and New Therapies Program, Epilepsy Foundation of America, Maryland, USA
| | - Jacqueline A. French
- Comprehensive Epilepsy Center, New York University, New York City, New York, USA
| |
Collapse
|
10
|
Goldenholz DM, Westover MB. Flexible realistic simulation of seizure occurrence recapitulating statistical properties of seizure diaries. Epilepsia 2023; 64:396-405. [PMID: 36401798 PMCID: PMC9905290 DOI: 10.1111/epi.17471] [Citation(s) in RCA: 10] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/11/2022] [Revised: 11/12/2022] [Accepted: 11/17/2022] [Indexed: 11/21/2022]
Abstract
OBJECTIVE A realistic seizure diary simulator is currently unavailable for many research needs, including clinical trial analysis and evaluation of seizure detection and seizure-forecasting tools. In recent years, important statistical features of seizure diaries have been characterized. These include (1) heterogeneity of individual seizure frequencies, (2) the relation between average seizure rate and standard deviation, (3) multiple risk cycles, (4) seizure clusters, and (5) limitations on inter-seizure intervals. The present study unifies these features into a single model. METHODS Our approach, Cyclic Heterogeneous Overdispersed Clustered Open-source L-relationship Adjustable Temporally limited E-diary Simulator (CHOCOLATES) is based on a hierarchical model centered on a gamma Poisson generator with several modifiers. This model accounts for the aforementioned statistical properties. The model was validated by simulating 10 000 randomized clinical trials (RCTs) of medication to compare with 23 historical RCTs. Metrics of 50% responder rate (RR50) and median percent change (MPC) were evaluated. We also used CHOCOLATES as input to a seizure-forecasting tool to test the flexibility of the model. We examined the area under the receiver-operating characteristic (ROC) curve (AUC) for test data with and without cycles and clusters. RESULTS The model recapitulated typical findings in 23 historical RCTs without the necessity of introducing an additional "placebo effect." The model produced the following RR50 values: placebo: 17 ± 4%; drug 38 ± 5%; and the following MPC values: placebo: 13 ± 6%; drug 40 ± 4%. These values are similar to historical data: for RR50: placebo, 21 ± 10%, drug: 43 ± 13%; and for MPC: placebo: 17 ± 10%, drug: 41 ± 11%. The seizure forecasts achieved an AUC of 0.68 with cycles and clusters, whereas without them the AUC was 0.51. SIGNIFICANCE CHOCOLATES represents the most realistic seizure occurrence simulator to date, based on observations from thousands of patients in different contexts. This tool is open source and flexible, and can be used for many applications, including clinical trial simulation and testing of seizure-forecasting tools.
Collapse
Affiliation(s)
- Daniel M. Goldenholz
- Dept. of Neurology, Beth Israel Deaconess Medical Center, Boston 02215 MA
- Dept. of Neurology, Harvard Medical School, Boston 02215 MA
| | - M. Brandon Westover
- Dept. of Neurology, Beth Israel Deaconess Medical Center, Boston 02215 MA
- Dept. of Neurology, Harvard Medical School, Boston 02215 MA
- Dept. of Neurology, Massachusetts General Hospital, Boston 02114 MA
- McCance Center for Brain Health, Boston, 02114 MA
| |
Collapse
|
11
|
Kerr WT, Brandt C, Ngo LY, Patten A, Cheng JY, Kramer L, French JA. Time to exceed pre-randomization monthly seizure count for perampanel in participants with primary generalized tonic-clonic seizures: A potential clinical end point. Epilepsia 2022; 63:2994-3004. [PMID: 36106379 PMCID: PMC9828687 DOI: 10.1111/epi.17411] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/08/2022] [Revised: 09/13/2022] [Accepted: 09/13/2022] [Indexed: 01/12/2023]
Abstract
OBJECTIVE To evaluate the exploratory time to exceed pre-randomization seizure count (T-PSC) in the determination of efficacy of adjunctive perampanel in participants with primary generalized tonic-clonic (PGTC) seizures in generalized-onset epilepsy. METHODS In this multicenter, double-blind study (ClinicalTrials.gov identifier: NCT01393743), participants ≥12 years of age with treatment-resistant idiopathic generalized epilepsy were randomized to receive placebo or adjunctive perampanel (≤8 mg/day) across a 17-week double-blind treatment phase (4-week titration; 13-week maintenance). We evaluated the pre-planned exploratory end point of the T-PSC using a Kaplan-Meier analysis. We also re-evaluated the correspondence of the primary end points of median percent seizure frequency change (MPC) and 50% responder rate (50RR) calculated at T-PSC and at the end of the trial. RESULTS The exploratory end point of median T-PSC on placebo was 43 days and >120 days on perampanel (log-rank p < .001). The primary end points calculated at T-PSC did not differ significantly from the end points at the end of the trial (MPC -31% vs -42% at T-PSC; 50RR 32% vs 51% at T-PSC). After T-PSC was reached, participants had a median (interquartile range) of 5 (3-13) additional seizures on placebo and 5 (2-10) on perampanel. SIGNIFICANCE The exploratory end point of T-PSC demonstrated the effectiveness of perampanel despite a shorter duration of monitoring. The seizures that occurred after T-PSC did not influence the conclusions of the trial; therefore, T-PSC may be a viable alternative to traditional trial end points that reduces the risk to participants.
Collapse
Affiliation(s)
- Wesley T. Kerr
- Department of NeurologyUniversity of MichiganAnn ArborMichiganUSA
| | - Christian Brandt
- Bethel Epilepsy CenterUniversity Hospital for EpileptologyBielefeldGermany
| | - Leock Y. Ngo
- Department of NeurologyEisai Inc.NutleyNew JerseyUSA
| | | | | | - Lynn Kramer
- Department of NeurologyEisai Inc.NutleyNew JerseyUSA
| | | |
Collapse
|
12
|
Romero J, Chiang S, Goldenholz DM. Can machine learning improve randomized clinical trial analysis? Seizure 2021; 91:499-502. [PMID: 34365104 DOI: 10.1016/j.seizure.2021.07.033] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/10/2021] [Revised: 07/08/2021] [Accepted: 07/30/2021] [Indexed: 11/16/2022] Open
Abstract
PURPOSE Recently a realistic simulator of patient seizure diaries was developed that can reproduce effects seen in randomized clinical trials (RCTs). RCTs suffer from high costs and statistical inefficiencies. Using realistic simulation and machine learning this study aimed to identify a more statistically efficient outcome metric. METHODS Five candidate deep learning architectures with 54 permutations of hyperparameters were compared to the traditional standard, median percent change (MPC). Each were also tested for type 1 error. All models had similar outcomes, with appropriate low levels of type 1 error. RESULTS The simplest model was equivalent to a logistic regression of a histogram of individual percentage changes in seizure rate, requiring 21-22% less patients to discriminate drug from placebo at 90% power. This model was referred to as LPC. CONCLUSION Future studies to validate LPC may enable faster, cheaper and more efficient clinical trials.
Collapse
Affiliation(s)
- Juan Romero
- Harvard Beth Israel Deaconess Medical Center, Boston MA, United States.
| | - Sharon Chiang
- University of California San Francisco, San Francisco, CA, United States.
| | | |
Collapse
|
13
|
Romero J, Goldenholz DM. Statistical efficiency of patient data in randomized clinical trials of epilepsy treatments. Epilepsia 2020; 61:1659-1667. [DOI: 10.1111/epi.16609] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/19/2020] [Revised: 06/17/2020] [Accepted: 06/17/2020] [Indexed: 01/28/2023]
Affiliation(s)
- Juan Romero
- Neurology Beth Israel Deaconess Medical Center Boston Massachusetts
| | | |
Collapse
|