1
|
Mizuma K, Hashimoto T, Sakui S, Kuroda S. Principal quantile treatment effect estimation using principal scores. Stat Med 2024. [PMID: 39155816 DOI: 10.1002/sim.10178] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/14/2023] [Revised: 04/25/2024] [Accepted: 07/11/2024] [Indexed: 08/20/2024]
Abstract
Intercurrent events and estimands play a key role in defining the treatment effects of interest precisely. Sometimes the median or other quantiles of outcomes in a principal stratum according to potential occurrence of intercurrent events are of interest in randomized clinical trials. Naïve analyses such as those based on the observed occurrence of the intercurrent events lead to biased results. Therefore, we propose principal quantile treatment effect estimators that can nonparametrically estimate the distribution of potential outcomes by principal score weighting without relying on the exclusion restriction assumption. Our simulation studies show that the proposed method works in situations where the median or quantiles may be regarded as the preferred population-level summary over the mean. We illustrate our proposed method by using data from a randomized controlled trial conducted on patients with nonerosive reflux disease.
Collapse
Affiliation(s)
- Kotaro Mizuma
- Statistical & Quantitative Sciences, Data Science Institute, Takeda Pharmaceutical Company Limited, Osaka, Japan
| | - Takamasa Hashimoto
- Statistical & Quantitative Sciences, Data Science Institute, Takeda Pharmaceutical Company Limited, Osaka, Japan
| | - Sho Sakui
- Statistical & Quantitative Sciences, Data Science Institute, Takeda Pharmaceutical Company Limited, Osaka, Japan
| | - Shingo Kuroda
- Statistical & Quantitative Sciences, Data Science Institute, Takeda Pharmaceutical Company Limited, Osaka, Japan
| |
Collapse
|
2
|
Fierenz A, Zapf A. Current developments of the estimand concept. Pharm Stat 2024. [PMID: 38676433 DOI: 10.1002/pst.2395] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/07/2022] [Revised: 02/15/2024] [Accepted: 04/12/2024] [Indexed: 04/28/2024]
Abstract
Since the introduction of the estimand in therapeutical studies, several adaptions have been developed. This short article highlights the important aspects of the estimand concept. A literature research was conducted to identify different extensions to this framework. Different modified strategies for intercurrent events are presented, as well as examples of methods to implement the estimand in clinical studies. The article reflects that the estimand is an ongoing research field with further exploration.
Collapse
Affiliation(s)
- Alexander Fierenz
- Institute of Medical Biometry and Epidemiology, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
| | - Antonia Zapf
- Institute of Medical Biometry and Epidemiology, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
| |
Collapse
|
3
|
Kahan BC, Hindley J, Edwards M, Cro S, Morris TP. The estimands framework: a primer on the ICH E9(R1) addendum. BMJ 2024; 384:e076316. [PMID: 38262663 PMCID: PMC10802140 DOI: 10.1136/bmj-2023-076316] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 11/07/2023] [Indexed: 01/25/2024]
Affiliation(s)
- Brennan C Kahan
- MRC Clinical Trials Unit at UCL, University College London, London WC1V 6LJ, UK
| | - Joanna Hindley
- MRC Clinical Trials Unit at UCL, University College London, London WC1V 6LJ, UK
| | - Mark Edwards
- Department of Anaesthesia, University Hospital Southampton NHS Foundation Trust, Southampton, UK
- Southampton NIHR Biomedical Research Centre, University of Southampton, Southampton, UK
| | - Suzie Cro
- Imperial Clinical Trials Unit, School of Public Health, Imperial College London, London, UK
| | - Tim P Morris
- MRC Clinical Trials Unit at UCL, University College London, London WC1V 6LJ, UK
| |
Collapse
|
4
|
Lyu T, Bornkamp B, Mueller-Velten G, Schmidli H. Bayesian inference for a principal stratum estimand on recurrent events truncated by death. Biometrics 2023; 79:3792-3802. [PMID: 36647690 DOI: 10.1111/biom.13831] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/12/2022] [Accepted: 01/05/2023] [Indexed: 01/18/2023]
Abstract
Recurrent events are often important endpoints in randomized clinical trials. For example, the number of recurrent disease-related hospitalizations may be considered as a clinically meaningful endpoint in cardiovascular studies. In some settings, the recurrent event process may be terminated by an event such as death, which makes it more challenging to define and estimate a causal treatment effect on recurrent event endpoints. In this paper, we focus on the principal stratum estimand, where the treatment effect of interest on recurrent events is defined among subjects who would be alive regardless of the assigned treatment. For the estimation of the principal stratum effect in randomized clinical trials, we propose a Bayesian approach based on a joint model of the recurrent event and death processes with a frailty term accounting for within-subject correlation. We also present Bayesian posterior predictive check procedures for assessing the model fit. The proposed approaches are demonstrated in the randomized Phase III chronic heart failure trial PARAGON-HF (NCT01920711).
Collapse
Affiliation(s)
- Tianmeng Lyu
- Novartis Pharmaceuticals Corporation, East Hanover, New Jersey, USA
| | | | | | | |
Collapse
|
5
|
Seoni S, Jahmunah V, Salvi M, Barua PD, Molinari F, Acharya UR. Application of uncertainty quantification to artificial intelligence in healthcare: A review of last decade (2013-2023). Comput Biol Med 2023; 165:107441. [PMID: 37683529 DOI: 10.1016/j.compbiomed.2023.107441] [Citation(s) in RCA: 9] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/04/2023] [Revised: 08/27/2023] [Accepted: 08/29/2023] [Indexed: 09/10/2023]
Abstract
Uncertainty estimation in healthcare involves quantifying and understanding the inherent uncertainty or variability associated with medical predictions, diagnoses, and treatment outcomes. In this era of Artificial Intelligence (AI) models, uncertainty estimation becomes vital to ensure safe decision-making in the medical field. Therefore, this review focuses on the application of uncertainty techniques to machine and deep learning models in healthcare. A systematic literature review was conducted using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. Our analysis revealed that Bayesian methods were the predominant technique for uncertainty quantification in machine learning models, with Fuzzy systems being the second most used approach. Regarding deep learning models, Bayesian methods emerged as the most prevalent approach, finding application in nearly all aspects of medical imaging. Most of the studies reported in this paper focused on medical images, highlighting the prevalent application of uncertainty quantification techniques using deep learning models compared to machine learning models. Interestingly, we observed a scarcity of studies applying uncertainty quantification to physiological signals. Thus, future research on uncertainty quantification should prioritize investigating the application of these techniques to physiological signals. Overall, our review highlights the significance of integrating uncertainty techniques in healthcare applications of machine learning and deep learning models. This can provide valuable insights and practical solutions to manage uncertainty in real-world medical data, ultimately improving the accuracy and reliability of medical diagnoses and treatment recommendations.
Collapse
Affiliation(s)
- Silvia Seoni
- Biolab, PolitoBIOMedLab, Department of Electronics and Telecommunications, Politecnico di Torino, Turin, Italy
| | | | - Massimo Salvi
- Biolab, PolitoBIOMedLab, Department of Electronics and Telecommunications, Politecnico di Torino, Turin, Italy
| | - Prabal Datta Barua
- School of Business (Information System), University of Southern Queensland, Toowoomba, QLD, 4350, Australia; Faculty of Engineering and Information Technology, University of Technology Sydney, Sydney, NSW, 2007, Australia
| | - Filippo Molinari
- Biolab, PolitoBIOMedLab, Department of Electronics and Telecommunications, Politecnico di Torino, Turin, Italy.
| | - U Rajendra Acharya
- School of Mathematics, Physics and Computing, University of Southern Queensland, Springfield, Australia
| |
Collapse
|
6
|
Han S, Zhou XH. Defining estimands in clinical trials: A unified procedure. Stat Med 2023; 42:1869-1887. [PMID: 36883638 DOI: 10.1002/sim.9702] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/22/2021] [Revised: 02/09/2023] [Accepted: 02/17/2023] [Indexed: 03/09/2023]
Abstract
The ICH E9 (R1) addendum proposes five strategies to define estimands by addressing intercurrent events. However, mathematical forms of these targeted quantities are lacking, which might lead to discordance between statisticians who estimate these quantities and clinicians, drug sponsors, and regulators who interpret them. To improve the concordance, we provide a unified four-step procedure for constructing the mathematical estimands. We apply the procedure for each strategy to derive the mathematical estimands and compare the five strategies in practical interpretations, data collection, and analytical methods. Finally, we show that the procedure can help ease tasks of defining estimands in settings with multiple types of intercurrent events using two real clinical trials.
Collapse
Affiliation(s)
- Shasha Han
- School of Population Medicine and Public Health, Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing, China.,Beijing International Center for Mathematical Research, Peking University, Beijing, China
| | - Xiao-Hua Zhou
- Beijing International Center for Mathematical Research, Peking University, Beijing, China.,Department of Biostatistics, School of Public Health, Peking University, Beijing, China.,National Engineering Laboratory of Big Data Analysis and Applied Technology, Peking University, Beijing, China
| |
Collapse
|
7
|
Qu Y, Lipkovich I, Ruberg SJ. Assessing the commonly used assumptions in estimating the principal causal effect in clinical trials. Stat Biopharm Res 2023. [DOI: 10.1080/19466315.2023.2166097] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/13/2023]
Affiliation(s)
- Yongming Qu
- Department of Data and Analytics, Eli Lilly and Company, Indianapolis, Indiana, 46285, USA
| | - Ilya Lipkovich
- Department of Data and Analytics, Eli Lilly and Company, Indianapolis, Indiana, 46285, USA
| | - Stephen J. Ruberg
- Analytix Thinking, LCC, 11121 Bentgrass Court, Indianapolis, IN 46236, USA
| |
Collapse
|
8
|
Lipkovich I, Ratitch B, Qu Y, Zhang X, Shan M, Mallinckrodt C. Using principal stratification in analysis of clinical trials. Stat Med 2022; 41:3837-3877. [PMID: 35851717 DOI: 10.1002/sim.9439] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/18/2021] [Revised: 03/06/2022] [Accepted: 05/03/2022] [Indexed: 11/08/2022]
Abstract
The ICH E9(R1) addendum (2019) proposed principal stratification (PS) as one of five strategies for dealing with intercurrent events. Therefore, understanding the strengths, limitations, and assumptions of PS is important for the broad community of clinical trialists. Many approaches have been developed under the general framework of PS in different areas of research, including experimental and observational studies. These diverse applications have utilized a diverse set of tools and assumptions. Thus, need exists to present these approaches in a unifying manner. The goal of this tutorial is threefold. First, we provide a coherent and unifying description of PS. Second, we emphasize that estimation of effects within PS relies on strong assumptions and we thoroughly examine the consequences of these assumptions to understand in which situations certain assumptions are reasonable. Finally, we provide an overview of a variety of key methods for PS analysis and use a real clinical trial example to illustrate them. Examples of code for implementation of some of these approaches are given in Supplemental Materials.
Collapse
Affiliation(s)
| | | | - Yongming Qu
- Eli Lilly and Company, Indianapolis, Indiana, USA
| | - Xiang Zhang
- CSL Behring, King of Prussia, Pennsylvania, USA
| | | | | |
Collapse
|
9
|
Stensrud MJ, Dukes O. Translating questions to estimands in randomized clinical trials with intercurrent events. Stat Med 2022; 41:3211-3228. [PMID: 35578779 PMCID: PMC9321763 DOI: 10.1002/sim.9398] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/03/2021] [Revised: 03/02/2022] [Accepted: 03/14/2022] [Indexed: 11/08/2022]
Abstract
Intercurrent (post-treatment) events occur frequently in randomized trials, and investigators often express interest in treatment effects that suitably take account of these events. Contrasts that naively condition on intercurrent events do not have a straight-forward causal interpretation, and the practical relevance of other commonly used approaches is debated. In this work, we discuss how to formulate and choose an estimand, beyond the marginal intention-to-treat effect, from the point of view of a decision maker and drug developer. In particular, we argue that careful articulation of a practically useful research question should either reflect decision making at this point in time or future drug development. Indeed, a substantially interesting estimand is simply a formalization of the (plain English) description of a research question. A common feature of estimands that are practically useful is that they correspond to possibly hypothetical but well-defined interventions in identifiable (sub)populations. To illustrate our points, we consider five examples that were recently used to motivate consideration of principal stratum estimands in clinical trials. In all of these examples, we propose alternative causal estimands, such as conditional effects, sequential regime effects, and separable effects, that correspond to explicit research questions of substantial interest.
Collapse
Affiliation(s)
- Mats J. Stensrud
- Department of MathematicsÉcole Polytechnique Fédérale de LausanneLausanneSwitzerland
| | - Oliver Dukes
- Department of Statistics and Data Science, The Wharton SchoolUniversity of PennsylvaniaPhiladelphiaPennsylvaniaUSA
- Department of Applied Mathematics, Statistics and Computer ScienceGhent UniversityGhentBelgium
| |
Collapse
|
10
|
Beynon V, George IC, Elliott C, Arnold DL, Ke J, Chen H, Zhu L, Ke C, Giovannoni G, Scaramozza M, Campbell N, Bradley DP, Franchimont N, Gafson A, Belachew S. Chronic lesion activity and disability progression in secondary progressive multiple sclerosis. BMJ Neurol Open 2022; 4:e000240. [PMID: 35720980 PMCID: PMC9185385 DOI: 10.1136/bmjno-2021-000240] [Citation(s) in RCA: 15] [Impact Index Per Article: 7.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/02/2021] [Accepted: 05/15/2022] [Indexed: 11/04/2022] Open
Abstract
Objective Slowly expanding lesions (SELs), a subgroup of chronic white matter lesions that gradually expand over time, have been shown to predict disability accumulation in primary progressive multiple sclerosis (MS) disease. However, the relationships between SELs, acute lesion activity (ALA), overall chronic lesion activity (CLA) and disability progression are not well understood. In this study, we examined the ASCEND phase III clinical trial, which compared natalizumab with placebo in secondary progressive MS (SPMS). Methods Patients with complete imaging datasets between baseline and week 108 (N=600) were analysed for SEL prevalence (the number and volume of SELs), disability progression, ALA (assessed by gadolinium-enhancing lesions and new T2-hyperintense lesions) and CLA (assessed by T1-hypointense lesion volume increase within baseline T2-non-enhancing lesions identified as SELs and non-SELs). Results CLA in both SELs and non-SELs was greater in patients with SPMS with confirmed disability progression than in those with no progression. In the complete absence of ALA at baseline and on study, SEL prevalence was significantly lower, while CLA within non-SELs remained associated with disability progression. Natalizumab decreased SEL prevalence and CLA in SELs and non-SELs compared with placebo. Conclusions This study shows that CLA in patients with SPMS is decreased but persists in the absence of ALA and is associated with disability progression, highlighting the need for therapeutics targeting all mechanisms of CLA, including smouldering inflammation and neurodegeneration. Trial registration number NCT01416181.
Collapse
Affiliation(s)
- Vanessa Beynon
- Global Research & Development, Biogen, Cambridge, Massachusetts, USA
| | - Ilena C George
- Department of Neurology, Massachusetts General Hospital, Boston, Massachusetts, USA
| | | | - Douglas L Arnold
- NeuroRx Research, Montreal, Quebec, Canada.,McConnell Brain Imaging Centre, McGill University, Montreal, Quebec, Canada
| | - Jun Ke
- Biostatistics, Biogen Inc, Cambridge, Massachusetts, USA
| | - Huaihou Chen
- Biostatistics, Biogen Inc, Cambridge, Massachusetts, USA
| | - Li Zhu
- Biostatistics, Biogen Inc, Cambridge, Massachusetts, USA
| | - Chunlei Ke
- Biostatistics, Biogen Inc, Cambridge, Massachusetts, USA
| | - Gavin Giovannoni
- Neuroscience and Trauma, Barts and The London School of Medicine and Dentistry Blizard Institute, London, UK
| | | | - Nolan Campbell
- Global Medical, Biogen Inc, Cambridge, Massachusetts, USA
| | | | | | - Arie Gafson
- Digital Health, Biogen Inc, Cambridge, Massachusetts, USA
| | | |
Collapse
|
11
|
Assessing treatment benefit in the presence of placebo response using the sequential parallel comparison design. Stat Med 2022; 41:2166-2190. [DOI: 10.1002/sim.9349] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/11/2021] [Revised: 11/30/2021] [Accepted: 01/05/2022] [Indexed: 11/07/2022]
|
12
|
Luo J, Ruberg SJ, Qu Y. Estimating the treatment effect for adherers using multiple imputation. Pharm Stat 2021; 21:525-534. [PMID: 34927339 DOI: 10.1002/pst.2184] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/06/2021] [Revised: 11/23/2021] [Accepted: 11/30/2021] [Indexed: 11/07/2022]
Abstract
Randomized controlled trials are considered the gold standard to evaluate the treatment effect (estimand) for efficacy and safety. According to the recent International Council on Harmonization (ICH)-E9 addendum (R1), intercurrent events (ICEs) need to be considered when defining an estimand, and principal stratum is one of the five strategies to handle ICEs. Qu et al. (2020, Statistics in Biopharmaceutical Research 12:1-18) proposed estimators for the adherer average causal effect (AdACE) for estimating the treatment difference for those who adhere to one or both treatments based on the causal-inference framework, and demonstrated the consistency of those estimators; however, this method requires complex custom programming related to high-dimensional numeric integrations. In this article, we implemented the AdACE estimators using multiple imputation (MI) and constructed confidence intervals (CIs) through bootstrapping. A simulation study showed that the MI-based estimators provided consistent estimators with the nominal coverage probabilities of CIs for the treatment difference for the adherent populations of interest. As an illustrative example, the new method was applied to data from a real clinical trial comparing two types of basal insulin for patients with type 1 diabetes.
Collapse
Affiliation(s)
- Junxiang Luo
- Department of Biostatistics and Programming, Moderna, Inc., Cambridge, Massachusetts, USA
| | | | - Yongming Qu
- Department of Statistics, Data and Analytics, Eli Lilly and Company, Indianapolis, Indiana, USA
| |
Collapse
|
13
|
Manitz J, Kan-Dobrosky N, Buchner H, Casadebaig ML, Degtyarev E, Dey J, Haddad V, Jie F, Martin E, Mo M, Rufibach K, Shentu Y, Stalbovskaya V, Sammi Tang R, Yung G, Zhou J. Estimands for overall survival in clinical trials with treatment switching in oncology. Pharm Stat 2021; 21:150-162. [PMID: 34605168 DOI: 10.1002/pst.2158] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/21/2020] [Revised: 04/28/2021] [Accepted: 07/10/2021] [Indexed: 11/09/2022]
Abstract
An addendum of the ICH E9 guideline on Statistical Principles for Clinical Trials was released in November 2019 introducing the estimand framework. This new framework aims to align trial objectives and statistical analyses by requiring a precise definition of the inferential quantity of interest, that is, the estimand. This definition explicitly accounts for intercurrent events, such as switching to new anticancer therapies for the analysis of overall survival (OS), the gold standard in oncology. Traditionally, OS in confirmatory studies is analyzed using the intention-to-treat (ITT) approach comparing treatment groups as they were initially randomized regardless of whether treatment switching occurred and regardless of any subsequent therapy (treatment-policy strategy). Regulatory authorities and other stakeholders often consider ITT results as most relevant. However, the respective estimand only yields a clinically meaningful comparison of two treatment arms if subsequent therapies are already approved and reflect clinical practice. We illustrate different scenarios where subsequent therapies are not yet approved drugs and thus do not reflect clinical practice. In such situations the hypothetical strategy could be more meaningful from patient's and prescriber's perspective. The cross-industry Oncology Estimand Working Group (www.oncoestimand.org) was initiated to foster a common understanding and consistent implementation of the estimand framework in oncology clinical trials. This paper summarizes the group's recommendations for appropriate estimands in the presence of treatment switching, one of the key intercurrent events in oncology clinical trials. We also discuss how different choices of estimands may impact study design, data collection, trial conduct, analysis, and interpretation.
Collapse
Affiliation(s)
- Juliane Manitz
- Global Biostatistics, EMD Serono, Billerica, Massachusetts, USA
| | | | - Hannes Buchner
- Biostatistics and Data Science, Staburo GmbH, Munich, Germany
| | | | - Evgeny Degtyarev
- Clinical Development and Analytics, Novartis, Basel, Switzerland
| | - Jyotirmoy Dey
- Data and Statistical Sciences, AbbVie Inc., North Chicago, Illinois, USA
| | | | - Fei Jie
- Biostatistics and Data Management, Daiichi Sankyo Inc, Basking Ridge, New Jersey, USA
| | - Emily Martin
- Global Biostatistics, EMD Serono, Billerica, Massachusetts, USA
| | - Mindy Mo
- Oncology Clinical Statistics US, Bayer, Whippany, New Jersey, USA
| | - Kaspar Rufibach
- Methods, Collaboration, and Outreach, F. Hoffmann-La Roche Ltd, Basel, Switzerland
| | - Yue Shentu
- Biostatistics and Research Decision Sciences, Merck & Co., Inc., Kenilworth, New Jersey, USA
| | | | - Rui Sammi Tang
- Global Biometric, Servier Pharmaceuticals, Boston, Massachusetts, USA
| | - Godwin Yung
- Methods, Collaboration, and Outreach, Genentech, San Francisco, California, USA
| | | |
Collapse
|
14
|
Keene ON, Wright D, Phillips A, Wright M. Why ITT analysis is not always the answer for estimating treatment effects in clinical trials. Contemp Clin Trials 2021; 108:106494. [PMID: 34186242 PMCID: PMC8234249 DOI: 10.1016/j.cct.2021.106494] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/22/2021] [Revised: 05/25/2021] [Accepted: 06/24/2021] [Indexed: 10/25/2022]
Abstract
For many years there has been a consensus among the Clinical Research community that ITT analysis represents the correct approach for the vast majority of trials. Recent worldwide regulatory guidance for pharmaceutical industry trials has allowed discussion of alternatives to the ITT approach to analysis; different treatment effects can be considered which may be more clinically meaningful and more relevant to patients and prescribers. The key concept is of a trial "estimand", a precise description of the estimated treatment effect. The strategy chosen to account for patients who discontinue treatment or take alternative medications which are not part of the randomised treatment regimen are important determinants of this treatment effect. One strategy to account for these events is treatment policy, which corresponds to an ITT approach. Alternative equally valid strategies address what the treatment effect is if the patient actually takes the treatment or does not use specific alternative medication. There is no single right answer to which strategy is most appropriate, the solution depends on the key clinical question of interest. The estimands framework discussed in the new guidance has been particularly useful in the context of the current COVID-19 pandemic and has clarified what choices are available to account for the impact of COVID-19 on clinical trials. Specifically, an ITT approach addresses a treatment effect that may not be generalisable beyond the current pandemic.
Collapse
Affiliation(s)
- Oliver N Keene
- Biostatistics, GlaxoSmithKline Research and Development, Brentford, Middlesex, UK.
| | - David Wright
- Data Science & Artificial Intelligence, BioPharmaceuticals R&D, AstraZeneca, Cambridge, UK
| | - Alan Phillips
- Biostatistics, ICON Clinical Research UK Ltd, Marlow, Buckinghamshire, UK
| | - Melanie Wright
- Clinical Development and Analytics, Novartis Pharma AG, Basel, Switzerland
| |
Collapse
|
15
|
De Pretis F, Landes J. EA3: A softmax algorithm for evidence appraisal aggregation. PLoS One 2021; 16:e0253057. [PMID: 34138908 PMCID: PMC8211196 DOI: 10.1371/journal.pone.0253057] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/19/2021] [Accepted: 05/27/2021] [Indexed: 11/18/2022] Open
Abstract
Real World Evidence (RWE) and its uses are playing a growing role in medical research and inference. Prominently, the 21st Century Cures Act—approved in 2016 by the US Congress—permits the introduction of RWE for the purpose of risk-benefit assessments of medical interventions. However, appraising the quality of RWE and determining its inferential strength are, more often than not, thorny problems, because evidence production methodologies may suffer from multiple imperfections. The problem arises to aggregate multiple appraised imperfections and perform inference with RWE. In this article, we thus develop an evidence appraisal aggregation algorithm called EA3. Our algorithm employs the softmax function—a generalisation of the logistic function to multiple dimensions—which is popular in several fields: statistics, mathematical physics and artificial intelligence. We prove that EA3 has a number of desirable properties for appraising RWE and we show how the aggregated evidence appraisals computed by EA3 can support causal inferences based on RWE within a Bayesian decision making framework. We also discuss features and limitations of our approach and how to overcome some shortcomings. We conclude with a look ahead at the use of RWE.
Collapse
Affiliation(s)
- Francesco De Pretis
- Department of Biomedical Sciences and Public Health, School of Medicine and Surgery, Marche Polytechnic University, Ancona, Italy
- Department of Communication and Economics, University of Modena and Reggio Emilia, Reggio Emilia, Italy
- * E-mail:
| | - Jürgen Landes
- Munich Center for Mathematical Philosophy, Ludwig-Maximilians-Universität München, München, Germany
| |
Collapse
|
16
|
Ren T, Shen W, Zhang L, Zhao H. Bayesian phase II clinical trial design with noncompliance. Stat Med 2021; 40:4457-4472. [PMID: 34050539 DOI: 10.1002/sim.9041] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/03/2020] [Revised: 02/27/2021] [Accepted: 04/15/2021] [Indexed: 11/08/2022]
Abstract
Noncompliance issue is common in early phase clinical trials; and may lead to biased estimation of the intent-to-treat effect and incorrect conclusions for the clinical trial. In this work, we propose a Bayesian approach for sequentially monitoring the phase II randomized clinical trials that takes account for the noncompliance information. We adopt the principal stratification framework and propose to use Bayesian additive regression trees for selecting useful baseline covariates and estimating the complier average causal effect (CACE) for both efficacy and toxicity outcomes. The decision of early termination or not is then made adaptively based on the estimated CACE from the accumulated data. Simulation studies have confirmed the excellent performance of the proposed design in the presence of noncompliance.
Collapse
Affiliation(s)
- Tingyang Ren
- School of Statistics and Management, Shanghai University of Finance and Economics, Shanghai, China
| | - Weining Shen
- Department of Statistics, University of California, Irvine, California, USA
| | - Liwen Zhang
- School of Statistics and Management, Shanghai University of Finance and Economics, Shanghai, China
| | - Haibing Zhao
- School of Statistics and Management, Shanghai University of Finance and Economics, Shanghai, China
| |
Collapse
|
17
|
Schmidli H, Roger JH, Akacha M. Estimands for Recurrent Event Endpoints in the Presence of a Terminal Event. Stat Biopharm Res 2021. [DOI: 10.1080/19466315.2021.1895883] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/05/2023]
Affiliation(s)
| | - James H. Roger
- Medical Statistics Department, London School of Hygiene & Tropical Medicine, London, UK
| | - Mouna Akacha
- Statistical Methodology, Novartis, Basel, Switzerland, on behalf of the Recurrent Event Qualification Opinion Consortium
| |
Collapse
|
18
|
Alizadehsani R, Roshanzamir M, Hussain S, Khosravi A, Koohestani A, Zangooei MH, Abdar M, Beykikhoshk A, Shoeibi A, Zare A, Panahiazar M, Nahavandi S, Srinivasan D, Atiya AF, Acharya UR. Handling of uncertainty in medical data using machine learning and probability theory techniques: a review of 30 years (1991-2020). ANNALS OF OPERATIONS RESEARCH 2021; 339:1-42. [PMID: 33776178 PMCID: PMC7982279 DOI: 10.1007/s10479-021-04006-2] [Citation(s) in RCA: 21] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 02/23/2021] [Indexed: 05/17/2023]
Abstract
Understanding the data and reaching accurate conclusions are of paramount importance in the present era of big data. Machine learning and probability theory methods have been widely used for this purpose in various fields. One critically important yet less explored aspect is capturing and analyzing uncertainties in the data and model. Proper quantification of uncertainty helps to provide valuable information to obtain accurate diagnosis. This paper reviewed related studies conducted in the last 30 years (from 1991 to 2020) in handling uncertainties in medical data using probability theory and machine learning techniques. Medical data is more prone to uncertainty due to the presence of noise in the data. So, it is very important to have clean medical data without any noise to get accurate diagnosis. The sources of noise in the medical data need to be known to address this issue. Based on the medical data obtained by the physician, diagnosis of disease, and treatment plan are prescribed. Hence, the uncertainty is growing in healthcare and there is limited knowledge to address these problems. Our findings indicate that there are few challenges to be addressed in handling the uncertainty in medical raw data and new models. In this work, we have summarized various methods employed to overcome this problem. Nowadays, various novel deep learning techniques have been proposed to deal with such uncertainties and improve the performance in decision making.
Collapse
Affiliation(s)
- Roohallah Alizadehsani
- Institute for Intelligent Systems Research and Innovations (IISRI), Deakin University, Geelong, Australia
| | - Mohamad Roshanzamir
- Department of Computer Engineering, Faculty of Engineering, Fasa University, 74617-81189 Fasa, Iran
| | - Sadiq Hussain
- System Administrator, Dibrugarh University, Dibrugarh, Assam 786004 India
| | - Abbas Khosravi
- Institute for Intelligent Systems Research and Innovations (IISRI), Deakin University, Geelong, Australia
| | - Afsaneh Koohestani
- Institute for Intelligent Systems Research and Innovations (IISRI), Deakin University, Geelong, Australia
| | | | - Moloud Abdar
- Institute for Intelligent Systems Research and Innovations (IISRI), Deakin University, Geelong, Australia
| | - Adham Beykikhoshk
- Applied Artificial Intelligence Institute, Deakin University, Geelong, Australia
| | - Afshin Shoeibi
- Computer Engineering Department, Ferdowsi University of Mashhad, Mashhad, Iran
- Faculty of Electrical and Computer Engineering, Biomedical Data Acquisition Lab, K. N. Toosi University of Technology, Tehran, Iran
| | - Assef Zare
- Faculty of Electrical Engineering, Gonabad Branch, Islamic Azad University, Gonabad, Iran
| | - Maryam Panahiazar
- Institute for Computational Health Sciences, University of California, San Francisco, USA
| | - Saeid Nahavandi
- Institute for Intelligent Systems Research and Innovations (IISRI), Deakin University, Geelong, Australia
| | - Dipti Srinivasan
- Dept. of Electrical and Computer Engineering, National University of Singapore, Singapore, 117576 Singapore
| | - Amir F. Atiya
- Department of Computer Engineering, Faculty of Engineering, Cairo University, Cairo, 12613 Egypt
| | - U. Rajendra Acharya
- Department of Electronics and Computer Engineering, Ngee Ann Polytechnic, Singapore, Singapore
- Department of Biomedical Engineering, School of Science and Technology, Singapore University of Social Sciences, Singapore, Singapore
- Department of Bioinformatics and Medical Engineering, Asia University, Taichung, Taiwan
| |
Collapse
|
19
|
Bornkamp B, Rufibach K, Lin J, Liu Y, Mehrotra DV, Roychoudhury S, Schmidli H, Shentu Y, Wolbers M. Principal stratum strategy: Potential role in drug development. Pharm Stat 2021; 20:737-751. [PMID: 33624407 DOI: 10.1002/pst.2104] [Citation(s) in RCA: 30] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/12/2020] [Revised: 12/01/2020] [Accepted: 02/05/2021] [Indexed: 12/12/2022]
Abstract
A randomized trial allows estimation of the causal effect of an intervention compared to a control in the overall population and in subpopulations defined by baseline characteristics. Often, however, clinical questions also arise regarding the treatment effect in subpopulations of patients, which would experience clinical or disease related events post-randomization. Events that occur after treatment initiation and potentially affect the interpretation or the existence of the measurements are called intercurrent events in the ICH E9(R1) guideline. If the intercurrent event is a consequence of treatment, randomization alone is no longer sufficient to meaningfully estimate the treatment effect. Analyses comparing the subgroups of patients without the intercurrent events for intervention and control will not estimate a causal effect. This is well known, but post-hoc analyses of this kind are commonly performed in drug development. An alternative approach is the principal stratum strategy, which classifies subjects according to their potential occurrence of an intercurrent event on both study arms. We illustrate with examples that questions formulated through principal strata occur naturally in drug development and argue that approaching these questions with the ICH E9(R1) estimand framework has the potential to lead to more transparent assumptions as well as more adequate analyses and conclusions. In addition, we provide an overview of assumptions required for estimation of effects in principal strata. Most of these assumptions are unverifiable and should hence be based on solid scientific understanding. Sensitivity analyses are needed to assess robustness of conclusions.
Collapse
Affiliation(s)
- Björn Bornkamp
- Clinical Development and Analytics, Novartis, Basel, Switzerland
| | - Kaspar Rufibach
- Methods, Collaboration, and Outreach Group (MCO), Department of Biostatistics, Hoffmann-La Roche Ltd, Basel, Switzerland
| | - Jianchang Lin
- Statistical & Quantitative Sciences (SQS), Takeda Pharmaceuticals, Cambridge, Massachusetts, USA
| | - Yi Liu
- Nektar Therapeutics, San Francisco, California, USA
| | - Devan V Mehrotra
- Clinical Biostatistics, Merck & Co., Inc., North Wales, Pennsylvania, USA
| | | | - Heinz Schmidli
- Clinical Development and Analytics, Novartis, Basel, Switzerland
| | - Yue Shentu
- Merck & Co., Inc., Rahway, New Jersey, USA
| | - Marcel Wolbers
- Methods, Collaboration, and Outreach Group (MCO), Department of Biostatistics, Hoffmann-La Roche Ltd, Basel, Switzerland
| |
Collapse
|
20
|
Cree BA, Magnusson B, Rouyrre N, Fox RJ, Giovannoni G, Vermersch P, Bar-Or A, Gold R, Piani Meier D, Karlsson G, Tomic D, Wolf C, Dahlke F, Kappos L. Siponimod: Disentangling disability and relapses in secondary progressive multiple sclerosis. Mult Scler 2020; 27:1564-1576. [PMID: 33205682 PMCID: PMC8414818 DOI: 10.1177/1352458520971819] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
Background: In multiple sclerosis, impact of treatment on disability progression can be
confounded if treatment also reduces relapses. Objective: To distinguish siponimod’s direct effects on disability progression from
those on relapses in the EXPAND phase 3 trial. Methods: Three estimands, one based on principal stratum and two on hypothetical
scenarios (no relapses, or equal relapses in both treatment arms), were
defined to determine the extent to which siponimod’s effects on 3- and
6-month confirmed disability progression were independent of on-study
relapses. Results: Principal stratum analysis estimated that siponimod reduced the risk of 3-
and 6-month confirmed disability progression by 14%–20% and 29%–33%,
respectively, compared with placebo in non-relapsing patients. In the
hypothetical scenarios, risk reductions independent of relapses were 14%–18%
and 23% for 3- and 6-month confirmed disability progression,
respectively. Conclusion: By controlling the confounding impact of on-study relapses on confirmed
disability progression, these statistical approaches provide a
methodological framework to assess treatment effects on disability
progression in relapsing and non-relapsing patients. The analyses support
that siponimod may be useful for treating secondary progressive multiple
sclerosis in patients with or without relapses.
Collapse
Affiliation(s)
- Bruce Ac Cree
- Department of Neurology, UCSF Weill Institute for Neurosciences, University of California San Francisco, San Francisco, CA, USA
| | | | | | - Robert J Fox
- Mellen Center for Multiple Sclerosis Treatment and Research, Neurological Institute, Cleveland Clinic, Cleveland, OH, USA
| | - Gavin Giovannoni
- Blizard Institute, Barts and The London School of Medicine and Dentistry, Queen Mary University of London, London, UK
| | | | - Amit Bar-Or
- Center for Neuroinflammation and Experimental Therapeutics and Multiple Sclerosis Division, Department of Neurology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA/Neuroimmunology Unit, Montreal Neurological Institute and Hospital, McGill University, Montreal, QC, Canada
| | - Ralf Gold
- Department of Neurology, St. Josef-Hospital, Ruhr-University Bochum, Bochum, Germany
| | | | | | | | | | | | - Ludwig Kappos
- Neurologic Clinic and Policlinic, Departments of Medicine, Clinical Research, Biomedicine and Biomedical Engineering, University Hospital, University of Basel, Basel, Switzerland
| |
Collapse
|
21
|
Jin M, Liu G. Estimand framework: Delineating what to be estimated with clinical questions of interest in clinical trials. Contemp Clin Trials 2020; 96:106093. [PMID: 32777382 DOI: 10.1016/j.cct.2020.106093] [Citation(s) in RCA: 13] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/27/2020] [Revised: 07/21/2020] [Accepted: 07/26/2020] [Indexed: 12/01/2022]
Abstract
ICH (International Council for Harmonization) E9 R1 (2019) proposes a framework to define estimands in clinical trials. Although the concept of estimand was proposed previously when US Food and Drug Administration (FDA) issued the panel report on handling missing data in clinical trials, many details including attributes and different strategies have not been developed until the recent ICH E9 (R1) addendum. A clearly defined estimand should include considerations of five attributes including patient population, treatment regimen of interest, endpoint/variables, handling of intercurrent events (IEs), and summary measures for assessing treatment effect. To evaluate the underlying treatment effects of a new investigational drug or biologic product, it is desirable to consider estimands that are aligned with the objectives of the study and that are meaningful to the stakeholders such as physicians or patients, health authority administration, and payers, etc.. In this paper, the concepts, attributes and strategies of the estimand framework will be reviewed and illustrated with clinical trial examples. Some common estimands and their associated scientific questions are discussed within a causal inference framework for longitudinal clinical trials.
Collapse
Affiliation(s)
- Man Jin
- AbbVie Inc., North Chicago, IL, USA.
| | | |
Collapse
|
22
|
Qu Y, Luo J, Ruberg SJ. Implementation of tripartite estimands using adherence causal estimators under the causal inference framework. Pharm Stat 2020; 20:55-67. [PMID: 33442928 DOI: 10.1002/pst.2054] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/19/2020] [Revised: 05/28/2020] [Accepted: 07/01/2020] [Indexed: 11/06/2022]
Abstract
Intercurrent events (ICEs) and missing values are inevitable in clinical trials of any size and duration, making it difficult to assess the treatment effect for all patients in randomized clinical trials. Defining the appropriate estimand that is relevant to the clinical research question is the first step in analyzing data. The tripartite estimands, which evaluate the treatment differences in the proportion of patients with ICEs due to adverse events, the proportion of patients with ICEs due to lack of efficacy, and the primary efficacy outcome for those who can adhere to study treatment under the causal inference framework, are of interest to many stakeholders in understanding the totality of treatment effects. In this manuscript, we discuss the details of how to estimate tripartite estimands based on a causal inference framework and how to interpret tripartite estimates through a phase 3 clinical study evaluating a basal insulin treatment for patients with type 1 diabetes.
Collapse
Affiliation(s)
- Yongming Qu
- Department of Biometrics, Eli Lilly and Company, Indianapolis, Indiana, USA
| | - Junxiang Luo
- Biostatistics and Programming, Sanofi, Bridgewater, New Jersey, USA
| | | |
Collapse
|
23
|
Lipkovich I, Ratitch B, Mallinckrodt CH. Causal Inference and Estimands in Clinical Trials. Stat Biopharm Res 2020. [DOI: 10.1080/19466315.2019.1697739] [Citation(s) in RCA: 18] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/04/2023]
|