1
|
Moran JL, Linden A. Problematic meta-analyses: Bayesian and frequentist perspectives on combining randomized controlled trials and non-randomized studies. BMC Med Res Methodol 2024; 24:99. [PMID: 38678213 PMCID: PMC11056075 DOI: 10.1186/s12874-024-02215-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/25/2022] [Accepted: 04/10/2024] [Indexed: 04/29/2024] Open
Abstract
PURPOSE In the literature, the propriety of the meta-analytic treatment-effect produced by combining randomized controlled trials (RCT) and non-randomized studies (NRS) is questioned, given the inherent confounding in NRS that may bias the meta-analysis. The current study compared an implicitly principled pooled Bayesian meta-analytic treatment-effect with that of frequentist pooling of RCT and NRS to determine how well each approach handled the NRS bias. MATERIALS & METHODS Binary outcome Critical-Care meta-analyses, reflecting the importance of such outcomes in Critical-Care practice, combining RCT and NRS were identified electronically. Bayesian pooled treatment-effect and 95% credible-intervals (BCrI), posterior model probabilities indicating model plausibility and Bayes-factors (BF) were estimated using an informative heavy-tailed heterogeneity prior (half-Cauchy). Preference for pooling of RCT and NRS was indicated for Bayes-factors > 3 or < 0.333 for the converse. All pooled frequentist treatment-effects and 95% confidence intervals (FCI) were re-estimated using the popular DerSimonian-Laird (DSL) random effects model. RESULTS Fifty meta-analyses were identified (2009-2021), reporting pooled estimates in 44; 29 were pharmaceutical-therapeutic and 21 were non-pharmaceutical therapeutic. Re-computed pooled DSL FCI excluded the null (OR or RR = 1) in 86% (43/50). In 18 meta-analyses there was an agreement between FCI and BCrI in excluding the null. In 23 meta-analyses where FCI excluded the null, BCrI embraced the null. BF supported a pooled model in 27 meta-analyses and separate models in 4. The highest density of the posterior model probabilities for 0.333 < Bayes factor < 1 was 0.8. CONCLUSIONS In the current meta-analytic cohort, an integrated and multifaceted Bayesian approach gave support to including NRS in a pooled-estimate model. Conversely, caution should attend the reporting of naïve frequentist pooled, RCT and NRS, meta-analytic treatment effects.
Collapse
Affiliation(s)
- John L Moran
- The Queen Elizabeth Hospital, Woodville, SA, 5011, Australia.
| | - Ariel Linden
- Department of Medicine, School of Medicine, University of California, San Francisco, USA
| |
Collapse
|
2
|
Evrenoglou T, Metelli S, Thomas JS, Siafis S, Turner RM, Leucht S, Chaimani A. Sharing information across patient subgroups to draw conclusions from sparse treatment networks. Biom J 2024; 66:e2200316. [PMID: 38637311 DOI: 10.1002/bimj.202200316] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/26/2022] [Revised: 11/07/2023] [Accepted: 12/26/2023] [Indexed: 04/20/2024]
Abstract
Network meta-analysis (NMA) usually provides estimates of the relative effects with the highest possible precision. However, sparse networks with few available studies and limited direct evidence can arise, threatening the robustness and reliability of NMA estimates. In these cases, the limited amount of available information can hamper the formal evaluation of the underlying NMA assumptions of transitivity and consistency. In addition, NMA estimates from sparse networks are expected to be imprecise and possibly biased as they rely on large-sample approximations that are invalid in the absence of sufficient data. We propose a Bayesian framework that allows sharing of information between two networks that pertain to different population subgroups. Specifically, we use the results from a subgroup with a lot of direct evidence (a dense network) to construct informative priors for the relative effects in the target subgroup (a sparse network). This is a two-stage approach where at the first stage, we extrapolate the results of the dense network to those expected from the sparse network. This takes place by using a modified hierarchical NMA model where we add a location parameter that shifts the distribution of the relative effects to make them applicable to the target population. At the second stage, these extrapolated results are used as prior information for the sparse network. We illustrate our approach through a motivating example of psychiatric patients. Our approach results in more precise and robust estimates of the relative effects and can adequately inform clinical practice in presence of sparse networks.
Collapse
Affiliation(s)
- Theodoros Evrenoglou
- Center of Research in Epidemiology and Statistics (CRESS-U1153), Université Paris Cité, INSERM, Paris, France
| | - Silvia Metelli
- Center of Research in Epidemiology and Statistics (CRESS-U1153), Université Paris Cité, INSERM, Paris, France
| | - Johannes-Schneider Thomas
- Department of Psychiatry and Psychotherapy, School of Medicine, Technical University of Munich, Munchen, Germany
| | - Spyridon Siafis
- Department of Psychiatry and Psychotherapy, School of Medicine, Technical University of Munich, Munchen, Germany
| | | | - Stefan Leucht
- Department of Psychiatry and Psychotherapy, School of Medicine, Technical University of Munich, Munchen, Germany
| | - Anna Chaimani
- Center of Research in Epidemiology and Statistics (CRESS-U1153), Université Paris Cité, INSERM, Paris, France
| |
Collapse
|
3
|
Götte H, Kirchner M, Krisam J, Allignol A, Schüler A, Kieser M. Estimation of treatment effects in early-phase randomized clinical trials involving external control data. J Biopharm Stat 2023:1-20. [PMID: 37823377 DOI: 10.1080/10543406.2023.2256835] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/15/2022] [Accepted: 09/04/2023] [Indexed: 10/13/2023]
Abstract
There are good reasons to perform a randomized controlled trial (RCT) even in early phases of clinical development. However, the low sample sizes in those settings lead to high variability of the treatment effect estimate. The variability could be reduced by adding external control data if available. For the common setting of suitable subject-level control group data only available from one external (clinical trial or real-world) data source, we evaluate different analysis options for estimating the treatment effect via hazard ratios. The impact of the external control data is usually guided by the level of similarity with the current RCT data. Such level of similarity can be determined via outcome and/or baseline covariate data comparisons. We provide an overview over existing methods, propose a novel option for a combined assessment of outcome and baseline data, and compare a selected set of approaches in a simulation study under varying assumptions regarding observable and unobservable confounder distributions using a time-to-event model. Our various simulation scenarios also reflect the differences between external clinical trial and real-world data. Data combinations via simple outcome-based borrowing or simple propensity score weighting with baseline covariate data are not recommended. Analysis options which conflate outcome and baseline covariate data perform best in our simulation study.
Collapse
Affiliation(s)
- Heiko Götte
- Global Biostatistics, Merck Healthcare KGaA, Darmstadt, Germany
| | - Marietta Kirchner
- Institute of Medical Biometry, University of Heidelberg, Heidelberg, Germany
| | - Johannes Krisam
- Institute of Medical Biometry, University of Heidelberg, Heidelberg, Germany
| | - Arthur Allignol
- HTA and Medical Affairs, Daiichi Sankyo Europe GmbH, Munich, Germany
| | - Armin Schüler
- Global Biostatistics, Merck Healthcare KGaA, Darmstadt, Germany
| | - Meinhard Kieser
- Institute of Medical Biometry, University of Heidelberg, Heidelberg, Germany
| |
Collapse
|
4
|
Röver C, Sturtz S, Lilienthal J, Bender R, Friede T. Summarizing empirical information on between-study heterogeneity for Bayesian random-effects meta-analysis. Stat Med 2023; 42:2439-2454. [PMID: 37005007 DOI: 10.1002/sim.9731] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/25/2022] [Revised: 11/16/2022] [Accepted: 03/18/2023] [Indexed: 04/04/2023]
Abstract
In Bayesian meta-analysis, the specification of prior probabilities for the between-study heterogeneity is commonly required, and is of particular benefit in situations where only few studies are included. Among the considerations in the set-up of such prior distributions, the consultation of available empirical data on a set of relevant past analyses sometimes plays a role. How exactly to summarize historical data sensibly is not immediately obvious; in particular, the investigation of an empirical collection of heterogeneity estimates will not target the actual problem and will usually only be of limited use. The commonly used normal-normal hierarchical model for random-effects meta-analysis is extended to infer a heterogeneity prior. Using an example data set, we demonstrate how to fit a distribution to empirically observed heterogeneity data from a set of meta-analyses. Considerations also include the choice of a parametric distribution family. Here, we focus on simple and readily applicable approaches to then translate these into (prior) probability distributions.
Collapse
Affiliation(s)
- Christian Röver
- Department of Medical Statistics, University Medical Center Göttingen, Göttingen, Germany
| | - Sibylle Sturtz
- Department of Medical Biometry, Institute for Quality and Efficiency in Health Care (IQWiG), Köln, Germany
| | - Jona Lilienthal
- Department of Medical Biometry, Institute for Quality and Efficiency in Health Care (IQWiG), Köln, Germany
| | - Ralf Bender
- Department of Medical Biometry, Institute for Quality and Efficiency in Health Care (IQWiG), Köln, Germany
| | - Tim Friede
- Department of Medical Statistics, University Medical Center Göttingen, Göttingen, Germany
| |
Collapse
|
5
|
Walker R, Phillips B, Dias S. Comparison of Bayesian methods for incorporating adult clinical trial data to improve certainty of treatment effect estimates in children. PLoS One 2023; 18:e0281791. [PMID: 37319173 PMCID: PMC10270354 DOI: 10.1371/journal.pone.0281791] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/30/2023] [Accepted: 05/31/2023] [Indexed: 06/17/2023] Open
Abstract
There are challenges associated with recruiting children to take part in randomised clinical trials and as a result, compared to adults, in many disease areas we are less certain about which treatments are most safe and effective. This can lead to weaker recommendations about which treatments to prescribe in practice. However, it may be possible to 'borrow strength' from adult evidence to improve our understanding of which treatments work best in children, and many different statistical methods are available to conduct these analyses. In this paper we discuss four Bayesian methods for extrapolating adult clinical trial evidence to children. Using an exemplar dataset, we compare the effect of their modelling assumptions on the estimated treatment effect and associated heterogeneity. These modelling assumptions range from adult evidence being completely generalisable to being completely unrelated to the children's evidence. We finally discuss the appropriateness of these modelling assumptions in the context of estimating treatment effect in children.
Collapse
Affiliation(s)
- Ruth Walker
- Centre for Reviews and Dissemination, University of York, York, North Yorkshire, United Kingdom
| | - Bob Phillips
- Centre for Reviews and Dissemination, University of York, York, North Yorkshire, United Kingdom
- Department of Paediatric Haematology and Oncology, Leeds Teaching Hospitals NHS Trust, Leeds, West Yorkshire, United Kingdom
| | - Sofia Dias
- Centre for Reviews and Dissemination, University of York, York, North Yorkshire, United Kingdom
| |
Collapse
|
6
|
Röver C, Friede T. Using the bayesmeta R package for Bayesian random-effects meta-regression. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2023; 229:107303. [PMID: 36566650 DOI: 10.1016/j.cmpb.2022.107303] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/06/2022] [Revised: 11/25/2022] [Accepted: 12/07/2022] [Indexed: 06/17/2023]
Abstract
BACKGROUND Random-effects meta-analysis within a hierarchical normal modeling framework is commonly implemented in a wide range of evidence synthesis applications. More general problems may even be tackled when considering meta-regression approaches that in addition allow for the inclusion of study-level covariables. METHODS We describe the Bayesian meta-regression implementation provided in the bayesmetaR package including the choice of priors, and we illustrate its practical use. RESULTS A wide range of example applications are given, such as binary and continuous covariables, subgroup analysis, indirect comparisons, and model selection. Example R code is provided. CONCLUSIONS The bayesmeta package provides a flexible implementation. Due to the avoidance of MCMC methods, computations are fast and reproducible, facilitating quick sensitivity checks or large-scale simulation studies.
Collapse
Affiliation(s)
- Christian Röver
- Department of Medical Statistics, University Medical Center Göttingen, Humboldtallee 32, 37073 Göttingen, Germany.
| | - Tim Friede
- Department of Medical Statistics, University Medical Center Göttingen, Humboldtallee 32, 37073 Göttingen, Germany
| |
Collapse
|
7
|
Campbell H, Gustafson P. Bayes factors and posterior estimation: Two sides of the very same coin. AM STAT 2022. [DOI: 10.1080/00031305.2022.2139293] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/01/2022]
Affiliation(s)
| | - Paul Gustafson
- Department of Statistics, University of British Columbia
| |
Collapse
|
8
|
Wang Z, Lin L, Murray T, Hodges JS, Chu H. BRIDGING RANDOMIZED CONTROLLED TRIALS AND SINGLE-ARM TRIALS USING COMMENSURATE PRIORS IN ARM-BASED NETWORK META-ANALYSIS. Ann Appl Stat 2021; 15:1767-1787. [PMID: 36032933 PMCID: PMC9417056 DOI: 10.1214/21-aoas1469] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 09/27/2023]
Abstract
Network meta-analysis (NMA) is a powerful tool to compare multiple treatments directly and indirectly by combining and contrasting multiple independent clinical trials. Because many NMAs collect only a few eligible randomized controlled trials (RCTs), there is an urgent need to synthesize different sources of information, e.g., from both RCTs and single-arm trials. However, single-arm trials and RCTs may have different populations and quality, so that assuming they are exchangeable may be inappropriate. This article presents a novel method using a commensurate prior on variance (CPV) to borrow variance (rather than mean) information from single-arm trials in an arm-based (AB) Bayesian NMA. We illustrate the advantages of this CPV method by reanalyzing an NMA of immune checkpoint inhibitors in cancer patients. Comprehensive simulations investigate the impact on statistical inference of including single-arm trials. The simulation results show that the CPV method provides efficient and robust estimation even when the two sources of information are moderately inconsistent.
Collapse
Affiliation(s)
- Zhenxun Wang
- Division of Biostatistics, School of Public Health, University of Minnesota, Minneapolis, MN 55455, USA
| | - Lifeng Lin
- Department of Statistics, Florida State University, Tallahassee, FL 32306, USA
| | - Thomas Murray
- Division of Biostatistics, School of Public Health, University of Minnesota, Minneapolis, MN 55455, USA
| | - James S Hodges
- Division of Biostatistics, School of Public Health, University of Minnesota, Minneapolis, MN 55455, USA
| | - Haitao Chu
- Division of Biostatistics, School of Public Health, University of Minnesota, Minneapolis, MN 55455, USA
| |
Collapse
|
9
|
Bartoš F, Gronau QF, Timmers B, Otte WM, Ly A, Wagenmakers EJ. Bayesian model-averaged meta-analysis in medicine. Stat Med 2021; 40:6743-6761. [PMID: 34705280 PMCID: PMC9298250 DOI: 10.1002/sim.9170] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/04/2020] [Revised: 08/04/2021] [Accepted: 08/05/2021] [Indexed: 11/08/2022]
Abstract
We outline a Bayesian model-averaged (BMA) meta-analysis for standardized mean differences in order to quantify evidence for both treatment effectiveness δ and across-study heterogeneity τ . We construct four competing models by orthogonally combining two present-absent assumptions, one for the treatment effect and one for across-study heterogeneity. To inform the choice of prior distributions for the model parameters, we used 50% of the Cochrane Database of Systematic Reviews to specify rival prior distributions for δ and τ . The relative predictive performance of the competing models and rival prior distributions was assessed using the remaining 50% of the Cochrane Database. On average, ℋ 1 r -the model that assumes the presence of a treatment effect as well as across-study heterogeneity-outpredicted the other models, but not by a large margin. Within ℋ 1 r , predictive adequacy was relatively constant across the rival prior distributions. We propose specific empirical prior distributions, both for the field in general and for each of 46 specific medical subdisciplines. An example from oral health demonstrates how the proposed prior distributions can be used to conduct a BMA meta-analysis in the open-source software R and JASP. The preregistered analysis plan is available at https://osf.io/zs3df/.
Collapse
Affiliation(s)
- František Bartoš
- Department of Psychology, University of Amsterdam, Amsterdam, The Netherlands
| | - Quentin F Gronau
- Department of Psychology, University of Amsterdam, Amsterdam, The Netherlands
| | - Bram Timmers
- Department of Psychology, University of Amsterdam, Amsterdam, The Netherlands
| | - Willem M Otte
- Department of Pediatric Neurology, UMC Utrecht Brain Center, University Medical Center Utrecht and Utrecht University, Utrecht, The Netherlands.,Biomedical MR Imaging and Spectroscopy Group, Center for Image Sciences, University Medical Center Utrecht and Utrecht University, Utrecht, The Netherlands
| | - Alexander Ly
- Department of Psychology, University of Amsterdam, Amsterdam, The Netherlands.,Centrum Wiskunde & Informatica, Amsterdam, The Netherlands
| | | |
Collapse
|
10
|
Zhang H, Chiang AY, Branson M. On the Implementation of Robust Meta-Analytical-Predictive Prior. Stat Biopharm Res 2021. [DOI: 10.1080/19466315.2021.1917450] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/21/2022]
Affiliation(s)
- Hongtao Zhang
- Global Biometric and Data Sciences, Bristol Myers Squibb, Berkeley Heights, New Jersey
| | - Alan Y Chiang
- Global Biometric and Data Sciences, Bristol Myers Squibb, Berkeley Heights, New Jersey
| | - Mike Branson
- Statistical Sciences and Innovation, UCB Pharma, Brussels, Belgium
| |
Collapse
|
11
|
Nikolaidis GF, Woods B, Palmer S, Soares MO. Classifying information-sharing methods. BMC Med Res Methodol 2021; 21:107. [PMID: 34022810 PMCID: PMC8140466 DOI: 10.1186/s12874-021-01292-z] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/09/2020] [Accepted: 04/22/2021] [Indexed: 12/20/2022] Open
Abstract
BACKGROUND Sparse relative effectiveness evidence is a frequent problem in Health Technology Assessment (HTA). Where evidence directly pertaining to the decision problem is sparse, it may be feasible to expand the evidence-base to include studies that relate to the decision problem only indirectly: for instance, when there is no evidence on a comparator, evidence on other treatments of the same molecular class could be used; similarly, a decision on children may borrow-strength from evidence on adults. Usually, in HTA, such indirect evidence is either included by ignoring any differences ('lumping') or not included at all ('splitting'). However, a range of more sophisticated methods exists, primarily in the biostatistics literature. The objective of this study is to identify and classify the breadth of the available information-sharing methods. METHODS Forwards and backwards citation-mining techniques were used on a set of seminal papers on the topic of information-sharing. Papers were included if they specified (network) meta-analytic methods for combining information from distinct populations, interventions, outcomes or study-designs. RESULTS Overall, 89 papers were included. A plethora of evidence synthesis methods have been used for information-sharing. Most papers (n=79) described methods that shared information on relative treatment effects. Amongst these, there was a strong emphasis on methods for information-sharing across multiple outcomes (n=42) and treatments (n=25), with fewer papers focusing on study-designs (n=23) or populations (n=8). We categorise and discuss the methods under four 'core' relationships of information-sharing: functional, exchangeability-based, prior-based and multivariate relationships, and explain the assumptions made within each of these core approaches. CONCLUSIONS This study highlights the range of information-sharing methods available. These methods often impose more moderate assumptions than lumping or splitting. Hence, the degree of information-sharing that they impose could potentially be considered more appropriate. Our identification of four 'core' methods of information-sharing allows for an improved understanding of the assumptions underpinning the different methods. Further research is required to understand how the methods differ in terms of the strength of sharing they impose and the implications of this for health care decisions.
Collapse
Affiliation(s)
- Georgios F. Nikolaidis
- The University of York, Centre for Health Economics, Alcuin A Block, Heslington, York, YO10 5DD UK
- IQVIA, 210 Pentonville Road, London, N1 9JY UK
| | - Beth Woods
- The University of York, Centre for Health Economics, Alcuin A Block, Heslington, York, YO10 5DD UK
| | - Stephen Palmer
- The University of York, Centre for Health Economics, Alcuin A Block, Heslington, York, YO10 5DD UK
| | - Marta O. Soares
- The University of York, Centre for Health Economics, Alcuin A Block, Heslington, York, YO10 5DD UK
| |
Collapse
|
12
|
Bennett M, White S, Best N, Mander A. A novel equivalence probability weighted power prior for using historical control data in an adaptive clinical trial design: A comparison to standard methods. Pharm Stat 2021; 20:462-484. [PMID: 33474798 PMCID: PMC8611797 DOI: 10.1002/pst.2088] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/14/2019] [Revised: 10/05/2020] [Accepted: 10/14/2020] [Indexed: 11/08/2022]
Abstract
A standard two-arm randomised controlled trial usually compares an intervention to a control treatment with equal numbers of patients randomised to each treatment arm and only data from within the current trial are used to assess the treatment effect. Historical data are used when designing new trials and have recently been considered for use in the analysis when the required number of patients under a standard trial design cannot be achieved. Incorporating historical control data could lead to more efficient trials, reducing the number of controls required in the current study when the historical and current control data agree. However, when the data are inconsistent, there is potential for biased treatment effect estimates, inflated type I error and reduced power. We introduce two novel approaches for binary data which discount historical data based on the agreement with the current trial controls, an equivalence approach and an approach based on tail area probabilities. An adaptive design is used where the allocation ratio is adapted at the interim analysis, randomising fewer patients to control when there is agreement. The historical data are down-weighted in the analysis using the power prior approach with a fixed power. We compare operating characteristics of the proposed design to historical data methods in the literature: the modified power prior; commensurate prior; and robust mixture prior. The equivalence probability weight approach is intuitive and the operating characteristics can be calculated exactly. Furthermore, the equivalence bounds can be chosen to control the maximum possible inflation in type I error.
Collapse
Affiliation(s)
- Maxine Bennett
- MRC Biostatistics UnitUniversity of CambridgeCambridgeUK
| | - Simon White
- MRC Biostatistics UnitUniversity of CambridgeCambridgeUK
- Department of PsychiatryUniversity of CambridgeCambridgeUK
| | | | - Adrian Mander
- MRC Biostatistics UnitUniversity of CambridgeCambridgeUK
- Centre for Trials ResearchCardiff UniversityCardiffUK
| |
Collapse
|
13
|
Best N, Price RG, Pouliquen IJ, Keene ON. Assessing efficacy in important subgroups in confirmatory trials: An example using Bayesian dynamic borrowing. Pharm Stat 2021; 20:551-562. [PMID: 33475231 PMCID: PMC8247867 DOI: 10.1002/pst.2093] [Citation(s) in RCA: 17] [Impact Index Per Article: 5.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/14/2019] [Revised: 10/30/2020] [Accepted: 01/04/2021] [Indexed: 12/23/2022]
Abstract
Assessment of efficacy in important subgroups - such as those defined by sex, age, race and region - in confirmatory trials is typically performed using separate analysis of the specific subgroup. This ignores relevant information from the complementary subgroup. Bayesian dynamic borrowing uses an informative prior based on analysis of the complementary subgroup and a weak prior distribution centred on a mean of zero to construct a robust mixture prior. This combination of priors allows for dynamic borrowing of prior information; the analysis learns how much of the complementary subgroup prior information to borrow based on the consistency between the subgroup of interest and the complementary subgroup. A tipping point analysis can be carried out to identify how much prior weight needs to be placed on the complementary subgroup component of the robust mixture prior to establish efficacy in the subgroup of interest. An attractive feature of the tipping point analysis is that it enables the evidence from the source subgroup, the evidence from the target subgroup, and the combined evidence to be displayed alongside each other. This method is illustrated with an example trial in severe asthma where efficacy in the adolescent subgroup was assessed using a mixture prior combining an informative prior from the adult data in the same trial with a non-informative prior.
Collapse
Affiliation(s)
- Nicky Best
- Department of BiostatisticsGlaxoSmithKline Research and DevelopmentBrentfordUK
| | - Robert G. Price
- Department of Clinical Pharmacology Modelling & SimulationGlaxoSmithKline Research and DevelopmentStevenageHertsUK
| | | | - Oliver N. Keene
- Department of BiostatisticsGlaxoSmithKline Research and DevelopmentBrentfordUK
| |
Collapse
|
14
|
Röver C, Bender R, Dias S, Schmid CH, Schmidli H, Sturtz S, Weber S, Friede T. On weakly informative prior distributions for the heterogeneity parameter in Bayesian random-effects meta-analysis. Res Synth Methods 2021; 12:448-474. [PMID: 33486828 DOI: 10.1002/jrsm.1475] [Citation(s) in RCA: 44] [Impact Index Per Article: 14.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/16/2020] [Revised: 01/13/2021] [Accepted: 01/16/2021] [Indexed: 12/13/2022]
Abstract
The normal-normal hierarchical model (NNHM) constitutes a simple and widely used framework for meta-analysis. In the common case of only few studies contributing to the meta-analysis, standard approaches to inference tend to perform poorly, and Bayesian meta-analysis has been suggested as a potential solution. The Bayesian approach, however, requires the sensible specification of prior distributions. While noninformative priors are commonly used for the overall mean effect, the use of weakly informative priors has been suggested for the heterogeneity parameter, in particular in the setting of (very) few studies. To date, however, a consensus on how to generally specify a weakly informative heterogeneity prior is lacking. Here we investigate the problem more closely and provide some guidance on prior specification.
Collapse
Affiliation(s)
- Christian Röver
- Department of Medical Statistics, University Medical Center Göttingen, Göttingen, Germany
| | - Ralf Bender
- Department of Medical Biometry, Institute for Quality and Efficiency in Health Care (IQWiG), Köln, Germany
| | - Sofia Dias
- Centre for Reviews and Dissemination, University of York, York, UK
| | - Christopher H Schmid
- Department of Biostatistics and Center for Evidence Synthesis in Health, Brown University School of Public Health, Providence, Rhode Island, USA
| | - Heinz Schmidli
- Statistical Methodology, Development, Novartis Pharma AG, Basel, Switzerland
| | - Sibylle Sturtz
- Department of Medical Biometry, Institute for Quality and Efficiency in Health Care (IQWiG), Köln, Germany
| | - Sebastian Weber
- Advanced Exploratory Analytics, Novartis Pharma AG, Basel, Switzerland
| | - Tim Friede
- Department of Medical Statistics, University Medical Center Göttingen, Göttingen, Germany
| |
Collapse
|
15
|
A group-specific prior distribution for effect-size heterogeneity in meta-analysis. Behav Res Methods 2020; 52:2020-2030. [DOI: 10.3758/s13428-020-01382-8] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/05/2023]
|
16
|
Graham PL, Moran JL. ECMO, ARDS and meta-analyses: Bayes to the rescue? J Crit Care 2020; 59:49-54. [PMID: 32516642 DOI: 10.1016/j.jcrc.2020.05.009] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/25/2020] [Revised: 04/23/2020] [Accepted: 05/24/2020] [Indexed: 02/07/2023]
Abstract
PURPOSE A recent meta-analysis by Munshi et al. (Lancet Respiratory Medicine, 2019) claimed mortality treatment efficacy for extra corporeal membrane oxygenation (ECMO) in the acute respitratory syndrome (ARDS) despite very low meta-analytic study numbers (n = 2 (RCTs), risk-ratio (RR) 0·73 (95%CI: 0·58-0·92); n = 5 (2 RCT, 3 observational), RR 0·69 (95%CI: 0·50-0·95)). We explore this efficacy claim by a comprehensive re-analysis of the data. METHODS Data were sourced from the two- and five-study meta-analyses, conducted using the Der-Simonian & Laird (DSL) method. A variety of frequentist (DSL, restricted maximum likelihood (REML), Paul-Mandel (PM), with/without Hartung-Knapp-Sidik-Jonkman variance correction), a beta-binomial model (BBN)) and Bayesian models (2 finite-mixture and several Markov-Chain-Monte-Carlo) were used to estimate treatment effects. Fragility-indices, the minimum patients changing mortality outcome needed to induce a conclusion change were also applied. RESULTS For the 2-study and 5-study meta-analysis only the uncorrected frequentist estimators (DSL, REML, PM) demonstrated significant RR. Except for the BBN model, which was significant for the 2-study meta-analysis, intervals for all other models included the null. Both meta-analyses demonstrated fragility. CONCLUSIONS Having canvassed the conduct of both meta-analyses presented by Munshi et al. and proffered alternative methods, we find no certainty regarding the efficacy of ECMO in ARDS.
Collapse
Affiliation(s)
- Petra L Graham
- Centre for Economic Impacts of Genomic Medicine (GenIMPACT), Macquarie Business School and Department of Mathematics and Statistics, Faculty of Science and Engineering, Macquarie University, North Ryde, NSW 2109, Australia.
| | - John L Moran
- Department of Intensive Care Medicine, The Queen Elizabeth Hospital, Woodville, SA 5011, Australia.
| |
Collapse
|
17
|
Li W, Liu F, Snavely D. Revisit of test-then-pool methods and some practical considerations. Pharm Stat 2020; 19:498-517. [PMID: 32171048 DOI: 10.1002/pst.2009] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/03/2019] [Revised: 01/02/2020] [Accepted: 02/10/2020] [Indexed: 11/08/2022]
Abstract
Test-then-pool is a simple statistical method that borrows historical information to improve efficiency of the drug development process. The original test-then-pool method examines the difference between the historical and current information and then pools the information if there is no significant difference. One drawback of this method is that a nonsignificant difference may not always imply consistency between the historical and current information. As a result, the original test-then-pool method is more likely to incorrectly borrow information from the historical control when the current trial has a small sample size. Statistically, it is more natural to use an equivalence test for examining the consistency. This manuscript develops an equivalence-based test-then-pool method for a continuous endpoint, explains the relationship between the two test-then-pool methods, explores the choice of an equivalence margin through the overlap probability, and proposes an adjustment to the nominal testing level for controlling type I error under the true consistency scenario. Furthermore, the analytical forms of the type I error and power for the two test-then-pool methods are derived, and practical considerations for using them are presented.
Collapse
Affiliation(s)
- Wen Li
- Biostatistics and Research Decision Sciences, MRL, Merck & Co., Inc., Kenilworth, New Jersey, USA
| | - Frank Liu
- Biostatistics and Research Decision Sciences, MRL, Merck & Co., Inc., Kenilworth, New Jersey, USA
| | - Duane Snavely
- Biostatistics and Research Decision Sciences, MRL, Merck & Co., Inc., Kenilworth, New Jersey, USA
| |
Collapse
|
18
|
A multicenter, randomized, placebo-controlled, double-blind phase 3 trial with open-arm comparison indicates safety and efficacy of nephroprotective therapy with ramipril in children with Alport's syndrome. Kidney Int 2020; 97:1275-1286. [PMID: 32299679 DOI: 10.1016/j.kint.2019.12.015] [Citation(s) in RCA: 70] [Impact Index Per Article: 17.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/30/2019] [Revised: 12/05/2019] [Accepted: 12/13/2019] [Indexed: 01/09/2023]
Abstract
Children with Alport syndrome develop renal failure early in life. Since the safety and efficacy of preemptive nephroprotective therapy are uncertain we conducted a randomized, placebo-controlled, double-blind trial in 14 German sites of pediatric patients with ramipril for three to six years plus six months follow-up to determine these parameters. Pretreated children and those whose parents refused randomization became an open-arm control, which were compared to prospective real-world data from untreated children. The co-primary endpoints were safety (adverse drug reactions) and efficacy (time to progression). Out of 66 oligosymptomatic children, 22 were randomized and 44 joined the open-arm comparison. Ramipril therapy showed no safety issues (total of 216.4 patient-years on ramipril; adverse event rate-ratio 1.00; 95% confidence interval 0.66-1.53). Although not significant, our results cautiously showed that ramipril therapy was effective: in the randomized arm, Ramipril decreased the risk of disease progression by almost half (hazard ratio 0.51 (0.12-2.20)), diminished the slope of albuminuria progression and the decline in glomerular filtration. In adjusted analysis, indications of efficacy were supported by prospective data from participants treated open label compared with untreated children, in whom ramipril again seemed to reduce progression by almost half (0.53 (0.22-1.29)). Incorporating these results into the randomized data by Bayesian evidence synthesis resulted in a more precise estimate of the hazard-ratio of 0.52 (0.19-1.39). Thus, our study shows the safety of early initiation of therapy and supports the hope to slow renal failure by many years, emphasizing the value of preemptive therapy. Hence, screening programs for glomerular hematuria in children and young adults could benefit from inclusion of genetic testing for Alport-related gene-variants.
Collapse
|
19
|
Chapple LAS, Weinel L, Ridley EJ, Jones D, Chapman MJ, Peake SL. Clinical Sequelae From Overfeeding in Enterally Fed Critically Ill Adults: Where Is the Evidence? JPEN J Parenter Enteral Nutr 2019; 44:980-991. [PMID: 31736105 DOI: 10.1002/jpen.1740] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/14/2019] [Revised: 10/11/2019] [Accepted: 10/25/2019] [Indexed: 12/13/2022]
Abstract
Enteral energy delivery above requirements (overfeeding) is believed to cause adverse effects during critical illness, but the literature supporting this is limited. We aimed to quantify the reported frequency and clinical sequelae of energy overfeeding with enterally delivered nutrition in critically ill adult patients. A systematic search of MEDLINE, EMBASE, and CINAHL from conception to November 28, 2018, identified clinical studies of nutrition interventions in enterally fed critically ill adults that reported overfeeding in 1 or more study arms. Overfeeding was defined as energy delivery > 2000 kcal/d, > 25 kcal/kg/d, or ≥ 110% of energy prescription. Data were extracted on methodology, demographics, prescribed and delivered nutrition, clinical variables, and predefined outcomes. Cochrane "Risk of Bias" tool was used to assess the quality of randomized controlled trials (RCTs). Eighteen studies were included, of which 10 were randomized (n = 4386 patients) and 8 were nonrandomized (n = 223). Only 4 studies reported a separation in energy delivery between treatment groups whereby 1 arm met the definition of overfeeding, which reported no between-group differences in mortality, infectious complications, or ventilatory support. Overfeeding was associated with increased insulin administration (median 3 [interquartile range: 0-41.8] vs 0 [0-30.6] units/d) and upper-gastrointestinal intolerance in 1 large RCT and with duration of antimicrobial therapy in a small RCT. There are limited high-quality data to determine the impact of energy overfeeding of critically ill patients by the enteral route; however, based on available evidence, overfeeding does not appear to affect mortality or other important clinical outcomes.
Collapse
Affiliation(s)
- Lee-Anne S Chapple
- Intensive Care Research, Royal Adelaide Hospital, Adelaide, Australia.,Discipline of Acute Care Medicine, School of Medicine, University of Adelaide, Adelaide, Australia
| | - Luke Weinel
- Intensive Care Research, Royal Adelaide Hospital, Adelaide, Australia.,Discipline of Acute Care Medicine, School of Medicine, University of Adelaide, Adelaide, Australia
| | - Emma J Ridley
- Australaian and New Zealand Intensive Care Research Centre, School of Public Health and Preventive Medicine, Monash University Melbourne, Melbourne, Australia.,Nutrition Department, Alfred Health, Melbourne, Australia
| | - Daryl Jones
- Intensive Care Unit, Austin Health, Melbourne, Australia.,School of Public Health and Preventive Medicine, Monash University Melbourne, Melbourne, Australia
| | - Marianne J Chapman
- Intensive Care Research, Royal Adelaide Hospital, Adelaide, Australia.,Discipline of Acute Care Medicine, School of Medicine, University of Adelaide, Adelaide, Australia
| | - Sandra L Peake
- Discipline of Acute Care Medicine, School of Medicine, University of Adelaide, Adelaide, Australia.,Department of Intensive Care Medicine, The Queen Elizabeth Hospital, Adelaide, Australia
| |
Collapse
|