1
|
Petropoulou M, Rücker G, Weibel S, Kranke P, Schwarzer G. Model selection for component network meta-analysis in connected and disconnected networks: a simulation study. BMC Med Res Methodol 2023; 23:140. [PMID: 37316775 DOI: 10.1186/s12874-023-01959-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/11/2022] [Accepted: 05/29/2023] [Indexed: 06/16/2023] Open
Abstract
BACKGROUND Network meta-analysis (NMA) allows estimating and ranking the effects of several interventions for a clinical condition. Component network meta-analysis (CNMA) is an extension of NMA which considers the individual components of multicomponent interventions. CNMA allows to "reconnect" a disconnected network with common components in subnetworks. An additive CNMA assumes that component effects are additive. This assumption can be relaxed by including interaction terms in the CNMA. METHODS We evaluate a forward model selection strategy for component network meta-analysis to relax the additivity assumption that can be used in connected or disconnected networks. In addition, we describe a procedure to create disconnected networks in order to evaluate the properties of the model selection in connected and disconnected networks. We apply the methods to simulated data and a Cochrane review on interventions for postoperative nausea and vomiting in adults after general anaesthesia. Model performance is compared using average mean squared errors and coverage probabilities. RESULTS CNMA models provide good performance for connected networks and can be an alternative to standard NMA if additivity holds. For disconnected networks, we recommend to use additive CNMA only if strong clinical arguments for additivity exist. CONCLUSIONS CNMA methods are feasible for connected networks but questionable for disconnected networks.
Collapse
Affiliation(s)
- Maria Petropoulou
- Institute of Medical Biometry and Statistics, Faculty of Medicine and Medical Center - University of Freiburg, Stefan-Meier-Straße 26, 79104, Freiburg, Germany
| | - Gerta Rücker
- Institute of Medical Biometry and Statistics, Faculty of Medicine and Medical Center - University of Freiburg, Stefan-Meier-Straße 26, 79104, Freiburg, Germany
| | - Stephanie Weibel
- Department of Anaesthesiology, Intensive Care, Emergency and Pain Medicine, University Hospital Würzburg, Oberdürrbacher Straße 6, 97080, Würzburg, Germany
| | - Peter Kranke
- Department of Anaesthesiology, Intensive Care, Emergency and Pain Medicine, University Hospital Würzburg, Oberdürrbacher Straße 6, 97080, Würzburg, Germany
| | - Guido Schwarzer
- Institute of Medical Biometry and Statistics, Faculty of Medicine and Medical Center - University of Freiburg, Stefan-Meier-Straße 26, 79104, Freiburg, Germany.
| |
Collapse
|
2
|
Chalkou K, Vickers AJ, Pellegrini F, Manca A, Salanti G. Decision Curve Analysis for Personalized Treatment Choice between Multiple Options. Med Decis Making 2023; 43:337-349. [PMID: 36511470 PMCID: PMC10021120 DOI: 10.1177/0272989x221143058] [Citation(s) in RCA: 8] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/04/2022] [Accepted: 11/03/2022] [Indexed: 12/15/2022]
Abstract
BACKGROUND Decision curve analysis can be used to determine whether a personalized model for treatment benefit would lead to better clinical decisions. Decision curve analysis methods have been described to estimate treatment benefit using data from a single randomized controlled trial. OBJECTIVES Our main objective is to extend the decision curve analysis methodology to the scenario in which several treatment options exist and evidence about their effects comes from a set of trials, synthesized using network meta-analysis (NMA). METHODS We describe the steps needed to estimate the net benefit of a prediction model using evidence from studies synthesized in an NMA. We show how to compare personalized versus one-size-fit-all treatment decision-making strategies, such as "treat none" or "treat all patients with a specific treatment" strategies. First, threshold values for each included treatment need to be defined (i.e., the minimum risk difference compared with control that renders a treatment worth taking). The net benefit per strategy can then be plotted for a plausible range of threshold values to reveal the most clinically useful strategy. We applied our methodology to an NMA prediction model for relapsing-remitting multiple sclerosis, which can be used to choose between natalizumab, dimethyl fumarate, glatiramer acetate, and placebo. RESULTS We illustrated the extended decision curve analysis methodology using several threshold value combinations for each available treatment. For the examined threshold values, the "treat patients according to the prediction model" strategy performs either better than or close to the one-size-fit-all treatment strategies. However, even small differences may be important in clinical decision making. As the advantage of the personalized model was not consistent across all thresholds, improving the existing model (by including, for example, predictors that will increase discrimination) is needed before advocating its clinical usefulness. CONCLUSIONS This novel extension of decision curve analysis can be applied to NMA-based prediction models to evaluate their use to aid treatment decision making. HIGHLIGHTS Decision curve analysis is extended into a (network) meta-analysis framework.Personalized models predicting treatment benefit are evaluated when several treatment options are available and evidence about their effects comes from a set of trials.Detailed steps to compare personalized versus one-size-fit-all treatment decision-making strategies are outlined.This extension of decision curve analysis can be applied to (network) meta-analysis-based prediction models to evaluate their use to aid treatment decision making.
Collapse
Affiliation(s)
- Konstantina Chalkou
- Institute of Social and Preventive Medicine,
University of Bern, Bern, Switzerland
- Graduate School for Health Sciences, University
of Bern, Switzerland
| | - Andrew J. Vickers
- Department of Epidemiology and Biostatistics,
Memorial Sloan-Kettering Cancer Center, New York, NY, USA
| | | | - Andrea Manca
- Centre for Health Economics, University of
York, York, UK
| | - Georgia Salanti
- Institute of Social and Preventive Medicine,
University of Bern, Bern, Switzerland
| |
Collapse
|
3
|
Hamza T, Chalkou K, Pellegrini F, Kuhle J, Benkert P, Lorscheider J, Zecca C, Iglesias-Urrutia CP, Manca A, Furukawa TA, Cipriani A, Salanti G. Synthesizing cross-design evidence and cross-format data using network meta-regression. Res Synth Methods 2023; 14:283-300. [PMID: 36625736 DOI: 10.1002/jrsm.1619] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/15/2022] [Revised: 10/28/2022] [Accepted: 12/01/2022] [Indexed: 01/11/2023]
Abstract
In network meta-analysis (NMA), we synthesize all relevant evidence about health outcomes with competing treatments. The evidence may come from randomized clinical trials (RCT) or non-randomized studies (NRS) as individual participant data (IPD) or as aggregate data (AD). We present a suite of Bayesian NMA and network meta-regression (NMR) models allowing for cross-design and cross-format synthesis. The models integrate a three-level hierarchical model for synthesizing IPD and AD into four approaches. The four approaches account for differences in the design and risk of bias (RoB) in the RCT and NRS evidence. These four approaches variously ignoring differences in RoB, using NRS to construct penalized treatment effect priors and bias-adjustment models that control the contribution of information from high RoB studies in two different ways. We illustrate the methods in a network of three pharmacological interventions and placebo for patients with relapsing-remitting multiple sclerosis. The estimated relative treatment effects do not change much when we accounted for differences in design and RoB. Conducting network meta-regression showed that intervention efficacy decreases with increasing participant age. We also re-analysed a network of 431 RCT comparing 21 antidepressants, and we did not observe material changes in intervention efficacy when adjusting for studies' high RoB. We re-analysed both case studies accounting for different study RoB. In summary, the described suite of NMA/NMR models enables the inclusion of all relevant evidence while incorporating information on the within-study bias in both observational and experimental data and enabling estimation of individualized treatment effects through the inclusion of participant characteristics.
Collapse
Affiliation(s)
- Tasnim Hamza
- Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland.,Graduate School for Health Sciences, University of Bern, Bern, Switzerland
| | - Konstantina Chalkou
- Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland.,Graduate School for Health Sciences, University of Bern, Bern, Switzerland
| | | | - Jens Kuhle
- Department of Neurology, University Hospital Basel, University of Basel, Basel, Switzerland.,Departments of Biomedicine and Clinical Research, University Hospital Basel, University of Basel, Basel, Switzerland
| | - Pascal Benkert
- Department of Clinical Research, University Hospital Basel, University of Basel, Basel, Switzerland
| | - Johannes Lorscheider
- Departments of Biomedicine and Clinical Research, University Hospital Basel, University of Basel, Basel, Switzerland.,Research Center for Clinical Neuroimmunology and Neuroscience Basel (RC2NB), University Hospital Basel, University of Basel, Basel, Switzerland
| | - Chiara Zecca
- Multiple Sclerosis Center, Neurocenter of Southern Switzerland, EOC, Lugano, Switzerland.,Faculty of Biomedical Sciences, Università della Svizzera Italiana, Lugano, Switzerland
| | | | - Andrea Manca
- Centre for Health Economics, University of York, York, UK
| | - Toshi A Furukawa
- Department of Health Promotion and Human Behavior, Graduate School of Medicine/School of Public Health, Kyoto University, Kyoto, Japan.,Department of Clinical Epidemiology, Graduate School of Medicine/School of Public Health, Kyoto University, Kyoto, Japan
| | - Andrea Cipriani
- Department of Psychiatry, University of Oxford, Oxford, UK.,Oxford Health NHS Foundation Trust, Warneford Hospital, Oxford, UK
| | - Georgia Salanti
- Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland
| |
Collapse
|
4
|
Veroniki AA, Seitidis G, Stewart L, Clarke M, Tudur-Smith C, Mavridis D, Yu CH, Moja L, Straus SE, Tricco AC. Comparative efficacy and complications of long-acting and intermediate-acting insulin regimens for adults with type 1 diabetes: an individual patient data network meta-analysis. BMJ Open 2022; 12:e058034. [PMID: 36332950 PMCID: PMC9639076 DOI: 10.1136/bmjopen-2021-058034] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/06/2022] Open
Abstract
OBJECTIVE To examine the comparative efficacy and complications of long-acting and intermediate-acting insulin for different patient characteristics for type 1 diabetes mellitus (T1DM). DESIGN Systematic review and individual patient data (IPD) network meta-analysis (NMA). DATA SOURCES MEDLINE, EMBASE and the Cochrane Central Register of Controlled Trials were searched through June 2015. ELIGIBILITY CRITERIA Randomised controlled trials (RCTs) on adults with T1DM assessing glycosylated haemoglobin (A1c) and severe hypoglycaemia in long-acting and intermediate-acting insulin regimens. DATA EXTRACTION AND SYNTHESIS We requested IPD from authors and funders. When IPD were not available, we used aggregate data. We conducted a random-effects model, and specifically a one-stage IPD-NMA for those studies providing IPD and a two-stage IPD-NMA to incorporate those studies not providing IPD. RESULTS We included 28 RCTs plus one companion report, after screening 6680 titles/abstracts and 205 full-text articles. Of the 28 RCTs, 27 studies provided data for the NMA with 7394 participants, of which 12 RCTs had IPD on 4943 participants. The IPD-NMA for A1c suggested that glargine once daily (mean difference [MD]=-0.31, 95% confidence interval [CI]: -0.48 to -0.14) and detemir once daily (MD=-0.25, 95% CI: -0.41 to -0.09) were superior to neutral protamine Hagedorn (NPH) once daily. NPH once/two times per day improved A1c compared with NPH once daily (MD=-0.30, 95% CI: -0.50 to -0.11). Results regarding complications in severe hypoglycaemia should be considered with great caution due to inconsistency in the evidence network. Accounting for missing data, there was no evidence of inconsistency and long-acting insulin regimens ranked higher regarding reducing severe hypoglycaemia compared with intermediate-acting insulin regimens (two-stage NMA: glargine two times per day SUCRA (Surface Under the Cumulative Ranking curve)=89%, detemir once daily SUCRA=77%; one-stage NMA: detemir once daily/two times per day SUCRA=85%). Using multiple imputations and IPD only, complications in severe hypoglycaemia increased with diabetes-related comorbidities (regression coefficient: 1.03, 95% CI: 1.02 to 1.03). CONCLUSIONS Long-acting insulin regimens reduced A1c compared with intermediate-acting insulin regimens and were associated with lower severe hypoglycaemia. Of the observed differences, only glargine once daily achieved a clinically significant reduction of 0.30%. Results should be interpreted with caution due to very low quality of evidence. PROSPERO REGISTRATION NUMBER CRD42015023511.
Collapse
Affiliation(s)
- Areti Angeliki Veroniki
- Knowledge Translation Program, Li Ka Shing Knowledge Institute, St. Michael's Hospital, Toronto, Ontario, Canada
- Institute of Health Policy Management and Evaluation, University of Toronto, Toronto, Ontario, Canada
| | - Georgios Seitidis
- Department of Primary Education, University of Ioannina, Ioannina, Greece
| | - Lesley Stewart
- Centre for Reviews and Dissemination, University of York, York, UK
| | - Mike Clarke
- Northern Ireland Methodology Hub, Queen's University Belfast, Belfast, UK
| | | | - Dimitris Mavridis
- Department of Primary Education, University of Ioannina, Ioannina, Greece
| | - Catherine H Yu
- Knowledge Translation Program, Li Ka Shing Knowledge Institute, St. Michael's Hospital, Toronto, Ontario, Canada
- Department of Medicine, University of Toronto, Toronto, Ontario, Canada
| | - Lorenzo Moja
- Department of Biomedical Sciences for Health, University of Milan, Milan, Italy
- Essential Medicines and Health Products, WHO, Geneva, Switzerland
| | - Sharon E Straus
- Knowledge Translation Program, Li Ka Shing Knowledge Institute, St. Michael's Hospital, Toronto, Ontario, Canada
- Institute of Health Policy Management and Evaluation, University of Toronto, Toronto, Ontario, Canada
- Department of Geriatric Medicine, University of Toronto, Toronto, Ontario, Canada
| | - Andrea C Tricco
- Knowledge Translation Program, Li Ka Shing Knowledge Institute, St. Michael's Hospital, Toronto, Ontario, Canada
- Dalla Lana School of Public Health, University of Toronto, Toronto, Ontario, Canada
| |
Collapse
|
5
|
Thom H, Leahy J, Jansen JP. Network Meta-analysis on Disconnected Evidence Networks When Only Aggregate Data Are Available: Modified Methods to Include Disconnected Trials and Single-Arm Studies while Minimizing Bias. Med Decis Making 2022; 42:906-922. [PMID: 35531938 PMCID: PMC9459361 DOI: 10.1177/0272989x221097081] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
BACKGROUND Network meta-analysis (NMA) requires a connected network of randomized controlled trials (RCTs) and cannot include single-arm studies. Regulators or academics often have only aggregate data. Two aggregate data methods for analyzing disconnected networks are random effects on baseline and aggregate-level matching (ALM). ALM has been used only for single-arm studies, and both methods may bias effect estimates. METHODS We modified random effects on baseline to separate RCTs connected to and disconnected from the reference and any single-arm studies, minimizing the introduction of bias. We term our modified method reference prediction. We similarly modified ALM and extended it to include RCTs disconnected from the reference. We tested these methods using constructed data and a simulation study. RESULTS In simulations, bias for connected treatments for ALM ranged from -0.0158 to 0.051 and for reference prediction from -0.0107 to 0.083. These were low compared with the true mean effect of 0.5. Coverage ranged from 0.92 to 1.00. In disconnected treatments, bias of ALM ranged from -0.16 to 0.392 and of reference prediction from -0.102 to 0.40, whereas coverage of ALM ranged from 0.30 to 0.82 and of reference prediction from 0.64 to 0.94. Under fixed study effects for disconnected evidence, bias was similar, but coverage was 0.81 to 1.00 for reference prediction and 0.18 to 0.76 for ALM. Trends of similar bias but greater coverage for reference prediction with random study effects were repeated in constructed data. CONCLUSIONS Both methods with random study effects seem to minimize bias in treatment connected to the reference. They can estimate treatment effects for disconnected treatments but may be biased. Reference prediction has greater coverage and may be recommended overall. HIGHLIGHTS Two methods were modified for network meta-analysis on disconnected networks and for including single-arm observational or interventional studies in network meta-analysis using only aggregate data and for minimizing the bias of effect estimates for treatments only in trials connected to the reference.Reference prediction was developed as a modification of random effects on baseline that keeps analyses of trials connected to the reference separately from those disconnected from the reference and from single-arm studies. The method was further modified to account for correlation in trials with more than 2 arms and, under random study effects, to estimate variance in heterogeneity separately in connected and disconnected evidence.Aggregate-level matching was extended to include trials disconnected from the reference, rather than only single-arm studies. The method was further modified to separately estimate treatment effects and heterogeneity variance in the connected and disconnected evidence and to account for the correlation between arms in trials with more than 2 arms.Performance was assessed using a constructed data example and simulation study.The methods were found to have similar, and sometimes low, bias when estimating the relative effects for disconnected treatments, but reference prediction with random study effects had the greatest coverage.The use of reference prediction with random study effects for disconnected networks is recommended if no individual patient data or alternative real-world evidence is available.
Collapse
Affiliation(s)
- Howard Thom
- Howard Thom, Bristol Medical School,
University of Bristol, Canynge Hall, Rm 2.07, 39 Whatley Rd, Bristol, BS8 2PS,
UK; ()
| | - Joy Leahy
- National Centre for Pharmacoeconomic, Dublin,
Ireland
| | - Jeroen P. Jansen
- School of Pharmacy, University of California,
San Francisco, USA
| |
Collapse
|
6
|
Riley RD, Dias S, Donegan S, Tierney JF, Stewart LA, Efthimiou O, Phillippo DM. Using individual participant data to improve network meta-analysis projects. BMJ Evid Based Med 2022; 28:197-203. [PMID: 35948411 DOI: 10.1136/bmjebm-2022-111931] [Citation(s) in RCA: 11] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 07/01/2022] [Indexed: 11/04/2022]
Abstract
A network meta-analysis combines the evidence from existing randomised trials about the comparative efficacy of multiple treatments. It allows direct and indirect evidence about each comparison to be included in the same analysis, and provides a coherent framework to compare and rank treatments. A traditional network meta-analysis uses aggregate data (eg, treatment effect estimates and standard errors) obtained from publications or trial investigators. An alternative approach is to obtain, check, harmonise and meta-analyse the individual participant data (IPD) from each trial. In this article, we describe potential advantages of IPD for network meta-analysis projects, emphasising five key benefits: (1) improving the quality and scope of information available for inclusion in the meta-analysis, (2) examining and plotting distributions of covariates across trials (eg, for potential effect modifiers), (3) standardising and improving the analysis of each trial, (4) adjusting for prognostic factors to allow a network meta-analysis of conditional treatment effects and (5) including treatment-covariate interactions (effect modifiers) to allow relative treatment effects to vary by participant-level covariate values (eg, age, baseline depression score). A running theme of all these benefits is that they help examine and reduce heterogeneity (differences in the true treatment effect between trials) and inconsistency (differences in the true treatment effect between direct and indirect evidence) in the network. As a consequence, an IPD network meta-analysis has the potential for more precise, reliable and informative results for clinical practice and even allows treatment comparisons to be made for individual patients and targeted populations conditional on their particular characteristics.
Collapse
Affiliation(s)
| | - Sofia Dias
- Centre for Reviews and Dissemination, University of York, York, UK
| | - Sarah Donegan
- Department of Health Data Science, University of Liverpool, Liverpool, UK
| | | | - Lesley A Stewart
- Centre for Reviews and Dissemination, University of York, York, UK
| | - Orestis Efthimiou
- Institute of Social and Preventive Medicine (ISPMU), University of Bern, Bern, Switzerland
| | - David M Phillippo
- Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, UK
| |
Collapse
|
7
|
Singh J, Gsteiger S, Wheaton L, Riley RD, Abrams KR, Gillies CL, Bujkiewicz S. Bayesian network meta-analysis methods for combining individual participant data and aggregate data from single arm trials and randomised controlled trials. BMC Med Res Methodol 2022; 22:186. [PMID: 35818035 PMCID: PMC9275254 DOI: 10.1186/s12874-022-01657-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/24/2021] [Accepted: 05/23/2022] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND Increasingly in network meta-analysis (NMA), there is a need to incorporate non-randomised evidence to estimate relative treatment effects, and in particular in cases with limited randomised evidence, sometimes resulting in disconnected networks of treatments. When combining different sources of data, complex NMA methods are required to address issues associated with participant selection bias, incorporating single-arm trials (SATs), and synthesising a mixture of individual participant data (IPD) and aggregate data (AD). We develop NMA methods which synthesise data from SATs and randomised controlled trials (RCTs), using a mixture of IPD and AD, for a dichotomous outcome. METHODS We propose methods under both contrast-based (CB) and arm-based (AB) parametrisations, and extend the methods to allow for both within- and across-trial adjustments for covariate effects. To illustrate the methods, we use an applied example investigating the effectiveness of biologic disease-modifying anti-rheumatic drugs for rheumatoid arthritis (RA). We applied the methods to a dataset obtained from a literature review consisting of 14 RCTs and an artificial dataset consisting of IPD from two SATs and AD from 12 RCTs, where the artificial dataset was created by removing the control arms from the only two trials assessing tocilizumab in the original dataset. RESULTS Without adjustment for covariates, the CB method with independent baseline response parameters (CBunadjInd) underestimated the effectiveness of tocilizumab when applied to the artificial dataset compared to the original dataset, albeit with significant overlap in posterior distributions for treatment effect parameters. The CB method with exchangeable baseline response parameters produced effectiveness estimates in agreement with CBunadjInd, when the predicted baseline response estimates were similar to the observed baseline response. After adjustment for RA duration, there was a reduction in across-trial heterogeneity in baseline response but little change in treatment effect estimates. CONCLUSIONS Our findings suggest incorporating SATs in NMA may be useful in some situations where a treatment is disconnected from a network of comparator treatments, due to a lack of comparative evidence, to estimate relative treatment effects. The reliability of effect estimates based on data from SATs may depend on adjustment for covariate effects, although further research is required to understand this in more detail.
Collapse
Affiliation(s)
- Janharpreet Singh
- Biostatistics Research Group, Department of Health Sciences, University of Leicester, Leicester, UK.
| | | | - Lorna Wheaton
- Biostatistics Research Group, Department of Health Sciences, University of Leicester, Leicester, UK
| | - Richard D Riley
- Centre for Prognosis Research, School of Medicine, University of Keele, Staffordshire, UK
| | - Keith R Abrams
- Department of Statistics, University of Warwick, Coventry, UK
| | - Clare L Gillies
- Leicester Real World Evidence Unit, Diabetes Research Centre, University of Leicester, Leicester, UK
| | - Sylwia Bujkiewicz
- Biostatistics Research Group, Department of Health Sciences, University of Leicester, Leicester, UK
| |
Collapse
|
8
|
Wang Z, Lin L, Murray T, Hodges JS, Chu H. BRIDGING RANDOMIZED CONTROLLED TRIALS AND SINGLE-ARM TRIALS USING COMMENSURATE PRIORS IN ARM-BASED NETWORK META-ANALYSIS. Ann Appl Stat 2021; 15:1767-1787. [PMID: 36032933 PMCID: PMC9417056 DOI: 10.1214/21-aoas1469] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 09/27/2023]
Abstract
Network meta-analysis (NMA) is a powerful tool to compare multiple treatments directly and indirectly by combining and contrasting multiple independent clinical trials. Because many NMAs collect only a few eligible randomized controlled trials (RCTs), there is an urgent need to synthesize different sources of information, e.g., from both RCTs and single-arm trials. However, single-arm trials and RCTs may have different populations and quality, so that assuming they are exchangeable may be inappropriate. This article presents a novel method using a commensurate prior on variance (CPV) to borrow variance (rather than mean) information from single-arm trials in an arm-based (AB) Bayesian NMA. We illustrate the advantages of this CPV method by reanalyzing an NMA of immune checkpoint inhibitors in cancer patients. Comprehensive simulations investigate the impact on statistical inference of including single-arm trials. The simulation results show that the CPV method provides efficient and robust estimation even when the two sources of information are moderately inconsistent.
Collapse
Affiliation(s)
- Zhenxun Wang
- Division of Biostatistics, School of Public Health, University of Minnesota, Minneapolis, MN 55455, USA
| | - Lifeng Lin
- Department of Statistics, Florida State University, Tallahassee, FL 32306, USA
| | - Thomas Murray
- Division of Biostatistics, School of Public Health, University of Minnesota, Minneapolis, MN 55455, USA
| | - James S Hodges
- Division of Biostatistics, School of Public Health, University of Minnesota, Minneapolis, MN 55455, USA
| | - Haitao Chu
- Division of Biostatistics, School of Public Health, University of Minnesota, Minneapolis, MN 55455, USA
| |
Collapse
|
9
|
Jenkins DA, Hussein H, Martina R, Dequen-O'Byrne P, Abrams KR, Bujkiewicz S. Methods for the inclusion of real-world evidence in network meta-analysis. BMC Med Res Methodol 2021; 21:207. [PMID: 34627166 PMCID: PMC8502389 DOI: 10.1186/s12874-021-01399-3] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/08/2021] [Accepted: 09/07/2021] [Indexed: 11/26/2022] Open
Abstract
Background Network Meta-Analysis (NMA) is a key component of submissions to reimbursement agencies world-wide, especially when there is limited direct head-to-head evidence for multiple technologies from randomised controlled trials (RCTs). Many NMAs include only data from RCTs. However, real-world evidence (RWE) is also becoming widely recognised as a valuable source of clinical data. This study aims to investigate methods for the inclusion of RWE in NMA and its impact on the level of uncertainty around the effectiveness estimates, with particular interest in effectiveness of fingolimod. Methods A range of methods for inclusion of RWE in evidence synthesis were investigated by applying them to an illustrative example in relapsing remitting multiple sclerosis (RRMS). A literature search to identify RCTs and RWE evaluating treatments in RRMS was conducted. To assess the impact of inclusion of RWE on the effectiveness estimates, Bayesian hierarchical and adapted power prior models were applied. The effect of the inclusion of RWE was investigated by varying the degree of down weighting of this part of evidence by the use of a power prior. Results Whilst the inclusion of the RWE led to an increase in the level of uncertainty surrounding effect estimates in this example, this depended on the method of inclusion adopted for the RWE. ‘Power prior’ NMA model resulted in stable effect estimates for fingolimod yet increasing the width of the credible intervals with increasing weight given to RWE data. The hierarchical NMA models were effective in allowing for heterogeneity between study designs, however, this also increased the level of uncertainty. Conclusion The ‘power prior’ method for the inclusion of RWE in NMAs indicates that the degree to which RWE is taken into account can have a significant impact on the overall level of uncertainty. The hierarchical modelling approach further allowed for accommodating differences between study types. Consequently, further work investigating both empirical evidence for biases associated with individual RWE studies and methods of elicitation from experts on the extent of such biases is warranted. Supplementary Information The online version contains supplementary material available at 10.1186/s12874-021-01399-3.
Collapse
Affiliation(s)
- David A Jenkins
- Biostatistics Research Group, Department of Health Sciences, University of Leicester, University Road, Leicester, LE1 7RH, UK.,School of Health Sciences, Faculty of Biology, Medicine and Health, University of Manchester, Oxford Road, Manchester, M13 9PL, UK.,NIHR Greater Manchester Patient Safety Translational Research Centre, University of Manchester, Oxford Road, Manchester, M13 9PL, UK
| | - Humaira Hussein
- Biostatistics Research Group, Department of Health Sciences, University of Leicester, University Road, Leicester, LE1 7RH, UK.
| | - Reynaldo Martina
- Biostatistics Research Group, Department of Health Sciences, University of Leicester, University Road, Leicester, LE1 7RH, UK
| | - Pascale Dequen-O'Byrne
- Biostatistics Research Group, Department of Health Sciences, University of Leicester, University Road, Leicester, LE1 7RH, UK
| | - Keith R Abrams
- Biostatistics Research Group, Department of Health Sciences, University of Leicester, University Road, Leicester, LE1 7RH, UK.,Centre for Health Economics, University of York, York, YO10 5DD, UK
| | - Sylwia Bujkiewicz
- Biostatistics Research Group, Department of Health Sciences, University of Leicester, University Road, Leicester, LE1 7RH, UK
| |
Collapse
|
10
|
Tian J, Gao Y, Zhang J, Yang Z, Dong S, Zhang T, Sun F, Wu S, Wu J, Wang J, Yao L, Ge L, Li L, Shi C, Wang Q, Li J, Zhao Y, Xiao Y, Yang F, Fan J, Bao S, Song F. Progress and challenges of network meta-analysis. J Evid Based Med 2021; 14:218-231. [PMID: 34463038 DOI: 10.1111/jebm.12443] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/28/2021] [Revised: 08/03/2021] [Accepted: 08/03/2021] [Indexed: 11/28/2022]
Abstract
In the past years, network meta-analysis (NMA) has been widely used among clinicians, guideline makers, and health technology assessment agencies and has played an important role in clinical decision-making and guideline development. To inform further development of NMAs, we conducted a bibliometric analysis to assess the current status of published NMA methodological studies, summarized the methodological progress of seven types of NMAs, and discussed the current challenges of NMAs.
Collapse
Affiliation(s)
- Jinhui Tian
- Evidence-Based Medicine Center, School of Basic Medical Sciences, Lanzhou University, Lanzhou, China
- Key Laboratory of Evidence-Based Medicine and Knowledge Translation of Gansu Province, Lanzhou, China
| | - Ya Gao
- Evidence-Based Medicine Center, School of Basic Medical Sciences, Lanzhou University, Lanzhou, China
- Key Laboratory of Evidence-Based Medicine and Knowledge Translation of Gansu Province, Lanzhou, China
| | - Junhua Zhang
- Evidence-Based Medicine Center, Tianjin University of Traditional Chinese Medicine, Tianjin, China
| | - Zhirong Yang
- Primary Care Unit, Department of Public Health and Primary Care, School of Clinical Medicine, University of Cambridge, Cambridge, UK
| | - Shengjie Dong
- Orthopedic Department, Yantaishan Hospital, Yantai, Shandong, China
| | - Tiansong Zhang
- Department of Traditional Chinese Medicine, Jing'an District Central Hospital, Shanghai, China
| | - Feng Sun
- Department of Epidemiology and Biostatistics, School of Public Health, Peking University, Beijing, China
| | - Shanshan Wu
- National Clinical Research Center of Digestive Diseases, Beijing Friendship Hospital, Capital Medical University, Beijing, China
| | - Jiarui Wu
- Department of Clinical Chinese Pharmacy, School of Chinese Materia Medica, Beijing University of Chinese Medicine, Beijing, China
| | - Junfeng Wang
- Division of Pharmacoepidemiology and Clinical Pharmacology, Utrecht Institute for Pharmaceutical Sciences, Utrecht University, Utrecht, The Netherlands
| | - Liang Yao
- Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Canada
| | - Long Ge
- Key Laboratory of Evidence-Based Medicine and Knowledge Translation of Gansu Province, Lanzhou, China
- Evidence-Based Social Science Research Center, School of Public Health, Lanzhou University, Lanzhou, China
| | - Lun Li
- Department of Breast Cancer, The Second Xiangya Hospital, Central South University, Changsha, Hunan, China
| | - Chunhu Shi
- Division of Nursing, Midwifery and Social Work, School of Health Sciences, Faculty of Biology, Medicine and Health, University of Manchester, Manchester, UK
| | - Quan Wang
- Department of Gastrointestinal Surgery, Peking University People's Hospital, Beijing, China
| | - Jiang Li
- National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Ye Zhao
- First Clinical Medical College, Lanzhou University, Lanzhou, China
- Departments of Biochemistry and Molecular Biology, Melvin and Bren Simon Comprehensive Cancer Center, Indiana University School of Medicine, Indianapolis, Indiana
| | - Yue Xiao
- China National Health Development Research Center, Beijing, China
| | - Fengwen Yang
- Evidence-Based Medicine Center, Tianjin University of Traditional Chinese Medicine, Tianjin, China
| | - Jinchun Fan
- Epidemiology and Evidence Based-Medicine, School of Public Health, Gansu University of Chinese Medicine, Lanzhou, China
| | - Shisan Bao
- Epidemiology and Evidence Based-Medicine, School of Public Health, Gansu University of Chinese Medicine, Lanzhou, China
- Sydney, NSW, Australia
| | - Fujian Song
- Public Health and Health Services Research, Norwich Medical School, University of East Anglia, Norwich, UK
| |
Collapse
|
11
|
Singh J, Abrams KR, Bujkiewicz S. Incorporating single-arm studies in meta-analysis of randomised controlled trials: a simulation study. BMC Med Res Methodol 2021; 21:114. [PMID: 34082702 PMCID: PMC8176581 DOI: 10.1186/s12874-021-01301-1] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/13/2020] [Accepted: 04/21/2021] [Indexed: 01/09/2023] Open
Abstract
Background Use of real world data (RWD) from non-randomised studies (e.g. single-arm studies) is increasingly being explored to overcome issues associated with data from randomised controlled trials (RCTs). We aimed to compare methods for pairwise meta-analysis of RCTs and single-arm studies using aggregate data, via a simulation study and application to an illustrative example. Methods We considered contrast-based methods proposed by Begg & Pilote (1991) and arm-based methods by Zhang et al (2019). We performed a simulation study with scenarios varying (i) the proportion of RCTs and single-arm studies in the synthesis (ii) the magnitude of bias, and (iii) between-study heterogeneity. We also applied methods to data from a published health technology assessment (HTA), including three RCTs and 11 single-arm studies. Results Our simulation study showed that the hierarchical power and commensurate prior methods by Zhang et al provided a consistent reduction in uncertainty, whilst maintaining over-coverage and small error in scenarios where there was limited RCT data, bias and differences in between-study heterogeneity between the two sets of data. The contrast-based methods provided a reduction in uncertainty, but performed worse in terms of coverage and error, unless there was no marked difference in heterogeneity between the two sets of data. Conclusions The hierarchical power and commensurate prior methods provide the most robust approach to synthesising aggregate data from RCTs and single-arm studies, balancing the need to account for bias and differences in between-study heterogeneity, whilst reducing uncertainty in estimates. This work was restricted to considering a pairwise meta-analysis using aggregate data. Supplementary Information The online version contains supplementary material available at (10.1186/s12874-021-01301-1).
Collapse
Affiliation(s)
- Janharpreet Singh
- Biostatistics Research Group, Department of Health Sciences, University of Leicester, Leicester, UK.
| | - Keith R Abrams
- Biostatistics Research Group, Department of Health Sciences, University of Leicester, Leicester, UK.,Centre for Health Economics, University of York, York, UK
| | - Sylwia Bujkiewicz
- Biostatistics Research Group, Department of Health Sciences, University of Leicester, Leicester, UK
| |
Collapse
|
12
|
Ronellenfitsch U, Friedrichs J, Grilli M, Hofheinz RD, Jensen K, Kieser M, Kleeff J, Michalski CW, Michl P, Seide S, Vey J, Vordermark D, Proctor T. Preoperative chemoradiotherapy versus chemotherapy for adenocarcinoma of the esophagus and esophagogastric junction (AEG): systematic review with individual participant data (IPD) network meta-analysis (NMA). Hippokratia 2021. [DOI: 10.1002/14651858.cd014748] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/29/2022]
Affiliation(s)
- Ulrich Ronellenfitsch
- Department of Visceral, Vascular and Endocrine Surgery; Medical Faculty of the Martin Luther University Halle-Wittenberg and University Hospital Halle (Saale); Halle (Saale) Germany
| | - Juliane Friedrichs
- Department of Visceral, Vascular and Endocrine Surgery; Medical Faculty of the Martin Luther University Halle-Wittenberg and University Hospital Halle (Saale); Halle (Saale) Germany
| | - Maurizio Grilli
- Library of the Medical Faculty Mannheim; Heidelberg University; Mannheim Germany
| | - Ralf-Dieter Hofheinz
- Day Treatment Center, Interdisciplinary Tumor Center Mannheim and III Medical Clinic; University Medical Centre Mannheim, University of Heidelberg; Mannheim Germany
| | - Katrin Jensen
- Institute of Medical Biometry and Informatics; University of Heidelberg; Heidelberg Germany
| | - Meinhard Kieser
- Institute of Medical Biometry and Informatics; Heidelberg University Hospital; Heidelberg Germany
| | - Jörg Kleeff
- Department of Visceral, Vascular and Endocrine Surgery; University Hospital Halle (Saale); Halle (Saale) Germany
| | | | - Patrick Michl
- Department of Internal Medicine I; University Hospital Halle (Saale); Halle (Saale) Germany
| | - Svenja Seide
- Institute of Medical Biometry and Informatics; Heidelberg University Hospital; Heidelberg Germany
| | - Johannes Vey
- Institute of Medical Biometry and Informatics; Heidelberg University Hospital; Heidelberg Germany
| | - Dirk Vordermark
- Department of Radiotherapy; Medical Faculty of the Martin Luther University Halle-Wittenberg and University Hospital Halle (Saale); Halle (Saale) Germany
| | | |
Collapse
|
13
|
A review of the quantitative effectiveness evidence synthesis methods used in public health intervention guidelines. BMC Public Health 2021; 21:278. [PMID: 33535975 PMCID: PMC7860217 DOI: 10.1186/s12889-021-10162-8] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/22/2020] [Accepted: 01/04/2021] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND The complexity of public health interventions create challenges in evaluating their effectiveness. There have been huge advancements in quantitative evidence synthesis methods development (including meta-analysis) for dealing with heterogeneity of intervention effects, inappropriate 'lumping' of interventions, adjusting for different populations and outcomes and the inclusion of various study types. Growing awareness of the importance of using all available evidence has led to the publication of guidance documents for implementing methods to improve decision making by answering policy relevant questions. METHODS The first part of this paper reviews the methods used to synthesise quantitative effectiveness evidence in public health guidelines by the National Institute for Health and Care Excellence (NICE) that had been published or updated since the previous review in 2012 until the 19th August 2019.The second part of this paper provides an update of the statistical methods and explains how they address issues related to evaluating effectiveness evidence of public health interventions. RESULTS The proportion of NICE public health guidelines that used a meta-analysis as part of the synthesis of effectiveness evidence has increased since the previous review in 2012 from 23% (9 out of 39) to 31% (14 out of 45). The proportion of NICE guidelines that synthesised the evidence using only a narrative review decreased from 74% (29 out of 39) to 60% (27 out of 45).An application in the prevention of accidents in children at home illustrated how the choice of synthesis methods can enable more informed decision making by defining and estimating the effectiveness of more distinct interventions, including combinations of intervention components, and identifying subgroups in which interventions are most effective. CONCLUSIONS Despite methodology development and the publication of guidance documents to address issues in public health intervention evaluation since the original review, NICE public health guidelines are not making full use of meta-analysis and other tools that would provide decision makers with fuller information with which to develop policy. There is an evident need to facilitate the translation of the synthesis methods into a public health context and encourage the use of methods to improve decision making.
Collapse
|
14
|
Pedder H, Dias S, Bennetts M, Boucher M, Welton NJ. Joining the Dots: Linking Disconnected Networks of Evidence Using Dose-Response Model-Based Network Meta-Analysis. Med Decis Making 2021; 41:194-208. [PMID: 33448252 PMCID: PMC7879230 DOI: 10.1177/0272989x20983315] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/02/2020] [Accepted: 11/30/2020] [Indexed: 12/20/2022]
Abstract
BACKGROUND Network meta-analysis (NMA) synthesizes direct and indirect evidence on multiple treatments to estimate their relative effectiveness. However, comparisons between disconnected treatments are not possible without making strong assumptions. When studies including multiple doses of the same drug are available, model-based NMA (MBNMA) presents a novel solution to this problem by modeling a parametric dose-response relationship within an NMA framework. In this article, we illustrate several scenarios in which dose-response MBNMA can connect and strengthen evidence networks. METHODS We created illustrative data sets by removing studies or treatments from an NMA of triptans for migraine relief. We fitted MBNMA models with different dose-response relationships. For connected networks, we compared MBNMA estimates with NMA estimates. For disconnected networks, we compared MBNMA estimates with NMA estimates from an "augmented" network connected by adding studies or treatments back into the data set. RESULTS In connected networks, relative effect estimates from MBNMA were more precise than those from NMA models (ratio of posterior SDs NMA v. MBNMA: median = 1.13; range = 1.04-1.68). In disconnected networks, MBNMA provided estimates for all treatments where NMA could not and were consistent with NMA estimates from augmented networks for 15 of 18 data sets. In the remaining 3 of 18 data sets, a more complex dose-response relationship was required than could be fitted with the available evidence. CONCLUSIONS Where information on multiple doses is available, MBNMA can connect disconnected networks and increase precision while making less strong assumptions than alternative approaches. MBNMA relies on correct specification of the dose-response relationship, which requires sufficient data at different doses to allow reliable estimation. We recommend that systematic reviews for NMA search for and include evidence (including phase II trials) on multiple doses of agents where available.
Collapse
Affiliation(s)
- Hugo Pedder
- Department of Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, UK
| | - Sofia Dias
- Centre for Reviews and Dissemination, University of York, York, North Yorkshire, UK
| | | | | | - Nicky J. Welton
- Department of Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, UK
| |
Collapse
|
15
|
Phillippo DM, Dias S, Ades AE, Welton NJ. Assessing the performance of population adjustment methods for anchored indirect comparisons: A simulation study. Stat Med 2020; 39:4885-4911. [PMID: 33015906 PMCID: PMC8690023 DOI: 10.1002/sim.8759] [Citation(s) in RCA: 42] [Impact Index Per Article: 10.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/25/2020] [Revised: 06/25/2020] [Accepted: 09/04/2020] [Indexed: 12/21/2022]
Abstract
Standard network meta-analysis and indirect comparisons combine aggregate data from multiple studies on treatments of interest, assuming that any factors that interact with treatment effects (effect modifiers) are balanced across populations. Population adjustment methods such as multilevel network meta-regression (ML-NMR), matching-adjusted indirect comparison (MAIC), and simulated treatment comparison (STC) relax this assumption using individual patient data from one or more studies, and are becoming increasingly prevalent in health technology appraisals and the applied literature. Motivated by an applied example and two recent reviews of applications, we undertook an extensive simulation study to assess the performance of these methods in a range of scenarios under various failures of assumptions. We investigated the impact of varying sample size, missing effect modifiers, strength of effect modification and validity of the shared effect modifier assumption, validity of extrapolation and varying between-study overlap, and different covariate distributions and correlations. ML-NMR and STC performed similarly, eliminating bias when the requisite assumptions were met. Serious concerns are raised for MAIC, which performed poorly in nearly all simulation scenarios and may even increase bias compared with standard indirect comparisons. All methods incur bias when an effect modifier is missing, highlighting the necessity of careful selection of potential effect modifiers prior to analysis. When all effect modifiers are included, ML-NMR and STC are robust techniques for population adjustment. ML-NMR offers additional advantages over MAIC and STC, including extending to larger treatment networks and producing estimates in any target population, making this an attractive choice in a variety of scenarios.
Collapse
Affiliation(s)
- David M. Phillippo
- Bristol Medical School (Population Health Sciences)University of BristolBristolUK
| | - Sofia Dias
- Bristol Medical School (Population Health Sciences)University of BristolBristolUK
- Centre for Reviews and DisseminationUniversity of YorkYorkUK
| | - A. E. Ades
- Bristol Medical School (Population Health Sciences)University of BristolBristolUK
| | - Nicky J. Welton
- Bristol Medical School (Population Health Sciences)University of BristolBristolUK
| |
Collapse
|
16
|
Schlueter M, Beaudet A, Davies E, Gurung B, Karabis A. Evidence synthesis in pulmonary arterial hypertension: a systematic review and critical appraisal. BMC Pulm Med 2020; 20:202. [PMID: 32723397 PMCID: PMC7388228 DOI: 10.1186/s12890-020-01241-4] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/19/2019] [Accepted: 07/17/2020] [Indexed: 11/17/2022] Open
Abstract
BACKGROUND The clinical landscape of pulmonary arterial hypertension (PAH) has evolved in terms of disease definition and classification, trial designs, available therapies and treatment strategies as well as clinical guidelines. This study critically appraises published evidence synthesis studies, i.e. meta-analyses (MA) and network-meta-analyses (NMA), to better understand their quality, validity and discuss the impact of the findings from these studies on current decision-making in PAH. METHODS A systematic literature review to identify MA/NMA studies considering approved and available therapies for treatment of PAH was conducted. Embase, Medline and the Cochrane's Database of Systematic Reviews were searched from database inception to April 22, 2020, supplemented by searches in health technology assessment websites. The International Society for Pharmacoeconomics and Outcomes Research (ISPOR) checklist covering six domains (relevance, credibility, analysis, reporting quality and transparency, interpretation and conflict of interest) was selected for appraisal of the included MA/NMA studies. RESULTS Fifty-two full publications (36 MAs, 15 NMAs, and 1 MA/NMA) in PAH met the inclusion criteria. The majority of studies were of low quality, with none of the studies being scored as 'strong' across all checklist domains. Key limitations included the lack of a clearly defined, relevant decision problem, shortcomings in assessing and addressing between-study heterogeneity, and an incomplete or misleading interpretation of results. CONCLUSIONS This is the first critical appraisal of published MA/NMA studies in PAH, suggesting low quality and validity of published evidence synthesis studies in this therapeutic area. Besides the need for direct treatment comparisons assessed in long-term randomized controlled trials, future efforts in evidence synthesis in PAH should improve analysis quality and scrutiny in order to meaningfully address challenges arising from an evolving therapeutic landscape.
Collapse
Affiliation(s)
| | - Amélie Beaudet
- Actelion Pharmaceuticals Ltd, Gewerbestrasse 16, CH-4123, Allschwil, Switzerland
| | - Evan Davies
- Actelion Pharmaceuticals Ltd, Gewerbestrasse 16, CH-4123, Allschwil, Switzerland
| | - Binu Gurung
- IQVIA, 210 Pentonville Road, London, N1 9JY, UK
| | - Andreas Karabis
- IQVIA, Herikerbergweg 314, 1101 CT, Amsterdam, Netherlands
- Department of Statistical Science, University College London, London, WC1E 6BT, UK
| |
Collapse
|
17
|
Phillippo DM, Dias S, Ades AE, Belger M, Brnabic A, Schacht A, Saure D, Kadziola Z, Welton NJ. Multilevel network meta-regression for population-adjusted treatment comparisons. JOURNAL OF THE ROYAL STATISTICAL SOCIETY. SERIES A, (STATISTICS IN SOCIETY) 2020; 183:1189-1210. [PMID: 32684669 PMCID: PMC7362893 DOI: 10.1111/rssa.12579] [Citation(s) in RCA: 59] [Impact Index Per Article: 14.8] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/09/2023]
Abstract
Standard network meta-analysis (NMA) and indirect comparisons combine aggregate data from multiple studies on treatments of interest, assuming that any effect modifiers are balanced across populations. Population adjustment methods relax this assumption using individual patient data from one or more studies. However, current matching-adjusted indirect comparison and simulated treatment comparison methods are limited to pairwise indirect comparisons and cannot predict into a specified target population. Existing meta-regression approaches incur aggregation bias. We propose a new method extending the standard NMA framework. An individual level regression model is defined, and aggregate data are fitted by integrating over the covariate distribution to form the likelihood. Motivated by the complexity of the closed form integration, we propose a general numerical approach using quasi-Monte-Carlo integration. Covariate correlation structures are accounted for by using copulas. Crucially for decision making, comparisons may be provided in any target population with a given covariate distribution. We illustrate the method with a network of plaque psoriasis treatments. Estimated population-average treatment effects are similar across study populations, as differences in the distributions of effect modifiers are small. A better fit is achieved than a random effects NMA, uncertainty is substantially reduced by explaining within- and between-study variation, and estimates are more interpretable.
Collapse
Affiliation(s)
| | - Sofia Dias
- University of York and University of Bristol UK
| | | | | | | | | | | | | | | |
Collapse
|
18
|
Statistical analyses and quality of individual participant data network meta-analyses were suboptimal: a cross-sectional study. BMC Med 2020; 18:120. [PMID: 32475340 PMCID: PMC7262764 DOI: 10.1186/s12916-020-01591-0] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/16/2020] [Accepted: 04/14/2020] [Indexed: 01/08/2023] Open
Abstract
BACKGROUND Network meta-analyses using individual participant data (IPD-NMAs) have been increasingly used to compare the effects of multiple interventions. Although there have been many studies on statistical methods for IPD-NMAs, it is unclear whether there are statistical defects in published IPD-NMAs and whether the reporting of statistical analyses has improved. This study aimed to investigate statistical methods used and assess the reporting and methodological quality of IPD-NMAs. METHODS We searched four bibliographic databases to identify published IPD-NMAs. The methodological quality was assessed using AMSTAR-2 and reporting quality assessed based on PRISMA-IPD and PRISMA-NMA. We performed stratified analyses and correlation analyses to explore the factors that might affect quality. RESULTS We identified 21 IPD-NMAs. Only 23.8% of the included IPD-NMAs reported statistical techniques used for missing participant data, 42.9% assessed the consistency, and none assessed the transitivity. None of the included IPD-NMAs reported sources of funding for trials included, only 9.5% stated pre-registration of protocols, and 28.6% assessed the risk of bias in individual studies. For reporting quality, compliance rates were lower than 50.0% for more than half of the items. Less than 15.0% of the IPD-NMAs reported data integrity, presented the network geometry, or clarified risk of bias across studies. IPD-NMAs with statistical or epidemiological authors often better assessed the inconsistency (P = 0.017). IPD-NMAs with a priori protocol were associated with higher reporting quality in terms of search (P = 0.046), data collection process (P = 0.031), and syntheses of results (P = 0.006). CONCLUSIONS The reporting of statistical methods and compliance rates of methodological and reporting items of IPD-NMAs were suboptimal. Authors of future IPD-NMAs should address the identified flaws and strictly adhere to methodological and reporting guidelines.
Collapse
|
19
|
Zhang K, Arora P, Sati N, Béliveau A, Troke N, Veroniki AA, Rodrigues M, Rios P, Zarin W, Tricco AC. Characteristics and methods of incorporating randomized and nonrandomized evidence in network meta-analyses: a scoping review. J Clin Epidemiol 2019; 113:1-10. [DOI: 10.1016/j.jclinepi.2019.04.023] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/21/2018] [Revised: 03/04/2019] [Accepted: 04/04/2019] [Indexed: 12/19/2022]
|
20
|
A practical approach to predict expansion of evidence networks: a case study in treatment-naive advanced melanoma. Melanoma Res 2019; 29:13-18. [PMID: 30273234 DOI: 10.1097/cmr.0000000000000513] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
Abstract
Network meta-analysis (NMA) is a statistical method used to produce comparable estimates of efficacy across a range of treatments that may not be compared directly within any single trial. NMA feasibility is determined by the comparability of the data and presence of a connected network. In rapidly evolving treatment landscapes, evidence networks can change substantially in a short period of time. We investigate methods to determine the optimum time to conduct or update a NMA based on anticipated available evidence. We report the results of a systematic review conducted in treatment-naive advanced melanoma and compare networks of evidence available at retrospective, current, and prospective time points. For included publications, we compared the primary completion date of trials from clinical trials registries (CTRs) with the date of their first available publication to provide an estimate of publication lag. Using CTRs we were able to produce anticipated networks for future time points based on projected study completion dates and average publication lags which illustrated expansion and strengthening of the initial network. We found that over a snapshot of periods between 2015 and 2018, evidence networks in melanoma changed substantively, adding new comparators and increasing network connectedness. Searching CTRs for ongoing trials demonstrates it is possible to anticipate future networks at a certain time point. Armed with this information, sensible decisions can be made over when best to conduct or update a NMA. Incorporating new and upcoming interventions in a NMA enables presentation of a complete, up-to-date and evolving picture of the evidence.
Collapse
|
21
|
Leahy J, Thom H, Jansen JP, Gray E, O'Leary A, White A, Walsh C. Incorporating single-arm evidence into a network meta-analysis using aggregate level matching: Assessing the impact. Stat Med 2019; 38:2505-2523. [PMID: 30895655 DOI: 10.1002/sim.8139] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/01/2017] [Revised: 11/27/2018] [Accepted: 02/14/2019] [Indexed: 01/21/2023]
Abstract
Increasingly, single-armed evidence is included in health technology assessment submissions when companies are seeking reimbursement for new drugs. While it is recognized that randomized controlled trials provide a higher standard of evidence, these are not available for many new agents that have been granted licenses in recent years. Therefore, it is important to examine whether alternative strategies for assessing this evidence may be used. In this work, we examine approaches to incorporating single-armed evidence formally in the evaluation process. We consider matching aggregate level covariates to comparator arms or trials and including this evidence in a network meta-analysis. We consider two methods of matching: (i) we include the chosen matched arm in the data set itself as a comparator for the single-arm trial; (ii) we use the baseline odds of an event in a chosen matched trial to use as a plug-in estimator for the single-arm trial. We illustrate that the synthesis of evidence resulting from such a setup is sensitive to the between-study variability, formulation of the prior for the between-design effect, weight given to the single-arm evidence, and extent of the bias in single-armed evidence. We provide a flowchart for the process involved in such a synthesis and highlight additional sensitivity analyses that should be carried out. This work was motivated by a hepatitis C data set, where many agents have only been examined in single-arm studies. We present the results of our methods applied to this data set.
Collapse
Affiliation(s)
- Joy Leahy
- School of Computer Science and Statistics, Trinity College Dublin, The University of Dublin, Dublin, Ireland.,National Centre for Pharmacoeconomics, St. James's Hospital, Dublin, Ireland
| | - Howard Thom
- Bristol Medical School: Population Health Sciences, University of Bristol, Bristol, UK
| | - Jeroen P Jansen
- Department of Health Research and Policy Epidemiology, Stanford University School of Medicine, Stanford, California
| | - Emma Gray
- School of Medicine, Trinity College Dublin, The University of Dublin, Dublin, Ireland
| | - Aisling O'Leary
- National Centre for Pharmacoeconomics, St. James's Hospital, Dublin, Ireland
| | - Arthur White
- School of Computer Science and Statistics, Trinity College Dublin, The University of Dublin, Dublin, Ireland.,National Centre for Pharmacoeconomics, St. James's Hospital, Dublin, Ireland
| | - Cathal Walsh
- National Centre for Pharmacoeconomics, St. James's Hospital, Dublin, Ireland.,Department of Mathematics and Statistics, Health Research Institute and MACSI, University of Limerick, Limerick, Ireland
| |
Collapse
|
22
|
Zoratti MJ, Devji T, Levine O, Thabane L, Xie F. Network meta-analysis of therapies for previously untreated advanced BRAF-mutated melanoma. Cancer Treat Rev 2019; 74:43-48. [DOI: 10.1016/j.ctrv.2019.02.001] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/22/2018] [Revised: 01/31/2019] [Accepted: 02/04/2019] [Indexed: 12/27/2022]
|
23
|
Schmitz S, Maguire Á, Morris J, Ruggeri K, Haller E, Kuhn I, Leahy J, Homer N, Khan A, Bowden J, Buchanan V, O’Dwyer M, Cook G, Walsh C. The use of single armed observational data to closing the gap in otherwise disconnected evidence networks: a network meta-analysis in multiple myeloma. BMC Med Res Methodol 2018; 18:66. [PMID: 29954322 PMCID: PMC6022299 DOI: 10.1186/s12874-018-0509-7] [Citation(s) in RCA: 22] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/11/2017] [Accepted: 05/09/2018] [Indexed: 12/17/2022] Open
Abstract
BACKGROUND Network meta-analysis (NMA) allows for the estimation of comparative effectiveness of treatments that have not been studied in head-to-head trials; however, relative treatment effects for all interventions can only be derived where available evidence forms a connected network. Head-to-head evidence is limited in many disease areas, regularly resulting in disconnected evidence structures where a large number of treatments are available. This is also the case in the evidence of treatments for relapsed or refractory multiple myeloma. METHODS Randomised controlled trials (RCTs) identified in a systematic literature review form two disconnected evidence networks. Standard Bayesian NMA models are fitted to obtain estimates of relative effects within each network. Observational evidence was identified to fill the evidence gap. Single armed trials are matched to act as each other's control group based on a distance metric derived from covariate information. Uncertainty resulting from including this evidence is incorporated by analysing the space of possible matches. RESULTS Twenty five randomised controlled trials form two disconnected evidence networks; 12 single armed observational studies are considered for bridging between the networks. Five matches are selected to bridge between the networks. While significant variation in the ranking is observed, daratumumab in combination with dexamethasone and either lenalidomide or bortezomib, as well as triple therapy of carfilzomib, ixazomib and elozumatab, in combination with lenalidomide and dexamethasone, show the highest effects on progression free survival, on average. CONCLUSIONS The analysis shows how observational data can be used to fill gaps in the existing networks of RCT evidence; allowing for the indirect comparison of a large number of treatments, which could not be compared otherwise. Additional uncertainty is accounted for by scenario analyses reducing the risk of over confidence in interpretation of results.
Collapse
Affiliation(s)
- Susanne Schmitz
- Department of Population Health, Luxembourg Institute of Health, Strassen, Luxembourg
| | - Áine Maguire
- Department of Psychiatry, Trinity College Dublin, Dublin, Ireland
- Department of Psychology, University of Cambridge, Cambridge, UK
| | | | - Kai Ruggeri
- Department of Psychology, University of Cambridge, Cambridge, UK
- Department of Health Policy & Management, Mailman School of Public Health, Columbia University, New York, USA
| | - Elisa Haller
- Department of Psychology, University of Zurich, Zürich, Switzerland
| | - Isla Kuhn
- University Library: Medical Library, University of Cambridge, Cambridge, UK
| | - Joy Leahy
- Department of Computer Science and Statistics, Trinity College Dublin, Dublin, Ireland
| | | | | | - Jack Bowden
- School of Social and Community Medicine, University of Bristol, Bristol, UK
| | | | | | - Gordon Cook
- Professor of Haematology & Myeloma Studies, Clinical Director NIHR MIC-DEL, St James’s University Hospital, Leeds, England
| | - Cathal Walsh
- Health Research Institute, University of Limerick, Limerick, Ireland
| |
Collapse
|
24
|
Stevens JW, Fletcher C, Downey G, Sutton A. A review of methods for comparing treatments evaluated in studies that form disconnected networks of evidence. Res Synth Methods 2017; 9:148-162. [DOI: 10.1002/jrsm.1278] [Citation(s) in RCA: 12] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/06/2016] [Revised: 10/04/2017] [Accepted: 10/13/2017] [Indexed: 12/28/2022]
Affiliation(s)
- John W. Stevens
- School of Health and Related Research; University of Sheffield; Regent Court, 30 Regent Street Sheffield UK
| | - Christine Fletcher
- Amgen Ltd, Global Biostatistical Science; 240 Cambridge Science Park, Milton Road Cambridge Cambridgeshire UK
| | - Gerald Downey
- Amgen Ltd, Global Biostatistical Science; 240 Cambridge Science Park, Milton Road Cambridge Cambridgeshire UK
| | - Anthea Sutton
- School of Health and Related Research; University of Sheffield; Regent Court, 30 Regent Street Sheffield UK
| |
Collapse
|
25
|
Riley RD, Jackson D, Salanti G, Burke DL, Price M, Kirkham J, White IR. Multivariate and network meta-analysis of multiple outcomes and multiple treatments: rationale, concepts, and examples. BMJ 2017; 358:j3932. [PMID: 28903924 PMCID: PMC5596393 DOI: 10.1136/bmj.j3932] [Citation(s) in RCA: 155] [Impact Index Per Article: 22.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
Abstract
Organisations such as the National Institute for Health and Care Excellence require the synthesis of evidence from existing studies to inform their decisions—for example, about the best available treatments with respect to multiple efficacy and safety outcomes. However, relevant studies may not provide direct evidence about all the treatments or outcomes of interest. Multivariate and network meta-analysis methods provide a framework to address this, using correlated or indirect evidence from such studies alongside any direct evidence. In this article, the authors describe the key concepts and assumptions of these methods, outline how correlated and indirect evidence arises, and illustrate the contribution of such evidence in real clinical examples involving multiple outcomes and multiple treatments
Collapse
Affiliation(s)
- Richard D Riley
- Research Institute for Primary Care and Health Sciences, Keele University, Staffordshire, UK
| | | | - Georgia Salanti
- Institute of Social and Preventive Medicine, University of Bern, Switzerland
- University of Ioannina School of Medicine, Ioannina, Greece
| | - Danielle L Burke
- Research Institute for Primary Care and Health Sciences, Keele University, Staffordshire, UK
| | - Malcolm Price
- Institute of Applied Health Research, University of Birmingham, UK
| | - Jamie Kirkham
- MRC North West Hub for Trials Methodology Research, Department of Biostatistics, University of Liverpool, Liverpool, UK
| | - Ian R White
- MRC Biostatistics Unit, Cambridge, UK
- MRC Clinical Trials Unit at UCL, London, UK
| |
Collapse
|
26
|
Phillippo DM, Ades AE, Dias S, Palmer S, Abrams KR, Welton NJ. Methods for Population-Adjusted Indirect Comparisons in Health Technology Appraisal. Med Decis Making 2017; 38:200-211. [PMID: 28823204 PMCID: PMC5774635 DOI: 10.1177/0272989x17725740] [Citation(s) in RCA: 199] [Impact Index Per Article: 28.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/15/2022]
Abstract
Standard methods for indirect comparisons and network meta-analysis are based on aggregate data, with the key assumption that there is no difference between the trials in the distribution of effect-modifying variables. Methods which relax this assumption are becoming increasingly common for submissions to reimbursement agencies, such as the National Institute for Health and Care Excellence (NICE). These methods use individual patient data from a subset of trials to form population-adjusted indirect comparisons between treatments, in a specific target population. Recently proposed population adjustment methods include the Matching-Adjusted Indirect Comparison (MAIC) and the Simulated Treatment Comparison (STC). Despite increasing popularity, MAIC and STC remain largely untested. Furthermore, there is a lack of clarity about exactly how and when they should be applied in practice, and even whether the results are relevant to the decision problem. There is therefore a real and present risk that the assumptions being made in one submission to a reimbursement agency are fundamentally different to—or even incompatible with—the assumptions being made in another for the same indication. We describe the assumptions required for population-adjusted indirect comparisons, and demonstrate how these may be used to generate comparisons in any given target population. We distinguish between anchored and unanchored comparisons according to whether a common comparator arm is used or not. Unanchored comparisons make much stronger assumptions, which are widely regarded as infeasible. We provide recommendations on how and when population adjustment methods should be used, and the supporting analyses that are required to provide statistically valid, clinically meaningful, transparent and consistent results for the purposes of health technology appraisal. Simulation studies are needed to examine the properties of population adjustment methods and their robustness to breakdown of assumptions.
Collapse
Affiliation(s)
- David M Phillippo
- School of Social and Community Medicine, University of Bristol, Bristol, UK (DMP, AEA, SD, NJW)
| | - Anthony E Ades
- School of Social and Community Medicine, University of Bristol, Bristol, UK (DMP, AEA, SD, NJW)
| | - Sofia Dias
- School of Social and Community Medicine, University of Bristol, Bristol, UK (DMP, AEA, SD, NJW)
| | - Stephen Palmer
- Centre for Health Economics, University of York, York, UK (SP)
| | - Keith R Abrams
- Department of Health Sciences, University of Leicester, Leicester, UK (KPA)
| | - Nicky J Welton
- School of Social and Community Medicine, University of Bristol, Bristol, UK (DMP, AEA, SD, NJW)
| |
Collapse
|
27
|
Saramago P, Woods B, Weatherly H, Manca A, Sculpher M, Khan K, Vickers AJ, MacPherson H. Methods for network meta-analysis of continuous outcomes using individual patient data: a case study in acupuncture for chronic pain. BMC Med Res Methodol 2016; 16:131. [PMID: 27716074 PMCID: PMC5053345 DOI: 10.1186/s12874-016-0224-1] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/20/2016] [Accepted: 09/09/2016] [Indexed: 11/13/2022] Open
Abstract
BACKGROUND Network meta-analysis methods, which are an extension of the standard pair-wise synthesis framework, allow for the simultaneous comparison of multiple interventions and consideration of the entire body of evidence in a single statistical model. There are well-established advantages to using individual patient data to perform network meta-analysis and methods for network meta-analysis of individual patient data have already been developed for dichotomous and time-to-event data. This paper describes appropriate methods for the network meta-analysis of individual patient data on continuous outcomes. METHODS This paper introduces and describes network meta-analysis of individual patient data models for continuous outcomes using the analysis of covariance framework. Comparisons are made between this approach and change score and final score only approaches, which are frequently used and have been proposed in the methodological literature. A motivating example on the effectiveness of acupuncture for chronic pain is used to demonstrate the methods. Individual patient data on 28 randomised controlled trials were synthesised. Consistency of endpoints across the evidence base was obtained through standardisation and mapping exercises. RESULTS Individual patient data availability avoided the use of non-baseline-adjusted models, allowing instead for analysis of covariance models to be applied and thus improving the precision of treatment effect estimates while adjusting for baseline imbalance. CONCLUSIONS The network meta-analysis of individual patient data using the analysis of covariance approach is advocated to be the most appropriate modelling approach for network meta-analysis of continuous outcomes, particularly in the presence of baseline imbalance. Further methods developments are required to address the challenge of analysing aggregate level data in the presence of baseline imbalance.
Collapse
Affiliation(s)
- Pedro Saramago
- Centre for Health Economics, University of York, York, UK
| | - Beth Woods
- Centre for Health Economics, University of York, York, UK
| | | | - Andrea Manca
- Centre for Health Economics, University of York, York, UK
| | - Mark Sculpher
- Centre for Health Economics, University of York, York, UK
| | - Kamran Khan
- Warwick Clinical Trials Unit, Division of Health Sciences, Warwick Medical School, University of Warwick, Coventry, UK
| | - Andrew J. Vickers
- Department of Epidemiology and Biostatistics, Memorial Sloan-Kettering Cancer Center, New York, USA
| | | |
Collapse
|
28
|
Veroniki AA, Straus SE, Soobiah C, Elliott MJ, Tricco AC. A scoping review of indirect comparison methods and applications using individual patient data. BMC Med Res Methodol 2016; 16:47. [PMID: 27116943 PMCID: PMC4847203 DOI: 10.1186/s12874-016-0146-y] [Citation(s) in RCA: 30] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/14/2015] [Accepted: 04/12/2016] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND Several indirect comparison methods, including network meta-analyses (NMAs), using individual patient data (IPD) have been developed to synthesize evidence from a network of trials. Although IPD indirect comparisons are published with increasing frequency in health care literature, there is no guidance on selecting the appropriate methodology and on reporting the methods and results. METHODS In this paper we examine the methods and reporting of indirect comparison methods using IPD. We searched MEDLINE, Embase, the Cochrane Library, and CINAHL from inception until October 2014. We included published and unpublished studies reporting a method, application, or review of indirect comparisons using IPD and at least three interventions. RESULTS We identified 37 papers, including a total of 33 empirical networks. Of these, only 9 (27 %) IPD-NMAs reported the existence of a study protocol, whereas 3 (9 %) studies mentioned that protocols existed without providing a reference. The 33 empirical networks included 24 (73 %) IPD-NMAs and 9 (27 %) matching adjusted indirect comparisons (MAICs). Of the 21 (64 %) networks with at least one closed loop, 19 (90 %) were IPD-NMAs, 13 (68 %) of which evaluated the prerequisite consistency assumption, and only 5 (38 %) of the 13 IPD-NMAs used statistical approaches. The median number of trials included per network was 10 (IQR 4-19) (IPD-NMA: 15 [IQR 8-20]; MAIC: 2 [IQR 3-5]), and the median number of IPD trials included in a network was 3 (IQR 1-9) (IPD-NMA: 6 [IQR 2-11]; MAIC: 2 [IQR 1-2]). Half of the networks (17; 52 %) applied Bayesian hierarchical models (14 one-stage, 1 two-stage, 1 used IPD as an informative prior, 1 unclear-stage), including either IPD alone or with aggregated data (AD). Models for dichotomous and continuous outcomes were available (IPD alone or combined with AD), as were models for time-to-event data (IPD combined with AD). CONCLUSIONS One in three indirect comparison methods modeling IPD adjusted results from different trials to estimate effects as if they had come from the same, randomized, population. Key methodological and reporting elements (e.g., evaluation of consistency, existence of study protocol) were often missing from an indirect comparison paper.
Collapse
Affiliation(s)
- Areti Angeliki Veroniki
- />Li Ka Shing Knowledge Institute, St. Michael’s Hospital, 209 Victoria Street, East Building, Toronto, ON M5B 1T8 Canada
| | - Sharon E. Straus
- />Li Ka Shing Knowledge Institute, St. Michael’s Hospital, 209 Victoria Street, East Building, Toronto, ON M5B 1T8 Canada
- />Department of Geriatric Medicine, Faculty of Medicine, University of Toronto, 27 King’s College Circle, Toronto, ON M5S 1A1 Canada
| | - Charlene Soobiah
- />Li Ka Shing Knowledge Institute, St. Michael’s Hospital, 209 Victoria Street, East Building, Toronto, ON M5B 1T8 Canada
- />Institute of Health Policy, Management and Evaluation, University of Toronto, Health Sciences Building, 155 College Street, 4th floor, Toronto, ON M5T 3M6 Canada
| | - Meghan J. Elliott
- />Li Ka Shing Knowledge Institute, St. Michael’s Hospital, 209 Victoria Street, East Building, Toronto, ON M5B 1T8 Canada
| | - Andrea C. Tricco
- />Li Ka Shing Knowledge Institute, St. Michael’s Hospital, 209 Victoria Street, East Building, Toronto, ON M5B 1T8 Canada
- />Epidemiology Division, Dalla Lana School of Public Health, University of Toronto, 155 College Street, 6th floor, Toronto, ON M5T 3M7 Canada
| |
Collapse
|