Bayesian bivariate meta-analysis of correlated effects: Impact of the prior distributions on the between-study correlation, borrowing of strength, and joint inferences

Predictive values of procalcitonin for coinfections in patients with COVID-19: a systematic review and meta-analysis

OBJECTIVES

To assess the ability of procalcitonin (PCT)-a promising marker for coinfections-to predict coinfections in patients with COVID-19.

METHODS

In this systematic review and meta-analysis, PubMed, Embase, Web of Science, Cochrane, the China National Knowledge Infrastructure (CNKI), and Wanfang were searched to identify eligible studies (up to August 30, 2021). Articles that reported the predictive value of PCT for coinfections in patients with COVID-19 were included. Individual and pooled sensitivities and specificities were reported, and I² was used to test heterogeneity. This study was prospectively registered on the International Prospective Register of Systematic Reviews (PROSPERO) database (registration number: CRD42021283344).

RESULTS

Five studies involving a total of 2775 patients reported the predictive value of PCT for coinfections in patients with COVID-19. The sensitivity, specificity, and area under the curve of PCT in predicting coinfections in the pooled studies were 0.60 (95% CI 0.35-0.81, I² = 88.85), 0.71 (95% CI 0.58-0.81, I² = 87.82), and 0.72(95% CI 0.68-0.76) respectively.

CONCLUSIONS

Although PCT has limited predictive value for coinfections in patients with COVID-19, lower PCT levels seem to indicate a decreased probability of having a coinfection.

To seal or not to seal following an antimicrobial infusion at dry-off? A systematic review and multivariate meta-analysis of the incidence and prevalence of intramammary infections post-calving in dairy cows

Teat sealants (TSs) consist of sterile formulations with no antibacterial activity. Alone or in combination with antimicrobial (AM) or non-AM treatments, TSs have been commonly used in dairy cows at dry-off to prevent intra-mammary infections (IMIs) during the dry period. This study aimed to identify and synthesise the available evidence on the efficacy of combining TSs with AM treatments on the incidence and prevalence of IMIs. A comprehensive search of three electronic databases, two relevant conference proceedings, and reference lists of reviews and eligible articles was conducted to retrieve and identify studies that could answer the following question: in dairy cows, how does the efficacy of an AM-TS combination administered at dry-off compare with an AM alone for preventing new IMI? In addition to the general IMIs, bacterial species-specific data were extracted and combined into nine distinct pathogen groups: coagulase-positive and negative staphylococci; S. dysgalactiae; non-dysgalactiae Streptococci; E. coli; non-E. coli Enterobacteriaceae; Corynebacterium spp.; yeast and other frequent mastitis pathogens. The structural relationship between each study's prevalence and incidence, as the new (incidence) and persistent (uncured) infections make up the prevalence, was utilised to approximate a variance-covariance matrix for the within-study correlation between their study-specific log odds ratios (ORs). A bivariate random-effects meta-analysis was employed, utilising the within- and between-study correlations to synthesise both outcomes simultaneously. The risk of bias was assessed using the Cochrane ROBINS-I tool, and the quality of the body of evidence was rated using the GRADE approach. A total of 17 trials (16 studies), providing either IMIs incidence (n = 4), prevalence (n = 3) or both (n = 10), were identified. Overall, quarters infused with AM-TS combinations showed lower odds of new IMIs post-calving (OR=0.70; 95% CI=0.57-0.86; Wald test P < 0.001) than those which received only AMs. Across the pathogen groups, varying levels of reduction of new IMIs were found, where administration of TSs was most effective against S. dysgalactiae (OR=0.47; 95% CI=0.23-0.98), non-dysgalactiae streptococci (OR=0.60; 95% CI=0.49-0.74), E. coli (OR=0.62; 95% CI=0.50-0.77), Corynebacterium spp. (OR=0.68; 95% CI=0.52-0.90) and coagulase-negative staphylococci (OR=0.85; 95% CI=0.76-0.94). However, additional TS infusion did not significantly reduce new IMIs in the remaining pathogen groups. The current meta-analytic evidence supports the efficacy of using TS add-on infusions in dairy cows at dry-off for reducing the incidence and prevalence of IMIs post-calving; however, pathogen group differences should be considered.

Bayesian hierarchical models for adaptive basket trial designs

Basket trials evaluate a single drug targeting a single genetic variant in multiple cancer cohorts. Empirical findings suggest that treatment efficacy across baskets may be heterogeneous. Most modern basket trial designs use Bayesian methods. These methods require the prior specification of at least one parameter that permits information sharing across baskets. In this study, we provide recommendations for selecting a prior for scale parameters for adaptive basket trials by using Bayesian hierarchical modeling. Heterogeneity among baskets attracts much attention in basket trial research, and substantial heterogeneity challenges the basic assumption of exchangeability of Bayesian hierarchical approach. Thus, we also allowed each stratum-specific parameter to be exchangeable or nonexchangeable with similar strata by using data observed in an interim analysis. Through a simulation study, we evaluated the overall performance of our design based on statistical power and type I error rates. Our research contributes to the understanding of the properties of Bayesian basket trial designs.

Incorporating historical control information in ANCOVA models using the meta-analytic-predictive approach

The meta-analytic-predictive (MAP) approach is a Bayesian meta-analytic method to synthesize and incorporate information from historical controls in the analysis of a new trial. Classically, only a single parameter, typically the intercept or rate, is assumed to vary across studies, which may not be realistic in more complex models. Analysis of covariance (ANCOVA) is often used to analyze trials with a pretest-posttest design, where both the intercept and the baseline effect (coefficient of the outcome at baseline) affect the estimated treatment effect. We extended the MAP approach to ANCOVA, to allow for variation in the intercept and the baseline effect across studies, and possibly also correlation between these parameters. The method was illustrated using data from the Alzheimer's Disease Cooperative Study (ADCS) and assessed with a simulation study. In the ADCS data, the proposed multivariate MAP approach yielded a prior effective sample size of 79 and 58 for the intercept and the baseline effect respectively and reduced the posterior standard deviation of the treatment effect by 12.6%. The result was robust to the choice of prior for the between-study variation. In the simulations, the proposed approach yielded power gains with a good control of the type I error rate. Ignoring the between-study correlation of the parameters or assuming no variation in the baseline effect generally led to less power gain. In conclusion, the MAP approach can be extended to a multivariate version for ANCOVA, which may improve the estimation of the treatment effect.

Multivariate meta-analysis of multiple outcomes: characteristics and predictors of borrowing of strength from Cochrane reviews

OBJECTIVES

Multivariate meta-analysis allows the joint synthesis of multiple outcomes accounting for their correlation. This enables borrowing of strength (BoS) across outcomes, which may lead to greater efficiency and even different conclusions compared to separate univariate meta-analyses. However, multivariate meta-analysis is complex to apply, so guidance is needed to flag (in advance of analysis) when the approach is most useful.

STUDY DESIGN AND SETTING

We use 43 Cochrane intervention reviews to empirically investigate the characteristics of meta-analysis datasets that are associated with a larger BoS statistic (from 0 to 100%) when applying a bivariate meta-analysis of binary outcomes.

RESULTS

Four characteristics were identified as strongly associated with BoS: the total number of studies, the number of studies with the outcome of interest, the percentage of studies missing the outcome of interest, and the largest absolute within-study correlation. Using these characteristics, we then develop a model for predicting BoS in a new dataset, which is shown to have good performance (an adjusted R² of 50%). Applied examples are used to illustrate the use of the BoS prediction model.

CONCLUSIONS

Cochrane reviewers mainly use univariate meta-analysis methods, but the identified characteristics associated with BoS and our subsequent prediction model for BoS help to flag when a multivariate meta-analysis may also be beneficial in Cochrane reviews with multiple binary outcomes. Extension to non-Cochrane reviews and other outcome types is still required.

Challenges of modelling approaches for network meta-analysis of time-to-event outcomes in the presence of non-proportional hazards to aid decision making: Application to a melanoma network

BACKGROUND

Synthesis of clinical effectiveness from multiple trials is a well-established component of decision-making. Time-to-event outcomes are often synthesised using the Cox proportional hazards model assuming a constant hazard ratio over time. However, with an increasing proportion of trials reporting treatment effects where hazard ratios vary over time and with differing lengths of follow-up across trials, alternative synthesis methods are needed.

OBJECTIVES

To compare and contrast five modelling approaches for synthesis of time-to-event outcomes and provide guidance on key considerations for choosing between the modelling approaches.

METHODS

The Cox proportional hazards model and five other methods of estimating treatment effects from time-to-event outcomes, which relax the proportional hazards assumption, were applied to a network of melanoma trials reporting overall survival: restricted mean survival time, generalised gamma, piecewise exponential, fractional polynomial and Royston-Parmar models.

RESULTS

All models fitted the melanoma network acceptably well. However, there were important differences in extrapolations of the survival curve and interpretability of the modelling constraints demonstrating the potential for different conclusions from different modelling approaches.

CONCLUSION

The restricted mean survival time, generalised gamma, piecewise exponential, fractional polynomial and Royston-Parmar models can accommodate non-proportional hazards and differing lengths of trial follow-up within a network meta-analysis of time-to-event outcomes. We recommend that model choice is informed using available and relevant prior knowledge, model transparency, graphically comparing survival curves alongside observed data to aid consideration of the reliability of the survival estimates, and consideration of how the treatment effect estimates can be incorporated within a decision model.

Bayesian multivariate network meta-analysis model for the difference in restricted mean survival times

Network meta-analysis (NMA) is essential for clinical decision-making. NMA enables inference for all pair-wise comparisons between interventions available for the same indication, by using both direct evidence and indirect evidence. In randomized trials with time-to event outcome data, such as lung cancer data, conventional NMA methods rely on the hazard ratio and the proportional hazards assumption, and ignore the varying follow-up durations across trials. We introduce a novel multivariate NMA model for the difference in restricted mean survival times (RMST). Our model synthesizes all the available evidence from multiple time points simultaneously and borrows information across time points through within-study covariance and between-study covariance for the differences in RMST. We propose an estimator of the within-study covariance and we then assume it to be known. We estimate the model under the Bayesian framework. We evaluated our model by conducting a simulation study. Our multiple-time-point model yields lower mean squared error over the conventional single-time-point model at all time points, especially when the availability of evidence decreases. We illustrated the model on a network of randomized trials of second-line treatments of advanced non-small-cell lung cancer. Our multiple-time-point model yielded increased precision and detected evidence of benefit at earlier time points as compared to the single-time-point model. Our model has the advantage of providing clinically interpretable measures of treatment effects.

Measurement error in meta-analysis (MEMA)-A Bayesian framework for continuous outcome data subject to non-differential measurement error

Ideally, a meta-analysis will summarize data from several unbiased studies. Here we look into the less than ideal situation in which contributing studies may be compromised by non-differential measurement error in the exposure variable. Specifically, we consider a meta-analysis for the association between a continuous outcome variable and one or more continuous exposure variables, where the associations may be quantified as regression coefficients of a linear regression model. A flexible Bayesian framework is developed which allows one to obtain appropriate point and interval estimates with varying degrees of prior knowledge about the magnitude of the measurement error. We also demonstrate how, if individual-participant data (IPD) are available, the Bayesian meta-analysis model can adjust for multiple participant-level covariates, these being measured with or without measurement error.

Accuracy of Heparin-Binding Protein in Diagnosing Sepsis: A Systematic Review and Meta-Analysis

OBJECTIVES

Existing studies evaluating the accuracy of heparin-binding protein for the diagnosis of sepsis have been inconsistent. We conducted a systematic review and meta-analysis to assess the totality of current evidence regarding the utility of heparin-binding protein to diagnose sepsis in patients with presumed systemic infection.

DATA SOURCE

PubMed, Embase, the China National Knowledge infrastructure, and WangFang electronic database were searched from inception to December of 2019.

STUDY SELECTION

Two independent reviewers identified eligible studies. Cohort and case-control studies, which measured serum levels of heparin-binding protein among adult patients with suspected sepsis, were eligible for inclusion.

DATA EXTRACTION

Two reviewers independently extracted data elements from the selected studies. A bivariate random-effects meta-analysis model was used to synthesize the prognostic accuracy measures. Risk of bias of studies was assessed with Quality Assessment of Diagnostic Accuracy Studies 2 tool.

DATA SYNTHESIS

We identified 26 studies with 3,868 patients in the meta-analysis. Heparin-binding protein had a pooled sensitivity of 0.85 (95% CI, 0.79-0.90) and a pooled specificity of 0.91 (95% CI, 0.82-0.96) for the diagnosis of sepsis. There was low heterogeneity between the studies (I2 = 12%), and no evidence of publication bias was detected. Heparin-binding protein had a higher sensitivity and specificity when compared with procalcitonin (0.75 [95% CI, 0.62-0.85] and 0.85 [95% CI, 0.73-0.92]) as well as C-reactive protein (0.75 [95% CI, 0.65-0.84] and 0.71 [95% CI, 0.63-0.77]). Serial measurements of heparin-binding protein also showed that heparin-binding protein levels rose significantly at least 24 hours before a diagnosis of sepsis.

CONCLUSIONS

The diagnostic ability of heparin-binding protein is favorable, demonstrating both high sensitivity and specificity in predicting progression to sepsis in critically ill patients. Future studies could assess the incremental value that heparin-binding protein may add to a multimodal sepsis identification and prognostication algorithm for critically ill patients.

Clinical prediction models to predict the risk of multiple binary outcomes: a comparison of approaches

Clinical prediction models (CPMs) can predict clinically relevant outcomes or events. Typically, prognostic CPMs are derived to predict the risk of a single future outcome. However, there are many medical applications where two or more outcomes are of interest, meaning this should be more widely reflected in CPMs so they can accurately estimate the joint risk of multiple outcomes simultaneously. A potentially naïve approach to multi-outcome risk prediction is to derive a CPM for each outcome separately, then multiply the predicted risks. This approach is only valid if the outcomes are conditionally independent given the covariates, and it fails to exploit the potential relationships between the outcomes. This paper outlines several approaches that could be used to develop CPMs for multiple binary outcomes. We consider four methods, ranging in complexity and conditional independence assumptions: namely, probabilistic classifier chain, multinomial logistic regression, multivariate logistic regression, and a Bayesian probit model. These are compared with methods that rely on conditional independence: separate univariate CPMs and stacked regression. Employing a simulation study and real-world example, we illustrate that CPMs for joint risk prediction of multiple outcomes should only be derived using methods that model the residual correlation between outcomes. In such a situation, our results suggest that probabilistic classification chains, multinomial logistic regression or the Bayesian probit model are all appropriate choices. We call into question the development of CPMs for each outcome in isolation when multiple correlated or structurally related outcomes are of interest and recommend more multivariate approaches to risk prediction.

Combining tumour response and progression free survival as surrogate endpoints for overall survival in advanced colorectal cancer

BACKGROUND

Progression free survival (PFS) and tumour response (TR) have been investigated as surrogate endpoints for overall survival (OS) in advanced colorectal cancer (aCRC), however their validity has been shown to be suboptimal. In recent years, meta-analytic methods allowing for use of multiple surrogate endpoints jointly have been proposed. Our aim was to assess if PFS and TR used jointly as surrogate endpoints to OS improve their predictive value.

METHODS

Data were obtained from a systematic review of randomised controlled trials investigating effectiveness of pharmacological therapies in aCRC, including systemic chemotherapies, anti-epidermal growth factor receptor therapies and anti-angiogenic agents. Multivariate meta-analysis was used to model the association patterns between treatment effects on the surrogate endpoints (TR, PFS) and the final outcome (OS).

RESULTS

Analysis of 33 trials reporting treatment effects on all three outcomes showed reasonably strong association between treatment effects on PFS and OS, however the association parameters were obtained with a large uncertainty. A weak surrogate relationship was noted between the treatment effects on TR and OS. Modelling the two surrogate endpoints, TR and PFS, jointly as predictors of treatment effect on OS gave no marked improvement to surrogate association patterns. Modest improvement in the precision of the predicted treatment effects on the final outcome was noted in studies investigating anti-angiogenic therapy, however it was likely due to chance.

CONCLUSION

The joint use of two surrogate endpoints did not lead to marked improvement in the association between treatment effects on surrogate and final endpoints in advanced colorectal cancer.

A framework for meta-analysis of prediction model studies with binary and time-to-event outcomes

It is widely recommended that any developed-diagnostic or prognostic-prediction model is externally validated in terms of its predictive performance measured by calibration and discrimination. When multiple validations have been performed, a systematic review followed by a formal meta-analysis helps to summarize overall performance across multiple settings, and reveals under which circumstances the model performs suboptimal (alternative poorer) and may need adjustment. We discuss how to undertake meta-analysis of the performance of prediction models with either a binary or a time-to-event outcome. We address how to deal with incomplete availability of study-specific results (performance estimates and their precision), and how to produce summary estimates of the c-statistic, the observed:expected ratio and the calibration slope. Furthermore, we discuss the implementation of frequentist and Bayesian meta-analysis methods, and propose novel empirically-based prior distributions to improve estimation of between-study heterogeneity in small samples. Finally, we illustrate all methods using two examples: meta-analysis of the predictive performance of EuroSCORE II and of the Framingham Risk Score. All examples and meta-analysis models have been implemented in our newly developed R package "metamisc".

Bivariate network meta-analysis for surrogate endpoint evaluation

Surrogate endpoints are very important in regulatory decision making in healthcare, in particular if they can be measured early compared to the long-term final clinical outcome and act as good predictors of clinical benefit. Bivariate meta-analysis methods can be used to evaluate surrogate endpoints and to predict the treatment effect on the final outcome from the treatment effect measured on a surrogate endpoint. However, candidate surrogate endpoints are often imperfect, and the level of association between the treatment effects on the surrogate and final outcomes may vary between treatments. This imposes a limitation on methods which do not differentiate between the treatments. We develop bivariate network meta-analysis (bvNMA) methods, which combine data on treatment effects on the surrogate and final outcomes, from trials investigating multiple treatment contrasts. The bvNMA methods estimate the effects on both outcomes for all treatment contrasts individually in a single analysis. At the same time, they allow us to model the trial-level surrogacy patterns within each treatment contrast and treatment-level surrogacy, thus enabling predictions of the treatment effect on the final outcome either for a new study in a new population or for a new treatment. Modelling assumptions about the between-studies heterogeneity and the network consistency, and their impact on predictions, are investigated using an illustrative example in advanced colorectal cancer and in a simulation study. When the strength of the surrogate relationships varies across treatment contrasts, bvNMA has the advantage of identifying treatment comparisons for which surrogacy holds, thus leading to better predictions.

Random-effects meta-analysis of the clinical utility of tests and prediction models

The use of data from multiple studies or centers for the validation of a clinical test or a multivariable prediction model allows researchers to investigate the test's/model's performance in multiple settings and populations. Recently, meta-analytic techniques have been proposed to summarize discrimination and calibration across study populations. Here, we rather consider performance in terms of net benefit, which is a measure of clinical utility that weighs the benefits of true positive classifications against the harms of false positives. We posit that it is important to examine clinical utility across multiple settings of interest. This requires a suitable meta-analysis method, and we propose a Bayesian trivariate random-effects meta-analysis of sensitivity, specificity, and prevalence. Across a range of chosen harm-to-benefit ratios, this provides a summary measure of net benefit, a prediction interval, and an estimate of the probability that the test/model is clinically useful in a new setting. In addition, the prediction interval and probability of usefulness can be calculated conditional on the known prevalence in a new setting. The proposed methods are illustrated by 2 case studies: one on the meta-analysis of published studies on ear thermometry to diagnose fever in children and one on the validation of a multivariable clinical risk prediction model for the diagnosis of ovarian cancer in a multicenter dataset. Crucially, in both case studies the clinical utility of the test/model was heterogeneous across settings, limiting its usefulness in practice. This emphasizes that heterogeneity in clinical utility should be assessed before a test/model is routinely implemented.

Bayesian one-step IPD network meta-analysis of time-to-event data using Royston-Parmar models

Network meta‐analysis (NMA) combines direct and indirect evidence from trials to calculate and rank treatment estimates. While modelling approaches for continuous and binary outcomes are relatively well developed, less work has been done with time‐to‐event outcomes. Such outcomes are usually analysed using Cox proportional hazard (PH) models. However, in oncology with longer follow‐up time, and time‐dependent effects of targeted treatments, this may no longer be appropriate. Network meta‐analysis conducted in the Bayesian setting has been increasing in popularity. However, fitting the Cox model is computationally intensive, making it unsuitable for many datasets. Royston‐Parmar models are a flexible alternative that can accommodate time‐dependent effects. Motivated by individual participant data (IPD) from 37 cervical cancer trials (5922 women) comparing surgery, radiotherapy, and chemotherapy, this paper develops an IPD Royston‐Parmar Bayesian NMA model for overall survival. We give WinBUGS code for the model. We show how including a treatment‐ln(time) interaction can be used to conduct a global test for PH, illustrate how to test for consistency of direct and indirect evidence, and assess within‐design heterogeneity. Our approach provides a computationally practical, flexible Bayesian approach to NMA of IPD survival data, which readily extends to include additional complexities, such as non‐PH, increasingly found in oncology trials.

Bayesian bivariate meta-analysis of diagnostic test studies with interpretable priors

In a bivariate meta-analysis, the number of diagnostic studies involved is often very low so that frequentist methods may result in problems. Using Bayesian inference is particularly attractive as informative priors that add a small amount of information can stabilise the analysis without overwhelming the data. However, Bayesian analysis is often computationally demanding and the selection of the prior for the covariance matrix of the bivariate structure is crucial with little data. The integrated nested Laplace approximations method provides an efficient solution to the computational issues by avoiding any sampling, but the important question of priors remain. We explore the penalised complexity (PC) prior framework for specifying informative priors for the variance parameters and the correlation parameter. PC priors facilitate model interpretation and hyperparameter specification as expert knowledge can be incorporated intuitively. We conduct a simulation study to compare the properties and behaviour of differently defined PC priors to currently used priors in the field. The simulation study shows that the PC prior seems beneficial for the variance parameters. The use of PC priors for the correlation parameter results in more precise estimates when specified in a sensible neighbourhood around the truth. To investigate the usage of PC priors in practice, we reanalyse a meta-analysis using the telomerase marker for the diagnosis of bladder cancer and compare the results with those obtained by other commonly used modelling approaches. Copyright © 2017 John Wiley & Sons, Ltd.