Deriving percentage study weights in multi-parameter meta-analysis models: with application to meta-regression, network meta-analysis and one-stage individual participant data models

Shortest path or random walks? A framework for path weights in network meta-analysis

Quantifying the contributions, or weights, of comparisons or single studies to the estimates in a network meta-analysis (NMA) is an active area of research. We extend this work to include the contributions of paths of evidence. We present a general framework, based on the path-design matrix, that describes the problem of finding path contributions as a linear equation. The resulting solutions may have negative coefficients. We show that two known approaches, called shortestpath and randomwalk, are special solutions of this equation, and both meet an optimization criterion, as they minimize the sum of absolute path contributions. In general, there is an infinite set of solutions, which can be identified using the generalized inverse (Moore-Penrose pseudoinverse). We consider two further special approaches. For large networks we find that shortestpath is superior with respect to run time and variability, compared to the other approaches, and is thus recommended in practice. The path-weights framework also has the potential to answer more general research questions in NMA.

Transcanal Transpromontorial Approaches to the Internal Auditory Canal: A Systematic Review

OBJECTIVE

Exclusive endoscopic (EETTA) and expanded (ExpTTA) transcanal transpromontorial approaches have shown promising results for treating internal auditory canal (IAC) lesions. We reviewed the literature to answer the question: "Do EETTA and ExpTTA achieve high rates of complete resection and low rates of complications in treating patients with IAC pathologies?"

DATA SOURCES

PubMed, EMBASE, Scopus, Web of Science, and Cochrane were searched.

REVIEW METHODS

Studies reporting EETTA/ExpTTA for IAC pathologies were included. Indications and techniques were discussed and meta-analyzed rates of outcomes and complications were obtained with random-effect model meta-analyses.

RESULTS

We included 16 studies comprising 173 patients, all with non-serviceable hearing. Baseline FN function was mostly House-Brackmann-I (96.5%; 95% CI: 94.9-98.1%). Most lesions were vestibular/cochlear schwannomas (98.3%; 95% CI: 96.7-99.8%) of Koos-I (45.9%; 95% CI: 41.3-50.3%) or II (47.1%; 95% CI: 43-51.1%). EETTA was performed in 101 patients (58.4%; 95% CI: 52.4-64.3%) and ExpTTA in 72 (41.6%; 95% CI: 35.6-47.6%), achieving gross-total resection in all cases. Transient complications occurred in 30 patients (17.3%; 95% CI: 13.9-20.5%), with meta-analyzed rates of 9% (95% CI: 4-15%), comprising FN palsy with spontaneous resolution (10.4%; 95% CI: 7.7-13.1%). Persistent complications occurred in 34 patients (19.6%; 95% CI: 17.1-22.2%), with meta-analyzed rates of 12% (95% CI: 7-19%), comprising persistent FN palsy in 22 patients (12.7%; 95% CI: 10.2-15.2%). Mean follow-up was 16 months (range, 1-69; 95% CI: 14.7-17.4). Post-surgery FN function was stable in 131 patients (75.8%; 95% CI: 72.1-79.5%), worsened in 38 (21.9%; 95% CI: 18.8-25%), and improved in 4 (2.3%; 95% CI: 0.7-3.9%), with meta-analyzed rates of improved/stable response of 84% (95% CI: 76-90%).

CONCLUSION

Transpromontorial approaches offer newer routes for IAC surgery, but their restricted indications and unfavorable FN outcomes currently limit their use. Laryngoscope, 133:2856-2867, 2023.

Blueprint for the ideal microplastic effect study: Critical issues of current experimental approaches and envisioning a path forward

This article presents a novel conceptual blueprint for an 'ideal', i.e., ecologically relevant, microplastic effect study. The blueprint considers how microplastics should be characterized and applied in laboratory experiments, and how biological responses should be measured to assure unbiased data that reliably reflect the effects of microplastics on aquatic biota. This 'ideal' experiment, although practically unachievable, serves as a backdrop to improve specific aspects of experimental research on microplastic effects. In addition, a systematic and quantitative literature review identified and quantified departures of published experiments from the proposed 'ideal' design. These departures are related mainly to the experimental design of microplastic effect studies failing to mimic natural environments, and experiments with limited potential to be scaled-up to ecosystem level. To produce a valid and generalizable assessment of the effect of microplastics on biota, a quantitative meta-analysis was performed that incorporated the departure of studies from the 'ideal' experiment (a measure of experimental quality) and inverse variance (a measure of the study precision) as weighting coefficients. Greater weights were assigned to experiments with higher quality and/or with lower variance in the response variables. This double-weighting captures jointly the technical quality, ecological relevance and precision of estimates provided in each study. The blueprint and associated meta-analysis provide an improved baseline for the design of ecologically relevant and technically sound experiments to understand the effects of microplastics on single species, populations and, ultimately, entire ecosystems.

Graphical enhancements to summary receiver operating characteristic plots to facilitate the analysis and reporting of meta-analysis of diagnostic test accuracy data

Diagnostic test accuracy (DTA) systematic reviews are conducted to summarize evidence on the accuracy of a diagnostic test including a critical evaluation of the primary studies. Where appropriate, the evidence is meta-analyzed to obtain pooled estimates of effectiveness.In this study, we reviewed and critiqued three DTA guidance documents with respect to the graphical presentation of DTA meta-analysis results. All three documents recommended the use of two forms of graphical presentation: (a) forest plots displaying meta-analysis results for sensitivity (ie, the true positive rate) and specificity (ie, true negative rate) separately, and (b) Summary Receiver Operating Characteristic (SROC) curve to provide a global summary of test performance. Two primary shortcomings were identified: (a) lack of incorporation of quality assessment results into the main analysis and; (b) ambiguity with which the contribution of individual studies is represented on SROC curves. In response, two alternative graphical approaches were developed: A quality assessment enhanced SROC plot which displays the results from individual studies in the meta-analysis with multiple indicators of quality assessed using QUADAS-2; and A percentage study weights enhanced SROC plot which accurately portrays the percentage contribution each study makes to the meta-analysis. The proposed enhanced SROC curves facilitate the exploration of DTA data, leading to a deeper understanding of the primary studies included in a DTA meta-analysis including identifying reasons for between study heterogeneity and why specific study results may be divergent. Both plots can easily be produced in the free online interactive application, MetaDTA (https://crsu.shinyapps.io/dta_ma/).

The statistical importance of a study for a network meta-analysis estimate

Background

In pairwise meta-analysis, the contribution of each study to the pooled estimate is given by its weight, which is based on the inverse variance of the estimate from that study. For network meta-analysis (NMA), the contribution of direct (and indirect) evidence is easily obtained from the diagonal elements of a hat matrix. It is, however, not fully clear how to generalize this to the percentage contribution of each study to a NMA estimate.

Methods

We define the importance of each study for a NMA estimate by the reduction of the estimate’s variance when adding the given study to the others. An equivalent interpretation is the relative loss in precision when the study is left out. Importances are values between 0 and 1. An importance of 1 means that the study is an essential link of the pathway in the network connecting one of the treatments with another.

Results

Importances can be defined for two-stage and one-stage NMA. These numbers in general do not add to one and thus cannot be interpreted as ‘percentage contributions’. After briefly discussing other available approaches, we question whether it is possible to obtain unique percentage contributions for NMA.

Conclusions

Importances generalize the concept of weights in pairwise meta-analysis in a natural way. Moreover, they are uniquely defined, easily calculated, and have an intuitive interpretation. We give some real examples for illustration.

Statistical analyses and quality of individual participant data network meta-analyses were suboptimal: a cross-sectional study

BACKGROUND

Network meta-analyses using individual participant data (IPD-NMAs) have been increasingly used to compare the effects of multiple interventions. Although there have been many studies on statistical methods for IPD-NMAs, it is unclear whether there are statistical defects in published IPD-NMAs and whether the reporting of statistical analyses has improved. This study aimed to investigate statistical methods used and assess the reporting and methodological quality of IPD-NMAs.

METHODS

We searched four bibliographic databases to identify published IPD-NMAs. The methodological quality was assessed using AMSTAR-2 and reporting quality assessed based on PRISMA-IPD and PRISMA-NMA. We performed stratified analyses and correlation analyses to explore the factors that might affect quality.

RESULTS

We identified 21 IPD-NMAs. Only 23.8% of the included IPD-NMAs reported statistical techniques used for missing participant data, 42.9% assessed the consistency, and none assessed the transitivity. None of the included IPD-NMAs reported sources of funding for trials included, only 9.5% stated pre-registration of protocols, and 28.6% assessed the risk of bias in individual studies. For reporting quality, compliance rates were lower than 50.0% for more than half of the items. Less than 15.0% of the IPD-NMAs reported data integrity, presented the network geometry, or clarified risk of bias across studies. IPD-NMAs with statistical or epidemiological authors often better assessed the inconsistency (P = 0.017). IPD-NMAs with a priori protocol were associated with higher reporting quality in terms of search (P = 0.046), data collection process (P = 0.031), and syntheses of results (P = 0.006).

CONCLUSIONS

The reporting of statistical methods and compliance rates of methodological and reporting items of IPD-NMAs were suboptimal. Authors of future IPD-NMAs should address the identified flaws and strictly adhere to methodological and reporting guidelines.

Estimating the contribution of studies in network meta-analysis: paths, flows and streams

In network meta-analysis, it is important to assess the influence of the limitations or other characteristics of individual studies on the estimates obtained from the network. The proportion contribution matrix, which shows how much each direct treatment effect contributes to each treatment effect estimate from network meta-analysis, is crucial in this context. We use ideas from graph theory to derive the proportion that is contributed by each direct treatment effect. We start with the 'projection' matrix in a two-step network meta-analysis model, called the H matrix, which is analogous to the hat matrix in a linear regression model. We develop a method to translate H entries to proportion contributions based on the observation that the rows of H  can be interpreted as flow networks, where a stream is defined as the composition of a path and its associated flow. We present an algorithm that identifies the flow of evidence in each path and decomposes it into direct comparisons. To illustrate the methodology, we use two published networks of interventions. The first compares no treatment, quinolone antibiotics, non-quinolone antibiotics and antiseptics for underlying eardrum perforations and the second compares 14 antimanic drugs. We believe that this approach is a useful and novel addition to network meta-analysis methodology, which allows the consistent derivation of the proportion contributions of direct evidence from individual studies to network treatment effects.

Estimating the contribution of studies in network meta-analysis: paths, flows and streams

In network meta-analysis, it is important to assess the influence of the limitations or other characteristics of individual studies on the estimates obtained from the network. The proportion contribution matrix, which shows how much each direct treatment effect contributes to each treatment effect estimate from network meta-analysis, is crucial in this context. We use ideas from graph theory to derive the proportion that is contributed by each direct treatment effect. We start with the ‘projection’ matrix in a two-step network meta-analysis model, called the H matrix, which is analogous to the hat matrix in a linear regression model. We develop a method to translate H entries to proportion contributions based on the observation that the rows of H can be interpreted as flow networks, where a stream is defined as the composition of a path and its associated flow. We present an algorithm that identifies the flow of evidence in each path and decomposes it into direct comparisons. To illustrate the methodology, we use two published networks of interventions. The first compares no treatment, quinolone antibiotics, non-quinolone antibiotics and antiseptics for underlying eardrum perforations and the second compares 14 antimanic drugs. We believe that this approach is a useful and novel addition to network meta-analysis methodology, which allows the consistent derivation of the proportion contributions of direct evidence from individual studies to network treatment effects.

Graphs of study contributions and covariate distributions for network meta-regression

Background

Meta‐regression results must be interpreted taking into account the range of covariate values of the contributing studies. Results based on interpolation or extrapolation may be unreliable. In network meta‐regression (NMR) models, which include covariates in network meta‐analyses, results are estimated using direct and indirect evidence; therefore, it may be unclear which studies and covariate values contribute to which result. We propose graphs to help understand which trials and covariate values contribute to each NMR result and to highlight extrapolation or interpolation.

Methods

We introduce methods to calculate the contribution that each trial and covariate value makes to each result and compare them with existing methods. We show how to construct graphs including a network covariate distribution diagram, covariate‐contribution plot, heat plot, contribution‐NMR plot, and heat‐NMR plot. We demonstrate the methods using a dataset with treatments for malaria using the covariate average age and a dataset of topical fluoride interventions for preventing dental caries using the covariate randomisation year.

Results

For the malaria dataset, no contributing trials had an average age between 7–25 years and therefore results were interpolated within this range. For the fluoride dataset, there are no contributing trials randomised between 1954–1959 for most comparisons therefore, within this range, results would be extrapolated.

Conclusions

Even in a fully connected network, an NMR result may be estimated from trials with a narrower covariate range than the range of the whole dataset. Calculating contributions and graphically displaying them aids interpretation of NMR result by highlighting extrapolated or interpolated results.

Multivariate and network meta-analysis of multiple outcomes and multiple treatments: rationale, concepts, and examples

Organisations such as the National Institute for Health and Care Excellence require the synthesis of evidence from existing studies to inform their decisions—for example, about the best available treatments with respect to multiple efficacy and safety outcomes. However, relevant studies may not provide direct evidence about all the treatments or outcomes of interest. Multivariate and network meta-analysis methods provide a framework to address this, using correlated or indirect evidence from such studies alongside any direct evidence. In this article, the authors describe the key concepts and assumptions of these methods, outline how correlated and indirect evidence arises, and illustrate the contribution of such evidence in real clinical examples involving multiple outcomes and multiple treatments