Statistical analyses and quality of individual participant data network meta-analyses were suboptimal: a cross-sectional study

Transitivity, coherence, and reliability of network meta-analyses comparing proximal humerus fracture treatments: a meta-epidemiological study

BACKGROUND

Network meta-analyses can be valuable for decision-makers in guiding clinical practice. However, for network meta-analysis results to be reliable, the assumptions of both transitivity and coherence must be met, and the methodology should adhere to current best practices. We aimed to assess whether network meta-analyses of randomized controlled trials (RCTs) comparing interventions for proximal humerus fractures provide reliable estimates of intervention effects.

METHODS

We searched PubMed, EMBASE, The Cochrane Library, and Web of Science for network meta-analyses comparing interventions for proximal humerus fractures. We critically assessed the methodology regarding the development of a protocol, search strategy, trial inclusion, outcome extraction, and the methods used to conduct the network meta-analyses. We assessed the transitivity and coherence of the network graphs for the Constant score (CS), Disabilities of the Arm, Shoulder, and Hand score (DASH), and additional surgery. Transitivity was assessed by comparing probable effect modifiers (age, gender, fracture morphology, and comorbidities) across intervention comparisons. Coherence was assessed using Separating Indirect from Direct Evidence (SIDE) (Separating Indirect from Direct Evidence) and the design-by-treatment interaction test. We used CINeMA (Confidence in Network Meta-analyses) to assess the confidence in the results.

RESULTS

None of the three included network meta-analyses had a publicly available protocol or data-analysis plan, and they all had methodological flaws that could threaten the validity of their results. Although we did not detect incoherence for most comparisons, the transitivity assumption was violated for CS, DASH, and additional surgery in all three network meta-analyses. Additionally, the confidence in the results was 'very low' primarily due to within-study bias, reporting bias, intransitivity, imprecision, and heterogeneity.

CONCLUSIONS

Current network meta-analyses of RCTs comparing interventions for proximal humerus fractures do not provide reliable estimates of intervention effects. We advise caution in using these network meta-analyses to guide clinical practice. To improve the utility of network meta-analyses to guide clinical practice, journal editors should require that network meta-analyses are done according to a predefined analysis plan in a publicly available protocol and that both coherence and transitivity have been adequately assessed and reported.

Reporting quality and risk of bias of Cochrane individual participant data meta-analyses: A cross-sectional study

OBJECTIVES

This study aimed to assess the reporting quality and risk of bias of Cochrane individual participant data meta-analyses (IPD-MAs).

METHODS

We searched the Cochrane Library and identified the Cochrane IPD-MAs. We used the Preferred Reporting Items for Systematic Review and Meta-Analyses of individual participant data (PRISMA-IPD) assessed the reporting quality of included Cochrane IPD- MAs, and the Risk Of Bias In Systematic reviews (ROBIS) tool was used to assess the risk of bias. We performed stratified and correlation analyses to explore factors affecting the quality.

RESULTS

Forty-six Cochrane IPD-MAs were included in our study. Twenty-six Cochrane IPD-MAs (56.5%) had statistical or epidemiological authors involved, and 31 (67.4%) contained only IPD data. Thirty-five studies (76.1%) did not report whether they used 1-stage or 2-stage methods, and forty (87.0%) did not report the statistical techniques used for missing participant data. We found that the entire compliance reported PRISMA-IPD items of Cochrane IPD-MAs published after 2015 (n = 18; Mean ± SD: 26.61 ± 2.75) was higher than those studies published in 2015 and before (n = 28; Mean ± SD: 22.61 ± 4.73), the difference was statistically significant (p = 0.002). A strong positive correlation was found between the fully reported PRISMA-IPD items and fully accordance ROBIS items (Spearman's: ρ = 0.653, p < 0.001).

CONCLUSIONS

The quality of Cochrane IPD-MAs is not high, especially in the reporting of statistical methods. There was room for further improvement in IPD retrieval, IPD integrity and statistical analyses.

Progress and challenges of network meta-analysis

In the past years, network meta-analysis (NMA) has been widely used among clinicians, guideline makers, and health technology assessment agencies and has played an important role in clinical decision-making and guideline development. To inform further development of NMAs, we conducted a bibliometric analysis to assess the current status of published NMA methodological studies, summarized the methodological progress of seven types of NMAs, and discussed the current challenges of NMAs.

Prespecification of subgroup analyses and examination of treatment-subgroup interactions in cancer individual participant data meta-analyses are suboptimal

OBJECTIVES

This study aimed to explore the prespecification and conduct of subgroup analyses in cancer individual participant data meta-analyses (IPDMAs).

STUDY DESIGN AND SETTING

We searched PubMed, Embase.com, Cochrane Library, and Web of Science to identify IPDMAs of randomized controlled trials evaluating intervention effects for cancer. We evaluated how often cancer IPDMAs prespecify subgroup analyses and statistical approaches for examining treatment-subgroup interactions and handling continuous subgroup variables.

RESULTS

We included 89 IPDMAs, of which 41 (46.1%) reported a statistically significant treatment-subgroup interaction (P < 0.05) in at least one subgroup analysis. 47 (52.8%) IPDMAs prespecified methods for conducting subgroup analyses and the remaining 42 (47.2%) did not prespecify subgroup analyses. Of the 47 IPDMAs prespecified subgroup analyses, 19 performed the planned subgroup analyses, 21 added subgroup analyses, 7 reduced subgroup analyses. Eighty IPDMAs examined treatment-subgroup interactions, but 72 IPDMAs did not provide enough information to determine whether an appropriate approach that avoided aggregation bias was used. 85 IPDMAs that used continuous variables in subgroup analyses categorized continuous variables and only 1 IPDMA examined non-linear relationships.

CONCLUSION

Many cancer IPDMAs did not prespecify subgroup analyses, nor did they fully perform planned subgroup analyses. Lack of details for the test of treatment-subgroup interactions and examination of non-linear interactions was suboptimal.