Reasons for missing evidence in rehabilitation meta-analyses: a cross-sectional meta-research study

BACKGROUND

Systematic reviews of randomized controlled trials are the best evidence for informing on intervention effectiveness. Their results, however, can be biased due to omitted evidence in the quantitative analyses. We aimed to assess the proportion of randomized controlled trials omitted from meta-analyses in the rehabilitation field and explore related reasons.

METHODS

This is a cross-sectional meta-research study. For each systematic review included in a published selected sample in the rehabilitation field, we identified an index meta-analysis on the primary outcome and the main comparison. We then looked at all the studies considered eligible for the chosen comparison in the systematic review and identified those trials that have been omitted (i.e., not included) from each index meta-analysis. Reasons for omission were collected based on an eight-reason classification. We used descriptive statistics to describe the proportion of omitted trials overall and according to each reason.

RESULTS

Starting from a cohort of 827 systematic reviews, 131 index meta-analyses comprising a total of 1761 eligible trials were selected. Only 16 index meta-analyses included all eligible studies while 15 omitted studies without providing references. From the remaining 100 index meta-analyses, 717 trials (40,7%) were omitted overall. Specific reasons for omission were: "unable to distinguish between selective reporting and inadequate planning" (39,3%, N = 282), "inadequate planning" (17%, N = 122), "justified to be not included" (15,1%, N = 108), "incomplete reporting" (8,4%, N = 60), "selective reporting" (3,3%, N = 24) and other situations (e.g., outcome present but no motivation for omission) (5,2%, N = 37). The 11,7% (N = 84) of omitted trials were not assessed due to non-English language or full text not available.

CONCLUSIONS

Almost half of the eligible trials were omitted from their index meta-analyses. Better reporting, protocol registration, definition and adoption of core outcome sets are needed to prevent omission of evidence in systematic reviews.

A Tailored Approach: A model for literature searching in complex systematic reviews

Our previous work identified that nine leading guidance documents for seven different types of systematic review advocated the same process of literature searching. We defined and illustrated this process and we named it ‘the Conventional Approach’. The Conventional Approach appears to meet the needs of researchers undertaking literature searches for systematic reviews of clinical interventions. In this article, we report a new and alternate process model of literature searching called ‘A Tailored Approach’. A Tailored Approach is indicated as a search process for complex reviews which do not focus on the evaluation of clinical interventions. The aims of this article are to (1) explain the rationale for, and the theories behind, the design of A Tailored Approach; (2) report the current conceptual illustration of A Tailored Approach and to describe a user’s interaction with the process model; and (3) situate the elements novel to A Tailored Approach (when compared with the Conventional Approach) in the relevant literature. A Tailored Approach suggests investing time at the start of a review, to develop the information needs from the research objectives, and to tailor the search approach to studies or data. Tailored Approaches should be led by the information specialist (librarian) but developed by the research team. The aim is not necessarily to focus on comprehensive retrieval. Further research is indicated to evaluate the use of supplementary search methods, methods of team-working to define search approaches, and to evaluate the use of conceptual models of information retrieval for testing and evaluation.

What do cochrane systematic reviews say about interventions for enuresis in children and adolescents? An overview of systematic reviews

OBJECTIVE

To conduct an overview of Cochrane systematic reviews about treatment alternatives for children and/or adolescents with enuresis.

SOURCES

An overview of Cochrane systematic reviews about interventions for enuresis in children/adolescents was developed between September/2021 and December/2021. The protocol was registered on PROSPERO and the search was conducted only in the Cochrane Library database without any restriction. Reviews involving any type of intervention for the treatment of enuresis in children/adolescents were included. The risk of bias was assessed using Risk of Bias in Systematic Reviews (ROBIS) and the quality of reviews was assessed using A Measurement Tool to Assess Systematic Reviews (AMSTAR-2).

SUMMARY OF THE FINDINGS

Seven systematic reviews were identified. Based on the ROBIS assessment, all reviews were classified as low risk of bias. According to the AMSTAR-2 assessment, the three oldest systematic reviews were rated as critically low quality, one review was moderate quality, and the three most recent systematic reviews were rated as high quality. No difference was shown between alarm and desmopressin for a complete response to therapy after treatment (RR = 1.30; 95%CI: 0.92 to 1.84), but alarm use is related to a lower risk of adverse events (RR = 0.38; 95%CI: 0.20 to 0.71). There is a moderate certainty that the association between imipramine and oxybutynin is better than placebo to reduce the risk of children who do not achieve 14 consecutive dry nights after treatment (RR = 0.43; 95%CI: 0.23 to 0.78).

CONCLUSIONS

There is no difference between alarm and desmopressin for enuresis treatment. However, alarm therapy had fewer adverse events than desmopressin. Moreover, combination therapy between imipramine and oxybutynin is better than placebo.

Recommendations from Cochrane reviews for improving future trials on anesthesia and pain: a meta-research study

Background: Cochrane systematic reviews (CSRs) have a section 'Implications for research' where authors make suggestions for improving future research. The authors of the present study assessed the prevalence and time dynamics of different recommendations in the CSRs about anesthesia and pain. Methods: The authors included all CSRs published by the Cochrane Anaesthesia Group and Cochrane Pain and Palliative Care Group before 17 July 2020. The authors analyzed recommendations for improving future research listed in the 'Implications for research' section of these CSRs and categorized recommendations for improvements. Results: They analyzed 370 reviews. Four categories of recommendations were present in more than 40% of the reviews. Most reviews recommended a larger sample size and better outcome choice, study design and choice of future intervention. These recommendations gradually increased in frequency in the Cochrane Pain and Palliative Care Group and mainly decreased in the Cochrane Anaesthesia Group. Conclusion: Recommendations from CSRs offer useful advice for trialists designing new trials.

Evaluating the effects of maternal positions in childbirth: An overview of Cochrane Systematic Reviews

INTRODUCTION

The purpose of this study is to conduct an overview of Cochrane systematic reviews (SRs) evaluating the effects of maternal positions in childbirth in order to compile existing evidence for relevant research questions that have been addressed by more than one review, to provide a succinct summary of the up-to-date evidence and to identify areas for future research.

METHODS

An electronic search was conducted in the Cochrane database. Two primary outcomes were the duration of labor and birth, and operative birth. The quality of included reviews was assessed using the AMSTAR criteria, and the quality of the evidence was rated using the GRADE criteria.

RESULTS

We included 3 Cochrane SRs. There was a significant mean difference (MD) found in the duration of the first stage by 1 hour and 22 minutes (MD= -1.21; 95% CI: -2.35 - -0.07, I2=94%) and reduction in caesarean section rates (RR=0.71; 95% CI: 0.54-0.94, I2=0%) in the upright birth position group compared with the horizontal. Also, there was a statistically significant difference in the duration (minutes) of the second stage of labor (MD= -6.16; 95% CI: -9.74 - -2.59, I2=91%) and a reduction in assisted vaginal birth rates (RR=0.75, 95% CI: 0.66-0.86, I2=29%) in the upright group compared with the horizontal without epidural analgesia. The quality of evidence within the reviews was very low to moderate.

CONCLUSIONS

There is currently a limited body of evidence to clearly assess the benefits and risks of assuming upright positions during childbirth. The overview highlights the need for high-quality research studies, involving better definition and comprehensive assessment of the effects of squatting during childbirth.

Performance of conceptual framework elements for the retrieval of qualitative health literature: a case study

Objective

A growing volume of studies address methods for performing systematic reviews of qualitative studies. One such methodological aspect is the conceptual framework used to structure the review question and plan the search strategy for locating relevant studies. The purpose of this case study was to evaluate the retrieval potential of each element of conceptual frameworks in qualitative systematic reviews in the health sciences.

Methods

The presence of elements from conceptual frameworks in publication titles, abstracts, and controlled vocabulary in CINAHL and PubMed was analyzed using a set of qualitative reviews and their included studies as a gold standard. Using a sample of 101 publications, we determined whether particular publications could be retrieved if a specific element from the conceptual framework was used in the search strategy.

Results

We found that the relative recall of conceptual framework elements varied considerably, with higher recall for patient/population (99%) and research type (97%) and lower recall for intervention/phenomenon of interest (74%), outcome (79%), and context (61%).

Conclusion

The use of patient/population and research type elements had high relative recall for qualitative studies. However, other elements should be used with great care due to lower relative recall.

Methodologies for systematic reviews with meta-analysis of randomised clinical trials in pain, anaesthesia, and perioperative medicine

Systematic reviews and meta-analyses (SRMAs) are increasing in popularity, but should they be used to inform clinical decision-making in anaesthesia? We present evidence that the certainty of evidence from SRMAs in anaesthesia (and in general) may be unacceptably low because of risks of bias exaggerating treatment effects, unexplained heterogeneity reducing certainty in estimates, random errors, and widespread prevalence of publication bias. We also present the latest methodological advances to help improve the certainty of evidence from SRMAs. The target audience includes both review authors and practising clinicians to help with SRMA appraisal. Issues discussed include minimising risks of bias from included trials, trial sequential analysis to reduce random error, updated methods for presenting effect estimates, and novel publication bias tests for commonly used outcome measures. These methods can help to reduce spurious conclusions on clinical significance, explain statistical heterogeneity, and reduce false positives when evaluating small-study effects. By reducing concerns in these domains of Grading of Recommendations, Assessment, Development and Evaluation, it should help improve the certainty of evidence from SRMAs used for decision-making in anaesthesia, pain, and perioperative medicine.