Are Neonatal Trials Better Conducted and Reported over the Last 6 Decades? An Analysis on Their Risk-of-Bias Status in Cochrane Reviews

Blinding Assessments in Neonatal Ventilation Meta-Analyses: A Systematic Meta-Epidemiological Review

INTRODUCTION

Randomization and blinding are generally important in randomized trials. In neonatology, blinding of ventilation strategies is unfeasible if not impossible and we hypothesized that its importance has been overestimated, while the peculiarities of the neonatal patient and the specific outcomes have not been considered.

METHODS

For this meta-epidemiological review, we searched PubMed and Scopus databases in November 2023. We included all meta-analyses focusing on ventilation, published in past 5 years, and reporting either mortality or bronchopulmonary dysplasia (BPD) as an outcome. We extracted the information about how the authors had analyzed risk of bias and evidence certainty.

RESULTS

We screened 494 abstracts and included 40 meta-analyses. Overall, 13 of the 40 reviews assessed blinding properly. Australian and European authors were most likely to perform correct assessment of the blinding (p = 0.03) and the use of RoB 2.0 tool was also associated with proper assessment (p < 0.001). In multivariate regression, the use of RoB 2.0 was the only factor associated with a proper assessment (Beta 0.57 [95% confidence interval: 0.29-0.99]). GRADE ratings were performed in 25 reviews, and the authors downgraded the evidence certainty due to risk of bias in 19 of these and none of these reviews performed the blinding assessment correctly.

CONCLUSION

In past neonatal evidence syntheses, the role of blinding has been mostly overestimated, which has led to downgrading of evidence certainty. Objective outcomes (such as mortality and BPD) do not need to be downgraded due to lack of blinding, as the knowledge of the received intervention does not influence the outcome assessment.

Success of blinding a procedural intervention in a randomised controlled trial in preterm infants receiving respiratory support

BACKGROUND

Blinding of treatment allocation from treating clinicians in neonatal randomised controlled trials can minimise performance bias, but its effectiveness is rarely assessed.

METHODS

To examine the effectiveness of blinding a procedural intervention from treating clinicians in a multicentre randomised controlled trial of minimally invasive surfactant therapy versus sham treatment in preterm infants of gestation 25-28 weeks with respiratory distress syndrome. The intervention (minimally invasive surfactant therapy or sham) was performed behind a screen within the first 6 h of life by a 'study team' uninvolved in clinical care including decision-making. Procedure duration and the study team's words and actions during the sham treatment mimicked those of the minimally invasive surfactant therapy procedure. Post-intervention, three clinicians completed a questionnaire regarding perceived group allocation, with the responses matched against actual intervention and categorised as correct, incorrect, or unsure. Success of blinding was calculated using validated blinding indices applied to the data overall (James index, successful blinding defined as > 0.50), or to the two treatment allocation groups (Bang index, successful blinding: -0.30 to 0.30). Blinding success was measured within staff role, and the associations between blinding success and procedural duration and oxygenation improvement post-procedure were estimated.

RESULTS

From 1345 questionnaires in relation to a procedural intervention in 485 participants, responses were categorised as correct in 441 (33%), incorrect in 142 (11%), and unsure in 762 (57%), with similar proportions for each of the response categories in the two treatment arms. The James index indicated successful blinding overall 0.67 (95% confidence interval (CI) 0.65-0.70). The Bang index was 0.28 (95% CI 0.23-0.32) in the minimally invasive surfactant therapy group and 0.17 (95% CI 0.12-0.21) in the sham arm. Neonatologists more frequently guessed the correct intervention (47%) than bedside nurses (36%), neonatal trainees (31%), and other nurses (24%). For the minimally invasive surfactant therapy intervention, the Bang index was linearly related to procedural duration and oxygenation improvement post-procedure. No evidence of such relationships was seen in the sham arm.

CONCLUSION

Blinding of a procedural intervention from clinicians is both achievable and measurable in neonatal randomised controlled trials.

Evaluation of the Quality of Perinatal Trials: Making the GRADE

BACKGROUND

Assessing the quality of clinical research is a key evidence-based practice skill. Clinicians, guideline producers, policy makers, service commissioners, and families need to have a sense of the validity, applicability, and certainty of research evidence when determining how it should inform their decision-making and practice.

METHODS

We consider the various methodological and study design factors that contribute to the validity and applicability of clinical research findings. We describe the "Grading of Recommendations Assessment, Development and Evaluation" (GRADE) methodology and discuss how this approach is used to assess and report certainty of evidence and strength of recommendations.

RESULTS

The randomized controlled trial (RCT) is the gold standard method for assessing interventions because randomization balances prognostic characteristics between comparison groups. The GRADE approach considers evidence from RCTs as high quality, but acknowledges that the quality and level of certainty of trial evidence may be "downgraded" based on consideration of threats across 5 domains: risk of bias in included trials, inconsistency between trials in outcome estimates, indirectness of the evidence, imprecision of estimates, and likelihood of publication bias.

CONCLUSIONS

Structured critical appraisal using GRADE methods to assess risk of bias and other threats to the internal and external validity of RCTs and systematic reviews and meta-analyses of their data facilitates transparency and consistency in using evidence to inform policy and practice.

Fake news and fake research: Why meta-research matters more than ever

Research is in a crisis of credibility, and this is to the peril of all paediatricians. Billions of dollars are being wasted each year because research is not planned, badly conducted or poorly reported, and this is on a background of rapidly reducing research budgets. How can paediatricians, families and patients make informed treatment choices if the evidence base is absent or not trustworthy? This article discusses why meta-research now matters more than ever, how it can help solve this crisis of credibility and how this should lead to more efficient and effective clinical care. The field of meta-research or research-on-research is the ultimate big picture approach to identifying and solving issues of bias, error, misconduct and waste in research. Meta-researchers value authenticity over aesthetics and quality over quantity. The utility of meta-research does not rely on accusations or critical assessments of individual research, but through highlighting where and how the scientific method and research standards across all fields can be improved. Meta-researchers study, analyse and critique the research pathway, focusing on elements such as methods (how to conduct), evaluation (how to test), reporting (how to communicate), reproducibility (how to verify) and incentives (how to reward). In the current climate it is now more critical than ever that we make use of meta-research and prioritise high-quality high-impact research, ultimately leading to improved patient outcomes.