Social media attention and citations of published outputs from re-use of clinical trial data: a matched comparison with articles published in the same journals

BACKGROUND

Data-sharing policies in randomized clinical trials (RCTs) should have an evaluation component. The main objective of this case-control study was to assess the impact of published re-uses of RCT data in terms of media attention (Altmetric) and citation rates.

METHODS

Re-uses of RCT data published up to December 2019 (cases) were searched for by two reviewers on 3 repositories (CSDR, YODA project, and Vivli) and matched to control papers published in the same journal. The Altmetric Attention Score (primary outcome), components of this score (e.g. mention of policy sources, media attention) and the total number of citations were compared between these two groups.

RESULTS

89 re-uses were identified: 48 (53.9%) secondary analyses, 34 (38.2%) meta-analyses, 4 (4.5%) methodological analyses and 3 (3.4%) re-analyses. The median (interquartile range) Altmetric Attention Scores were 5.9 (1.3-22.2) for re-use and 2.8 (0.3-12.3) for controls (p = 0.14). No statistical difference was found on any of the components of in the Altmetric Attention Score. The median (interquartile range) numbers of citations were 3 (1-8) for reuses and 4 (1 - 11.5) for controls (p = 0.30). Only 6/89 re-uses (6.7%) were cited in a policy source.

CONCLUSIONS

Using all available re-uses of RCT data to date from major data repositories, we were not able to demonstrate that re-uses attracted more attention than a matched sample of studies published in the same journals. Small average differences are still possible, as the sample size was limited. However matching choices have some limitations so results should be interpreted very cautiously. Also, citations by policy sources for re-uses were rare.

TRIAL REGISTRATION

Registration: osf.io/fp62e.

GRIPP2 reporting checklists: tools to improve reporting of patient and public involvement in research

BACKGROUND

While the patient and public involvement (PPI) evidence base has expanded over the past decade, the quality of reporting within papers is often inconsistent, limiting our understanding of how it works, in what context, for whom, and why.

OBJECTIVE

To develop international consensus on the key items to report to enhance the quality, transparency, and consistency of the PPI evidence base. To collaboratively involve patients as research partners at all stages in the development of GRIPP2.

METHODS

The EQUATOR method for developing reporting guidelines was used. The original GRIPP (Guidance for Reporting Involvement of Patients and the Public) checklist was revised, based on updated systematic review evidence. A three round Delphi survey was used to develop consensus on items to be included in the guideline. A subsequent face-to-face meeting produced agreement on items not reaching consensus during the Delphi process.

RESULTS

One hundred forty-three participants agreed to participate in round one, with an 86% (123/143) response for round two and a 78% (112/143) response for round three. The Delphi survey identified the need for long form (LF) and short form (SF) versions. GRIPP2-LF includes 34 items on aims, definitions, concepts and theory, methods, stages and nature of involvement, context, capture or measurement of impact, outcomes, economic assessment, and reflections and is suitable for studies where the main focus is PPI. GRIPP2-SF includes five items on aims, methods, results, outcomes, and critical perspective and is suitable for studies where PPI is a secondary focus.

CONCLUSIONS

GRIPP2-LF and GRIPP2-SF represent the first international evidence based, consensus informed guidance for reporting patient and public involvement in research. Both versions of GRIPP2 aim to improve the quality, transparency, and consistency of the international PPI evidence base, to ensure PPI practice is based on the best evidence. In order to encourage its wide dissemination this article is freely accessible on The BMJ and Research Involvement and Engagement journal websites.

GRIPP2 reporting checklists: tools to improve reporting of patient and public involvement in research

Background While the patient and public involvement (PPI) evidence base has expanded over the past decade, the quality of reporting within papers is often inconsistent, limiting our understanding of how it works, in what context, for whom, and why.Objective To develop international consensus on the key items to report to enhance the quality, transparency, and consistency of the PPI evidence base. To collaboratively involve patients as research partners at all stages in the development of GRIPP2.Methods The EQUATOR method for developing reporting guidelines was used. The original GRIPP (Guidance for Reporting Involvement of Patients and the Public) checklist was revised, based on updated systematic review evidence. A three round Delphi survey was used to develop consensus on items to be included in the guideline. A subsequent face-to-face meeting produced agreement on items not reaching consensus during the Delphi process.Results 143 participants agreed to participate in round one, with an 86% (123/143) response for round two and a 78% (112/143) response for round three. The Delphi survey identified the need for long form (LF) and short form (SF) versions. GRIPP2-LF includes 34 items on aims, definitions, concepts and theory, methods, stages and nature of involvement, context, capture or measurement of impact, outcomes, economic assessment, and reflections and is suitable for studies where the main focus is PPI. GRIPP2-SF includes five items on aims, methods, results, outcomes, and critical perspective and is suitable for studies where PPI is a secondary focus.Conclusions GRIPP2-LF and GRIPP2-SF represent the first international evidence based, consensus informed guidance for reporting patient and public involvement in research. Both versions of GRIPP2 aim to improve the quality, transparency, and consistency of the international PPI evidence base, to ensure PPI practice is based on the best evidence. In order to encourage its wide dissemination this article is freely accessible on The BMJ and Research Involvement and Engagement journal websites.

Comparison of pharmacological and non-pharmacological interventions to prevent delirium in critically ill patients: a protocol for a systematic review incorporating network meta-analyses

BACKGROUND

Delirium is characterized by acute changes in mental status including inattention, disorganized thinking, and altered level of consciousness, and is highly prevalent in critically ill adults. Delirium has adverse consequences for both patients and the healthcare system; however, at this time, no effective treatment exists. The identification of effective prevention strategies is therefore a clinical and research imperative. An important limitation of previous reviews of delirium prevention is that interventions were considered in isolation and only direct evidence was used. Our systematic review will synthesize all existing data using network meta-analysis, a powerful statistical approach that enables synthesis of both direct and indirect evidence.

METHODS

We will search Ovid MEDLINE, CINAHL, Embase, PsycINFO, and Web of Science from 1980 to March 2016. We will search the PROSPERO registry for protocols and the Cochrane Library for published systematic reviews. We will examine reference lists of pertinent reviews and search grey literature and the International Clinical Trials Registry Platform for unpublished studies and ongoing trials. We will include randomized and quasi-randomized trials of critically ill adults evaluating any pharmacological, non-pharmacological, or multi-component intervention for delirium prevention, administered in or prior to (i.e., peri-operatively) transfer to the ICU. Two authors will independently screen search results and extract data from eligible studies. Risk of bias assessments will be completed on all included studies. To inform our network meta-analysis, we will first conduct conventional pair-wise meta-analyses for primary and secondary outcomes using random-effects models. We will generate our network meta-analysis using a Bayesian framework, assuming a common heterogeneity parameter across all comparisons, and accounting for correlations in multi-arm studies. We will perform analyses using WinBUGS software.

DISCUSSION

This systematic review will address the existing knowledge gap regarding best practices for delirium prevention in critically ill adults by synthesizing evidence from trials of pharmacological, non-pharmacological, and multi-component interventions administered in or prior to transfer to the ICU. Use of network meta-analysis will clarify which delirium prevention strategies are most effective in improving clinical outcomes while causing least harm. The network meta-analysis is a novel approach and will provide knowledge users and decision makers with comparisons of multiple interventions of delirium prevention strategies.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO CRD42016036313.

[Better Reporting of Interventions: Template for Intervention Description and Replication (TIDieR) Checklist and Guide]

Without a complete published description of interventions, clinicians and patients cannot reliably implement interventions that are shown to be useful, and other researchers cannot replicate or build on research findings. The quality of description of interventions in publications, however, is remarkably poor. To improve the completeness of reporting, and ultimately the replicability, of interventions, an international group of experts and stakeholders developed the Template for Intervention Description and Replication (TIDieR) checklist and guide. The process involved a literature review for relevant checklists and research, a Delphi survey of an international panel of experts to guide item selection, and a face-to-face panel meeting. The resultant 12-item TIDieR checklist (brief name, why, what (materials), what (procedure), who intervened, how, where, when and how much, tailoring, modifications, how well (planned), how well (actually carried out)) is an extension of the CONSORT 2010 statement (item 5) and the SPIRIT 2013 statement (item 11). While the emphasis of the checklist is on trials, the guidance is intended to apply across all evaluative study designs. This paper presents the TIDieR checklist and guide, with a detailed explanation of each item, and examples of good reporting. The TIDieR checklist and guide should improve the reporting of interventions and make it easier for authors to structure the accounts of their interventions, reviewers and editors to assess the descriptions, and readers to use the information.

Protocol for the development of a CONSORT-equity guideline to improve reporting of health equity in randomized trials

BACKGROUND

Health equity concerns the absence of avoidable and unfair differences in health. Randomized controlled trials (RCTs) can provide evidence about the impact of an intervention on health equity for specific disadvantaged populations or in general populations; this is important for equity-focused decision-making. Previous work has identified a lack of adequate reporting guidelines for assessing health equity in RCTs. The objective of this study is to develop guidelines to improve the reporting of health equity considerations in RCTs, as an extension of the Consolidated Standards of Reporting Trials (CONSORT).

METHODS/DESIGN

A six-phase study using integrated knowledge translation governed by a study executive and advisory board will assemble empirical evidence to inform the CONSORT-equity extension. To create the guideline, the following steps are proposed: (1) develop a conceptual framework for identifying "equity-relevant trials," (2) assess empirical evidence regarding reporting of equity-relevant trials, (3) consult with global methods and content experts on how to improve reporting of health equity in RCTs, (4) collect broad feedback and prioritize items needed to improve reporting of health equity in RCTs, (5) establish consensus on the CONSORT-equity extension: the guideline for equity-relevant trials, and (6) broadly disseminate and implement the CONSORT-equity extension.

DISCUSSION

This work will be relevant to a broad range of RCTs addressing questions of effectiveness for strategies to improve practice and policy in the areas of social determinants of health, clinical care, health systems, public health, and international development, where health and/or access to health care is a primary outcome. The outcomes include a reporting guideline (CONSORT-equity extension) for equity-relevant RCTs and a knowledge translation strategy to broadly encourage its uptake and use by journal editors, authors, and funding agencies.

Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement

Economic evaluations of health interventions pose a particular challenge for reporting. There is also a need to consolidate and update existing guidelines and promote their use in a user friendly manner. The Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement is an attempt to consolidate and update previous health economic evaluation guidelines efforts into one current, useful reporting guidance.The primary audiences for the CHEERS statement are researchers reporting economic evaluations and the editors and peer reviewers assessing them for publication. The need for new reporting guidance was identified by a survey of medical editors. A list of possible items based on a systematic review was created. A two round, modified Delphi panel consisting of representatives from academia, clinical practice, industry, government, and the editorial community was conducted. Out of 44 candidate items, 24 items and accompanying recommendations were developed. The recommendations are contained in a user friendly, 24 item checklist. A copy of the statement, accompanying checklist, and this report can be found on the ISPOR Health Economic Evaluations Publication Guidelines Task Force website (www.ispor.org/TaskForces/EconomicPubGuidelines.asp). We hope CHEERS will lead to better reporting, and ultimately, better health decisions. To facilitate dissemination and uptake, the CHEERS statement is being co-published across 10 health economics and medical journals. We encourage other journals and groups, to endorse CHEERS. The author team plans to review the checklist for an update in 5 years.

Influence of reported study design characteristics on intervention effect estimates from randomised controlled trials: combined analysis of meta-epidemiological studies

BACKGROUND

The design of randomised controlled trials (RCTs) should incorporate characteristics (such as concealment of randomised allocation and blinding of participants and personnel) that avoid biases resulting from lack of comparability of the intervention and control groups. Empirical evidence suggests that the absence of such characteristics leads to biased intervention effect estimates, but the findings of different studies are not consistent.

OBJECTIVES

To examine the influence of unclear or inadequate random sequence generation and allocation concealment, and unclear or absent double blinding, on intervention effect estimates and between-trial heterogeneity, and whether or not these influences vary with type of clinical area, intervention, comparison and outcome measure.

DATA SOURCES AND METHODS

Data were combined from seven contributing meta-epidemiological studies (collections of meta-analyses in which trial characteristics are assessed and results recorded). The resulting database was used to identify and remove overlapping meta-analyses. Outcomes were coded such that odds ratios < 1 correspond to beneficial intervention effects. Outcome measures were classified as mortality, other objective or subjective. We examined agreement between assessments of trial characteristics in trials assessed in more than one contributing study. We used hierarchical Bayesian bias models to estimate the effect of trial characteristics on average bias [quantified as ratios of odds ratios (RORs) with 95% credible intervals (CrIs) comparing trials with and without a characteristic] and in increasing between-trial heterogeneity.

RESULTS

The analysis data set contained 1973 trials included in 234 meta-analyses. Median kappa statistics for agreement between assessments of trial characteristics were: sequence generation 0.60, allocation concealment 0.58 and blinding 0.87. Intervention effect estimates were exaggerated by an average 11% in trials with inadequate or unclear (compared with adequate) sequence generation (ROR 0.89, 95% CrI 0.82 to 0.96); between-trial heterogeneity was higher among such trials. Bias associated with inadequate or unclear sequence generation was greatest for subjective outcomes (ROR 0.83, 95% CrI 0.74 to 0.94) and the increase in heterogeneity was greatest for such outcomes [standard deviation (SD) 0.20, 95% CrI 0.03 to 0.32]. The effect of inadequate or unclear (compared with adequate) allocation concealment was greatest among meta-analyses with a subjectively assessed outcome intervention effect (ROR 0.85, 95% CrI 0.75 to 0.95), and the increase in between-trial heterogeneity was also greatest for such outcomes (SD 0.20, 95% CrI 0.02 to 0.33). Lack of, or unclear, double blinding (compared with double blinding) was associated with an average 13% exaggeration of intervention effects (ROR 0.87, 95% CrI 0.79 to 0.96), and between-trial heterogeneity was increased for such studies (SD 0.14, 95% CrI 0.02 to 0.30). Average bias (ROR 0.78, 95% CrI 0.65 to 0.92) and between-trial heterogeneity (SD 0.37, 95% CrI 0.19 to 0.53) were greatest for meta-analyses assessing subjective outcomes. Among meta-analyses with subjectively assessed outcomes, the effect of lack of blinding appeared greater than the effect of inadequate or unclear sequence generation or allocation concealment.

CONCLUSIONS

Bias associated with specific reported study design characteristics leads to exaggeration of beneficial intervention effect estimates and increases in between-trial heterogeneity. For each of the three characteristics assessed, these effects were greatest for subjectively assessed outcomes. Assessments of the risk of bias in RCTs should account for these findings. Further research is needed to understand the effects of attrition bias, as well as the relative importance of blinding of patients, care-givers and outcome assessors, and thus separate the effects of performance and detection bias.

FUNDING

National Institute for Health Research Health Technology Assessment programme.

Skeletal findings in children recently initiating glucocorticoids for the treatment of nephrotic syndrome

SUMMARY

Eighty children with nephrotic syndrome underwent lumbar spine densitometry and vertebral morphometry soon after glucocorticoid initiation. We found an inverse relationship between glucocorticoid exposure and spine areal bone mineral density (BMD) Z-score and a low rate of vertebral deformities (8%).

INTRODUCTION

Vertebral fractures are an under-recognized complication of childhood glucocorticoid-treated illnesses. Our goal was to study the relationships among glucocorticoid exposure, lumbar spine areal BMD (LS BMD), and vertebral shape in glucocorticoid-treated children with new-onset nephrotic syndrome.

METHODS

Lateral thoracolumbar spine radiography and LS BMD were performed in 80 children with nephrotic syndrome (median age 4.4 years; 46 boys) within the first 37 days of glucocorticoid therapy. Genant semiquantitative grading was used as the primary method for vertebral morphometry; the algorithm-based qualitative (ABQ) method was used for secondary vertebral deformity analysis.

RESULTS

Six of the 78 children with usable radiographs (8%; 95% confidence interval 4 to 16%) manifested a single Genant grade 1 deformity each. All deformities were mild anterior wedging (two at each of T6, T7, and T8). Four of the 78 children (5%; 95% confidence interval 2 to 13%) showed one ABQ sign of fracture each (loss of endplate parallelism; two children at T6 and two at T8). Two of the children with ABQ signs also had a Genant grade 1 deformity in the same vertebral body. None of the children with a Genant or ABQ deformity reported back pain. An inverse relationship was identified between LS BMD Z-score and glucocorticoid exposure.

CONCLUSIONS

Although we identified an inverse relationship between steroid exposure and LS BMD soon after glucocorticoid initiation for childhood nephrotic syndrome, there was only a low rate of vertebral deformities. The clinical significance of these findings requires further study.

Prevalent vertebral fractures among children initiating glucocorticoid therapy for the treatment of rheumatic disorders

OBJECTIVE

Vertebral fractures are an under-recognized problem in children with inflammatory disorders. We studied spine health among 134 children (87 girls) with rheumatic conditions (median age 10 years) within 30 days of initiating glucocorticoid therapy.

METHODS

Children were categorized as follows: juvenile dermatomyositis (n = 30), juvenile idiopathic arthritis (n = 28), systemic lupus erythematosus and related conditions (n = 26), systemic arthritis (n = 22), systemic vasculitis (n = 16), and other conditions (n = 12). Thoracolumbar spine radiograph and dual x-ray absorptiometry for lumbar spine (L-spine) areal bone mineral density (BMD) were performed within 30 days of glucocorticoid initiation. Genant semiquantitative grading was used for vertebral morphometry. Second metacarpal morphometry was carried out on a hand radiograph. Clinical factors including disease and physical activity, calcium and vitamin D intake, cumulative glucocorticoid dose, underlying diagnosis, L-spine BMD Z score, and back pain were analyzed for association with vertebral fracture.

RESULTS

Thirteen vertebral fractures were noted in 9 children (7%). Of these, 6 patients had a single vertebral fracture and 3 had 2-3 fractures. Fractures were clustered in the mid-thoracic region (69%). Three vertebral fractures (23%) were moderate (grade 2); the others were mild (grade 1). For the entire cohort, mean +/- SD L-spine BMD Z score was significantly different from zero (-0.55 +/- 1.2, P < 0.001) despite a mean height Z score that was similar to the healthy average (0.02 +/- 1.0, P = 0.825). Back pain was highly associated with increased odds for fracture (odds ratio 10.6 [95% confidence interval 2.1-53.8], P = 0.004).

CONCLUSION

In pediatric rheumatic conditions, vertebral fractures can be present prior to prolonged glucocorticoid exposure.

Is staging beneficial for Fowler-Stephens orchiopexy? A systematic review

PURPOSE

Fowler and Stephens showed that by dividing the spermatic vessels a high intra-abdominal testis could be placed in the scrotum. Testicular atrophy is a potential complication of this technique. We conducted a systematic review to determine whether single or 2-stage Fowler-Stephens orchiopexy results in better testicular viability.

MATERIALS AND METHODS

We searched electronic databases, clinical trial registries and gray literature. We included reports describing boys younger than 18 years with a primary outcome of "testicular viability and position." We performed a meta-analysis using random effects models. Heterogeneity was assessed using forest plot and I(2) statistic.

RESULTS

We identified 1,807 citations and included 61 articles. Single stage Fowler-Stephens orchiopexy was discussed in 9 articles, a 2-stage procedure in 36 and both approaches in 16. There were no randomized controlled trials, and most studies were cohort or case series. The pooled estimate of success rates was 80% for single stage Fowler-Stephens orchiopexy (95% CI 75 to 86) and 85% for 2-stage Fowler-Stephens orchiopexy (95% CI 81 to 90). The pooled odds ratio of single stage vs 2-stage Fowler-Stephens orchiopexy was 2.0 (95% CI 1.1 to 3.9) favoring the 2-stage procedure. There was no difference in the success rate between laparoscopic and open techniques in either single or 2-stage Fowler-Stephens orchiopexy. There was no evidence of asymmetry on the funnel plot. There were no complications reported with single stage, while ileus, hematoma and infection were the most common complications with 2-stage Fowler-Stephens orchiopexy.

CONCLUSIONS

Both techniques have a fairly high success rate but 2-stage Fowler-Stephens orchiopexy appears to carry a higher rate of success than the single stage approach (85% vs 80%, OR 2 in favor of 2-stage). Laparoscopic and open techniques had the same success rate. However, the level of evidence of the studies was low, and a study of a more robust design, such as a randomized controlled trial, should be performed.

A catalogue of reporting guidelines for health research

Growing evidence demonstrates widespread deficiencies in the reporting of health research studies. The EQUATOR Network is an international initiative that aims to enhance the reliability and value of the published health research literature. EQUATOR provides resources, education and training to facilitate good research reporting and assists in the development, dissemination and implementation of robust reporting guidelines. This paper presents a collection of tools and guidelines available on the EQUATOR website (http://www.equator-network.org) that have been developed to increase the accuracy and transparency of health research reporting.