Short email with attachment versus long email without attachment when contacting authors to request unpublished data for a systematic review: a nested randomised trial

Should we adjudicate outcomes in stroke trials? A systematic review

BACKGROUND

Central adjudication of outcomes is common in randomized clinical trials in stroke. The rationale for adjudication is clear; centrally adjudicated outcomes should have less random and systematic errors than outcomes assessed locally by site investigators. However, adjudication brings added complexities to a clinical trial and can be costly.

AIM

To assess the evidence for outcome adjudication in stroke trials.

SUMMARY OF REVIEW

We identified 12 studies evaluating central adjudication in stroke trials. The majority of these were secondary analyses of trials, and the results of all of these would have remained unchanged had central adjudication not taken place, even for trials without sufficient blinding. The largest differences between site-assessed and adjudicator-assessed outcomes were between the most subjective outcomes, such as causality of serious adverse events. We found that the cost of adjudication could be upward of £100,000 for medium to large prevention trials. These findings suggest that the cost of central adjudication may outweigh the advantages it brings in many cases. However, through simulation, we found that only a small amount of bias is required in site investigators' outcome assessments before adjudication becomes important.

CONCLUSION

Central adjudication may not be necessary in stroke trials with blinded outcome assessment. However, for open-label studies, central adjudication may be more important.

Status, use and impact of sharing individual participant data from clinical trials: a scoping review

OBJECTIVES

To explore the impact of data-sharing initiatives on the intent to share data, on actual data sharing, on the use of shared data and on research output and impact of shared data.

ELIGIBILITY CRITERIA

All studies investigating data-sharing practices for individual participant data (IPD) from clinical trials.

SOURCES OF EVIDENCE

We searched the Medline database, the Cochrane Library, the Science Citation Index Expanded and the Social Sciences Citation Index via Web of Science, and preprints and proceedings of the International Congress on Peer Review and Scientific Publication. In addition, we inspected major clinical trial data-sharing platforms, contacted major journals/publishers, editorial groups and some funders.

CHARTING METHODS

Two reviewers independently extracted information on methods and results from resources identified using a standardised questionnaire. A map of the extracted data was constructed and accompanied by a narrative summary for each outcome domain.

RESULTS

93 studies identified in the literature search (published between 2001 and 2020, median: 2018) and 5 from additional information sources were included in the scoping review. Most studies were descriptive and focused on early phases of the data-sharing process. While the willingness to share IPD from clinical trials is extremely high, actual data-sharing rates are suboptimal. A survey of journal data suggests poor to moderate enforcement of the policies by publishers. Metrics provided by platforms suggest that a large majority of data remains unrequested. When requested, the purpose of the reuse is more often secondary analyses and meta-analyses, rarely re-analyses. Finally, studies focused on the real impact of data-sharing were rare and used surrogates such as citation metrics.

CONCLUSIONS

There is currently a gap in the evidence base for the impact of IPD sharing, which entails uncertainties in the implementation of current data-sharing policies. High level evidence is needed to assess whether the value of medical research increases with data-sharing practices.

Frequency of receiving requested data for a systematic review and associated factors: A cross-sectional study

This study aimed to assess the frequency of receiving requested data for a systematic review and associated factors. We contacted the authors of studies in need of additional data via e-mail. The primary outcome was the success in receiving the requested data according to the time until receipt. We estimated the hazard ratio (HR) and 95% confidence interval (CI) for success in each variable compared to the reference category, with weighted Cox proportional hazards models using Stata (version 14.2). Out of 164 studies contacted, 110 replied (67.1%), and 51 sent requested data (31.1%). Median time to receive a response or withdraw contact was 36.0 days (interquartile range: 17.5, 142.5). Higher success ratio was observed in studies published as scientific papers (HR = 3.01, 95% CI = [1.18, 7.70]), in more than one publication (HR = 2.00, 95% CI = [1.14, 3.51]), and contacted by personal e-mail (HR = 2.85, 95% CI = [1.34, 6.07]). Three or more contact attempts led to lower success ratio (HR = 0.19, 95% CI = [0.11, 0.35]) than one or two. Requesting data for a systematic review was time-consuming and effective in three out of ten studies. Fewer contacts were more successful than insisting.

Author queries via email text elicited high response and took less reviewer time than data forms - a randomised study within a review

OBJECTIVE

To compare two strategies for requesting additional information for systematic reviews (SR) from study authors.

STUDY DESIGN AND SETTING

Randomised study within a SR of hospital volume-outcome relationships in total knee arthroplasty. We sent personalized email requests for additional information to study authors as either email text ("Email" group) or attachment with self-developed, personalised data request forms ("Attachment" group). The primary outcome was the response rate, the secondary outcomes were the data completeness rate and the reviewer time invested in author contact.

RESULTS

Of 57 study authors, 29 were randomised to the Email group and 28 to the Attachment group. The response rate was 93% for Email and 75% for Attachment (odds ratio 4.5, 95% confidence interval [0.9-24.0]). Complete data were provided by 55% (Email) vs. 36% (Attachment) of authors (odds ratio 2.2 [0.8-6.4]). The mean reviewer time was shorter in the Email (mean ± standard deviation of 20.2±14.4 minutes/author) than the Attachment group (31.8±14.4 minutes/author) with a mean difference of 11.6 [4.1-19.1] minutes/author.

CONCLUSION

Personalised email requests elicited high response but only moderate data completeness rates regardless of the method (email text or attachment). Email requests as text took less reviewer time than creating attachments.

Outcome Assessment by Central Adjudicators Versus Site Investigators in Stroke Trials: A Systematic Review and Meta-Analysis

Background and Purpose- In randomized stroke trials, central adjudication of a trial's primary outcome is regularly implemented. However, recent evidence questions the importance of central adjudication in randomized trials. The aim of this review was to compare outcomes assessed by central adjudicators with outcomes assessed by site investigators. Methods- We included randomized stroke trials where the primary outcome had undergone an assessment by site investigators and central adjudicators. We searched MEDLINE, EMBASE, CENTRAL (Cochrane Central Register of Controlled Trials), Web of Science, PsycINFO, and Google Scholar for eligible studies. We extracted information about the adjudication process as well as the treatment effect for the primary outcome, assessed both by central adjudicators and by site investigators. We calculated the ratio of these treatment effects so that a ratio of these treatment effects >1 indicated that central adjudication resulted in a more beneficial treatment effect than assessment by the site investigator. A random-effects meta-analysis model was fitted to estimate a pooled effect. Results- Fifteen trials, comprising 69 560 participants, were included. The primary outcomes included were stroke (8/15, 53%), a composite event including stroke (6/15, 40%) and functional outcome after stroke measured on the modified Rankin Scale (1/15, 7%). The majority of site investigators were blind to treatment allocation (9/15, 60%). On average, there was no difference in treatment effect estimates based on data from central adjudicators and site investigators (pooled ratio of these treatment effects=1.02; 95% CI, [0.95-1.09]). Conclusions- We found no evidence that central adjudication of the primary outcome in stroke trials had any impact on trial conclusions. This suggests that potential advantages of central adjudication may not outweigh cost and time disadvantages in stroke studies if the primary purpose of adjudication is to ensure validity of trial findings.