Use of publication procedures to improve research integrity by addiction journals

Perceptions and Opinions Towards Data-Sharing: A Survey of Addiction Journal Editorial Board Members

Background

We surveyed addiction journal editorial board members to better understand their opinions towards data-sharing.

Methods

Survey items consisted of Likert-type (e.g., one to five scale), multiple-choice, and free-response questions. Journal websites were searched for names and email addresses. Emails were distributed using SurveyMonkey. Descriptive statistics were used to characterize the responses.

Results

We received 178 responses (of 1039; 17.1%). Of these, 174 individuals agreed to participate in our study (97.8%). Most respondents did not know whether their journal had a data-sharing policy. Board members "somewhat agree" that addiction journals should recommend but not require data-sharing for submitted manuscripts [M=4.09 (SD=0.06); 95% CI: 3.97-4.22]. Items with the highest perceived benefit ratings were "secondary data use (e.g., meta-analysis)" [M=3.44 (SD=0.06); 95% CI: 3.31-3.56] and "increased transparency" [M=3.29 (SD=0.07); 95% CI: 3.14-3.43]. Items perceived to be the greatest barrier to data-sharing included "lack of metadata standards" [M=3.21 (SD=0.08); 95% CI: 3.06-3.36], "no incentive" [M=3.43 (SD=0.07); 95% CI: 3.30-3.57], "inadequate resources" [M=3.53 (SD=0.05); 95% CI: 3.42-3.63], and "protection of privacy"[M=3.22 (SD=0.07); 95% CI: 3.07-3.36].

Conclusion

Our results suggest addiction journal editorial board members believe data-sharing has a level of importance within the research community. However, most board members are unaware of their journals' data-sharing policies, and most data-sharing should be recommended but not required. Future efforts aimed at better understanding common reservations and benefits towards data-sharing, as well as avenues to optimize data-sharing while minimizing potential risks, are warranted.

Registration and primary outcome reporting in behavioral health trials

BACKGROUND

Registration of research studies is designed to lock investigators into a data collection and analysis plan before a study starts and thereby limit their ability to engage in flexible data analysis and selective outcome reporting. Studies of registered clinical trials show that one- to two-thirds are registered after the study has started and that non-adherence to important design and analytic features, such as reporting data pertaining to all primary outcomes, remains high. Less is known about the effects of registration on research transparency and integrity outside of clinical trials. To address this gap in knowledge, the current study examined the effects of registration on the reporting of research findings in a sample of behavioral health trials published in BMC Public Health.

METHODS

Registered trials published in the BMC Public Health section "Health Behavior, Health Promotion and Society" between 2011 and 2015 were included in the study. For each trial, we reviewed associated online submissions from 13 different registration sites. For those determined to have been prospectively registered, we used the trial registry, MEDLINE (Pubmed), PsychINFO, Web of Science and e-mails to investigators to identify subsequent publications from the study that reported results pertaining to primary outcomes. The two investigators then independently reviewed the outcome publication(s) and compared the primary outcomes reported in these to the registered primary outcomes.

RESULTS

The final analytic sample comprised 136 locatable, registered trials with an identifiable start date. Sixty-eight of the 136 were prospectively registered. Among these prospectively registered trials, only 16 published manuscripts reported outcomes and methods that were concordant with their registrations.

CONCLUSIONS

Retrospective submission of protocols for publication and retrospective registration remain common in public health research, and adherence to prespecified outcomes is rare. In its current form, registration of behavioral and health promotion trials is likely to have minimal effect on preventing selective outcome reporting in publications, and the pervasiveness of vague and incomplete registry entries means that registries will have limited utility in terms of facilitating replication studies.

Assessing open science and citizen science in addictions and substance use research: A scoping review

BACKGROUND

The EU promotes 'Open Science' as a public good. Complementary to its implementation is Citizen Science, which redefines the relationship between the scientific community, civic society and the individual. Open Science and Citizen Science poses challenges for the substance use and addictions research community but may provide positive opportunities for future European addiction research. This paper explores both current barriers and potential facilitators for the implementation of Open Science and Citizen Science in substance use and addictions research.

METHODOLOGY

A scoping review was used to examine barriers and facilitators identified in the substance use and addiction research literature for the adoption of Open Science and Citizen Science.

RESULTS

'Technical' facilitators included the pre-registration of study protocols; publication of open-source datasets; open peer review and online tools. 'Motivational' facilitators included enhanced reputation; embracing co-creation; engaged citizenship and gamification. 'Economic' facilitators included the use of free tools and balanced remuneration of crowdworkers. 'Political' facilitators included better informed debates through the 'triple helix' approach and trust-generating transparency. 'Legal' facilitators included epidemiologically informed law enforcement; better policy surveillance and the validation of other datasets. 'Ethical' facilitators included the 'democratisation of science' and opportunities to explore new concepts of ethics in addiction research.

CONCLUSION

Open Science and Citizen Science in substance use and addictions research may provide a range of benefits in relation to the democratisation of science; transparency; efficiency and the reliability/validity of data. However, its implementation raises a range of research integrity and ethical issues that need be considered. These include issues related to participant recruitment; privacy; confidentiality; security; cost and industry involvement. Progressive journal policies to support Open Science practices; a shift in researcher norms; the use of free tools and the greater availability of methodological and ethical standards are likely to increase adoption in the field.

Raising the bar: improving methodological rigour in cognitive alcohol research

BACKGROUND AND AIMS

A range of experimental paradigms claim to measure the cognitive processes underpinning alcohol use, suggesting that heightened attentional bias, greater approach tendencies and reduced cue-specific inhibitory control are important drivers of consumption. This paper identifies methodological shortcomings within this broad domain of research and exemplifies them in studies focused specifically on alcohol-related attentional bias.

ARGUMENT AND ANALYSIS

We highlight five main methodological issues: (i) the use of inappropriately matched control stimuli; (ii) opacity of stimulus selection and validation procedures; (iii) a credence in noisy measures; (iv) a reliance on unreliable tasks; and (v) variability in design and analysis. This is evidenced through a review of alcohol-related attentional bias (64 empirical articles, 68 tasks), which reveals the following: only 53% of tasks use appropriately matched control stimuli; as few as 38% report their stimulus selection and 19% their validation procedures; less than 28% used indices capable of disambiguating attentional processes; 22% assess reliability; and under 2% of studies were pre-registered.

CONCLUSIONS

Well-matched and validated experimental stimuli, the development of reliable cognitive tasks and explicit assessment of their psychometric properties, and careful consideration of behavioural indices and their analysis will improve the methodological rigour of cognitive alcohol research. Open science principles can facilitate replication and reproducibility in alcohol research.

Status, use and impact of sharing individual participant data from clinical trials: a scoping review

OBJECTIVES

To explore the impact of data-sharing initiatives on the intent to share data, on actual data sharing, on the use of shared data and on research output and impact of shared data.

ELIGIBILITY CRITERIA

All studies investigating data-sharing practices for individual participant data (IPD) from clinical trials.

SOURCES OF EVIDENCE

We searched the Medline database, the Cochrane Library, the Science Citation Index Expanded and the Social Sciences Citation Index via Web of Science, and preprints and proceedings of the International Congress on Peer Review and Scientific Publication. In addition, we inspected major clinical trial data-sharing platforms, contacted major journals/publishers, editorial groups and some funders.

CHARTING METHODS

Two reviewers independently extracted information on methods and results from resources identified using a standardised questionnaire. A map of the extracted data was constructed and accompanied by a narrative summary for each outcome domain.

RESULTS

93 studies identified in the literature search (published between 2001 and 2020, median: 2018) and 5 from additional information sources were included in the scoping review. Most studies were descriptive and focused on early phases of the data-sharing process. While the willingness to share IPD from clinical trials is extremely high, actual data-sharing rates are suboptimal. A survey of journal data suggests poor to moderate enforcement of the policies by publishers. Metrics provided by platforms suggest that a large majority of data remains unrequested. When requested, the purpose of the reuse is more often secondary analyses and meta-analyses, rarely re-analyses. Finally, studies focused on the real impact of data-sharing were rare and used surrogates such as citation metrics.

CONCLUSIONS

There is currently a gap in the evidence base for the impact of IPD sharing, which entails uncertainties in the implementation of current data-sharing policies. High level evidence is needed to assess whether the value of medical research increases with data-sharing practices.

An evaluation of the practice of transparency and reproducibility in addiction medicine literature

BACKGROUND

Credible research emphasizes transparency, openness, and reproducibility. These characteristics are fundamental to promoting and maintaining research integrity. The aim of this study was to evaluate the current state of reproducibility in the field of addiction science.

DESIGN

Cross-sectional design.

MEASUREMENTS

The National Library of Medicine catalog was searched for all journals using the subject terms tag: Substance-Related Disorders [ST]. Journals were then searched via PubMed to identify publications from January 1, 2014, to December 31, 2018; 300 publications were randomly selected from among those identified. A pilot-tested Google form containing reproducibility/transparency characteristics was used for data extraction in a duplicated and blinded fashion by two investigators.

FINDINGS

Slightly more than half of the publications were open access (152/300, 50.70%). Few publications had pre-registration (7/244, 2.87%), material availability (2/237, 0.84%), protocol availability (3/244, 1.23%), data availability (28/244, 11.48%), or analysis script availability (2/244, 0.82%). Most publications provided a conflict of interest statement (221/293, 75.43%) and funding sources (268/293, 91.47%). One replication study was reported (1/244, 0.4%).

CONCLUSION

Our study found that current practices that promote transparency and reproducibility are lacking, thus, there is room for improvement. In particular, investigators should pre-register studies prior to commencement. Researchers should also make the materials, data, and analysis script publicly available. Further, individuals should be transparent about funding sources for the project and financial conflicts of interest. Research stakeholders should work together toward improvements on these matters. With such protections, the field of addiction medicine can better disseminate the information necessary to treat patients.

High impact nutrition and dietetics journals' use of publication procedures to increase research transparency

BACKGROUND

The rigor and integrity of the published research in nutrition studies has come into serious question in recent years. Concerns focus on the use of flexible data analysis practices and selective reporting and the failure of peer review journals to identify and correct these practices. In response, it has been proposed that journals employ editorial procedures designed to improve the transparency of published research.

OBJECTIVE

The present study examines the adoption of editorial procedures designed to improve the reporting of empirical studies in the field of nutrition and dietetics research.

DESIGN

The instructions for authors of 43 journals included in Quartiles 1 and 2 of the Clarivate Analytics' 2018 Journal Citation Report category Nutrition and Dietetics were reviewed. For journals that published original research, conflict of interest disclosure, recommendation of reporting guidelines, registration of clinical trials, registration of other types of studies, encouraging data sharing, and use of the Registered Reports were assessed. For journals that only published reviews, all of the procedures except clinical trial registration were assessed.

RESULTS

Thirty-three journals published original research and 10 published only reviews. Conflict of interest disclosure was required by all 33 original research journals. Use of guidelines, trial registration and encouragement of data sharing were mentioned by 30, 27 and 25 journals, respectively. Registration of other studies was required by eight and none offered Registered Reports as a publication option at the time of the review. All 10 review journals required conflict of interest disclosure, four recommended data sharing and three the use of guidelines. None mentioned the other two procedures.

CONCLUSIONS

While nutrition journals have adopted a number of procedures designed to improve the reporting of research findings, their limited effects likely result from the mechanisms through which they influence analytic flexibility and selective reporting and the extent to which they are properly implemented and enforced by journals.

Availability of Research Data in High-Impact Addiction Journals with Data Sharing Policies

Although data sharing is one of the primary measures proposed to improve the integrity and quality of published research, studies show it remains the exception not the rule. The current study examines the availability of data in papers reporting the results of analyses of empirical data from original research in high-impact addiction journals. Thirteen high-impact journals with data sharing policies were selected from those included in the substance abuse category of the 2018 Clarivate Analytics' Journal Citation Report. The first 10 full or short original research reports that included empirical data in the most recent complete issue of each journal were electronically searched and reviewed for reference to where their data can be obtained and for a formal data sharing statement. Only eight of the 130 papers contained a data sharing statement in their text or supplementary online materials, and just one contained a direct link to the data analyzed. Data sharing was rare in the 13 high-impact addiction journals reviewed. The nature of the data reported in addiction journals might partly explain this. Currently, data sharing is not a procedure likely to improve the quality and integrity of published addiction research.

Evaluation of selective outcome reporting and trial registration practices among addiction clinical trials

BACKGROUND AND AIMS

Selective outcome reporting occurs when trialists pre-specify primary and secondary outcomes during trial planning but alter the definitions in the published report. Here, we investigate selective outcome reporting in published addiction randomized controlled trials (RCTs) and evaluate whether particular funding sources are associated with an increased likelihood of selective outcome reporting.

DESIGN

We conducted a cross-sectional study of published addiction clinical trials. A PubMed search was performed to identify RCTs in addiction journals from 2013 to 2017. Included studies used a randomized design to address one of the following topics: (1) drug, alcohol and tobacco addiction prevention, (2) stabilization following excessive use of a substance, (3) relapse prevention or (4) recovery maintenance.

SETTING

Single-center, medical research institution.

PARTICIPANTS

Our sample included 162 RCTs that were prospectively registered with a clearly defined primary outcome.

MEASUREMENT

We extracted the following items from addiction RCTs: journal, funding source, trial registry number (if included), sample size, dates of subject enrollment, whether primary and secondary outcomes were denoted, all published outcomes, P-value for all outcomes and whether authors mentioned any deviations from the trial protocol as it related to RCT outcomes.

FINDINGS

In total, 47 of 162 RCTs (29.0%) had at least one major discrepancy between the trial registry and published RCT. Overall, these 47 RCTs included 54 major discrepancies. The most common major discrepancy was demotion of a primary registered outcome (19/54, 35.2%). The majority of RCTs (132/162, 81.5%) were funded from public sources. Additionally, 166 RCTs were excluded from our sample because registration could not be confirmed.

CONCLUSIONS

There is evidence suggestive of selective outcome reporting in addiction randomized controlled trials (RCTs). The most common major discrepancies pertained to the primary outcome.

Scratch Card Near-Miss Outcomes Increase the Urge to Gamble, but Do Not Impact Further Gambling Behaviour: A Pre-registered Replication and Extension

Scratch card near-misses, outcomes in which two out of three required jackpot symbols are uncovered, have been shown to erroneously increase the urge to continue gambling. It remains unknown if and how these outcomes influence further gambling behaviour. Previous studies examining the influence of near-misses on purchasing behaviour offered a low-stakes gamble to participants after experiencing a near-miss or a regular loss. We sought to investigate the influence of these outcomes on scratch card purchasing behaviour with a stronger test of participants’ gambling behavior by having them either “cash out” or risk all of their winnings to purchase another card. Additionally, we sought to test an original hypothesis that endorsement of the illusion of control might influence the decision to purchase additional scratch cards. We pre-registered our hypotheses, sample size, and data analysis plan. 138 subjects experienced two custom-made scratch card games that included a win on the first card (for all participants) and either a regular loss or a near-miss in the final outcome position on the second card (between-subjects manipulation). Although near-miss outcomes increased the urge to continue gambling relative to regular losses, no differences in the rates of purchasing were found between the conditions. Additionally, no support for our hypotheses concerning the influence of the illusion of control in near-miss outcomes was found. These results are discussed in terms of previous studies on scratch card gambling behaviour and subjective reactivity.