Systematic review: Outcome reporting bias is a problem in high impact factor neurology journals

Evaluating guideline and registration policies among neurology journals: a cross-sectional analysis

BACKGROUND

Neurological disorders have had a substantial rise the last three decades, imposing substantial burdens on both patients and healthcare costs. Consequently, the demand for high-quality research has become crucial for exploring effective treatment options. However, current neurology research has some limitations in terms of transparency, reproducibility, and reporting bias. The adoption of reporting guidelines (RGs) and trial registration policies has been proven to address these issues and improve research quality in other medical disciplines. It is unclear the extent to which these policies are being endorsed by neurology journals. Therefore, our study aims to evaluate the publishing policies of top neurology journals regarding RGs and trial registration.

METHODS

For this cross-sectional study, neurology journals were identified using the 2021 Scopus CiteScore Tool. The top 100 journals were listed and screened for eligibility for our study. In a masked, duplicate fashion, investigators extracted data on journal characteristics, policies on RGs, and policies on trial registration using information from each journal's Instruction for Authors webpage. Additionally, investigators contacted journal editors to ensure information was current and accurate. No human participants were involved in this study. Our data collection and analyses were performed from December 14, 2022, to January 9, 2023.

RESULTS

Of the 356 neurology journals identified, the top 100 were included into our sample. The five-year impact of these journals ranged from 50.844 to 2.226 (mean [SD], 7.82 [7.01]). Twenty-five (25.0%) journals did not require or recommend a single RG within their Instructions for Authors webpage, and a third (33.0%) did not require or recommend clinical trial registration. The most frequently mentioned RGs were CONSORT (64.6%), PRISMA (52.5%), and ARRIVE (53.1%). The least mentioned RG was QUOROM (1.0%), followed by MOOSE (9.0%), and SQUIRE (17.9%).

CONCLUSIONS

While many top neurology journals endorse the use of RGs and trial registries, there are still areas where their adoption can be improved. Addressing these shortcomings leads to further advancements in the field of neurology, resulting in higher-quality research and better outcomes for patients.

Selective outcome reporting in trials of behavioural health interventions in health psychology and behavioural medicine journals: a review

Selective outcome reporting can result in overestimation of treatment effects, research waste, and reduced openness and transparency. This review aimed to examine selective outcome reporting in trials of behavioural health interventions and determine potential outcome reporting bias. A review of nine health psychology and behavioural medicine journals was conducted to identify randomised controlled trials of behavioural health interventions published since 2019. Discrepancies in outcome reporting were observed in 90% of the 29 trials with corresponding registrations/protocols. Discrepancies included 72% of trials omitting prespecified outcomes; 55% of trials introduced new outcomes. Thirty-eight percent of trials omitted prespecified and introduced new outcomes. Three trials (10%) downgraded primary outcomes in registrations/protocols to secondary outcomes in final reports; downgraded outcomes were not statistically significant in two trials. Five trials (17%) upgraded secondary outcomes to primary outcomes; upgraded outcomes were statistically significant in all trials. In final reports, three trials (7%) omitted outcomes from the methods section; three trials (7%) introduced new outcomes in results that were not in the methods. These findings indicate that selective outcome reporting is a problem in behavioural health intervention trials. Journal- and trialist-level approaches are needed to minimise selective outcome reporting in health psychology and behavioural medicine.

Outcome Reporting Bias in Clinical Trials Researching Disease-Modifying Therapy in Patients With Multiple Sclerosis

BACKGROUND AND OBJECTIVES

Outcome reporting bias occurs when publication of trial results is dependent on clinical significance, thereby threatening the validity of trial results. Research on immunomodulatory drugs in multiple sclerosis has thrived in recent years. We aim to comprehensively examine to what extent outcome reporting bias is present in these trials and the possible underlying factors.

METHODS

We identified clinical trials evaluating the efficacy and safety of immunomodulatory drugs in patients with multiple sclerosis (MS) registered in ClinicalTrials.gov after September 2007 and completed before the end of 2018. Information about study design, type of funding, and primary and secondary outcome measures was extracted from the registry. Timing of registration in relation to study initiation and subsequent amendments to the planned outcomes were reviewed. Publications related to these trials were identified in several bibliographic databases using the trial registration number. Registered primary and secondary outcomes were recorded for each trial and compared with outcomes in the publication describing the main outcomes of the trial.

RESULTS

A search of ClinicalTrials.gov identified 535 eligible registered clinical trials; of these, 101 had a matching publication. Discrepancies between registered and published primary and secondary outcomes were found in 95% of the trials, including discrepancies between the registered and published primary outcomes in 26 publications. Forty-four percent of the published secondary outcomes were not included in the registry. A similar proportion of registered and nonregistered reported primary efficacy outcomes were positive (favoring the intervention). Nonindustry-funded and open-label trials in MS were more prone to selective primary outcome reporting, although these findings did not reach statistical significance. Only two-thirds of the trials were registered in ClinicalTrials.gov before the trial start date, and 62% of trials made amendments in registered outcomes during or after the trial period.

DISCUSSION

Selective outcome reporting is prevalent in trials of disease-modifying drugs in people with MS. We propose methods to diminish the occurrence of this bias in future research.

Estimating the prevalence of discrepancies between study registrations and publications: a systematic review and meta-analyses

OBJECTIVES

Prospectively registering study plans in a permanent time-stamped and publicly accessible document is becoming more common across disciplines and aims to reduce risk of bias and make risk of bias transparent. Selective reporting persists, however, when researchers deviate from their registered plans without disclosure. This systematic review aimed to estimate the prevalence of undisclosed discrepancies between prospectively registered study plans and their associated publication. We further aimed to identify the research disciplines where these discrepancies have been observed, whether interventions to reduce discrepancies have been conducted, and gaps in the literature.

DESIGN

Systematic review and meta-analyses.

DATA SOURCES

Scopus and Web of Knowledge, published up to 15 December 2019.

ELIGIBILITY CRITERIA

Articles that included quantitative data about discrepancies between registrations or study protocols and their associated publications.

DATA EXTRACTION AND SYNTHESIS

Each included article was independently coded by two reviewers using a coding form designed for this review (osf.io/728ys). We used random-effects meta-analyses to synthesise the results.

RESULTS

We reviewed k=89 articles, which included k=70 that reported on primary outcome discrepancies from n=6314 studies and, k=22 that reported on secondary outcome discrepancies from n=1436 studies. Meta-analyses indicated that between 29% and 37% (95% CI) of studies contained at least one primary outcome discrepancy and between 50% and 75% (95% CI) contained at least one secondary outcome discrepancy. Almost all articles assessed clinical literature, and there was considerable heterogeneity. We identified only one article that attempted to correct discrepancies.

CONCLUSIONS

Many articles did not include information on whether discrepancies were disclosed, which version of a registration they compared publications to and whether the registration was prospective. Thus, our estimates represent discrepancies broadly, rather than our target of undisclosed discrepancies between prospectively registered study plans and their associated publications. Discrepancies are common and reduce the trustworthiness of medical research. Interventions to reduce discrepancies could prove valuable.

REGISTRATION

osf.io/ktmdg. Protocol amendments are listed in online supplemental material A.

Assessment of Spin in the Abstracts of Systematic Reviews and Meta-analyses on Platelet-Rich Plasma Treatment in Orthopaedics: A Cross-sectional Analysis

Background

Systematic reviews on the use of platelet-rich plasma (PRP) in orthopaedic surgery are abundant in current published literature. However, a beautification of results (referred to as spin) has been noted in abstracts across various aspects of medicine.

Purpose

To determine the prevalence of spin in systematic reviews of PRP-related orthopaedic surgery abstracts.

Study Design

Cross-sectional study.

Methods

Following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) and Murad and Wang guidelines, we conducted a search in Medline, Embase, and the Cochrane Database for reviews on PRP-related orthopaedic surgery. The search included studies published from inception until June 30, 2021. Included were systematic reviews written in English that involved the use of PRP in the treatment of orthopaedic injuries in human participants. The abstracts of the included reviews were evaluated for the top 9 types of spin as described by Yavchitz et al in 2016. We determined the relationship between spin and study characteristics using odds ratios.

Results

Of an initial 1560 studies, 176 were included. We found that 50 studies (28.4%) contained at least 1 form of spin. The 2 most common forms of spin found in our sample were type 5 ("Conclusion claims the beneficial effect of treatment despite high risk of bias"; n = 27 [15.3%]) and type 3 ("Selective reporting or overemphasis of efficacy in outcomes favoring beneficial effect of intervention"; n = 18 [10.2%]). No statistical significance was found between study characteristics and the presence of spin.

Conclusion

Spin was present in 28% of the systematic reviews that covered PRP-related orthopaedic treatments. Spin was not associated with general study characteristics, including adherence to PRISMA guidelines or funding. Journals and authors should be aware of spin in articles and avoid its usage.

Clinical trial data-sharing policies among journals, funding agencies, foundations, and other professional organizations: a scoping review

BACKGROUND AND OBJECTIVES

To identify the similarities and differences in data-sharing policies for clinical trial data that are endorsed by biomedical journals, funding agencies, and other professional organizations. Additionally, to determine the beliefs, and opinions regarding data-sharing policies for clinical trials discussed in articles published in biomedical journals.

METHODS

Two searches were conducted, a bibliographic search for published articles that present beliefs, opinions, similarities, and differences regarding policies governing the sharing of clinical trial data. The second search analyzed the gray literature (non-peer-reviewed publications) to identify important data-sharing policies in selected biomedical journals, foundations, funding agencies, and other professional organizations.

RESULTS

A total of 471 articles were included after database search and screening, with 45 from the bibliographic search and 426 from the gray literature search. A total of 424 data-sharing policies were included. Fourteen of the 45 published articles from the bibliographic search (31.1%) discussed only advantages specific to data-sharing policies, 27 (27/45; 60%) discussed both advantages and disadvantages, and 4 (4/45; 8.9%) discussed only disadvantages specific. A total of 216 journals (of 270; 80%) specified a data-sharing policy provided by the journal itself. One hundred industry data-sharing policies were included, and 32 (32%) referenced a data-sharing policy on their website. One hundred and thirty-six (42%) organizations (of 327) specified a data-sharing policy.

CONCLUSION

We found many similarities listed as advantages to data-sharing and fewer disadvantages were discussed within the literature. Additionally, we found a wide variety of commonalities and differences-such as the lack of standardization between policies, and inadequately addressed details regarding the accessibility of research data-that exist in data-sharing policies endorsed by biomedical journals, funding agencies, and other professional organizations. Our study may not include information on all data sharing policies and our data is limited to the entities' descriptions of each policy.

Inconsistent Outcome Reporting in Heart Failure Randomized Controlled Trials

BACKGROUND

Randomized controlled trials (RCTs) may report outcomes different from those prespecified on trial-registration websites, protocols and statistical analysis plans (SAPs). This study sought to investigate the prevalence and characteristics of heart failure (HF) RCTs that report outcomes different from those prespecified.

METHODS AND RESULTS

MEDLINE via PubMed was searched to include phase II-IV HF RCTs in 9 high-impact journals from 2010 to 2020. Outcomes reported in trial publications were compared with prespecified outcomes in protocols, registration websites and SAPs. We used the χ² or Fisher exact test to analyze correlations between trial characteristics and inconsistencies. Among 216 trials, 32 inconsistencies were observed in 28 trials (13.0%). Among 32 inconsistencies, 2 (6.3%) pertained to omission of prespecified primary outcomes, 4 (12.5%) to omission of prespecified secondary outcomes, 2 (6.3%) to changing prespecified primary outcomes to secondary outcomes, and 2 (6.3%) to changing prespecified secondary outcomes to primary outcomes. Of the inconsistencies, 3 (9.4%) pertained to addition of new primary outcomes, 17 (53.1%) to addition of new secondary outcomes, and 2 (6.3%,) to changes in the timing of assessment of primary outcomes. The majority of the inconsistencies favored statistically significant findings; 78 (36.1%) were registered retrospectively. Single-center recruitment was associated with outcome inconsistencies (β = -0.14; 95% CI, -0.22 - -0.01; P = 0.035).

CONCLUSIONS

More than 1 in 10 trials reported outcomes inconsistent with those specified in trial registration websites, SAPs and protocols. An action plan is warranted to minimize selective reporting and improve transparency.

A review of reproducible and transparent research practices in urology publications from 2014 to2018

Background

Reproducibility is essential for the integrity of scientific research. Reproducibility is measured by the ability of different investigators to replicate the outcomes of an original publication using the same materials and procedures. Unfortunately, reproducibility is not currently a standard being met by most scientific research.

Methods

For this review, we sampled 300 publications in the field of urology to assess for 14 indicators of reproducibility including material availability, raw data availability, analysis script availability, pre-registration information, links to protocols, and if the publication was available free to the public. Publications were also assessed for statements about conflicts of interest and funding sources.

Results

Of the 300 sample publications, 171 contained empirical data available for analysis of reproducibility. Of the 171 articles with empirical data to analyze, 0.58% provided links to protocols, 4.09% provided access to raw data, 3.09% provided access to materials, and 4.68% were pre-registered. None of the studies provided analysis scripts. Our review is cross-sectional in nature, including only PubMed indexed journals-published in English-and within a finite time period. Thus, our results should be interpreted in light of these considerations.

Conclusion

Current urology research does not consistently provide the components needed to reproduce original studies. Collaborative efforts from investigators and journal editors are needed to improve research quality while minimizing waste and patient risk.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12894-022-01059-8.

Registry versus publication: discrepancy of primary outcomes and possible outcome reporting bias in child and adolescent mental health

Outcome reporting bias is one of the fundamental forms of publication bias. It implies publishing only outcomes that have positive results. The aim of this observational study was to explore primary outcome discrepancies between registry of clinical trials and their corresponding publications, since these can indicate outcome reporting bias in child mental health. Data were extracted from completed interventional clinical trials from ClinicalTrial.gov registry and its Archive site. Trials were registered under "Behaviours and Mental Disorders" category, and conducted on underage participants (0-17 years). Their primary outcomes were compared to those published in publication which had a corresponding NCT number stated in the text. Sixteen percent of trials did not have the minimum information on primary outcome stated in the registry-neither the measure used nor the measurement time points; 38.9% of trials had the minimum information stated to describe primary outcome, while only 3.3% of trials had all the necessary elements stated in the registry. Most of the publication in our sample had positive results (66.4%). Half of the trials registered before completion had non-matching primary outcomes in the registry and publication; 85.4% of trials with non-matching outcomes indicated possible outcome reporting bias for some of the primary outcome. Middle-sized trials and industry-funded trials were related with higher quality of primary outcome registration. Industry funding was related with positive findings in publication. Non-industry funding proved to be the only significant predictor of discrepancy between registered and published primary outcomes, and possible outcome reporting bias. Journal impact factor was not related with any of the outcome measures. The main limitation of the study is that it primarily offers an insight into discrepancy of registered and published outcomes. The methodology does not imply an access to results of unpublished outcomes - therefore, it was not possible to determine the presence of the bias with sufficient certainty in large number of trials. Further research should be done with improved methodology and additional data.

Evaluating the reproducibility of research in obstetrics and gynecology

OBJECTIVE

Reproducibility is a core tenet of scientific research. A reproducible study is one where the results can be recreated by using the same methodology and materials as the original researchers. Unfortunately, reproducibility is not a standard to which the majority of research is currently adherent.

METHODS

Our cross-sectional survey evaluated 300 trials in the field of Obstetrics and Gynecology. Our primary objective was to identify nine indicators of reproducibility and transparency. These indicators include availability of data, analysis scripts, pre-registration information, study protocols, funding source, conflict of interest statements and whether or not the study was available via Open Access.

RESULTS

Of the 300 trials in our sample, 208 contained empirical data that could be assessed for reproducibility. None of the trials in our sample provided a link to their protocols or provided a statement on availability of materials. None were replication studies. Just 10.58% provided a statement regarding their data availability, while only 5.82% provided a statement on preregistration. 25.85% failed to report the presence or absence of conflicts of interest and 54.08% did not state the origin of their funding.

CONCLUSION

In the studies we examined, research in the field of Obstetrics and Gynecology is not consistently reproducible and frequently lacks conflict of interest disclosure. Consequences of this could be far-reaching and include increased research waste, widespread acceptance of misleading results and erroneous conclusions guiding clinical decision-making.

Sharing Government Health Data With the Private Sector: Community Attitudes Survey

Background

The use of government health data for secondary purposes, such as monitoring the quality of hospital services, researching the health needs of populations, and testing how well new treatments work, is increasing. This increase in the secondary uses of health data has led to increased interest in what the public thinks about data sharing, in particular, the possibilities of sharing with the private sector for research and development. Although international evidence demonstrates broad public support for the secondary use of health data, this support does not extend to sharing health data with the private sector. If governments intend to share health data with the private sector, knowing what the public thinks will be important. This paper reports a national survey to explore public attitudes in Australia toward sharing health data with private companies for research on and development of therapeutic drugs and medical devices.

Objective

This study aims to explore public attitudes in Australia toward sharing government health data with the private sector.

Methods

A web-based survey tool was developed to assess attitudes about sharing government health data with the private sector. A market research company was employed to administer the web-based survey in June 2019.

Results

The survey was completed by 2537 individuals residing in Australia. Between 51.8% and 57.98% of all participants were willing to share their data, with slightly fewer in favor of sharing to improve health services (51.99%) and a slightly higher proportion in favor of sharing for research and development (57.98%). There was a preference for opt-in consent (53.44%) and broad support for placing conditions on sharing health information with private companies (62% to 91.99%). Wide variability was also observed in participants’ views about the extent to which the private sector could be trusted and how well they would behave if entrusted with people’s health information. In their qualitative responses, the participants noted concerns about private sector corporate interests, corruption, and profit making and expressed doubt about the Australian government’s capacity to manage data sharing safely. The percentages presented are adjusted against the Australian population.

Conclusions

This nationally representative survey provides preliminary evidence that Australians are uncertain about sharing their health data with the private sector. Although just over half of all the respondents supported sharing health data with the private sector, there was also strong support for strict conditions on sharing data and for opt-in consent and significant concerns about how well the private sector would manage government health data. Addressing public concern about sharing government health data with the private sector will require more and better engagement to build community understanding about how agencies can collect, share, protect, and use their personal data.

Clinical relevance assessment of animal preclinical research (RAA) tool: development and explanation

Background

Only a small proportion of preclinical research (research performed in animal models prior to clinical trials in humans) translates into clinical benefit in humans. Possible reasons for the lack of translation of the results observed in preclinical research into human clinical benefit include the design, conduct, and reporting of preclinical studies. There is currently no formal domain-based assessment of the clinical relevance of preclinical research. To address this issue, we have developed a tool for the assessment of the clinical relevance of preclinical studies, with the intention of assessing the likelihood that therapeutic preclinical findings can be translated into improvement in the management of human diseases.

Methods

We searched the EQUATOR network for guidelines that describe the design, conduct, and reporting of preclinical research. We searched the references of these guidelines to identify further relevant publications and developed a set of domains and signalling questions. We then conducted a modified Delphi-consensus to refine and develop the tool. The Delphi panel members included specialists in evidence-based (preclinical) medicine specialists, methodologists, preclinical animal researchers, a veterinarian, and clinical researchers. A total of 20 Delphi-panel members completed the first round and 17 members from five countries completed all three rounds.

Results

This tool has eight domains (construct validity, external validity, risk of bias, experimental design and data analysis plan, reproducibility and replicability of methods and results in the same model, research integrity, and research transparency) and a total of 28 signalling questions and provides a framework for researchers, journal editors, grant funders, and regulatory authorities to assess the potential clinical relevance of preclinical animal research.

Conclusion

We have developed a tool to assess the clinical relevance of preclinical studies. This tool is currently being piloted.

Evaluation of Spin in Abstracts of Systematic Reviews and Meta-analyses Focused on Treatments of Erectile Dysfunction: A Cross-sectional Analysis

INTRODUCTION

It is predicted that erectile dysfunction will affect around 322 million men worldwide by 2025. Because of the large volume of literature on the topic, physicians often turn to systematic reviews and meta-analyses-and particularly abstracts of such articles-for clinical guidance. Thus, it is crucial that findings are not misrepresented in abstracts. In this study, we evaluated the use of spin (ie, the misreporting of study findings by overstating or selectively reporting efficacy results, minimizing harms, or making unwarranted clinical recommendations) in the abstracts of systematic reviews on erectile dysfunction.

METHODS

A search strategy was developed using the MEDLINE and Embase databases to retrieve systematic reviews focused on treatments for erectile dysfunction. 2 investigators independently screened the titles and abstracts from the reviews for study inclusion. Investigators analyzed the included systematic reviews for 9 of the most severe types of spin using a previously developed classification scheme and rated them for methodological quality using the revised A MeaSurement Tool to Assess systematic Reviews (AMSTAR) in a masked, duplicate manner. Study characteristics for each review were also extracted in duplicate.

RESULTS

Our search returned 2,224 articles, of which 102 systematic reviews and meta-analyses were included in the final analysis. A total of 31.4% (32/102) of systematic reviews contained spin. 8 types of spin were identified in our sample. Type 3 (selective reporting of or overemphasis on efficacy outcomes) and type 5 (conclusion claims beneficial effect despite high risk of bias) were the most common types of spin, each occurring in 10.8% (11/102) of abstracts. There was no significant association between the presence of spin and the extracted study characteristics or methodological quality.

CONCLUSION

Spin was present in systematic reviews and meta-analyses covering erectile dysfunction treatments. Steps should be taken to improve the reporting quality of abstracts on erectile dysfunction treatment. Reddy AK, Lulkovich K, Ottwell R, et al. Evaluation of Spin in Abstracts of Systematic Reviews and Meta-analyses Focused on Treatments of Erectile Dysfunction: A Cross-sectional Analysis. Sex Med 2020;9:100284.

Reporting Quality of Randomized Controlled Trials for the Treatment of Eczema with Chinese Patent Medicine Based on the CONSORT-CHM Formulas 2017

OBJECTIVE

Chinese patent medicine (CPM) has been widely used to treat eczema in mainland China for decades. This study aims to investigate circulating CPM for eczema in mainland China and to evaluate the reporting quality of randomized controlled trials (RCTs) of them by using the CONSORT-CHM formulas 2017 (Consolidated Standards of Reporting Trials for Chinese herbal medicine formulas 2017).

METHODS

Circulating CPM with the indication for eczema was selected by searching three drug databases and confirmed by contacting the manufacturers. RCTs for the treatment of eczema with CPM were selected in four Chinese literature databases and four English literature databases from their inception to August 31, 2019. The reporting quality of included RCTs was assessed based on the CONSORT-CHM formulas 2017. A univariate analysis was conducted to identify the factors associated with the reporting quality.

RESULTS

A total of 70 circulating CPMs had the indication for eczema. Among them, 21 CPMs with 144 RCTs reached the eligible criteria. The mean overall quality score (OQS) of 144 RCTs was 19.85 ± 2.73, which was much less than the maximum score of 38. Of the 38 items, 12 items were reported in over 70% of the trials, 6 items were reported in 50%-70% of the trials, and 16 items were reported in less than 50% of the trials. Publication after 2015 (P < 0.001) and the first author from a university hospital (P=0.010) were associated with the better reporting quality.

CONCLUSION

There are a lot of circulating CPMs with the indication for eczema in mainland China, but both the quantity and the reporting quality of RCTs regarding those CPMs are suboptimal. It is necessary that authors and journal editors learn and adhere to the CONSORT-CHM formulas 2017 to enhance the reporting quality of RCTs for the treatment of eczema with CPM.

Evaluation of selective outcome reporting and trial registration practices among addiction clinical trials

BACKGROUND AND AIMS

Selective outcome reporting occurs when trialists pre-specify primary and secondary outcomes during trial planning but alter the definitions in the published report. Here, we investigate selective outcome reporting in published addiction randomized controlled trials (RCTs) and evaluate whether particular funding sources are associated with an increased likelihood of selective outcome reporting.

DESIGN

We conducted a cross-sectional study of published addiction clinical trials. A PubMed search was performed to identify RCTs in addiction journals from 2013 to 2017. Included studies used a randomized design to address one of the following topics: (1) drug, alcohol and tobacco addiction prevention, (2) stabilization following excessive use of a substance, (3) relapse prevention or (4) recovery maintenance.

SETTING

Single-center, medical research institution.

PARTICIPANTS

Our sample included 162 RCTs that were prospectively registered with a clearly defined primary outcome.

MEASUREMENT

We extracted the following items from addiction RCTs: journal, funding source, trial registry number (if included), sample size, dates of subject enrollment, whether primary and secondary outcomes were denoted, all published outcomes, P-value for all outcomes and whether authors mentioned any deviations from the trial protocol as it related to RCT outcomes.

FINDINGS

In total, 47 of 162 RCTs (29.0%) had at least one major discrepancy between the trial registry and published RCT. Overall, these 47 RCTs included 54 major discrepancies. The most common major discrepancy was demotion of a primary registered outcome (19/54, 35.2%). The majority of RCTs (132/162, 81.5%) were funded from public sources. Additionally, 166 RCTs were excluded from our sample because registration could not be confirmed.

CONCLUSIONS

There is evidence suggestive of selective outcome reporting in addiction randomized controlled trials (RCTs). The most common major discrepancies pertained to the primary outcome.

Reproducible and transparent research practices in published neurology research

BACKGROUND

The objective of this study was to evaluate the nature and extent of reproducible and transparent research practices in neurology publications.

METHODS

The NLM catalog was used to identify MEDLINE-indexed neurology journals. A PubMed search of these journals was conducted to retrieve publications over a 5-year period from 2014 to 2018. A random sample of publications was extracted. Two authors conducted data extraction in a blinded, duplicate fashion using a pilot-tested Google form. This form prompted data extractors to determine whether publications provided access to items such as study materials, raw data, analysis scripts, and protocols. In addition, we determined if the publication was included in a replication study or systematic review, was preregistered, had a conflict of interest declaration, specified funding sources, and was open access.

RESULTS

Our search identified 223,932 publications meeting the inclusion criteria, from which 400 were randomly sampled. Only 389 articles were accessible, yielding 271 publications with empirical data for analysis. Our results indicate that 9.4% provided access to materials, 9.2% provided access to raw data, 0.7% provided access to the analysis scripts, 0.7% linked the protocol, and 3.7% were preregistered. A third of sampled publications lacked funding or conflict of interest statements. No publications from our sample were included in replication studies, but a fifth were cited in a systematic review or meta-analysis.

CONCLUSIONS

Currently, published neurology research does not consistently provide information needed for reproducibility. The implications of poor research reporting can both affect patient care and increase research waste. Collaborative intervention by authors, peer reviewers, journals, and funding sources is needed to mitigate this problem.

Assessment of transparent and reproducible research practices in the psychiatry literature

BACKGROUND

Reproducibility is a cornerstone of scientific advancement; however, many published works may lack the core components needed for study reproducibility.

AIMS

In this study, we evaluate the state of transparency and reproducibility in the field of psychiatry using specific indicators as proxies for these practices.

METHODS

An increasing number of publications have investigated indicators of reproducibility, including research by Harwicke et al, from which we based the methodology for our observational, cross-sectional study. From a random 5-year sample of 300 publications in PubMed-indexed psychiatry journals, two researchers extracted data in a duplicate, blinded fashion using a piloted Google form. The publications were examined for indicators of reproducibility and transparency, which included availability of: materials, data, protocol, analysis script, open-access, conflict of interest, funding and online preregistration.

RESULTS

This study ultimately evaluated 296 randomly-selected publications with a 3.20 median impact factor. Only 107 were available online. Most primary authors originated from USA, UK and the Netherlands. The top three publication types were cohort studies, surveys and clinical trials. Regarding indicators of reproducibility, 17 publications gave access to necessary materials, four provided in-depth protocol and one contained raw data required to reproduce the outcomes. One publication offered its analysis script on request; four provided a protocol availability statement. Only 107 publications were publicly available: 13 were registered in online repositories and four, ten and eight publications included their hypothesis, methods and analysis, respectively. Conflict of interest was addressed by 177 and reported by 31 publications. Of 185 publications with a funding statement, 153 publications were funded and 32 were unfunded.

CONCLUSIONS

Currently, Psychiatry research has significant potential to improve adherence to reproducibility and transparency practices. Thus, this study presents a reference point for the state of reproducibility and transparency in Psychiatry literature. Future assessments are recommended to evaluate and encourage progress.

Using the CONSORT statement to evaluate the completeness of reporting of addiction randomised trials: a cross-sectional review

OBJECTIVES

Evaluate the completeness of reporting of addiction randomised controlled trials (RCTs) using the Consolidated Standards of Reporting Trials (CONSORT) statement.

SETTING

Not applicable.

PARTICIPANTS

RCTs identified using a PubMed search of 15 addiction journals and a 5-year cross-section.

OUTCOME MEASURES

Completeness of reporting.

RESULTS

Our analysis of 394 addiction RCTs found that the mean number of CONSORT items reported was 19.2 (SD 5.2), out of a possible 31. Twelve items were reported in <50% of RCTs; similarly, 12 items were reported in >75% of RCTs. Journal endorsement of CONSORT was found to improve the number of CONSORT items reported.

CONCLUSIONS

Poor reporting quality may prohibit readers from critically appraising the methodological quality of addiction trials. We recommend journal endorsement of CONSORT since our study and those previous have shown that CONSORT endorsement improves the quality of reporting.

Effect of funding source on reporting bias in studies of intravitreal anti-vascular endothelial growth factor therapy for retinal vein occlusion

PURPOSE

To examine the relationship between industry funding and outcome reporting bias in high-quality studies investigating the use of intravitreal anti-vascular endothelial growth factor (VEGF) agents for patients with macular oedema secondary to branch or central retinal vein occlusion (RVO).

METHODS

This systematic review in PubMed and Ovid MEDLINE examined all randomized clinical trials and meta-analyses published in journals with impact factor of ≥2 that investigated effectiveness of intravitreal anti-VEGF therapy in patients with RVO. The main outcome measure was correspondence between statistical outcome and abstract conclusion wording.

RESULTS

Forty-five studies met inclusion criteria; 38 (84%) showed correspondence between outcome and abstract conclusion without difference between industry-funded and nonindustry-funded publications (p = 0.39) or between publications in journals with impact factor ≥3 versus <3 (p = 0.96).

CONCLUSION

In high-quality studies of intravitreal anti-VEGF therapy for RVO, neither industry funding nor journal impact factor affected the rate of outcome reporting bias.

Orthognathic Surgery as a Treatment for Temporomandibular Disorders

Well-controlled clinical trials supporting orthognathic surgery as the primary management for temporomandibular disorders (TMDs) are lacking. Most published studies lack an adequate experimental design to minimize biases. Studies that did minimize some biases do support an overall reduction in the frequency of TMD signs and symptoms in some Class III and Class II patients who had orthognathic surgery. However, Class II correction with counter-clockwise rotation of the mandible increased TMD. Individual variability precludes the ability to predict TMD outcome after surgery. Irreversible therapies such as orthognathic surgery should not be primary treatments in the management or prevention of TMDs.

An Institutional Program to Increase Compliance with Clinicaltrials.gov Requirements

Recent National Institutes of Health policy changes have expanded the number of research studies that must be registered in clinicaltrials.gov beyond the requirements of the Food and Drug Administration Amendments Act of 2007. The International Committee of Medical Journal Editors has also adopted a policy that requires registration of research in a public database. The goal was to increase the transparency of research by reporting the original endpoints of a study, and to discern whether primary endpoints were excluded in subsequent publications. Efforts to increase openness and accountability in clinical trials are likely to strengthen public trust. However, first investigators and study staff must be educated about the requirements, and staff must be prepared to offer support to researchers in navigating the clinicaltrials.gov system. For academic institutions, maintaining compliance requires continuous oversight so that problems can be identified centrally and addressed with investigators. At Wake Forest University Health Sciences, because researchers often did not realize they were out of compliance, we implemented a program to assist them and provide oversight. We introduced standard operating procedures, provided education and assistance to investigators, and engaged leadership about consequences of compliance, resulting in increased budget support for a full-time employee in this role. As a result of these changes, compliance increased from 22% to 92% over 4 months. These approaches may help other institutions become compliant with registration requirements more quickly.

Inclusion and definition of acute renal dysfunction in critically ill patients in randomized controlled trials: a systematic review

BACKGROUND

In evidence-based medicine, multicenter, prospective, randomized controlled trials (RCTs) are the gold standard for evaluating treatment benefits and ensuring the effectiveness of interventions. Patient-centered outcomes, such as mortality, are most often the preferred evaluated outcomes. While there is currently agreement on how to classify renal dysfunction in critically ill patients , the application frequency of this new classification system in RCTs has not previously been evaluated. In this study, we aim to assess the definition of renal dysfunction in multicenter RCTs involving critically ill patients that included mortality as a primary endpoint.

METHODS

A comprehensive search was conducted for publications reporting multicenter randomized controlled trials (RCTs) involving adult patients in intensive care units (ICUs) that included mortality as a primary outcome. MEDLINE and PUBMED were queried for relevant articles in core clinical journals published between May 2004 and December 2017.

RESULTS

Of 418 articles reviewed, 46 multicenter RCTs with a primary endpoint related to mortality were included. Thirty-six (78.3%) of the trial reports provided information on renal function in the participants. Only seven articles (15.2%) included mean or median serum creatinine levels, mean creatinine clearance or estimated glomerular filtration rates. Sequential organ failure assessment (SOFA) score was the most commonly used definition of renal dysfunction (20 studies; 43.5%). Risk, Injury, Failure, Loss, End-stage renal disease (RIFLE), Acute Kidney Injury Network (AKIN) and Kidney Disease Improving Global Outcomes (KDIGO) criteria were used in five (10.9%) trials. In thirteen trials (28.3%), no renal dysfunction criteria were reported. Only one trial excluded patients with renal dysfunction, and it used urinary output or need for renal replacement therapy (RRT) as criteria for this diagnosis.

CONCLUSION

The presence of renal dysfunction was included as a baseline patient characteristic in most RCTs. The RIFLE, AKIN and KDIGO classification systems were infrequently used; renal dysfunction was generally defined using the SOFA score.

Inconsistent selection of outcomes and measurement devices found in shoulder arthroplasty research: An analysis of studies on ClinicalTrials.gov

Introduction

Recent evidence suggests a lack of standardization of shoulder arthroplasty outcomes. This issue is a limiting factor in systematic reviews. Core outcome set (COS) methodology could address this problem by delineating a minimum set of outcomes for measurement in all shoulder arthroplasty trials.

Methods

A ClinicalTrials.gov search yielded 114 results. Eligible trials were coded on the following characteristics: study status, study type, arthroplasty type, sample size, measured outcomes, outcome measurement device, specific metric of measurement, method of aggregation, outcome classification, and adverse events.

Results

Sixty-six trials underwent data abstraction and data synthesis. Following abstraction, 383 shoulder arthroplasty outcomes were organized into 11 outcome domains. The most commonly reported outcomes were shoulder outcome score (n = 58), pain (n = 33), and quality of life (n = 15). The most common measurement devices were the Constant-Murley Shoulder Outcome Score (n = 38) and American Shoulder and Elbow Surgeons Shoulder Score (n = 33). Temporal patterns of outcome use was also found.

Conclusion

Our study suggests the need for greater standardization of outcomes and instruments. The lack of consistency across trials indicates that developing a core outcome set for shoulder arthroplasty trials would be worthwhile. Such standardization would allow for more effective comparison across studies in systematic reviews, while at the same time consider important outcomes that may be underrepresented otherwise. This review of outcomes provides an evidence-based foundation for the development of a COS for shoulder arthroplasty.