The impact of the CONSORT statement on reporting of randomized clinical trials in psychiatry

Quality of reporting and concordance between sources of adverse events in the treatment of moderate-to-severe psoriasis: a cross-sectional study of RCTs from a Cochrane systematic review

OBJECTIVES

Incomplete reporting of safety outcomes in quality and availability of safety reporting in published articles of randomized controlled trials (RCTs) were described in different medical areas. The number of RCTs assessing systemic treatments for psoriasis has increased considerably. Complete and precise reporting of safety is mandatory for the efficacy/harms balance evaluation. We aimed to assess the quality and availability of safety reporting in published RCTs assessing systemic treatments for psoriasis, as well as the concordance of data between published trials and ClinicalTrials.gov (CT).

STUDY DESIGN AND SETTING

We included all RCTs in adults initiated after September 2009, assessing systemic psoriasis treatments compared with placebo or with an active comparator. All trials were selected in duplicate by 2 independent authors from the latest search of the dedicated Cochrane review. We described quality of safety reporting for all published RCTs, using a modified Consolidated Standards of Reporting Trials harms scale by using descriptive analysis, and a composite score of 3 key items of safety report. For each RCT, data on adverse events (AEs)/serious AEs (SAEs) were extracted from the publication and CT: total number of AEs/SAEs, patients with AEs/SAEs, SAEs by system organ class classification and deaths. These data were compared between sources for each RCT.

RESULTS

In total, 128 trials were included in the analysis of reporting quality, and 76 in the analysis of data concordance between sources. The median number of reported Consolidated Standards of Reporting Trials harms items per article was 9 out of 18 (IQR 7-10), and mean number was 8.39 (SD = 3.02). Items in the methods section were the least frequently reported. The proportion of RCTs reporting the number of SAEs and death were significantly higher on CT than in the published article ((100% (76/76) vs 88.2%, McNemar test, P < .0016). At least 1 discrepancy between sources for SAE safety data was found in 30/76 (39.5%) RCTs.

CONCLUSION

Shortcomings and gaps in the quality of safety reporting in publications of RCTs of systemic psoriasis treatments have been identified. A lack of data in published articles and discrepancies between published articles and CT data complete this finding.

Survey of quality and clarity of methods and results reporting in 1 year of intervention studies published in high-impact medical and psychiatric journals

OBJECTIVE

We assessed how well articles in major medical and psychiatric journals followed best reporting practices in presenting results of intervention studies.

METHOD

Standardised data collection was used to review studies in high-impact and widely read medical (JAMA, Lancet and New England Journal of Medicine) and psychiatric (American Journal of Psychiatry, JAMA Psychiatry, Journal of Clinical Psychiatry and Lancet Psychiatry) journals, published between 1 September 2018 and 31 August 2019. Two team members independently reviewed each article.

MEASURES

The primary outcome measure was proportion of papers reporting consensus elements required to understand and evaluate the results of the intervention. The secondary outcome measure was comparison of complete and accessible reporting in the major medical versus the major psychiatric journals.

RESULTS

One hundred twenty-seven articles were identified for inclusion. At least 90% of articles in both medical and psychiatric journals included sample size, statistical significance, randomisation method, elements of study flow, and age, sex, and illness severity by randomisation group. Selected elements less frequently reported by either journal type were confidence intervals in the abstract, reported in 93% (95% CI 84% to 97%) of medical journal articles and 58% (95% CI 45% to 69%) of psychiatric journal articles, and sample size method (93%, 95% CI 84% to 97% medical; 69%, 95% CI 57% to 80% psychiatric), race and ethnicity by randomisation group (51%, 95% CI 40% to 63% medical; 73%, 95% CI 60% to 83% psychiatric), and adverse events (94%; 95% CI 86% to 98% medical; 80%, 95% CI 68% to 88% psychiatric) in the main text. CIs were included less often in psychiatric than medical journals (p<0.004 abstract, p=0.04 main text, after multiple-testing correction).

CONCLUSIONS

Recommendations include standard inclusion of a table specifying the outcome(s) designated as primary, and the sample size, effect size(s), CI(s) and p value(s) corresponding to the primary test(s) for efficacy.

Reporting quality for abstracts of randomised trials on child and adolescent depression prevention: a meta-epidemiological study on adherence to CONSORT for abstracts

OBJECTIVES

This study aimed to investigate adherence to Consolidated Standards of Reporting Trials (CONSORT) for abstracts in reports of randomised trials on child and adolescent depression prevention. Secondary objective was to examine factors associated with overall reporting quality.

DESIGN

Meta-epidemiological study.

DATA SOURCES

We searched MEDLINE, EMBASE, PsycINFO, PsycArticles and CENTRAL.

ELIGIBILITY CRITERIA

Trials were eligible if the sample consisted of children and adolescents under 18 years with or without an increased risk for depression or subthreshold depression. We included reports published from 1 January 2003 to 8 August 2020 on randomised controlled trials (RCTs) and cluster randomised trials (CRTs) assessing universal, selective and indicated interventions aiming to prevent the onset of depression or reducing depressive symptoms.

DATA EXTRACTION AND SYNTHESIS

As the primary outcome measure, we assessed for each trial abstract whether information recommended by CONSORT was adequately reported, inadequately reported or not reported. Moreover, we calculated a summative score of overall reporting quality and analysed associations with trial and journal characteristics.

RESULTS

We identified 169 eligible studies, 103 (61%) RCTs and 66 (39%) CRTs. Adequate reporting varied considerably across CONSORT items: while 9 out of 10 abstracts adequately reported the study objective, no abstract adequately provided information on blinding. Important adverse events or side effects were only adequately reported in one out of 169 abstracts. Summative scores for the abstracts' overall reporting quality ranged from 17% to 83%, with a median of 40%. Scores were associated with the number of authors, abstract word count, journal impact factor, year of publication and abstract structure.

CONCLUSIONS

Reporting quality for abstracts of trials on child and adolescent depression prevention is suboptimal. To help health professionals make informed judgements, efforts for improving adherence to reporting guidelines for abstracts are needed.

Evaluating reporting of patient-reported outcomes in randomized controlled trials regarding inflammatory bowel disease: a methodological study

Patient-reported outcomes (PROs) in randomized controlled trials pertaining to inflammatory bowel disease are important in identifying patients' perspective of treatment. Incompletely reported PROs within trials could misrepresent information for clinicians and may contribute to treatment which lacks accommodation of patient input. Our study evaluates completeness of reporting of PROs and risk of bias (RoB) to identify how well trialists are adhering to known resources for trials. We used MEDLINE, Embase, and the Cochrane Central Register of Controlled Trials to identify eligible trials from 2006 to 2020 with at least 1 PRO measure related to inflammatory bowel disease. The trials were screened in duplicate using Rayyan. We then compared trial completion of reporting to the Consolidated Standards of Reporting Trials (CONSORT)-PRO adaptation, and assessed RoB using the Cochrane Collaboration RoB 2.0 tool. To measure trial and reporting characteristics, we performed bivariate regression analyses. Among a sample of 29 trials, the mean completion percentage for CONSORT-PRO was 46.77%. We found PROs as a secondary outcome had significantly lower CONSORT-PRO reporting (p<0.05). In addition, per cent completeness of reporting was significantly higher with both a 'therapy' intervention, and trials published following the development of CONSORT-PRO (p<0.05). Incomplete PRO reporting is common in trials focused on inflammatory bowel disease. This suboptimal reporting indicates the need for adherence to reporting guidelines. Trialists should use the CONSORT-PRO checklist, as endorsed by Patient-Reported Outcomes Tools: Engaging Users and Stakeholders, to assess their studies in order to enhance reporting adherence.

Comparing the reporting and conduct quality of exercise and pharmacological randomised controlled trials: a systematic review

OBJECTIVE

Evaluate the quality of exercise randomised controlled trial (RCT) reporting and conduct in clinical populations (ie, adults with or at risk of chronic conditions) and compare with matched pharmacological RCTs.

DESIGN

Systematic review.

DATA SOURCES

Embase (Elsevier), PubMed (NLM) and CINAHL (EBSCO).

STUDY SELECTION

RCTs of exercise in clinical populations with matching pharmacological RCTs published in leading clinical, medical and specialist journals with impact factors ≥15.

REVIEW METHODS

Overall RCT quality was evaluated by two independent reviewers using three research reporting guidelines (ie, Consolidated Standards of Reporting Trials (CONSORT; pharmacological RCTs)/CONSORT for non-pharmacological treatments; exercise RCTs), CONSORT-Harms, Template for Intervention Description and Replication) and two risk of bias assessment (research conduct) tools (ie, Cochrane Risk of Bias, Jadad Scale). We compared research reporting and conduct quality within exercise RCTs with matched pharmacological RCTs, and examined factors associated with quality in exercise and pharmacological RCTs, separately.

FINDINGS

Forty-eight exercise RCTs (11 658 patients; median sample n=138) and 48 matched pharmacological RCTs were evaluated (18 501 patients; median sample n=160). RCTs were conducted primarily in cardiovascular medicine (43%) or oncology (31%). Overall quality score (composite of all research reporting and conduct quality scores; primary endpoint) for exercise RCTs was 58% (median score 46 of 80; IQR: 39-51) compared with 77% (53 of 68; IQR: 47-58) in the matched pharmacological RCTs (p≤0.001). Individual quality scores for trial reporting and conduct were lower in exercise RCTs compared with matched pharmacological RCTs (p≤0.03). Factors associated with higher overall quality scores for exercise RCTs were journal impact factor (≥25), sample size (≥152) and publication year (≥2013).

CONCLUSIONS AND RELEVANCE

Research reporting and conduct quality within exercise RCTs is inferior to matched pharmacological RCTs. Suboptimal RCT reporting and conduct impact the fidelity, interpretation, and reproducibility of exercise trials and, ultimately, implementation of exercise in clinical populations.

PROSPERO REGISTRATION NUMBER

CRD42018095033.

Development and Validation of a Natural Language Processing Tool to Generate the CONSORT Reporting Checklist for Randomized Clinical Trials

IMPORTANCE

Adherence to the Consolidated Standards of Reporting Trials (CONSORT) for randomized clinical trials is associated with improvingquality because inadequate reporting in randomized clinical trials may complicate the interpretation and the application of findings to clinical care.

OBJECTIVE

To evaluate an automated reporting checklist generation tool that uses natural language processing (NLP), called CONSORT-NLP.

DESIGN, SETTING, AND PARTICIPANTS

This study used published journal articles as training, testing, and validation sets to develop, refine, and evaluate the CONSORT-NLP tool. Articles reporting randomized clinical trials were selected from 25 high-impact-factor journals under the following categories: (1) general and internal medicine, (2) oncology, and (3) cardiac and cardiovascular systems.

MAIN OUTCOMES AND MEASURES

For an evaluation of the performance of this tool, an accuracy metric defined as the number of correct assessments divided by all assessments was calculated.

RESULTS

The CONSORT-NLP tool uses the widely used Portable Document Format as an input file. Of the 37 CONSORT reporting items, 34 (92%) were included in the tool. Of these 34 reporting items, 30 were fully implemented; 28 (93%) of the fully implemented CONSORT reporting items had an accuracy of more than 90% for the validation set. The remaining 2 (7%) had an accuracy between 80% and 90% for the validation set. Two to 5 articles were selected from each of these journals for a total of 158 articles to establish a training set of 111 articles to train CONSORT-NLP for CONSORT reporting items, a testing set of 25 articles to refine CONSORT-NLP, and a validation set of 22 articles to assess the performance of CONSORT-NLP. The CONSORT-NLP tool used the Portable Document Format of the articles as input files. A CONSORT-NLP graphical user interface was built using Java in 2019. The time required to complete the CONSORT checklist manually vs using the CONSORT-NLP tool was compared for 30 articles. Two case studies for randomized clinical trials are provided as an illustration for the CONSORT-NLP tool. For the 30 articles investigated, CONSORT-NLP required a mean (SD) 23.0 (4.1) seconds, whereas the manual reviewer required a mean (SD) 11.9 (2.2), 22.6 (4.6), and 57.6 (7.1) minutes, for 3 reviewers, respectively.

CONCLUSIONS AND RELEVANCE

The CONSORT-NLP tool is designed to assist in the reporting of randomized clinical trials. Potential users of CONSORT-NLP include clinicians, researchers, and scientists who plan to publish a randomized trial study in a peer-reviewed journal. The use of CONSORT-NLP may help them save substantial time when generating the CONSORT checklist. This tool may also be useful for manuscript reviewers and journal editors who review these articles.

Evaluation of intervention reporting in published emergency medicine clinical trials

OBJECTIVE

The extent of intervention reporting in emergency medicine journals remains unclear. The primary objective is to assess overall completion of the Template for Intervention Description and Replication (TIDieR) checklist described in emergency medicine randomized clinical trials (RCTs). The secondary outcomes were to (1) compare reporting before and after TIDieR publication; (2) evaluate factors associated with intervention reporting.

METHODS

Our cross-sectional study used Google Scholar's metrics to identify seven emergency medicine journals; of which, we randomly sampled 300 articles. Using two PubMed searches, we extracted 150 RCTs before and after publications of TIDieR. Two investigators independently extracted data. The primary analysis to measure overall completion included descriptive statistics for each checklist item. Our secondary analysis used an interrupted time series analysis and generalized estimating equations to determine the effect of TIDieR publication on intervention reporting.

RESULTS

Our initial search yielded 635 articles; from which, we randomly sampled 300 articles. We excluded 67 articles, leaving 233 for analysis. The mean number of TIDieR items reported was 5.4 (standard deviation = 1.18). Of the 233 trials, 42.9% provided information about materials, 67% provided intervention procedures, and 99.1% provided intervention delivery. The least reported items were intervention modifications (2.6%), intervention adherence assessment methods (3.4%), and intervention adherence assessment outcomes (2.2%).

CONCLUSIONS

The completeness of intervention reporting is suboptimal in emergency medicine journals, necessitating improvement. The current state of adherence could be improved through the combined efforts of journal editors, major editorial organizations, and authors.

Scoping review on interventions to improve adherence to reporting guidelines in health research

OBJECTIVES

The goal of this study is to identify, analyse and classify interventions to improve adherence to reporting guidelines in order to obtain a wide picture of how the problem of enhancing the completeness of reporting of biomedical literature has been tackled so far.

DESIGN

Scoping review.

SEARCH STRATEGY

We searched the MEDLINE, EMBASE and Cochrane Library databases and conducted a grey literature search for (1) studies evaluating interventions to improve adherence to reporting guidelines in health research and (2) other types of references describing interventions that have been performed or suggested but never evaluated. The characteristics and effect of the evaluated interventions were analysed. Moreover, we explored the rationale of the interventions identified and determined the existing gaps in research on the evaluation of interventions to improve adherence to reporting guidelines.

RESULTS

109 references containing 31 interventions (11 evaluated) were included. These were grouped into five categories: (1) training on the use of reporting guidelines, (2) improving understanding, (3) encouraging adherence, (4) checking adherence and providing feedback, and (5) involvement of experts. Additionally, we identified lack of evaluated interventions (1) on training on the use of reporting guidelines and improving their understanding, (2) at early stages of research and (3) after the final acceptance of the manuscript.

CONCLUSIONS

This scoping review identified a wide range of strategies to improve adherence to reporting guidelines that can be taken by different stakeholders. Additional research is needed to assess the effectiveness of many of these interventions.

The impact of STRICTA and CONSORT on reporting of randomised control trials of acupuncture: a systematic methodological evaluation

BACKGROUND

Clear and unambiguous reporting is essential for researchers and clinicians to be able to assess the quality of research. To enhance the quality of reporting, consensus-based reporting guidelines are commonly used.

OBJECTIVES

To update and extend previous research by evaluating the more recent impact of STRICTA (STandards for Reporting Interventions in Controlled Trials of Acupuncture) and CONSORT (CONsolidated Standards Of Reporting Trials) guidelines on the quality of reporting of acupuncture trials.

METHODS

By random sampling, approximately 45 trials from each of five 2-year time periods between 1994 and 2015 were included in the study. Using scoring sheets based on the STRICTA and CONSORT checklist items (range 0 to 7 and 0 to 5, respectively), the distribution of items reported over time was investigated, with changes shown using scatterplots. The primary analysis used a before-and-after t-test to compare time periods. A meta-analysis investigated whether or not trials published in journals that endorsed STRICTA were associated with better reporting.

RESULTS

The study included 207 trials. Improved reporting of items over time was observed, as represented by changes in the scatterplot slope and intercept. The mean STRICTA score increased from 4.27 in the 1994-1995 period to 5.53 in 2014-2015, an 18% improvement. The mean CONSORT score rose from 1.01 in the 1994-1995 period to 3.32 in 2014-2015, an increment of 46%. There was proportionately lower reporting for items related to practitioner background (STRICTA) and for randomisation implementation and allocation concealment (CONSORT). Trials published in journals that endorsed STRICTA had statistically significantly superior reporting of both STRICTA and CONSORT items overall.

CONCLUSION

This study has provided evidence of an improvement in reporting of STRICTA and CONSORT items over the time period from 1994 to 2015. Journals that endorse STRICTA have a better record in terms of reporting quality. Some evidence suggests that the publication of STRICTA has had a positive impact on reporting quality.

Scientific Reporting: Raising the Standards

This article is based on a presentation that was made at the 2014 annual meeting of the editorial board of Health Education & Behavior. The article addresses critical issues related to standards of scientific reporting in journals, including concerns about external and internal validity and reporting bias. It reviews current reporting guidelines, effects of adopting guidelines, and offers suggestions for improving reporting. The evidence about the effects of guideline adoption and implementation is briefly reviewed. Recommendations for adoption and implementation of appropriate guidelines, including considerations for journals, are provided.

The limitations of using randomised controlled trials as a basis for developing treatment guidelines

Randomised controlled trials (RCTs) are considered the 'gold standard' by which novel psychotropic medications and psychological interventions are evaluated and consequently adopted into widespread clinical practice. However, there are some limitations to using RCTs as the basis for developing treatment guidelines. While RCTs allow researchers to determine whether a given medication or intervention is effective in a specific patient sample, for practicing clinicians it is more important to know whether it will work for their particular patient in their particular setting. This information cannot be garnered from an RCT. These inherent limitations are exacerbated by biases in design, recruitment, sample populations and data analysis that are inevitable in real-world studies. While trial registration and CONSORT have been implemented to correct and improve these issues, it is worrying that many trials fail to achieve such standards and yet their findings are used to inform clinical decision making. This perspective piece questions the assumptions of RCTs and highlights the widespread distortion of findings that currently undermine the credibility of this powerful design. It is recommended that the clinical guidelines include advice as to what should be considered good and relevant evidence and that external bodies continue to monitor RCTs to ensure that the outcomes published indeed reflect reality.

Successful recruitment to trials: findings from the SCIMITAR+ Trial

BACKGROUND

Randomised controlled trials (RCT) can struggle to recruit to target on time. This is especially the case with hard to reach populations such as those with severe mental ill health. The SCIMITAR+ trial, a trial of a bespoke smoking cessation intervention for people with severe mental ill health achieved their recruitment ahead of time and target. This article reports strategies that helped us to achieve this with the aim of aiding others recruiting from similar populations.

METHODS

SCIMITAR+ is a multi-centre pragmatic two-arm parallel-group RCT, which aimed to recruit 400 participants with severe mental ill health who smoke and would like to cut down or quit. The study recruited primarily in secondary care through community mental health teams and psychiatrists with a smaller number of participants recruited through primary care. Recruitment opened in October 2015 and closed in December 2016, by which point 526 participants had been recruited. We gathered information from recruiting sites on strategies which led to the successful recruitment in SCIMITAR+ and in this article present our approach to trial management along with the strategies employed by the recruiting sites.

RESULTS

Alongside having a dedicated trial manager and trial management team, we identified three main themes that led to successful recruitment. These were: clinicians with a positive attitude to research; researchers and clinicians working together; and the use of NHS targets. The overriding theme was the importance of relationships between both the researchers and the recruiting clinicians and the recruiting clinicians and the participants.

CONCLUSIONS

This study makes a significant contribution to the limited evidence base of real-world cases of successful recruitment to RCTs and offers practical guidance to those planning and conducting trials. Building positive relationships between clinicians, researchers and participants is crucial to successful recruitment.

Revisiting the Quality of Reporting Randomized Controlled Trials in Nursing Literature

PURPOSE

To examine and update the literature on the quality of randomized controlled trials (RCTs) as reported in top nursing journals, based on manuscripts' adherence to the CONsolidated Standards of Reporting Trials (CONSORT) guidelines.

DESIGN

Descriptive review of adherence of RCT manuscript to CONSORT guidelines.

METHODS

Top 40 International Scientific Indexing (ISI) ranked nursing journals that published 20 or more RCTs between 2010 and 2014, were included in the study. Selected articles were randomly assigned to four reviewers who assessed the quality of the articles using the CONSORT checklist. Data were analyzed using descriptive and inferential statistics.

FINDINGS

A total of 119 articles were included in the review. The mean CONSORT score significantly differed by journal but did not differ based on year of publication. The least consistently reported items included random allocation, who randomly assigned participants and whether those administering the interventions were blinded to group assignment.

CONCLUSIONS

Although progress has been made, there is still room for improvement in the quality of RCT reporting in nursing journals. Special attention must be paid to how adequately studies adhere to the CONSORT prior to publication in nursing journals.

CLINICAL RELEVANCE

Evidence from (RCTs) are thought to provide the best evidence for evaluating the impact of treatments and interventions by the U.S. Preventive Services Task Force. Since the evidence may be used for the development of clinical practice guidelines, it is critical that RCTs be designed, conducted, and reported appropriately and precisely.

Assessing reporting quality of randomized controlled trial abstracts in psychiatry: Adherence to CONSORT for abstracts: A systematic review

BACKGROUND

Reporting quality of randomized controlled trial (RCT) abstracts is important as readers often make their first judgments based on the abstracts. This study aims to assess the reporting quality of psychiatry RCT abstracts published before and after the release of Consolidated Standards of Reporting Trials for Abstracts (CONSORT-A) guidelines.

METHODS

MEDLINE/PubMed search was conducted to identify psychiatric RCTs published during 2005-2007 (pre-CONSORT) and 2012-2014 (post-CONSORT). Two independent reviewers assessed abstracts using a 18-point overall quality score (OQS) based on the CONSORT-A guidelines. Linear regression analysis was conducted to analyze factors associated with reporting quality.

RESULTS

Among 1,927 relevant articles, 285 pre-CONSORT and 214 post-CONSORT psychiatric RCT abstracts were included for analysis. The mean OQS improved from 6.9 (range: 3-13; 95% confidence interval (CI): 6.7-7.2) to 8.2 (range: 4-16; 95% CI: 7.8-8.5) after the CONSORT-A guidelines. Despite improvement, methods of randomization, allocation concealment, and funding source remained to be insufficiently reported (<5%) even after the release of CONSORT-A. High-impact general medical journals, multicenter design, positive outcome, and structured abstracts were associated with better reporting quality.

CONCLUSIONS

The reporting quality in psychiatric RCT abstracts, although improved, remains suboptimal. To improve reporting quality of psychiatry RCT abstracts, greater efforts by both investigators and journal editors are required to enhance better adherence to the CONSORT-A guidelines.

Adherence to the CONSORT Statement in the Reporting of Randomized Controlled Trials on Pharmacological Interventions Published in Iranian Medical Journals

BACKGROUND

Among manuscripts submitted to biomedical journals, randomized controlled trials (RCTs) form the backbone of evidence-based medicine. Hence, their protocol should be designed rigorously and their results should be reported clearly. To improve the quality of RCT reporting, researchers developed the CONSORT Statement in 1996 and updated it in 2010. This study was designed to assess the quality of RCT reporting vis-à-vis adherence to CONSORT among articles published in Iranian medical journals (English, Persian, CONSORT-endorsing, and non-CONSORT-endorsing).

METHODS

In this cross-sectional study, all RCTs published in all Iranian medical journals from September 2012 to September 2013 were retrieved to evaluate their adherence to CONSORT. The journals' instructions for authors were also reviewed to find out whether or not they endorsed CONSORT. The CONSORT 2010 Checklist was used. Microsoft Excel 2007 was applied to analyze the data, and MedCalc was employed to compare the groups.

RESULTS

Totally, 492 pharmacological RCTs that met our inclusion criteria were identified. Twenty-five items were reported in fewer than 50% of the articles. The differences between the articles published in Persian and English language journals were statistically significant in 17 items. The differences between the articles published in the CONSORT-endorsing and non-CONSORT-endorsing journals were significant in 8 items.

CONCLUSION

Our findings showed very weak adherence to CONSORT. Authors, reviewers, and editors should be trained to use standards expressed by the CONSORT Group in reporting RCTs.

The Current Status of Quality of Reporting in Acupuncture Treatment Case Reports: An Analysis of the Core Journal in Korea

OBJECTIVES

The present study aimed to evaluate the overall quality of case reports concerning acupuncture treatment in Korea.

METHODS

We selected a representative Korean journal and retrieved eligible case reports on acupuncture treatment published from 2009 to 2015. We assessed the quality of reporting based on CAse REport (CARE) and STandards for Reporting Interventions in Clinical Trials of Acupuncture (STRICTA) guideline checklists.

RESULTS

A total of 93 eligible case reports of acupuncture treatment were identified among the 107 articles screened. Overall quality of reporting in the case reports was generally acceptable (75.4% on CARE, 67.7% on STRICTA), but several crucial items remained substantially underreported.

CONCLUSIONS

Endorsement of the CARE and STRICTA guidelines is needed to improve the completeness of reporting. Our findings will be helpful in developing a more appropriate reporting guideline for case reports in acupuncture treatment.

Endorsement of PRISMA statement and quality of systematic reviews and meta-analyses published in nursing journals: a cross-sectional study

OBJECTIVE

Systematic reviews (SRs) often poorly report key information, thereby diminishing their usefulness. Previous studies evaluated published SRs and determined that they failed to meet explicit criteria or characteristics. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement was recommended as a reporting guideline for SR and meta-analysis (MA), but previous studies showed that adherence to the statement was not high for SRs published in different medical fields. Thus, the aims of this study are twofold: (1) to investigate the number of nursing journals that have required or recommended the use of the PRISMA statement for reporting SR, and (2) to examine the adherence of SRs and/or meta-analyses to the PRISMA statement published in nursing journals.

DESIGN

A cross-sectional study.

METHODS

Nursing journals listed in the ISI journal citation report were divided into 2 groups based on the recommendation of PRISMA statement in their 'Instruction for Authors'. SRs and meta-analyses published in 2014 were searched in 3 databases. 37 SRs and meta-analyses were randomly selected in each group. The adherence of each item to the PRISMA was examined and summarised using descriptive statistics. The quality of the SRs was assessed by Assessing the Methodological Quality of Systematic Reviews. The differences between the 2 groups were compared using the Mann-Whitney U test.

RESULTS

Out of 107 nursing journals, 30 (28.0%) recommended or required authors to follow the PRISMA statement when they submit SRs or meta-analyses. The median rates of adherence to the PRISMA statement for reviews published in journals with and without PRISMA endorsement were 64.9% (IQR: 17.6-92.3%) and 73.0% (IQR: 59.5-94.6%), respectively. No significant difference was observed in any of the items between the 2 groups.

CONCLUSIONS

The median adherence of SRs and meta-analyses in nursing journals to PRISMA is low at 64.9% and 73.0%, respectively. Nonetheless, the adherence level of nursing journals to the PRISMA statement does not significantly vary whether they endorse or recommend such a guideline.

Evaluation of the Endorsement of the STrengthening the REporting of Genetic Association Studies (STREGA) Statement on the Reporting Quality of Published Genetic Association Studies

The STrengthening the REporting of Genetic Association studies (STREGA) statement was based on the STrengthening the REporting of OBservational studies in Epidemiology (STROBE) statement, and it was published in 2009 in order to improve the reporting of genetic association (GA) studies. Our aim was to evaluate the impact of STREGA endorsement on the quality of reporting of GA studies published in journals in the field of genetics and heredity (GH). Quality of reporting was evaluated by assessing the adherence of papers to the STREGA checklist. After identifying the GH journals that endorsed STREGA in their instructions for authors, we randomly appraised papers published in 2013 from journals endorsing STREGA that published GA studies (Group A); in GH journals that never endorsed STREGA (Group B); in GH journals endorsing STREGA, but in the year preceding its endorsement (Group C); and in the same time period as Group C from GH journals that never endorsed STREGA (Group D). The STREGA statement was referenced in 29 (18.1%) of 160 GH journals, of which 18 (62.1%) journals published GA studies. Among the 18 journals endorsing STREGA, we found a significant increase in the overall adherence to the STREGA checklist over time (A vs C; P < 0.0001). Adherence to the STREGA checklist was significantly higher in journals endorsing STREGA compared to those that did not endorse the statement (A vs B; P = 0.04). No significant improvement was detected in the adherence to STREGA items in journals not endorsing STREGA over time (B vs D; P > 0.05). The endorsement of STREGA resulted in an increase in quality of reporting of GA studies over time, while no similar improvement was reported for journals that never endorsed STREGA.

Use of adjectives in abstracts when reporting results of randomized, controlled trials from industry and academia

Objective

Accurate representation of study findings is crucial to preserve public trust. The language used to describe results could affect perceptions of the efficacy or safety of interventions. We sought to compare the adjectives used in clinical trial reports of industry-authored and non-industry-authored research.

Methods

We included studies in PubMed that were randomized trials and had an abstract. Studies were classified as “non-industry-authored” when all authors had academic or governmental affiliations, or as “industry-authored” when any of the authors had industry affiliations. Abstracts were analyzed using a part-of-speech tagger to identify adjectives. To reduce the risk of false positives, the analysis was restricted to adjectives considered relevant to “coloring” (influencing interpretation) of trial results. Differences between groups were determined using exact tests, stratifying by journal.

Results

A total of 306,007 publications met the inclusion criteria. We were able to classify 16,789 abstracts; 9,085 were industry-authored research, and 7,704 were non-industry-authored research. We found a differential use of adjectives between industry-authored and non-industry-authored reports. Adjectives such as “well tolerated” and “meaningful” were more commonly used in the title or conclusion of the abstract by industry authors, while adjectives such as “feasible” were more commonly used by non-industry authors.

Conclusions

There are differences in the adjectives used when study findings are described in industry-authored reports compared with non-industry-authored reports. Authors should avoid overusing adjectives that could be inaccurate or result in misperceptions. Editors and peer reviewers should be attentive to the use of adjectives and assess whether the usage is context appropriate.

Longitudinal analysis of meta-analysis literatures in the database of ISI Web of Science

The meta-analysis is regarded as an important evidence for making scientific decision. The database of ISI Web of Science collected a great number of high quality literatures including meta-analysis literatures. However, it is significant to understand the general characteristics of meta-analysis literatures to outline the perspective of meta-analysis. In this present study, we summarized and clarified some features on these literatures in the database of ISI Web of Science. We retrieved the meta-analysis literatures in the database of ISI Web of Science including SCI-E, SSCI, A&HCI, CPCI-S, CPCI-SSH, CCR-E, and IC. The annual growth rate, literature category, language, funding, index citation, agencies and countries/territories of the meta-analysis literatures were analyzed, respectively. A total of 95,719 records, which account for 0.38% (99% CI: 0.38%-0.39%) of all literatures, were found in the database. From 1997 to 2012, the annual growth rate of meta-analysis literatures was 18.18%. The literatures involved in many categories, languages, fundings, citations, publication agencies, and countries/territories. Interestingly, the index citation frequencies of the meta-analysis were significantly higher than that of other type literatures such as multi-centre study, randomize controlled trial, cohort study, case control study, and cases report (P<0.0001). The increasing numbers, intensively global influence and high citations revealed that the meta-analysis has been becoming more and more prominent in recent years. In future, in order to promote the validity of meta-analysis, the CONSORT and PRISMA standard should be continuously popularized in the field of evidence-based medicine.

Randomized controlled trials in adult traumatic brain injury: a review of compliance to CONSORT statement

OBJECTIVE

To describe the extent to which adherence to Consolidated Standards of Reporting Trials (CONSORT) statement in randomized controlled trials (RCTs) in adult traumatic brain injury (TBI) has improved over time.

DATA SOURCES

MEDLINE, PsycINFO, and CINAHL databases were searched from inception to September 2013.

STUDY SELECTION

Primary report of RCTs in adult TBI. The quality of reporting on CONSORT checklist items was examined and compared over time. Study selection was conducted by 2 researchers independently. Any disagreements were solved by discussion.

DATA EXTRACTION

Two reviewers independently conducted data extraction based on a set of structured data extraction forms. Data regarding the publication years, size, locations, participation centers, intervention types, intervention groups, and CONSORT checklist items were extracted from the including trials.

DATA SYNTHESIS

Of 105 trials reviewed, 38.1%, 5.7%, and 32.4% investigated drugs, surgical procedures, and rehabilitations as the intervention of interest, respectively. Among reports published between the 2 periods 2002 and 2010 (n=51) and 2011 and September 2013 (n=16), the median sample sizes were 99 and 118; 39.2% and 37.5% of all reports detailed implementation of the randomization process; 60.8% and 43.8% provided information on the method of allocation concealment; 56.9% and 31.3% stated how blinding was achieved; 15.7% and 43.8% reported information regarding trial registration; and only 2.0% and 6.3% stated where the full trial protocol could be accessed, all respectively.

CONCLUSIONS

Reporting of several important methodological aspects of RCTs conducted in adult TBI populations improved over the years; however, the quality of reporting remains below an acceptable level. The small sample sizes suggest that many RCTs are likely underpowered. Further improvement is recommended in designing and reporting RCTs.

The reporting quality of parallel randomised controlled trials in ophthalmic surgery in 2011: a systematic review

PURPOSE

Randomised controlled trials (RCTs) represent a gold standard for evaluating therapeutic interventions. However, poor reporting clarity can prevent readers from assessing potential bias that can arise from a lack of methodological rigour. The Consolidated Standards of Reporting Trials statement for non-pharmacological interventions 2008 (CONSORT NPT) was developed to aid reporting. RCTs in ophthalmic surgery pose particular challenges in study design and implementation. We aim to provide the first assessment of the compliance of RCTs in ophthalmic surgery to the CONSORT NPT statement.

METHOD

In August 2012, the Medline database was searched for RCTs in ophthalmic surgery reported between 1 January 2011 and 31 December 2011. Results were searched by two authors and relevant papers selected. Papers were scored against the 23-item CONSORT NPT checklist and compared against surrogate markers of paper quality. The CONSORT score was also compared between different RCT designs.

RESULTS

In all, 186 papers were retrieved. Sixty-five RCTs, involving 5803 patients, met the inclusion criteria. The mean CONSORT score was 8.9 out of 23 (39%, range 3.0-14.7, SD 2.49). The least reported items related to the title and abstract (1.6%), reporting intervention adherence (3.1%), and interpretation of results (4.7%). No significant correlation was found between CONSORT score and journal impact factor (R=0.14, P=0.29), number of authors (R=0.01, P=0.93), or whether the RCT used paired-eye, one-eye, or two-eye designs in their randomisation (P=0.97).

CONCLUSIONS

The reporting of RCTs in ophthalmic surgery is suboptimal. Further work is needed by trial groups, funding agencies, authors, and journals to improve reporting clarity.

Assessment of the quality of reporting in randomised controlled trials of acupuncture in the Korean literature using the CONSORT statement and STRICTA guidelines

OBJECTIVES

This study aims to assess the completeness of reporting of randomised controlled trials (RCTs) of acupuncture in the Korean literature.

DESIGN

Systematic review.

METHODS

We searched 12 Korean databases and 7 Korean journals to identify eligible RCTs of acupuncture published from 1996 to July 2011. We used the Consolidated Standards of Reporting Trials (CONSORT) checklist for parallel RCTs and the revised STandards for Reporting Interventions in Clinical Trials of Acupuncture (STRICTA) to assess the quality of reporting in Korean RCTs. We compared the completeness of reporting of CONSORT and STRICTA items in RCTs published in two time periods (1996-2004 referred to as the early period and 2005-2011 referred to as the late period).

RESULTS

We analysed 146 eligible RCTs using the CONSORT statement concerning RCTs of both needling and non-needling acupuncture and the STRICTA guidelines for 90 trials of needling acupuncture. Among the 103 RCTs in the late period, the proportion of RCTs that completely reported the CONSORT items of outcome definition (15.5%), sample size calculation (2.9%), randomisation (56%), allocation concealment (5.8%), implementation of allocation (11.7%), outcome assessor blinding (20.3%), flow of participants (25.2%), number of participants analysed (19.4%), ancillary analyses (0.0%), adverse events (24.3%), generalisability of findings (1.9%) and overall evidence (32.0%) remained small. Among the 61 RCTs of needling acupuncture in the late period, the STRICTA items of setting/context (24.6%) and practitioner background (27.9%) showed incomplete reporting. The completeness of reporting improved over time in several CONSORT and STRICTA items.

CONCLUSIONS

The completeness of reporting of Korean RCTs of acupuncture was suboptimal according to the CONSORT and revised STRICTA statements. Trial authors and journal editors should use the CONSORT statement and STRICTA guidelines for transparent reporting in Korean RCTs of acupuncture. The endorsement of the CONSORT and revised STRICTA statements in author instructions is also required.

Evaluation of the endorsement of the preferred reporting items for systematic reviews and meta-analysis (PRISMA) statement on the quality of published systematic review and meta-analyses

INTRODUCTION

PRISMA statement was published in 2009 in order to set standards in the reporting of systematic reviews and meta-analyses. Our aim was to evaluate the impact of PRISMA endorsement on the quality of reporting and methodological quality of systematic reviews and meta-analyses, published in journals in the field of gastroenterology and hepatology (GH).

METHODS

Quality of reporting and methodological quality were evaluated by assessing the adherence of papers to PRISMA checklist and AMSTAR quality scale. After identifying the GH journals which endorsed PRISMA in instructions for authors (IA), we appraised: 15 papers published in 2012 explicitly mentioning PRISMA in the full text (Group A); 15 papers from the same journals published in 2012 not explicitly mentioning PRISMA in the full text (Group B); 30 papers published the year preceding PRISMA endorsement from the same journals as above (Group C); 30 papers published in 2012 on the 10 highest impact factor journals in GH which not endorsed PRISMA (Group D).

RESULTS

PRISMA statement was referred in the IA in 9 out of 70 GH journals (12.9%). We found significant increase in overall adherence to PRISMA checklist (Group A, 90.1%; Group C, 83.1%; p = 0.003) and compliance to AMSTAR scale (Group A, 85.0%; Group C, 74.6%; p = 0.002), following the PRISMA endorsement from the nine GH journals. Explicit referencing of PRISMA in manuscript was not associated with increase in quality of reporting and methodological quality (Group A vs. B, p = 0.651, p = 0.900, respectively). Adherence to PRISMA checklist, and the compliance with AMSTAR were significantly higher in journals endorsing PRISMA compared to those not (Groups A+B vs. D; p = 0.003 and p = 0.016, respectively).

CONCLUSION

The endorsement of PRISMA resulted in increase of both quality of reporting and methodological quality. It is advised that an increasing number of medical journals include PRISMA in the instructions for authors.

How psychiatry journals support the unbiased translation of clinical research. A cross-sectional study of editorial policies

INTRODUCTION

Reporting guidelines (e.g. CONSORT) have been developed as tools to improve quality and reduce bias in reporting research findings. Trial registration has been recommended for countering selective publication. The International Committee of Medical Journal Editors (ICMJE) encourages the implementation of reporting guidelines and trial registration as uniform requirements (URM). For the last two decades, however, biased reporting and insufficient registration of clinical trials has been identified in several literature reviews and other investigations. No study has so far investigated the extent to which author instructions in psychiatry journals encourage following reporting guidelines and trial registration.

METHOD

Psychiatry Journals were identified from the 2011 Journal Citation Report. Information given in the author instructions and during the submission procedure of all journals was assessed on whether major reporting guidelines, trial registration and the ICMJE's URM in general were mentioned and adherence recommended.

RESULTS

We included 123 psychiatry journals (English and German language) in our analysis. A minority recommend or require 1) following the URM (21%), 2) adherence to reporting guidelines such as CONSORT, PRISMA, STROBE (23%, 7%, 4%), or 3) registration of clinical trials (34%). The subsample of the top-10 psychiatry journals (ranked by impact factor) provided much better but still improvable rates. For example, 70% of the top-10 psychiatry journals do not ask for the specific trial registration number.

DISCUSSION

Under the assumption that better reported and better registered clinical research that does not lack substantial information will improve the understanding, credibility, and unbiased translation of clinical research findings, several stakeholders including readers (physicians, patients), authors, reviewers, and editors might benefit from improved author instructions in psychiatry journals. A first step of improvement would consist in requiring adherence to the broadly accepted reporting guidelines and to trial registration.

The Oxford Implementation Index: a new tool for incorporating implementation data into systematic reviews and meta-analyses

OBJECTIVES

This article presents a new tool that helps systematic reviewers to extract and compare implementation data across primary trials. Currently, systematic review guidance does not provide guidelines for the identification and extraction of data related to the implementation of the underlying interventions.

STUDY DESIGN AND SETTING

A team of systematic reviewers used a multistaged consensus development approach to develop this tool. First, a systematic literature search on the implementation and synthesis of clinical trial evidence was performed. The team then met in a series of subcommittees to develop an initial draft index. Drafts were presented at several research conferences and circulated to methodological experts in various health-related disciplines for feedback. The team systematically recorded, discussed, and incorporated all feedback into further revisions. A penultimate draft was discussed at the 2010 Cochrane-Campbell Collaboration Colloquium to finalize its content.

RESULTS

The Oxford Implementation Index provides a checklist of implementation data to extract from primary trials. Checklist items are organized into four domains: intervention design, actual delivery by trial practitioners, uptake of the intervention by participants, and contextual factors. Systematic reviewers piloting the index at the Cochrane-Campbell Colloquium reported that the index was helpful for the identification of implementation data.

CONCLUSION

The Oxford Implementation Index provides a framework to help reviewers assess implementation data across trials. Reviewers can use this tool to identify implementation data, extract relevant information, and compare features of implementation across primary trials in a systematic review. The index is a work-in-progress, and future efforts will focus on refining the index, improving usability, and integrating the index with other guidance on systematic reviewing.

Reporting quality of social and psychological intervention trials: a systematic review of reporting guidelines and trial publications

BACKGROUND

Previous reviews show that reporting guidelines have improved the quality of trial reports in medicine, yet existing guidelines may not be fully suited for social and psychological intervention trials.

OBJECTIVE/DESIGN

We conducted a two-part study that reviewed (1) reporting guidelines for and (2) the reporting quality of social and psychological intervention trials.

DATA SOURCES

(1) To identify reporting guidelines, we systematically searched multiple electronic databases and reporting guideline registries. (2) To identify trials, we hand-searched 40 journals with the 10 highest impact factors in clinical psychology, criminology, education, and social work. ELIGIBILITY: (1) Reporting guidelines consisted of articles introducing a checklist of reporting standards relevant to social and psychological intervention trials. (2) Trials reported randomised experiments of complex interventions with psychological, social, or health outcomes.

RESULTS

(1) We identified 19 reporting guidelines that yielded 147 reporting standards relevant to social and psychological interventions. Social and behavioural science guidelines included 89 standards not found in CONSORT guidelines. However, CONSORT guidelines used more recommended techniques for development and dissemination compared to other guidelines. (2) Our review of trials (n = 239) revealed that many standards were poorly reported, such as identification as a randomised trial in titles (20% reported the information) and abstracts (55%); information about blinding (15%), sequence generation (23%), and allocation concealment (17%); and details about actual delivery of experimental (43%) and control interventions (34%), participant uptake (25%), and service environment (28%). Only 11 of 40 journals referenced reporting guidelines in "Instructions to Authors."

CONCLUSION

Existing reporting guidelines have important limitations in content, development, and/or dissemination. Important details are routinely missing from trial publications; most leading journals in social and behavioural sciences do not ask authors to follow reporting standards. Findings demonstrate a need to develop a CONSORT extension with updated standards for social and psychological intervention trials.

Consolidated standards of reporting trials (CONSORT) and the completeness of reporting of randomised controlled trials (RCTs) published in medical journals

BACKGROUND

An overwhelming body of evidence stating that the completeness of reporting of randomised controlled trials (RCTs) is not optimal has accrued over time. In the mid-1990s, in response to these concerns, an international group of clinical trialists, statisticians, epidemiologists, and biomedical journal editors developed the CONsolidated Standards Of Reporting Trials (CONSORT) Statement. The CONSORT Statement, most recently updated in March 2010, is an evidence-based minimum set of recommendations including a checklist and flow diagram for reporting RCTs and is intended to facilitate the complete and transparent reporting of trials and aid their critical appraisal and interpretation. In 2006, a systematic review of eight studies evaluating the "effectiveness of CONSORT in improving reporting quality in journals" was published.

OBJECTIVES

To update the earlier systematic review assessing whether journal endorsement of the 1996 and 2001 CONSORT checklists influences the completeness of reporting of RCTs published in medical journals.

SEARCH METHODS

We conducted electronic searches, known item searching, and reference list scans to identify reports of evaluations assessing the completeness of reporting of RCTs. The electronic search strategy was developed in MEDLINE and tailored to EMBASE. We searched the Cochrane Methodology Register and the Cochrane Database of Systematic Reviews using the Wiley interface. We searched the Science Citation Index, Social Science Citation Index, and Arts and Humanities Citation Index through the ISI Web of Knowledge interface. We conducted all searches to identify reports published between January 2005 and March 2010, inclusive.

SELECTION CRITERIA

In addition to studies identified in the original systematic review on this topic, comparative studies evaluating the completeness of reporting of RCTs in any of the following comparison groups were eligible for inclusion in this review: 1) Completeness of reporting of RCTs published in journals that have and have not endorsed the CONSORT Statement; 2) Completeness of reporting of RCTs published in CONSORT-endorsing journals before and after endorsement; or 3) Completeness of reporting of RCTs before and after the publication of the CONSORT Statement (1996 or 2001). We used a broad definition of CONSORT endorsement that includes any of the following: (a) requirement or recommendation in journal's 'Instructions to Authors' to follow CONSORT guidelines; (b) journal editorial statement endorsing the CONSORT Statement; or (c) editorial requirement for authors to submit a CONSORT checklist and/or flow diagram with their manuscript. We contacted authors of evaluations reporting data that could be included in any comparison group(s), but not presented as such in the published report and asked them to provide additional data in order to determine eligibility of their evaluation. Evaluations were not excluded due to language of publication or validity assessment.

DATA COLLECTION AND ANALYSIS

We completed screening and data extraction using standardised electronic forms, where conflicts, reasons for exclusion, and level of agreement were all automatically and centrally managed in web-based management software, DistillerSR(®). One of two authors extracted general characteristics of included evaluations and all data were verified by a second author. Data describing completeness of reporting were extracted by one author using a pre-specified form; a 10% random sample of evaluations was verified by a second author. Any discrepancies were discussed by both authors; we made no modifications to the extracted data. Validity assessments of included evaluations were conducted by one author and independently verified by one of three authors. We resolved all conflicts by consensus.For each comparison we collected data on 27 outcomes: 22 items of the CONSORT 2001 checklist, plus four items relating to the reporting of blinding, and one item of aggregate CONSORT scores. Where reported, we extracted and qualitatively synthesised data on the methodological quality of RCTs, by scale or score.

MAIN RESULTS

Fifty-three publications reporting 50 evaluations were included. The total number of RCTs assessed within evaluations was 16,604 (median per evaluation 123 (interquartile range (IQR) 77 to 226) published in a median of six (IQR 3 to 26) journals. Characteristics of the included RCT populations were variable, resulting in heterogeneity between included evaluations. Validity assessments of included studies resulted in largely unclear judgements. The included evaluations are not RCTs and less than 8% (4/53) of the evaluations reported adjusting for potential confounding factors.   Twenty-five of 27 outcomes assessing completeness of reporting in RCTs appeared to favour CONSORT-endorsing journals over non-endorsers, of which five were statistically significant. 'Allocation concealment' resulted in the largest effect, with risk ratio (RR) 1.81 (99% confidence interval (CI) 1.25 to 2.61), suggesting that 81% more RCTs published in CONSORT-endorsing journals adequately describe allocation concealment compared to those published in non-endorsing journals. Allocation concealment was reported adequately in 45% (393/876) of RCTs in CONSORT-endorsing journals and in 22% (329/1520) of RCTs in non-endorsing journals. Other outcomes with results that were significant include: scientific rationale and background in the 'Introduction' (RR 1.07, 99% CI 1.01 to 1.14); 'sample size' (RR 1.61, 99% CI 1.13 to 2.29); method used for 'sequence generation' (RR 1.59, 99% CI 1.38 to 1.84); and an aggregate score over reported CONSORT items, 'total sum score' (standardised mean difference (SMD) 0.68 (99% CI 0.38 to 0.98)).

AUTHORS' CONCLUSIONS

Evidence has accumulated to suggest that the reporting of RCTs remains sub-optimal. This review updates a previous systematic review of eight evaluations. The findings of this review are similar to those from the original review and demonstrate that, despite the general inadequacies of reporting of RCTs, journal endorsement of the CONSORT Statement may beneficially influence the completeness of reporting of trials published in medical journals. Future prospective studies are needed to explore the influence of the CONSORT Statement dependent on the extent of editorial policies to ensure adherence to CONSORT guidance.

How transparent is behavioral intervention research on pathological gambling and other gambling-related disorders? A systematic literature review

AIMS

To review the transparency of reports of behavioral interventions for pathological gambling and other gambling-related disorders.

METHODS

We used the Transparent Reporting of Evaluations with Nonrandomized Designs (TREND) Statement to develop the 59-question adapted TREND questionnaire (ATQ). Each ATQ question corresponds to a transparency guideline and asks how clearly a study reports its objectives, research design, analytical methods and conclusions. A subset of 23 ATQ questions is considered particularly important. We searched PubMed, PsychINFO and Web of Science to identify experimental evaluations published between 2000 and 2011 aiming to reduce problem gambling behaviors or decrease problems caused by gambling. Twenty-six English-language reports met the inclusion criteria and were reviewed by three abstractors using the ATQ.

RESULTS

The average report adhered to 38.4 (65.1%) of the 59 ATQ transparency guidelines. Each of the 59 ATQ questions received positive responses from an average of 16.9 (63.8%) of the reports. The subset of 23 particularly relevant questions received an average of 15.3 (66.5%) positive responses. Thirty-two of 59 (54%) ATQ questions were answered positively by 75% or more of the study reports, while 12 (20.3%) received positive responses by 25% or fewer. Publication year did not affect these findings.

CONCLUSIONS

Gambling intervention reports need to improve their transparency by adhering to currently neglected and particularly relevant guidelines. Among them are recommendations for comparing study participants who are lost to follow-up and those who are retained, comparing study participants with the target population, describing methods used to minimize potential bias due to group assignment, and reporting adverse events or unintended effects.

[Quality of reporting in studies on bipolar disorders: implications for the development of guidelines]

BACKGROUND

Selective publishing as well as inadequate reporting of clinical trials entail a risk of bias in clinical decision making. Therefore the CONSORT statement was introduced to improve the quality of reporting of randomized controlled trials (RCT). This study aimed to assess the quality of reporting of RCTs on pharmacological treatment of bipolar disorder in relation to publication period and endorsement of publication guidelines.

METHODS

In the context of the development of the German evidence and consensus-based S3 guidelines for diagnosis and therapy of bipolar disorders a systematic literature search was carried out to identify all RCTs published between 2000 and 2010 relevant to the pharmacological treatment of bipolar disorders. An adapted checklist based on the CONSORT statement was used to assess the quality of reporting.

RESULTS

A total of 134 RCTs were included in this analysis. Of the 72 checklist items, 43% were generally reported adequately (reported in  ≥ 75% of all trials) and 25% inadequately (reported in  < 25% of all trials). Reporting was generally poor for randomization, effect size (reported in 22%) and number needed to treat (NNT 16%). No consistent trend could be shown for improvement in quality of reporting over time or for journals that do or do not endorse the URM (uniform requirements for manuscripts submitted to biomedical journals).

CONCLUSIONS

Clinical investigators as well as editors and reviewers should be further encouraged to follow publication guidelines otherwise trials have to be downgraded or excluded from systematic evaluations.

The reporting of harms in randomized controlled trials of hypertension using the CONSORT criteria for harm reporting

The aim of this study was to assess the quality of reporting of harms in hypertension clinical trials identified from the Cochrane Database using the Consolidated Standards of Reporting Trials (CONSORT) extension for harms reporting. Forty-one hypertension trials were included in the study. On average trials reported less than half of the items recommended by the CONSORT extension for harms (mean 9.83 items; 95% confidence interval = 8.06, 11.60). Trialists need to address the perceived shortcomings in measurement, analysis, and reporting of harms data so that the available trial data can be considered as a balanced and reliable source of evidence.

Method and reporting quality in health professions education research: a systematic review

CONTEXT

Studies evaluating reporting quality in health professions education (HPE) research have demonstrated deficiencies, but none have used comprehensive reporting standards. Additionally, the relationship between study methods and effect size (ES) in HPE research is unknown.

OBJECTIVES

This review aimed to evaluate, in a sample of experimental studies of Internet-based instruction, the quality of reporting, the relationship between reporting and methodological quality, and associations between ES and study methods.

METHODS

We conducted a systematic search of databases including MEDLINE, Scopus, CINAHL, EMBASE and ERIC, for articles published during 1990-2008. Studies (in any language) quantifying the effect of Internet-based instruction in HPE compared with no intervention or other instruction were included. Working independently and in duplicate, we coded reporting quality using the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement, and coded study methods using a modified Newcastle-Ottawa Scale (m-NOS), the Medical Education Research Study Quality Instrument (MERSQI), and the Best Evidence in Medical Education (BEME) global scale.

RESULTS

For reporting quality, articles scored a mean±standard deviation (SD) of 51±25% of STROBE elements for the Introduction, 58±20% for the Methods, 50±18% for the Results and 41±26% for the Discussion sections. We found positive associations (all p<0.0001) between reporting quality and MERSQI (ρ=0.64), m-NOS (ρ=0.57) and BEME (ρ=0.58) scores. We explored associations between study methods and knowledge ES by subtracting each study's ES from the pooled ES for studies using that method and comparing these differences between subgroups. Effect sizes in single-group pretest/post-test studies differed from the pooled estimate more than ESs in two-group studies (p=0.013). No difference was found between other study methods (yes/no: representative sample, comparison group from same community, randomised, allocation concealed, participants blinded, assessor blinded, objective assessment, high follow-up).

CONCLUSIONS

Information is missing from all sections of reports of HPE experiments. Single-group pre-/post-test studies may overestimate ES compared with two-group designs. Other methodological variations did not bias study results in this sample.

Core journals that publish clinical trials of physical therapy interventions

OBJECTIVE

The objective of this study was to identify core journals in physical therapy by identifying those that publish the most randomized controlled trials of physical therapy interventions, provide the highest-quality reports of randomized controlled trials, and have the highest journal impact factors.

DESIGN

This study was an audit of a bibliographic database.

METHODS

All trials indexed in the Physiotherapy Evidence Database (PEDro) were analyzed. Journals that had published at least 80 trials were selected. The journals were ranked in 4 ways: number of trials published; mean total PEDro score of the trials published in the journal, regardless of publication year; mean total PEDro score of the trials published in the journal from 2000 to 2009; and 2008 journal impact factor.

RESULTS

The top 5 core journals in physical therapy, ranked by the total number of trials published, were Archives of Physical Medicine and Rehabilitation, Clinical Rehabilitation, Spine, British Medical Journal (BMJ), and Chest. When the mean total PEDro score was used as the ranking criterion, the top 5 journals were Journal of Physiotherapy, Journal of the American Medical Association (JAMA), Stroke, Spine, and Clinical Rehabilitation. When the mean total PEDro score of the trials published from 2000 to 2009 was used as the ranking criterion, the top 5 journals were Journal of Physiotherapy, JAMA, Lancet, BMJ, and Pain. The most highly ranked physical therapy-specific journals were Physical Therapy (ranked eighth on the basis of the number of trials published) and Journal of Physiotherapy (ranked first on the basis of the quality of trials). Finally, when the 2008 impact factor was used for ranking, the top 5 journals were JAMA, Lancet, BMJ, American Journal of Respiratory and Critical Care Medicine, and Thorax. There were no significant relationships among the rankings on the basis of trial quality, number of trials, or journal impact factor.

CONCLUSIONS

Physical therapists who are trying to keep up-to-date by reading the best available evidence on the effects of physical therapy interventions have to read more broadly than just physical therapy-specific journals. Readers of articles on physical therapy trials should be aware that high-quality trials are not necessarily published in journals with high impact factors.

Achieving high research reporting quality through the use of computational ontologies

Systematic reviews and meta-analyses constitute one of the central pillars of evidence-based medicine. However, clinical trials are poorly reported which delays meta-analyses and consequently the translation of clinical research findings to clinical practice. We propose a Center of Excellence in Research Reporting in Neurosurgery (CERR-N) and the creation of a clinically significant computational ontology to encode Randomized Controlled Trials (RCT) studies in neurosurgery. A 128 element strong computational ontology was derived from the Trial Bank ontology by omitting classes which were not required to perform meta-analysis. Three researchers from our team tagged five randomly selected RCT's each, published in the last 5 years (2004-2008), in the Journal of Neurosurgery (JoN), Neurosurgery Journal (NJ) and Journal of Neurotrauma (JoNT). We evaluated inter and intra observer reliability for the ontology using percent agreement and kappa coefficient. The inter-observer agreement was 76.4%, 75.97% and 74.9% and intra-observer agreement was 89.8%, 80.8% and 86.56% for JoN, NJ and JoNT respectively. The inter-observer kappa coefficient was 0.60, 0.54 and 0.53 and the intra-observer kappa coefficient was 0.79, 0.82 and 0.79 for JoN, NJ and JoNT journals respectively. The high degree of inter and intra-observer agreement confirms tagging consistency in sections of a given scientific manuscript. Standardizing reporting for neurosurgery articles can be reliably achieved through the integration of a computational ontology within the context of a CERR-N. This approach holds potential for the overall improvement in the quality of reporting of RCTs in neurosurgery, ultimately streamlining the translation of clinical research findings to improvement in patient care.

Computational ontologies in orthopaedic surgery

Information Technology (IT) plays an important role in storing and collating the vast amounts of healthcare data. However, analyzing and integrating this data to extract useful information is difficult due to the heterogeneous, siloed, disparate, and unstructured nature of the data. WHERE ARE WE NOW?: Attempts to standardize data reporting by establishing reporting standards, checklists and guidelines have not been optimal. Moreover, efforts to integrate data through the use of registries, data sharing networks, vocabularies and data standards have also yielded limited results. These efforts, when applied to orthopaedics, where theoretical knowledge is scattered over subspecialties, make it a cognitively challenging and tedious process. WHERE DO WE NEED TO GO?: Implementing data standardization is an important step towards homogenizing the data so that it can be integrated. Once integrated, the next step would be data analysis for information extraction. This information would be useful in answering important questions, especially in orthopaedic clinical practice and research, and could even help optimize methodologies in the education field. HOW DO WE GET THERE?: With the ability to describe concepts in a standardized manner and define existing interrelationships, ontologies are a potential solution. They assist in standardizing and integrating data and also impart strong inferential capabilities at a granular level. When applied to orthopaedics, they can standardize data collection, link data sources, generate knowledge based on the assumptions present in the interlinked data, thus answering important questions regarding orthopaedic clinical practice, research and education.

Hypnosis in the treatment of depression: considerations in research design and methods

Depressive disorders constitute a serious problem in the United States and around the world. The appearance of practice guidelines and lists of evidenced-based therapies suggests that adequate treatments for depression exist. However, a careful consideration of what is known and not known about the treatment of depression leaves plenty of room for improved approaches to addressing this condition. Although there has been a dearth of research on the treatment of depression using hypnosis, there are several compelling arguments for the inclusion of hypnotic approaches in the array of current strategies for dealing with depression. However, traditional "gold-standard" research methods, namely randomized controlled trials, have many shortcomings for identifying the potential impact of hypnosis on depression. Other strategies, notably single-case design and benchmarking approaches, may offer a more practical solution to the problem of determining "what works for depression."