1
|
Chauvin A, Ravaud P, Moher D, Schriger D, Hopewell S, Shanahan D, Alam S, Baron G, Regnaux JP, Crequit P, Martinez V, Riveros C, Le Cleach L, Recchioni A, Altman DG, Boutron I. Accuracy in detecting inadequate research reporting by early career peer reviewers using an online CONSORT-based peer-review tool (COBPeer) versus the usual peer-review process: a cross-sectional diagnostic study. BMC Med 2019; 17:205. [PMID: 31744489 PMCID: PMC6864983 DOI: 10.1186/s12916-019-1436-0] [Citation(s) in RCA: 65] [Impact Index Per Article: 10.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/07/2019] [Accepted: 09/27/2019] [Indexed: 12/30/2022] Open
Abstract
BACKGROUND The peer review process has been questioned as it may fail to allow the publication of high-quality articles. This study aimed to evaluate the accuracy in identifying inadequate reporting in RCT reports by early career researchers (ECRs) using an online CONSORT-based peer-review tool (COBPeer) versus the usual peer-review process. METHODS We performed a cross-sectional diagnostic study of 119 manuscripts, from BMC series medical journals, BMJ, BMJ Open, and Annals of Emergency Medicine reporting the results of two-arm parallel-group RCTs. One hundred and nineteen ECRs who had never reviewed an RCT manuscript were recruited from December 2017 to January 2018. Each ECR assessed one manuscript. To assess accuracy in identifying inadequate reporting, we used two tests: (1) ECRs assessing a manuscript using the COBPeer tool (after completing an online training module) and (2) the usual peer-review process. The reference standard was the assessment of the manuscript by two systematic reviewers. Inadequate reporting was defined as incomplete reporting or a switch in primary outcome and considered nine domains: the eight most important CONSORT domains and a switch in primary outcome(s). The primary outcome was the mean number of domains accurately classified (scale from 0 to 9). RESULTS The mean (SD) number of domains (0 to 9) accurately classified per manuscript was 6.39 (1.49) for ECRs using COBPeer versus 5.03 (1.84) for the journal's usual peer-review process, with a mean difference [95% CI] of 1.36 [0.88-1.84] (p < 0.001). Concerning secondary outcomes, the sensitivity of ECRs using COBPeer versus the usual peer-review process in detecting incompletely reported CONSORT items was 86% [95% CI 82-89] versus 20% [16-24] and in identifying a switch in primary outcome 61% [44-77] versus 11% [3-26]. The specificity of ECRs using COBPeer versus the usual process to detect incompletely reported CONSORT domains was 61% [57-65] versus 77% [74-81] and to identify a switch in primary outcome 77% [67-86] versus 98% [92-100]. CONCLUSIONS Trained ECRs using the COBPeer tool were more likely to detect inadequate reporting in RCTs than the usual peer review processes used by journals. Implementing a two-step peer-review process could help improve the quality of reporting. TRIAL REGISTRATION Clinical.Trials.gov NCT03119376 (Registered April, 18, 2017).
Collapse
|
Evaluation Study |
6 |
65 |
2
|
Wilson C, Rooshenas L, Paramasivan S, Elliott D, Jepson M, Strong S, Birtle A, Beard DJ, Halliday A, Hamdy FC, Lewis R, Metcalfe C, Rogers CA, Stein RC, Blazeby JM, Donovan JL. Development of a framework to improve the process of recruitment to randomised controlled trials (RCTs): the SEAR (Screened, Eligible, Approached, Randomised) framework. Trials 2018; 19:50. [PMID: 29351790 PMCID: PMC5775609 DOI: 10.1186/s13063-017-2413-6] [Citation(s) in RCA: 62] [Impact Index Per Article: 8.9] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/16/2017] [Accepted: 12/14/2017] [Indexed: 11/24/2022] Open
Abstract
BACKGROUND Research has shown that recruitment to trials is a process that stretches from identifying potentially eligible patients, through eligibility assessment, to obtaining informed consent. The length and complexity of this pathway means that many patients do not have the opportunity to consider participation. This article presents the development of a simple framework to document, understand and improve the process of trial recruitment. METHODS Eight RCTs integrated a QuinteT Recruitment Intervention (QRI) into the main trial, feasibility or pilot study. Part of the QRI required mapping the patient recruitment pathway using trial-specific screening and recruitment logs. A content analysis compared the logs to identify aspects of the recruitment pathway and process that were useful in monitoring and improving recruitment. Findings were synthesised to develop an optimised simple framework that can be used in a wide range of RCTs. RESULTS The eight trials recorded basic information about patients screened for trial participation and randomisation outcome. Three trials systematically recorded reasons why an individual was not enrolled in the trial, and further details why they were not eligible or approached, or declined randomisation. A framework to facilitate clearer recording of the recruitment process and reasons for non-participation was developed: SEAR - Screening, to identify potentially eligible trial participants; Eligibility, assessed against the trial protocol inclusion/exclusion criteria; Approach, the provision of oral and written information and invitation to participate in the trial, and Randomised or not, with the outcome of randomisation or treatment received. CONCLUSIONS The SEAR framework encourages the collection of information to identify recruitment obstacles and facilitate improvements to the recruitment process. SEAR can be adapted to monitor recruitment to most RCTs, but is likely to add most value in trials where recruitment problems are anticipated or evident. Further work to test it more widely is recommended.
Collapse
|
research-article |
7 |
62 |
3
|
Wright N, Ivers N, Eldridge S, Taljaard M, Bremner S. A review of the use of covariates in cluster randomized trials uncovers marked discrepancies between guidance and practice. J Clin Epidemiol 2015; 68:603-9. [PMID: 25648791 PMCID: PMC4425474 DOI: 10.1016/j.jclinepi.2014.12.006] [Citation(s) in RCA: 38] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/23/2014] [Revised: 12/12/2014] [Accepted: 12/23/2014] [Indexed: 11/24/2022]
Abstract
OBJECTIVES Reviews of the handling of covariates in trials have explicitly excluded cluster randomized trials (CRTs). In this study, we review the use of covariates in randomization, the reporting of covariates, and adjusted analyses in CRTs. STUDY DESIGN AND SETTING We reviewed a random sample of 300 CRTs published between 2000 and 2008 across 150 English language journals. RESULTS Fifty-eight percent of trials used covariates in randomization. Only 69 (23%) included tables of cluster- and individual-level covariates. Fifty-eight percent reported significance tests of baseline balance. Of 207 trials that reported baseline measures of the primary outcome, 155 (75%) subsequently adjusted for these in analyses. Of 174 trials that used covariates in randomization, 30 (17%) included an analysis adjusting for all those covariates. Of 219 trial reports that included an adjusted analysis of the primary outcome, only 71 (32%) reported that covariates were chosen a priori. CONCLUSION There are some marked discrepancies between practice and guidance on the use of covariates in the design, analysis, and reporting of CRTs. It is essential that researchers follow guidelines on the use and reporting of covariates in CRTs, promoting the validity of trial conclusions and quality of trial reports.
Collapse
|
Review |
10 |
38 |
4
|
Korevaar DA, Cohen JF, Hooft L, Bossuyt PMM. Literature survey of high-impact journals revealed reporting weaknesses in abstracts of diagnostic accuracy studies. J Clin Epidemiol 2015; 68:708-15. [PMID: 25703213 DOI: 10.1016/j.jclinepi.2015.01.014] [Citation(s) in RCA: 24] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/30/2014] [Revised: 12/17/2014] [Accepted: 01/21/2015] [Indexed: 11/28/2022]
Abstract
OBJECTIVES Informative journal abstracts are crucial for the identification and initial appraisal of studies. We aimed to evaluate the informativeness of abstracts of diagnostic accuracy studies. STUDY DESIGN AND SETTING PubMed was searched for reports of studies that had evaluated the diagnostic accuracy of a test against a clinical reference standard, published in 12 high-impact journals in 2012. Two reviewers independently evaluated the information contained in included abstracts using 21 items deemed important based on published guidance for adequate reporting and study quality assessment. RESULTS We included 103 abstracts. Crucial information on study population, setting, patient sampling, and blinding as well as confidence intervals around accuracy estimates were reported in <50% of the abstracts. The mean number of reported items per abstract was 10.1 of 21 (standard deviation 2.2). The mean number of reported items was significantly lower for multiple-gate (case-control type) studies, in reports in specialty journals, and for studies with smaller sample sizes and lower abstract word counts. No significant differences were found between studies evaluating different types of tests. CONCLUSION Many abstracts of diagnostic accuracy study reports in high-impact journals are insufficiently informative. Developing guidelines for such abstracts could help the transparency and completeness of reporting.
Collapse
|
Review |
10 |
24 |
5
|
Zhong YQ, Fu JJ, Liu XM, Diao X, Mao B, Fan T, Yang HM, Liu GJ, Zhang WB. The reporting quality, scientific rigor, and ethics of randomized placebo-controlled trials of traditional Chinese medicine compound formulations and the differences between Chinese and non-Chinese trials. CURRENT THERAPEUTIC RESEARCH 2014; 71:30-49. [PMID: 24683249 DOI: 10.1016/j.curtheres.2010.02.001] [Citation(s) in RCA: 20] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 01/04/2010] [Indexed: 02/05/2023]
Abstract
BACKGROUND An increasing number of randomized placebo-controlled trials involving traditional Chinese medicine (TCM) compound formulations have been implemented worldwide. OBJECTIVE The aim of this study was to assess the reporting quality, scientific rigor, and ethics of randomized placebo-controlled trials of TCM compound formulations and compare these differences between Chinese and non-Chinese trials. METHODS English-language databases included the following: PubMed, OVID, EMBASE, and Science Citation Index Expanded. Chinese-language databases included the following: Chinese Biomedical Literature Database, Wanfang Database, Chinese Scientific and Technological Periodical Database, and the China National Knowledge Infrastructure. All were searched from respective inception to March 2009 to identify randomized placebo-controlled trials involving TCM compound prescriptions. Two reviewers independently assessed the retrieved trials via a modified Consolidated Standard of Reporting Trials (CONSORT) checklist and some evaluation indices that embodied the TCM characteristics or the scientific rigor and ethics of placebo-controlled trials. Trial publishing time was divided into 3 intervals: phase 1 (≤1999); phase 2 (2000-2004); and phase 3 (2005-2009). The number and percentage of trials reporting each item and the corresponding differences between Chinese (mainland China, Hong Kong, and Taiwan) and non-Chinese (eg, Japan, United States, Australia, Korea, and United Kingdom) trials were calculated. Moreover, the influence of trial publishing time on the reporting of CONSORT items and the differences in the number of items reported for each time interval between Chinese and non-Chinese trials were assessed. RESULTS A total of 324 trials from China and 51 trials from other countries were included. A mean of 39.7% of the CONSORT items across all Chinese trials and 50.2% of the items across all non-Chinese trials were reported. The number of the reported CONSORT items all increased over time in both groups and the gap between Chinese articles and non-Chinese articles gradually decreased. Additionally, of the 324 Chinese articles, 137 (42.28%) reported TCM syndrome type, 113 (34.88%) reported the diagnostic criteria of diseases for TCM, and 69 (21.30%) reported efficacy evaluation indices of TCM. Of the non-Chinese articles, 3 (5.88%) reported TCM syndrome type and 1 (1.96%) reported the diagnostic criteria of diseases and evaluation indices of efficacy for TCM. It was found that 45.37% and 6.17% of Chinese articles reported the standard intervention for the diseases being treated and the emergency plan, respectively, compared with 23.53% and 9.80% for the non-Chinese articles; 33.02% and 10.49% of Chinese articles reported informed consent and ethics committee approval, respectively, compared with 92.16% and 82.35% for the non-Chinese articles. With regard to placebo ethics, 38.89% of the Chinese trials and 23.53% of the non-Chinese trials found it would not be ethically acceptable to use placebo alone in the control group. CONCLUSIONS The data indicate that the reporting quality of the included trials on TCM compounds has improved over time, but still remains poor regardless of Chinese or non-Chinese trials. Across all trials, particularly Chinese trials, the reporting of the CONSORT items was inadequate (39.7%). The difference in the mean number of the reported CONSORT items between Chinese trials and non-Chinese trials narrowed from phase 1 (10.0 vs 13.8) to phase 3 (14.4 vs 17.4). Moreover, a large number of trials, especially non-Chinese trials (94.1%), were lacking syndrome differentiation of TCM. More importantly, in many placebo-controlled trials, especially Chinese trials, the use of placebo was not justified and was ethically contradictory.
Collapse
|
Journal Article |
11 |
20 |
6
|
Baulig C, Krummenauer F, Geis B, Tulka S, Knippschild S. Reporting quality of randomised controlled trial abstracts on age-related macular degeneration health care: a cross-sectional quantification of the adherence to CONSORT abstract reporting recommendations. BMJ Open 2018; 8:e021912. [PMID: 29789352 PMCID: PMC5988143 DOI: 10.1136/bmjopen-2018-021912] [Citation(s) in RCA: 20] [Impact Index Per Article: 2.9] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/25/2018] [Revised: 03/12/2018] [Accepted: 04/18/2018] [Indexed: 12/20/2022] Open
Abstract
OBJECTIVE To assess the reporting quality of randomised controlled trial (RCT) abstracts on age-related macular degeneration (AMD) healthcare, to evaluate the adherence to the Consolidated Standards of Reporting Trials (CONSORT) statement's recommendations on minimum abstract information and to identify journal characteristics associated with abstract reporting quality. DESIGN Cross-sectional evaluation of RCT abstracts on AMD healthcare. METHODS A PubMed search was implemented to identify RCT abstracts on AMD healthcare published in the English language between January 2004 and December 2013. Data extraction was performed by two parallel readers independently by means of a documentation format in accordance with the 16 items of the CONSORT checklist for abstracts. The total number of criteria fulfilled by an abstract was derived as primary endpoint of the investigation; incidence rate ratios (IRRs) with unadjusted 95% CI were estimated by means of multiple Poisson regression to identify journal and article characteristics (publication year, multicentre design, structured abstract recommendations, effective sample size, effective abstract word counts and journal impact factor) possibly associated with the total number of fulfilled items. STUDY CHARACTERISTICS 136 of 673 identified abstracts (published in 36 different journals) fulfilled all eligibility criteria. RESULTS The median number of fulfilled items was 7 (95% CI 7 to 8). No abstract reported all 16 recommended items; the maximum total number was 14, the minimum 3 of 16 items. Multivariate analysis only demonstrated the abstracts' word counts as being significantly associated with a better reporting of abstracts (Poisson regression-based IRR 1.002, 95% CI 1.001 to 1.003). CONCLUSIONS Reporting quality of RCT abstracts on AMD investigations showed a considerable potential for improvement to meet the CONSORT abstract reporting recommendations. Furthermore, word counts of abstracts were identified as significantly associated with the overall abstract reporting quality.
Collapse
|
research-article |
7 |
20 |
7
|
Chhapola V, Tiwari S, Brar R, Kanwal SK. Reporting quality of trial abstracts-improved yet suboptimal: A systematic review and meta-analysis. J Evid Based Med 2018; 11:89-94. [PMID: 29460397 DOI: 10.1111/jebm.12294] [Citation(s) in RCA: 17] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/19/2017] [Accepted: 01/23/2018] [Indexed: 01/11/2023]
Abstract
OBJECTIVES We conducted a systematic review and meta-analysis of literature to determine if the publication of the Consolidated Standards of Reporting Trials (CONSORT) abstract guideline in 2008 was followed by change in reporting quality of randomized controlled trial (RCT) abstracts. STUDY DESIGN AND SETTINGS Evaluations were included if they compared reporting quality of RCT abstracts before and after the publication of CONSORT-abstract guideline. The literature search was performed (January 2008 to April 2017) in Medline (Ovid), EMbase, CINAHL plus and Cochrane methodologies register. We assessed study validity with a special validity tool, adapted from a previous Cochrane review. RESULTS Initial search identified 4142 articles, of which total 10 evaluations including 5184 abstracts were included. Total 22 outcomes related to individual items of CONSORT-abstract guideline were assessed, and 14 showed significant effect sizes favoring CONSORT-abstract guideline. Despite significant effect size, the overall post-CONSORT reporting (PCR) was suboptimal for ten items: title (RR = 1.40, 95% CI 1.23 to 1.59, PCR = 53.4%), participants (RR = 1.58, 95% CI 1.11 to 2.26, PCR = 24.5%), primary outcome (RR = 1.12, 95% CI 1.02 to 1.23, PCR = 65%), blinding (RR = 2.13, 95% CI 1.20 to 3.76, PCR = 13.9%), trial status (RR = 1.81, 95% 1.39 to 2.35, PCR = 10.6%), numbers analyzed (RR = 1.51, 95% CI 1.15 to 1.98, PCR = 26.5%), outcome (RR = 1.40, 95% 1.05 to 1.86, PCR = 21.9%), effect size and precision (RR = 1.59, 95% CI 1.15 to 2.19, PCR = 58.9%), harms (RR = 1.24, 95% CI 1.04 to 1.48, PCR = 41.8%), trial registration (RR = 2.02, 95% CI 1.63 to 2.50, PCR = 33.8%). Three items with favorable effect size in addition had wide CIs: randomization (RR = -4.28, 95% CI 1.56 to 11.75, PCR = -3.3%), allocation concealment (RR = -19.89, 95% CI 1.54 to 256.69, PCR = -5.7%), and funding (RR = -22.61, 95% CI 8.13 to 62.67, PCR = -11.32%). CONCLUSION The change in reporting quality of RCT abstracts is far from satisfactory, as evidenced by suboptimal post-CONSORT rates and wide CIs of effect sizes for majority of improved items. Mere publication of CONSORT-abstract guideline, without strict endorsement has failed to translate into good quality abstracts.
Collapse
|
Meta-Analysis |
7 |
17 |
8
|
An interrupted time series analysis showed suboptimal improvement in reporting quality of trial abstract. J Clin Epidemiol 2015; 71:11-7. [PMID: 26524495 DOI: 10.1016/j.jclinepi.2015.10.013] [Citation(s) in RCA: 14] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/23/2015] [Revised: 10/16/2015] [Accepted: 10/26/2015] [Indexed: 11/20/2022]
Abstract
OBJECTIVE To assess and compare the immediate and long-term change in reporting quality of randomized controlled trial (RCT) abstracts published in Pediatrics, The Journal of Pediatrics, and JAMA Pediatrics before and after the publication of Consolidated Standards of Reporting Trial (CONSORT)-abstract statement. STUDY DESIGN AND SETTINGS Study had "Interrupted time-series" design. Eligible RCT abstracts were retrieved by PubMed search in two study periods from January 2003 to December 2007 (pre-CONSORT) and January 2010 to December 2014 (post-CONSORT). These abstracts were matched with the CONSORT checklist for abstracts. The primary outcome measure was CONSORT-abstract score defined as number of CONSORT items correctly reported divided by 18 and expressed as percentage. The mean percentage scores were used to compare reporting quality between pre- and post-CONSORT using segmented linear regression. RESULTS A total of 424 RCT abstracts in pre-CONSORT and 467 in post-CONSORT were analyzed. A significant change in slope of regression line between two time periods (0.151 [confidence interval CI, 0.004-0.298], P = 0.044) was observed. Intercepts did not show a significant difference (-2.39 [CI, 4.93-0.157], P = 0.065). CONCLUSION The overall reporting quality of RCT abstracts in the high-impact pediatrics journals was suboptimal (<50%); however, it improved when assessed over a 5-year period, implying slow but gradual adoption of guideline.
Collapse
|
Journal Article |
10 |
14 |
9
|
Maggi CB, Griebeler IH, Dal Pizzol TDS. Information on adverse events in randomised clinical trials assessing drug interventions published in four medical journals with high impact factors. INTERNATIONAL JOURNAL OF RISK & SAFETY IN MEDICINE 2014; 26:9-22. [PMID: 24796347 DOI: 10.3233/jrs-140609] [Citation(s) in RCA: 13] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
BACKGROUND The impact of the extension of the Consolidated Standards of Reporting Trials (CONSORT) statement, which was published in 2004 and aimed to improve the quality of the safety information presented in clinical trials, remains uncertain. OBJECTIVE To assess the incorporation of the CONSORT statement extension's recommendations in randomised clinical trials (RCTs) evaluating drug therapies published in high-impact medical journals. METHODS Using Medline, 122 RCTs published in 2009 were selected from BMJ, JAMA, Lancet, and NEJM. A structured form was used to identify the harms information reported in the RCTs, following the recommendations of the CONSORT statement extension. RESULTS The most frequently met CONSORT recommendation was the mention of harms in the title or abstract of the paper (72.1% of the papers analysed); the least-met recommendation was the reporting of how the harms information was collected (10.7%). The studies that focused on harms presented better information on safety, but only 10.8% met all recommendations in the CONSORT statement. CONCLUSION The adverse event information was insufficient for the RCTs published in four high-impact medical journals five years after the publication of the extension of the CONSORT statement.
Collapse
|
Review |
11 |
13 |
10
|
Carlton DA, Kocherginsky M, Langerman AJ. A systematic review of the quality of randomized controlled trials in head and neck oncology surgery. Laryngoscope 2014; 125:146-52. [PMID: 24729155 DOI: 10.1002/lary.24718] [Citation(s) in RCA: 9] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/30/2013] [Revised: 03/29/2014] [Accepted: 04/08/2014] [Indexed: 11/11/2022]
Abstract
OBJECTIVES/HYPOTHESIS To determine the quality of randomized controlled trials (RCTs) in head and neck surgery in which surgery was a primary intervention. DATA SOURCES Potential articles were identified in PubMed without publication date restrictions. REVIEW METHODS Articles were scored using the CONSORT checklist and the relationship between the checklist score and whether the first and/or last authors were surgeons was investigated. Differences in the checklist score based on how many surgeons were among the first and last authors of the study were analyzed using the Kruskal-Wallis test. Fisher's exact test was used to examine if there was a significant difference of the reporting of individual items from the checklist between surgeons and nonsurgeons. A nonparametric trend test was used to determine whether there was a difference in the reporting of individual items based on whether there were none, one, or two surgeons among first and last authors. RESULTS A total of 38 publications satisfied the inclusion criteria. There was a trend toward lower quality for studies in which surgeons were either first, last, or both first and last authors compared to studies that were first-authored and last-authored by nonsurgeons (P = 0.068). Nonsurgeons were more likely to report on critical elements regarding hypothesis, sample size determination, randomization, and eligibility of centers (P = 0.023-0.058). CONCLUSION The quality of RCTs in head and neck surgery is poor. Improved training in conducting and reporting clinical research is needed in otolaryngology residencies.
Collapse
|
Systematic Review |
11 |
9 |
11
|
Tardy MP, Gal J, Chamorey E, Almairac F, Vandenbos F, Bondiau PY, Saada-Bouzid E. Quality of Randomized Controlled Trials Reporting in the Treatment of Adult Patients with High-Grade Gliomas. Oncologist 2017; 23:337-345. [PMID: 29133516 DOI: 10.1634/theoncologist.2017-0196] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/03/2017] [Accepted: 09/26/2017] [Indexed: 12/13/2022] Open
Abstract
BACKGROUND The randomized controlled trial (RCT) is the gold standard to objectively assess the effect of treatments. To help improve the quality of RCTs, experts established a list of recommendations, the CONsolidated Standards of Reporting Trials (CONSORT) Statement. In this study, we evaluated the implementation of the CONSORT Statement in the field of high-grade gliomas in adult patients and looked for criteria associated with higher quality of RCTs. MATERIALS AND METHODS We searched all high-grade gliomas RCTs published in PubMed between January 1990 and December 2016. The quality of these RCTs was assessed by completing a modified CONSORT Score (CS). RESULTS Ninety-six published RCTs were identified. The median CS was 19.5 on a scale of 0-33. Items were not equally reported. Items regarding the method of randomization or the blinding were reported in less than 25% of RCTs. However, the CS has constantly improved over the years. Before the implementation of the CONSORT Statement in 1996, the median CS was 13, whereas it was 17 for the period 1996-2004 and 22 after 2005. A higher CS was observed when RCTs were published in a journal with an impact factor above 10 (p < .001) or after 2010 (p = .001), when the primary outcome was clearly defined (p < .001) and for RCTs that enrolled more than 200 patients (p = .004). CONCLUSION Although there has been a steady improvement in the CS over the years in the field of high-grade gliomas, a major effort must be made in the reporting methods for randomization and blinding. IMPLICATIONS FOR PRACTICE This study showed that the quality of reporting of randomized control trials (RCTs) concerning the treatment of high-grade gliomas is poor. Factors associated with a better quality of reports were identified and should be incorporated into the design of future RCTs. When clinicians read the results of RCTs, they should be aware of the possible inadequate reporting from these trials and take it into account for the management of their patients. This study identifies how RCTs can be improved in their reporting but also in their design, in order to advance care for patients with high-grade gliomas in the future.
Collapse
|
Journal Article |
8 |
8 |
12
|
Tabatabaei-Malazy O, Shadman Z, Ejtahed HS, Atlasi R, Abdollahi M, Larijani B. Quality of reporting of randomized controlled trials of herbal medicines conducted in metabolic disorders in Middle East countries: A systematic review. Complement Ther Med 2018; 38:61-66. [PMID: 29857881 DOI: 10.1016/j.ctim.2018.04.004] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/28/2017] [Revised: 01/11/2018] [Indexed: 01/31/2023] Open
Abstract
INTRODUCTION Based on WHO recommendation for considering herbal medicine as an inexpensive appropriate method to treat metabolic disorders, conducting randomized controlled trials (RCTs) is increasing worldwide. Since poor quality RCTs can lead to wrong conclusion, we assessed the quality of reporting of herbal medicines' RCTs conducted in Middle East in a systematic review study. MATERIALS & METHODS All herbal medicines' RCTs in metabolic disorders (diabetes mellitus, metabolic syndrome, hyperlipidemia, obesity and osteoporosis) conducted in Middle East countries and published before January 2017 were included. To obtain all related studies PubMed, Scopus, Web of Science, Cochran library, and Embase web databases were searched. Exclusion criteria were animal studies, non-herbal medicines' RCTs, RCTs conducted in Type 1 diabetes, in children or pregnant women. We used Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklist for reporting study selection processes as well as Consolidated Standards of Reporting Trials (CONSORT) statement for assessment of quality of reporting. RESULTS Out of 5319 identified studies, 215 RCTs were included. The proportion of published RCTs in the topic increased significantly over the time (P < 0.001). The total mean ± SD score for 37 items of CONSORT checklist was 21.15 ± 4.27. Most of RCTs (60%) were not reported randomization in the title. Some important items were incompletely reported including trial registration (42.3%), sample size estimation (38.1%), randomization method (35.3%), generation of allocation (27.9%), and concealment of allocation (13.5%). CONCLUSIONS Our findings indicate that the quality of reporting of herbal medicines' RCTs in metabolic disorders has improved over time in Middle East, but remains suboptimal.
Collapse
|
Review |
7 |
8 |
13
|
Jones BG, Streeter AJ, Baker A, Moyeed R, Creanor S. Bayesian statistics in the design and analysis of cluster randomised controlled trials and their reporting quality: a methodological systematic review. Syst Rev 2021; 10:91. [PMID: 33789717 PMCID: PMC8015172 DOI: 10.1186/s13643-021-01637-1] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/14/2019] [Accepted: 03/11/2021] [Indexed: 11/18/2022] Open
Abstract
BACKGROUND In a cluster randomised controlled trial (CRCT), randomisation units are "clusters" such as schools or GP practices. This has methodological implications for study design and statistical analysis, since clustering often leads to correlation between observations which, if not accounted for, can lead to spurious conclusions of efficacy/effectiveness. Bayesian methodology offers a flexible, intuitive framework to deal with such issues, but its use within CRCT design and analysis appears limited. This review aims to explore and quantify the use of Bayesian methodology in the design and analysis of CRCTs, and appraise the quality of reporting against CONSORT guidelines. METHODS We sought to identify all reported/published CRCTs that incorporated Bayesian methodology and papers reporting development of new Bayesian methodology in this context, without restriction on publication date or location. We searched Medline and Embase and the Cochrane Central Register of Controlled Trials (CENTRAL). Reporting quality metrics according to the CONSORT extension for CRCTs were collected, as well as demographic data, type and nature of Bayesian methodology used, journal endorsement of CONSORT guidelines, and statistician involvement. RESULTS Twenty-seven publications were included, six from an additional hand search. Eleven (40.7%) were reports of CRCT results: seven (25.9%) were primary results papers and four (14.8%) reported secondary results. Thirteen papers (48.1%) reported Bayesian methodological developments, the remaining three (11.1%) compared different methods. Four (57.1%) of the primary results papers described the method of sample size calculation; none clearly accounted for clustering. Six (85.7%) clearly accounted for clustering in the analysis. All results papers reported use of Bayesian methods in the analysis but none in the design or sample size calculation. CONCLUSIONS The popularity of the CRCT design has increased rapidly in the last twenty years but this has not been mirrored by an uptake of Bayesian methodology in this context. Of studies using Bayesian methodology, there were some differences in reporting quality compared to CRCTs in general, but this study provided insufficient data to draw firm conclusions. There is an opportunity to further develop Bayesian methodology for the design and analysis of CRCTs in order to expand the accessibility, availability, and, ultimately, use of this approach.
Collapse
|
Systematic Review |
4 |
7 |
14
|
Navas-Carretero S, Martinez JA. Cause-effect relationships in nutritional intervention studies for health claims substantiation: guidance for trial design. Int J Food Sci Nutr 2016; 66 Suppl 1:S53-61. [PMID: 26241012 DOI: 10.3109/09637486.2015.1025720] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/24/2022]
Abstract
The growing worldwide interest on functional food research has been accompanied by increasing regulatory guidelines in this area, with the aim of ensuring that any claimed effect in foods, beyond their nutritional role, is based on scientific unequivocal evidence. In order to assess the cause-effect relationship between the regular consumption of a food or a food component and the beneficial outcome for the consumer, an appropriate study design is required. Previous knowledge and research on the specific claimed food or product may be an adequate basis for defining a hypothesis and accurate objectives. Other key factors to take into account are based on the outcomes studied, the length of the trial, sample size and type, as well as the transparency on reporting the results obtained. Based on the Consolidated Standards on Reporting Trials statement (CONSORT), together with the specific guidelines published by the European Food Safety Authority (EFSA) Panel on Dietetic Products, Nutrition and Allergies, the present article aims at summarizing key questions conducting to the most appropriate study design for solid health claim substantiation.
Collapse
|
Review |
9 |
6 |
15
|
Sargeant JM, Plishka M, Ruple A, Selmic LE, Totton SC, Vriezen ER. Quality of reporting of clinical trials in dogs and cats: An update. J Vet Intern Med 2021; 35:1957-1971. [PMID: 34184331 PMCID: PMC8295703 DOI: 10.1111/jvim.16204] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/17/2021] [Revised: 06/08/2021] [Accepted: 06/16/2021] [Indexed: 12/15/2022] Open
Abstract
BACKGROUND Comprehensive reporting of clinical trials is essential to allow the trial reader to evaluate the methodological rigor of the trial and interpret the results. Since publication of the updated Consolidated Standards of Reporting Trials (CONSORT) guidelines for reporting of parallel clinical trials in humans, extensions for reporting of abstracts and crossover trials have been published. OBJECTIVES To describe the types of trials using dogs and cats published from 2015 to 2020 and to evaluate the quality of reporting of a sample of recently published parallel and crossover trials. ANIMALS None. METHODS A comprehensive search was conducted to identify parallel or crossover design clinical trials using dogs and cats published from January 1, 2015 onwards. Quality of reporting was evaluated on a subset of trials published during 2019. The reporting of items recommended in the CONSORT reporting guidelines for abstracts, parallel trials, and crossover trials was evaluated independently by 2 reviewers using standardized forms created for this study. Disagreements among reviewers were resolved by consensus. Results were tabulated descriptively. RESULTS The frequency of reporting of trial features varied from low to high. There remain deficiencies in the quality of reporting of key methodological features and information needed to evaluate and interpret trial results. CONCLUSIONS AND CLINICAL IMPORTANCE There is still a need for authors, peer-reviewers, and editors to follow reporting guidelines such as CONSORT to maximize the value of clinical trials and to increase confidence in the validity of the trial results.
Collapse
|
Journal Article |
4 |
6 |
16
|
Thoma A, Chew RT, Sprague S, Veltri K. Application of the CONSORT statement to randomized controlled trials comparing endoscopic and open carpal tunnel release. THE CANADIAN JOURNAL OF PLASTIC SURGERY = JOURNAL CANADIEN DE CHIRURGIE PLASTIQUE 2011; 14:205-10. [PMID: 19554136 DOI: 10.1177/229255030601400401] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/31/2022]
Abstract
BACKGROUND The CONSORT (Consolidated Standards of Reporting Trials) statement was developed by a group of clinical trialists, biostatisticians, epidemiologists and biomedical editors as a means to improve the quality of reports of randomized controlled trials (RCTs). The purpose of the present study is to assess the reporting quality of published RCTs that compare endoscopic carpal tunnel release (ECTR) with open carpal tunnel release (OCTR) using the CONSORT statement. METHODS A computerized literature search was conducted to identify all RCTs published from January 1989 to November 2004 that compared ECTR with OCTR. Foreign language studies were also included, and translated versions of these studies were obtained. Two investigators independently reviewed each eligible article and determined whether the authors reported on each of the 22 items of the CONSORT statement. Disagreements were resolved by consensus. The mean scores for studies published before the introduction of the CONSORT statement and those published afterward were compared. Similarly, a comparison was made between foreign language studies and those published in English. RESULTS Eighteen RCTs comparing ECTR with OCTR met the inclusion criteria. The total scores on the CONSORT checklist ranged from 3 to 20, with a mean score of 9.83+/-3.79 (the maximum possible score was 22). The six studies published in foreign language journals had a statistically significantly lower mean score than the studies published in English language journals (7.00+/-2.76 versus 11.25+/-3.49, respectively; P<0.05). The mean score was higher for studies published after 1996 than for those published in 1996 or earlier (12.14+/-3.80 versus 8.36+/-3.11, respectively; P<0.05). CONCLUSIONS The quality of reporting improved over time, but no study met all 22 criteria of the CONSORT statement. The CONSORT scores were higher for studies published after 1996 and for studies published in English language journals. Despite the improvement after 1996, most of these RCTs only reported one-half of the items listed on the CONSORT statement. Future investigators of surgical RCTs should make an effort to comply with the CONSORT checklist.
Collapse
|
Journal Article |
14 |
5 |
17
|
An assessment of the compliance of Randomised controlled trials published in craniofacial surgery journals with the CONSORT statement: A systematic review protocol. Int J Surg Protoc 2017; 5:1-4. [PMID: 31851730 PMCID: PMC6913545 DOI: 10.1016/j.isjp.2017.06.001] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/15/2017] [Revised: 05/29/2017] [Accepted: 06/02/2017] [Indexed: 11/20/2022] Open
Abstract
Introduction The role of clinical trials in medicine is expanding, particularly in surgery. Randomised controlled trials (RCTs) represent the gold standard evidence for high-quality assessment of healthcare interventions. The Consolidated Standards of Reporting Trials (CONSORT) guidance has been published to maximise RCT reporting transparency. This paper outlines the study protocol for a systematic review that will assess the current compliance of RCTs published within craniofacial surgery with the CONSORT criteria. The aims are to identify areas where reporting can be improved to ensure craniofacial surgery is guided by high-quality evidence. Methods and analysis This protocol is compliant with the Preferred Reporting Items for Systematic Review and meta-Analysis protocols (PRISMA-P) guidelines. Craniofacial surgery RCTs will be identified by searching within craniofacial surgery journals. Five journals from the Thomson Reuters Impact Factor Report 2016 included 'cranio' in their title and were included. MEDLINE PubMed will be used to search all RCTs published in these journals. The search strategy is described within this protocol. It will be limited to articles written in English, conducted on humans, and published in the last five years. Two independent researchers will assess each study for inclusion and will perform the data extraction. The researchers will assess compliance of each RCT with the 25-item CONSORT Statement checklist as the primary outcome. Discrepancies will be resolved through consensus or third author arbitration. Secondary outcomes to be extracted include the pathology and interventions examined, and indices of RCT quality. The systematic review will be compliant with PRISMA guidelines. The review has been registered a priori with the Registry of Systematic Reviews/meta-analyses (UIN: reviewregistry219). Ethics and dissemination This systematic review will be conducted in line with the Cochrane Handbook for Systematic Reviews and Interventions. The intent is to publish in a peer-reviewed journal and present the data at relevant conferences.
Collapse
|
Journal Article |
8 |
4 |
18
|
Du J, Zhang Y, Dong Y, Duan J, Bai H, Wang J, Xu J, Wang Z. Reporting quality of randomized, controlled trials evaluating immunotherapy in lung cancer. Thorac Cancer 2021; 12:2732-2739. [PMID: 34432361 PMCID: PMC8520800 DOI: 10.1111/1759-7714.14114] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/17/2021] [Revised: 08/01/2021] [Accepted: 08/02/2021] [Indexed: 12/16/2022] Open
Abstract
BACKGROUND With the improvement of therapeutic strategies from cytotoxic chemotherapy to immunotherapy, the possibility of achieving timely intervention for lung cancer has dramatically increased. This study aimed to systematically evaluate the reporting quality of randomized controlled trials (RCT) on immunotherapy in lung cancer. METHODS The RCTs evaluating the efficacy of immunotherapy in lung cancer published up to 2021 were searched and collected from PUBMED and EMBASE by two investigators. The 2010 Consolidated Standards for Test Reports (CONSORT) statement-based 28-point overall quality score (OQS) and the 2001 CONSORT statement-based 19-point OQS was utilized for assessing the overall quality of each report. RESULTS One hundred and fifty-two related RCTs were retrieved in this study, including 81,931 patients. The average OQS in 2010 was 17.89 (range, 7.5-24.5). Overall, studies have sufficiently reported the eligibility criteria (143/152; 94.07%), described the scientific background (150/152; 98.7%) and discussed interventions (147/152; 96.7%). However, the RCTs did not consistently report the changes to trial after commencement (48/152; 31.6%), allocation, enrollment and assignment personnel (34/152; 22.4%), blinding (48/152; 31.6%), or randomization method (58/152; 38.2%). CONCLUSIONS The overall reporting quality of RCTs on immunotherapy in lung cancer was found to be unsatisfactory despite the fact that the CONSORT statement was issued more than a decade ago. Furthermore, there was virtual selectivity and heterogeneity in reporting some key issues in these trials. This is the first study to enlighten lung cancer researchers especially focusing on immunotherapy, and also to remind editors and peer reviewers to strengthen their due diligence.
Collapse
|
|
4 |
3 |
19
|
Huang D, Jin X, Gao J, Li Y, Lu L, Sun F, Chen D, Zhao W, Luo W, Li H, Hu Y, Hu F. Quality evaluation of randomized controlled trials reports of laparoscopy compared with open colorectal resection for colorectal cancer. Expert Rev Anticancer Ther 2015; 15:727-32. [PMID: 26004141 DOI: 10.1586/14737140.2015.1043896] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/23/2023]
Abstract
OBJECTIVES Previously, there were no data looking at the quality evaluation of randomized controlled trials (RCTs) on effect comparison of laparoscopic surgery and open surgery for colorectal cancer in China. Here, we evaluate the completeness and transparency of RCT reports in this field. METHODS The following databases were searched: Medline, EMbase, SCI Expanded, China National Knowledge Infrastructure, the Chinese Biological Medicine Database, VIP database and Wan Fang databases) to search RCT reports on the effect comparison of laparoscopic surgery and open surgery for colorectal cancer in China. Our study evaluated the reporting quality of RCTs based on 22 standards of Consolidated Standards for Reporting Trials (CONSORT) 2010 Statement. Two reviewers responded with 'yes' or 'no' to each standard to judge whether the authors had reported or had recorded concrete details of the reports accomplished in accordance with the requirement of each standard. RESULTS A total of 40 relevant RCTs were included in the final analysis. For the 'Title and abstract', only three articles (7.5%) could be identified directly from its title as the report of RCTs. For the 'Methods', only three articles (7.5%) applied the method of random allocation of sequences; only two articles (5%) mentioned the type of randomization or gave the description of the mechanism of allocation concealment; no article referred the concrete implementation of random method. Only one article (2.5%) applied the method of blinding or sample size calculation; no article had analysis about the metaphase of an experiment or an explanation of its interruption. For 'results', only one article (2.5%) described participant flow, primary and secondary outcomes with estimated effect size or ancillary analyses. Only 13 articles (32.5%) showed baseline demographic and clinical characteristics, 10 (25%) referred to intention-to-treat analysis, and 12 (30%) mentioned important harms or unintended effects. For the 'discussion', only eight articles (20%) gave the description of trials' limitations and 13 (32.5%) presented the generalizability (external validity, applicability) of the trial findings. CONCLUSION The quality of the RCT reports on effect comparison of laparoscopic surgery and open surgery for colorectal cancer in China is poor. The reporting of RCTs in this field should be standardized, according to the specifications of the CONSORT 2010.
Collapse
|
Review |
10 |
2 |
20
|
Lombard N, Gasmi A, Sulpice L, Boudjema K, Naudet F, Bergeat D. Research transparency promotion by surgical journals publishing randomised controlled trials: a survey. Trials 2020; 21:824. [PMID: 33004055 PMCID: PMC7528240 DOI: 10.1186/s13063-020-04756-7] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/27/2019] [Accepted: 09/22/2020] [Indexed: 12/01/2022] Open
Abstract
Objective To describe surgical journals’ position statements on data-sharing policies (primary objective) and to describe key features of their research transparency promotion. Methods Only “SURGICAL” journals with an impact factor higher than 2 (Web of Science) were eligible for the study. They were included, if there were explicit instructions for clinical trial publication in the official instructions for authors (OIA) or if they had published randomised controlled trial (RCT) between 1 January 2016 and 31 December 2018. The primary outcome was the existence of a data-sharing policy included in the instructions for authors. Data-sharing policies were grouped into 3 categories, inclusion of data-sharing policy mandatory, optional, or not available. Details on research transparency promotion were also collected, namely the existence of a “prospective registration of clinical trials requirement policy”, a conflict of interests (COIs) disclosure requirement, and a specific reference to reporting guidelines, such as CONSORT for RCT. Results Among the 87 surgical journals identified, 82 were included in the study: 67 (82%) had explicit instructions for RCT and the remaining 15 (18%) had published at least one RCT. The median impact factor was 2.98 [IQR = 2.48–3.77], and in 2016 and 2017, the journals published a median of 11.5 RCT [IQR = 5–20.75]. The OIA of four journals (5%) stated that the inclusion of a data-sharing statement was mandatory, optional in 45% (n = 37), and not included in 50% (n = 41). No association was found between journal characteristics and the existence of data-sharing policies (mandatory or optional). A “prospective registration of clinical trials requirement” was associated with International Committee of Medical Journal Editors (ICMJE) allusion or affiliation and higher impact factors. Journals with specific RCT instructions in their OIA and journals referenced on the ICMJE website more frequently mandated the use of CONSORT guidelines. Conclusion Research transparency promotion is still limited in surgical journals. Standardisation of journal requirements according to ICMJE guidelines could be a first step forward for research transparency promotion in surgery.
Collapse
|
|
5 |
2 |
21
|
Jauch K, Kowark A, Coburn M, Clusmann H, Höllig A. Randomized Controlled Trials on Intracerebral Hemorrhage: A Cross Sectional Retrospective Analysis of CONSORT Item Adherence. Front Neurol 2019; 10:991. [PMID: 31616358 PMCID: PMC6763943 DOI: 10.3389/fneur.2019.00991] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/11/2019] [Accepted: 09/02/2019] [Indexed: 01/23/2023] Open
Abstract
Object: Intracranial hemorrhage (ICH) is the second most common cause of stroke but still there is little consolidated knowledge about the optimal treatment strategies (e.g., the benefit of surgical evacuation). We evaluated the current randomized controlled trials (RCTs) on primary ICH (01.2013–03.2017) according to their fulfillment of the CONSORT statement's criteria (published in 2010) –as a marker of transparency and quality of study planning and realization. Methods: A Pubmed and a Cochrane database (including clinicaltrials.gov) search was carried out (01.2014–3.2017, respectively 01.2013–12.2013). Abstracts were screened for inclusion. Eligible full text manuscripts were assessed for the implementation of the CONSORT criteria. Citation frequencies and impact factors of the journals were related to ratio of CONSORT criteria fulfillment. Further, the risk of bias according to the Risk of bias tool 2 (RoB 2) was assessed. Results: Overall 3097 abstracts were screened for inclusion; 39 studies were suitable for final analysis. A mean fulfillment ratio of 51% (±28%) was found. A high correlation between impact factor and adherence to CONSORT criteria was shown (r = 0.7664; p < 0.0001). Citation frequency per year was related to ratio of CONSORT item fulfillment (r = 0.6747; p < 0.0001) and to the impact factor of the publishing journal (r = 0.7310; p < 0.0001). Of note, the items 10 (randomization: implementation) and 21 (generalizability) showed particularly high rates of non-fulfillment (87 and 85%). The majority of studies (95%) complied with item 2b (specific objectives or hypotheses), but strikingly objectives were mostly described vaguely. Other essential criteria such as sample size determination, definition of outcome parameters, and participant flow were only fulfilled weakly (51, 54, and 39%). Conclusions: Over 20 years after its inception there is still weak adherence to the CONSORT statement. As a consequence, conclusions are hampered by inadequate planning and/or reporting. Particularly with respect to pathologies as ICH lacking clear, evidence-based guidelines adherence to the CONSORT statement might improve research quality in order to define valuable treatment strategies.
Collapse
|
Journal Article |
6 |
2 |
22
|
[Influence of impact factor on reporting sample size calculations in publications on studies exemplified by AMD treatment : Cross-sectional investigation on the presence of sample size calculations in publications of RCTs on AMD treatment in journals with low and high impact factors]. Ophthalmologe 2020; 117:125-131. [PMID: 31201561 DOI: 10.1007/s00347-019-0924-0] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/26/2022]
Abstract
BACKGROUND For scientific and ethical reasons randomized controlled clinical trials (RCTs) should be based on a sample size calculation. The CONSORT statement, an established publication guideline for transparent study reporting, requires a sample size calculation in every study publication. OBJECTIVE The availability of sample size calculations in RCT publications on treatment of age-related macular degeneration (AMD) was investigated. The primary hypothesis of this investigation compared the prevalence of reported sample size calculations between journals with higher (≥5) versus lower (<5) impact factors (IF). MATERIAL AND METHODS It was examined whether information on sample size calculation was available in a series of 97 publications of RTCs on AMD treatment published between 2004 and 2014. RESULTS Only 46 out of 97 (47%) study publications provided information on the reason for the number of patients enrolled. The comparison of publications from journals with an IF ≥ 5 (63%, 30) and from journals with an IF < 5 (40%, 67) showed a statistically significant difference of 23% in the frequencies of available sample size calculations (95% confidence interval, CI 2%; 44%). Of the publications published before 2010, 43% reported a sample size calculation versus 51% of the publications afterwards. CONCLUSION Publications in journals with higher IF more frequently reported a sample size calculation. More than 50% of the publications did not report any sample size calculation. Authors and reviewers of publications should pay more attention to the explicit reporting of sample size calculations.
Collapse
|
Journal Article |
5 |
1 |
23
|
Zhang X, Zhang G, Yuan Y, Zhang Y. Study on the Reporting Quality of Randomized Controlled Trials of Flipped Classroom in Medical Education. Clin Anat 2022; 35:592-597. [PMID: 35411588 DOI: 10.1002/ca.23871] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/13/2022] [Accepted: 03/31/2022] [Indexed: 11/06/2022]
Abstract
BACKGROUND With the deepening of medical education reform, the flipped classroom has become a commonly used teaching method. Many randomized controlled trials (RCTs) have shown that flipped classroom combined with problem-based learning (PBL) improves many students' abilities. It remains unclear whether the design and reports of these RCTs comply with specifications. The aim of this study was to provide a systematic assessment of the quality of RCTs investigating the effectiveness of flipped classroom teaching in medical education. METHOD Two investigators searched PUBMED and EMBASE and analyzed 15 filtered RCTs addressing flipped classroom in medical education, published between January 1, 2016 and December 31, 2021. The overall reporting quality was assessed using the overall quality score (OQS) according to the 2010 CONSORT statement. RESULTS The search revealed 15 RCTs closely related to flipped classrooms. After analysis, it appeared that least half of them made no mention of half of the OQS items in the CONSORT 2010 statement. Since the flipped classroom is an emerging teaching method, all the retrieved articles were published after 2016. Furthermore, a number of issues that we considered very important were not adequately addressed in the RCT reports. DISCUSSION Although the CONSORT 2010 statement was issued 12 years ago, the quality and content of many RCTs are not satisfactory. Our findings suggest that authors need to make more effort to meet the CONSORT statement requirements. This article is protected by copyright. All rights reserved.
Collapse
|
|
3 |
|
24
|
Zhang G, Kuang S, Zhang X. Assessing the reporting quality of randomized controlled trials on COVID-19 vaccines: a systematic review. Hum Vaccin Immunother 2022; 18:2031453. [PMID: 35176960 PMCID: PMC8993078 DOI: 10.1080/21645515.2022.2031453] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/04/2022] Open
Abstract
This systematic review evaluated the reporting quality of COVID-19 vaccine randomized controlled trials (RCTs). Relevant RCTs published between July 20, 2020 and June 11, 2021 were identified in the PubMed database by two independent reviewers. Study quality was evaluated with the 2010 AND 2001 Consolidated Standards of Reporting Trials (CONSORT) adherence scores. A total of 22 RCTs were included. The median CONSORT adherence score according to the 2010 criteria was 21 (range, 12-25), thus indicating that 75% of the items in more than half of the RCTs had clear reports. Univariate analysis showed that CONSORT adherence scores were not predicted by category; analysis of variance also showed no significant difference between groups. Our results indicated that the overall quality of COVID-19 vaccine RCTs was very good. Current evidence indicates that a variety of COVID-19 vaccines are effective. No RCTs have reported serious adverse effects such as mortality.
Collapse
|
|
3 |
|
25
|
Xiu WC, Meng X, Hu XY, Shi LJ, Gang WJ, Jing XH. [Evaluation of the report quality of Chinese and English randomized controlled trials of acupuncture based on CONSORT statement and STRICTA checklist]. ZHONGGUO ZHEN JIU = CHINESE ACUPUNCTURE & MOXIBUSTION 2023; 43:355-61. [PMID: 36858402 DOI: 10.13703/j.0255-2930.20220328-k0005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Subscribe] [Scholar Register] [Indexed: 03/03/2023]
Abstract
OBJECTIVE To evaluate the report quality of Chinese and English randomized controlled trials (RCTs) of acupuncture based on the CONSORT statement and STRICTA checklist. METHODS The Chinese and English RCTs of acupuncture published from January 1, 2015 to December 31, 2019 were searched in 7 databases including PubMed. The report quality of the included RCTs was evaluated with the CONSORT 2010 statement and STRICTA checklist. RESULTS A total of 506 Chinese RCTs and 76 English RCTs were included. According to the CONSORT statement, in Chinese RCTs, the items with report rate less than 50% accounted for 78.38% of all items, and the report rate of 25 items, such as background and reason, study design, outcome index, and sample size, was less than 10%. In English RCTs, the items with report rate less than 50% accounted for 35.14% of all items, and 5 items had a report rate of less than 10%. The difference of the report rate of 15 items, such as background, reason and study design, was more than 50% between Chinese and English RCTs. The report rate of all items of STRICTA checklist was relatively high in both Chinese and English RCTs. In Chinese RCTs, the items with report rate less than 50% accounted for 29.41% of all items, which included acupuncture rationale and depth of insertion. In English RCTs, only two items had a report rate less than 50%, which were acupuncture rationale, setting and context of treatment. The report rate of five items, including needle retention time, frequency and duration of treatment sessions, details of other interventions administered to the acupuncture group, setting and context of treatment, and precise description of the control or comparator in Chinese RCTs, were higher than in English RCTs. CONCLUSION The report quality of Chinese acupuncture RCT needs to be improved urgently, and corresponding measures should be taken to further standardize the writing and reporting of acupuncture clinical research.
Collapse
|
English Abstract |
2 |
|