Secular changes in the quality of published randomized clinical trials in rheumatology

Reproducibility and rigor in rheumatology research

The pillars of scientific progress in rheumatology are experimentation and observation, followed by the publication of reliable and credible results. These data must then be independently verified, validated, and replicated. Peer and journal-specific technical and statistical reviews are paramount to improving rigor and reproducibility. In addition, research integrity, ethics, and responsible conduct training can help to reduce research misconduct and improve scientific evidence. As the number of published articles in rheumatology grows, the field has become critical for determining reproducibility. Prospective, longitudinal, randomized controlled clinical trials are the gold standard for evaluating clinical intervention efficacy and safety in this space. However, their applicability to larger, more representative patient populations with rheumatological disorders worldwide could be limited due to time, technical, and cost constraints involved with large-scale clinical trials. Accordingly, analysis of real-world, patient-centered clinical data retrieved from established healthcare inventories, such as electronic health records, medical billing reports, and disease registries, are increasingly used to report patient outcomes. Unfortunately, it is unknown whether this clinical research paradigm in rheumatology could be deployed in medically underserved regions.

Adherence to the OARSI recommendations for designing, conducting, and reporting of clinical trials in knee osteoarthritis: a targeted literature review

Background

The Osteoarthritis Research Society International (OARSI) updated their guideline for clinical trials on knee osteoarthritis (KOA) in 2015, which contains recommendations for the conduct, design, and reporting of clinical trials. The purpose of this study was to assess the quality of clinical trials published between 2010 and 2020 investigating intra-articular interventions in patients with KOA using the OARSI recommendations.

Methods

A targeted literature review was conducted to identify randomized controlled trials in patients with KOA receiving intra-articular interventions, published between 2010 and 2020. Included studies were assessed using the OARSI recommendations. For a comparison between the time periods before and after the introduction of the new OARSI recommendations, the year 2016 was selected as the cut-off.

Results

One hundred forty-eight publications, representing 139 unique trials, were included in this review. Included studies adhered to between 9 and 24 recommendations (median: 19). The highest increase in adherence from studies published in 2016 or earlier compared to after 2016 was seen in the reporting and registration of trials and the use of structural outcome measures. Overall, adherence to the recommendations related to the collection of biochemical biomarkers and the use of structural outcome measures remained low.

Conclusion

An improvement can be made in the conduct, design, and reporting of clinical trials for intra-articular therapies in KOA. Despite proper guidelines, quality of clinical trials varies, and the methodological deficiencies found are preventable and can be corrected. The quality of research should be considered when making treatment decisions for patients with KOA in clinical practice.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12891-022-05116-z.

The Quality of Randomized Controlled Trials in High-impact Rheumatology Journals, 1998-2018

OBJECTIVE

Well-designed randomized controlled trials (RCT) mitigate bias and confounding, but previous evaluations of rheumatology trials found high rates of methodological flaws. Outside of rheumatoid arthritis, no studies in the modern era have assessed the quality of rheumatology RCT over time or regarding industry funding.

METHODS

We identified all RCT published in 3 high-impact rheumatology journals from 1998, 2008, and 2018. Quality metrics derived from a modified Jadad scale were analyzed by year of publication and by funding source.

RESULTS

Ninety-six publications met inclusion criteria; 82 of these described the primary analysis of an RCT. Over time (1998-2008-2018), trials were less likely to adequately report dropouts and withdrawals (100% vs 82% vs 60%; p < 0.01) or include an active comparator (44% vs 12% vs 13%; p = 0.01). Later trials were more likely to evaluate biologic therapy (11% vs 38% vs 83%; p < 0.01) and report adequate randomization procedures (39% vs 29% vs 60%; p = 0.04). Seventy-nine percent of trials received industry funding. Industry-funded trials were more likely to report double-blinding (86% vs 53%; p < 0.01), patient-reported outcome measures (77% vs 41%; p < 0.01), and intention-to-treat analyses (86% vs 65%; p = 0.04).

CONCLUSION

Industry-funded trials comprise the majority of RCT published in high-impact rheumatology journals and more frequently report metrics associated with RCT quality. RCT assessing active comparators and nonbiologic therapies have become less common in high-impact rheumatology journals.

Ethics and Industry Interactions: Impact on Specialty Training, Clinical Practice, and Research

Physicians in training and their mentors must be cognizant of ethical concerns related to industry interactions. Mentors perceived to have conflicts of interest or to be engaging in misconduct can unconsciously and profoundly affect the learning and academic environment by implying certain values and expectations. Despite increased awareness of ethical concerns related to industry interactions in clinical practice and research, there remains a need for interventions to prevent ethical transgressions. Ethics education is essential and a move in the right direction, but it alone is likely inadequate in preventing unethical behavior. Education should be supplemented with ethical environments at institutions.

Questioning a publication bias between industry-funded and non-industry-funded randomized controlled trials on biological and small molecule therapy for rheumatoid arthritis

BACKGROUND

There has been a significant increase in financial support of clinical research by the pharmaceutical industry.

METHODS

We performed a comprehensive systematic literature review to determine whether there is publication bias for rheumatoid arthritis (RA) studies between industry-funded and non-industry funded randomized controlled trials (RCTs), and between RCTs with positive results (PRs) and those with negative results (NRs) of FDAapproved biological and small molecule drug therapy for RA. Each RCT was classified as having either a PR or a NR, and as having received commercial funding or not.

RESULTS

Most (297/349, 85.18%) of the RCTs were commercially funded. There was no significant difference in PRs or association with publication between commercially and noncommercially funded RCTs. Sample size was significantly larger in commercially funded RCTs and in those with PRs, and it was the only significant parameter that predicted publication in higher impact factor journals in the field of RA.

CONCLUSION

There is no significant association between commercial funding and the publication of positive results or the publication of an RCT in higher impact factor journals.

Scoping review on interventions to improve adherence to reporting guidelines in health research

OBJECTIVES

The goal of this study is to identify, analyse and classify interventions to improve adherence to reporting guidelines in order to obtain a wide picture of how the problem of enhancing the completeness of reporting of biomedical literature has been tackled so far.

DESIGN

Scoping review.

SEARCH STRATEGY

We searched the MEDLINE, EMBASE and Cochrane Library databases and conducted a grey literature search for (1) studies evaluating interventions to improve adherence to reporting guidelines in health research and (2) other types of references describing interventions that have been performed or suggested but never evaluated. The characteristics and effect of the evaluated interventions were analysed. Moreover, we explored the rationale of the interventions identified and determined the existing gaps in research on the evaluation of interventions to improve adherence to reporting guidelines.

RESULTS

109 references containing 31 interventions (11 evaluated) were included. These were grouped into five categories: (1) training on the use of reporting guidelines, (2) improving understanding, (3) encouraging adherence, (4) checking adherence and providing feedback, and (5) involvement of experts. Additionally, we identified lack of evaluated interventions (1) on training on the use of reporting guidelines and improving their understanding, (2) at early stages of research and (3) after the final acceptance of the manuscript.

CONCLUSIONS

This scoping review identified a wide range of strategies to improve adherence to reporting guidelines that can be taken by different stakeholders. Additional research is needed to assess the effectiveness of many of these interventions.

Critical appraisal of RCTs by 3rd year undergraduates after short courses in EBM compared to expert appraisal

Introduction: An essential aim of courses in evidence-based medicine (EBM) is to improve the skills for reading and interpreting medical literature adequately. Regarding the conceptual framework, it is important to consider different educational levels. Aim: Our primary aim was to investigate the applicability of different instruments for the assessment of methodological study quality by 3rd grade students after short courses in EBM. Our secondary outcomes were agreement with expert assessments and student's knowledge and competences. Methods: We conducted four short courses in EBM of 90 minutes each for health care management and medical students focused on critical appraisal of the literature. At the end, the students assessed five publications about randomized controlled trials (RCTs) using five different instruments; the results were compared to expert assessments. Results: In total, 167 students participated in our EBM courses. Students' assessments showed a non-systematic over- and underestimation of risk of bias compared to expert assessments with no clear direction. Agreement with expert assessments ranged between 66% to over 80%. Across RCTs, evidence was found that the choice of instrument had an impact on agreement rates between expert and student assessments (p=0.0158). Three RCTs showed an influence of the instrument on the agreement rate (p<0.05 each). Discussion: Our results contrast sharply with those of many other comparable evaluations. Reasons may be a lack of students' motivation due to the compulsory courses, and the comparison to a reference standard in addition to self-ratings causing objectivity. Conclusion: Undergraduates should become familiar with the principles of EBM, including research methods, and the reading of scientific papers as soon as possible. For a deeper understanding, clinical experience seems to be an indispensable precondition. Based on our results, we would recommend an integration of lectures about EBM and critical appraisal at least twice during studies and with greater intensity shortly before graduation.

Quality of reporting of interventional animal studies in rheumatology: a systematic review using the ARRIVE guidelines

AIM

To systematically investigate the quality of reporting of published interventional animal studies in experimental rheumatology.

METHODS

Original scientific publications in Annals of the Rheumatic Diseases (ARD) and Arthritis and Rheumatism (A&R) from January to December 2012 were identified. Studies were included if they used animal experimental model(s) and involved a treatment intervention. Data were extracted regarding disease type, animal model, intervention type and funding. Each study was assessed for quality of reporting, using the ARRIVE guidelines as a checklist.

RESULTS

Forty-one studies (15 ARD, 26 A&R) were analyzed. Ethics approval was not reported or unclear in 22%. Randomization was not reported or unclear in 82.9% of the papers. Only 19.5% and 9.8% of papers reported attrition rate and important adverse events, respectively. Sample size calculation or allocation method was not reported in any paper. Only one study published negative results.

CONCLUSION

A number of key study design principles are poorly reported in experimental animal research investigating potential treatments in rheumatology. We support the widespread implementation of the ARRIVE guidelines in the rheumatology literature to promote the publication of manuscripts that allow rigorous appraisal of scientific quality.

Financial conflicts of interest and their association with outcome and quality of fibromyalgia drug therapy randomized controlled trials

AIMS

To evaluate the association of financial conflicts of interest (FCOI) with the characteristics, outcome and reported methodological quality of fibromyalgia drug therapy randomized controlled trials (FM-RCTs).

METHODS

A cross-sectional study of original, parallel-group, drug therapy FM-RCTs published between 1997 and 2011 from Medline and Cochrane Central Register of Controlled Trials was conducted. Two reviewers independently assessed each RCT for funding source, authors' FCOI(s), study characteristics, reporting of methodological measures important for internal validity and outcome (positive [statistically significant result favoring experimental drug for the primary outcome] or non-positive).

RESULTS

Forty-seven RCTs were eligible with funding source as: 26 (55.3%) industry; eight (17%) non-profit source(s); five (10.6%) mixed; and eight (17%) unspecified. Industry-funded RCTs were more likely to be multicenter and enroll greater number of patients. Reporting of key methodological measures was suboptimal; however, industry and non-profit funded RCTs did not differ in their reporting. Thirty (63.8%) RCTs had ≥ one author who disclosed an FCOI (receipt of research grant [21, 44.7%], industry sponsor employee [20, 42.6%], receipt of consultancy fee/honorarium [16, 34%] and stock ownership [11, 23.4%]). Although industry funding and certain authors' FCOIs (employment and receipt of consultancy fee/honorarium) were univariately associated with positive outcome, such association was not observed after adjusting for study sample size.

CONCLUSIONS

The majority of FM-RCTs were industry-sponsored, and had at least one author with an FCOI. Reporting of key methodological measures was suboptimal. After adjusting for study sample size, no association of industry funding or author's FCOI with study outcome was seen.

Brief report: inadequate description and discussion of enrolled patient characteristics and potential inter-study site differences in reports of randomized controlled trials: a systematic survey in six rheumatology journals

OBJECTIVE

Detailed information on patient recruitment and study settings is an important component of reports of randomized controlled trials (RCTs). We had the impression that many RCTs published in rheumatology journals lacked this information. This study was undertaken to systemically survey this matter in 6 leading journals.

METHODS

A hand search was conducted for RCTs published in 2011 and 2012 in Arthritis & Rheumatism, Annals of the Rheumatic Diseases, Rheumatology (Oxford), Arthritis Care & Research, The Journal of Rheumatology, and Clinical and Experimental Rheumatology. Using the Consolidated Standards of Reporting Trials (CONSORT) guidelines, 2 observers evaluated the articles for the inclusion of patient eligibility criteria, including method of recruitment; study health care settings; geographic location; and, among multicenter studies, a discussion of possible effects of intercenter differences on outcomes.

RESULTS

Among 118 articles, an informative account of the method of recruitment was available in 36 (30.5%). Information about the study health care setting was found in 56 (47.5%). Patient socioeconomic profile was available in 11 (9.3%), patient education level in 10 (8.4%), and patient race in 48 (40.7%). Among 76 multicenter studies, the potential effects of possible intercenter differences on outcome were discussed in 13 (17.1%). There were no important differences between the 3 journals that emphasized the use of CONSORT in their author guidelines and the remaining 3 journals.

CONCLUSION

Adequate information on patient recruitment, the trial setting, and a discussion of possible multicenter design effects on outcomes are lacking in the majority of RCT reports in rheumatology. This affects the validity of these reports and calls for closer attention of authors, journals, and reviewers.

Consolidated standards of reporting trials (CONSORT) and the completeness of reporting of randomised controlled trials (RCTs) published in medical journals

BACKGROUND

An overwhelming body of evidence stating that the completeness of reporting of randomised controlled trials (RCTs) is not optimal has accrued over time. In the mid-1990s, in response to these concerns, an international group of clinical trialists, statisticians, epidemiologists, and biomedical journal editors developed the CONsolidated Standards Of Reporting Trials (CONSORT) Statement. The CONSORT Statement, most recently updated in March 2010, is an evidence-based minimum set of recommendations including a checklist and flow diagram for reporting RCTs and is intended to facilitate the complete and transparent reporting of trials and aid their critical appraisal and interpretation. In 2006, a systematic review of eight studies evaluating the "effectiveness of CONSORT in improving reporting quality in journals" was published.

OBJECTIVES

To update the earlier systematic review assessing whether journal endorsement of the 1996 and 2001 CONSORT checklists influences the completeness of reporting of RCTs published in medical journals.

SEARCH METHODS

We conducted electronic searches, known item searching, and reference list scans to identify reports of evaluations assessing the completeness of reporting of RCTs. The electronic search strategy was developed in MEDLINE and tailored to EMBASE. We searched the Cochrane Methodology Register and the Cochrane Database of Systematic Reviews using the Wiley interface. We searched the Science Citation Index, Social Science Citation Index, and Arts and Humanities Citation Index through the ISI Web of Knowledge interface. We conducted all searches to identify reports published between January 2005 and March 2010, inclusive.

SELECTION CRITERIA

In addition to studies identified in the original systematic review on this topic, comparative studies evaluating the completeness of reporting of RCTs in any of the following comparison groups were eligible for inclusion in this review: 1) Completeness of reporting of RCTs published in journals that have and have not endorsed the CONSORT Statement; 2) Completeness of reporting of RCTs published in CONSORT-endorsing journals before and after endorsement; or 3) Completeness of reporting of RCTs before and after the publication of the CONSORT Statement (1996 or 2001). We used a broad definition of CONSORT endorsement that includes any of the following: (a) requirement or recommendation in journal's 'Instructions to Authors' to follow CONSORT guidelines; (b) journal editorial statement endorsing the CONSORT Statement; or (c) editorial requirement for authors to submit a CONSORT checklist and/or flow diagram with their manuscript. We contacted authors of evaluations reporting data that could be included in any comparison group(s), but not presented as such in the published report and asked them to provide additional data in order to determine eligibility of their evaluation. Evaluations were not excluded due to language of publication or validity assessment.

DATA COLLECTION AND ANALYSIS

We completed screening and data extraction using standardised electronic forms, where conflicts, reasons for exclusion, and level of agreement were all automatically and centrally managed in web-based management software, DistillerSR(®). One of two authors extracted general characteristics of included evaluations and all data were verified by a second author. Data describing completeness of reporting were extracted by one author using a pre-specified form; a 10% random sample of evaluations was verified by a second author. Any discrepancies were discussed by both authors; we made no modifications to the extracted data. Validity assessments of included evaluations were conducted by one author and independently verified by one of three authors. We resolved all conflicts by consensus.For each comparison we collected data on 27 outcomes: 22 items of the CONSORT 2001 checklist, plus four items relating to the reporting of blinding, and one item of aggregate CONSORT scores. Where reported, we extracted and qualitatively synthesised data on the methodological quality of RCTs, by scale or score.

MAIN RESULTS

Fifty-three publications reporting 50 evaluations were included. The total number of RCTs assessed within evaluations was 16,604 (median per evaluation 123 (interquartile range (IQR) 77 to 226) published in a median of six (IQR 3 to 26) journals. Characteristics of the included RCT populations were variable, resulting in heterogeneity between included evaluations. Validity assessments of included studies resulted in largely unclear judgements. The included evaluations are not RCTs and less than 8% (4/53) of the evaluations reported adjusting for potential confounding factors.   Twenty-five of 27 outcomes assessing completeness of reporting in RCTs appeared to favour CONSORT-endorsing journals over non-endorsers, of which five were statistically significant. 'Allocation concealment' resulted in the largest effect, with risk ratio (RR) 1.81 (99% confidence interval (CI) 1.25 to 2.61), suggesting that 81% more RCTs published in CONSORT-endorsing journals adequately describe allocation concealment compared to those published in non-endorsing journals. Allocation concealment was reported adequately in 45% (393/876) of RCTs in CONSORT-endorsing journals and in 22% (329/1520) of RCTs in non-endorsing journals. Other outcomes with results that were significant include: scientific rationale and background in the 'Introduction' (RR 1.07, 99% CI 1.01 to 1.14); 'sample size' (RR 1.61, 99% CI 1.13 to 2.29); method used for 'sequence generation' (RR 1.59, 99% CI 1.38 to 1.84); and an aggregate score over reported CONSORT items, 'total sum score' (standardised mean difference (SMD) 0.68 (99% CI 0.38 to 0.98)).

AUTHORS' CONCLUSIONS

Evidence has accumulated to suggest that the reporting of RCTs remains sub-optimal. This review updates a previous systematic review of eight evaluations. The findings of this review are similar to those from the original review and demonstrate that, despite the general inadequacies of reporting of RCTs, journal endorsement of the CONSORT Statement may beneficially influence the completeness of reporting of trials published in medical journals. Future prospective studies are needed to explore the influence of the CONSORT Statement dependent on the extent of editorial policies to ensure adherence to CONSORT guidance.

Association of industry funding with the outcome and quality of randomized controlled trials of drug therapy for rheumatoid arthritis

OBJECTIVE

To assess the association of industry funding with the characteristics, outcome, and reported quality of randomized controlled trials (RCTs) of drug therapy for rheumatoid arthritis (RA).

METHODS

The Medline and Cochrane Central Register of Controlled Trials databases were searched to identify original RA drug therapy RCTs published in 2002-2003 and 2006-2007. Two reviewers independently assessed each RCT for the funding source, characteristics, outcome (positive [statistically significant result favoring experimental drug for the primary outcome] or not positive), and reporting of methodologic measures whose inadequate performance may have biased the assessment of treatment effect. RCTs that were registered at ClinicalTrials.gov and completed during the study years were assessed for publication bias.

RESULTS

Of the 103 eligible RCTs identified, 58 (56.3%) were funded by industry, 19 (18.4%) were funded by nonprofit sources, 6 (5.8%) had mixed funding, and funding for 20 (19.4%) was not specified. Industry-funded RCTs had significantly more study centers and subjects, while nonprofit agency-funded RCTs had longer duration and were more likely to study different treatment strategies. Outcome could be assessed for 86 (83.5%) of the 103 RCTs studied. The funding source was not associated with a higher likelihood of positive outcomes favoring the sponsored experimental drug (75.5% of industry-funded RCTs had a positive outcome, compared with 68.8% of non-industry-funded RCTs, 40% of RCTs with mixed funding, and 81.2% of RCTs for which funding was not specified). Industry-funded RCTs showed a trend toward a higher likelihood of nonpublication (P=0.093). Industry-funded RCTs were more frequently associated with double-blinding, an adequate description of participant flow, and performance of an intent-to-treat analysis.

CONCLUSION

Industry funding was not associated with a higher likelihood of positive outcomes of published RCTs of drug therapy for RA, and industry-funded RCTs performed significantly better than non-industry-funded RCTs in terms of reporting the use of some key methodologic quality measures.

Reporting randomised clinical trials of analgesics after traumatic or orthopaedic surgery is inadequate: a systematic review

BACKGROUND

Several randomised clinical trials (RCTs) of analgesics in postoperative pain after traumatic or orthopaedic surgery (TOS) have been published, but no studies have assessed the quality of these reports. We aimed to examine the quality of reporting RCTs on analgesics for postoperative pain after TOS.

METHODS

Reports of RCTs assessing analgesics in postoperative pain after TOS were systematically searched from electronic databases. The quality of reports was assessed using the CONSORT checklist (scoring range from 0 to 22). The quality was considered poor when scoring was 12 or lesser. The publication year and the impact factor of journals were recorded.

RESULTS

A total of 92 reports of RCTs were identified and 69 (75%) scored 12 or lesser in CONSORT checklist (range 5-17). The mean (SD) CONSORT score of all reports was 10.6 (2.7). Missing CONSORT items included primary and secondary outcome measures (11%), the specific objectives and hypothesis definition (12%), the sample size calculation (12%), the dates defining the periods of recruitment (12%), the discussion of external validity of findings (14%), the allocation sequence generation (24%), and the interpretation of potential bias or imprecision of results (25%). There was a little improvement in CONSORT scores over time (r = 0.62; p < 0.001) and with impact factor of journals (r = 0.30; p < 0.001).

CONCLUSION

Quality of reporting RCTs on analgesics after TOS is poor. Reporting of those RCTs should be improved according to methodological standard checklists in the next years.

Randomized controlled trials in neurosurgery--how good are we?

BACKGROUND

The strongest evidence in medical clinical literature is represented by randomized controlled trials (RCTs). This study was designed to evaluate neurosurgically relevant RCTs published recently by neurosurgeons.

METHOD

A literature search in MEDLINE and EMBASE included all clinical studies published up to 30 June 2006. RCTs with neurosurgical relevance published by at least one author with affiliation to a neurosurgical department were selected. The number and characteristics of individual trials were recorded, and the quality of the trials with regard to study design, quality of reporting, and relevance for clinical practice was assessed by two different investigators using a modification of the Scottish Intercollegiate Guidelines Network methodology checklist. Changes of RCT quality over time as well as factors influencing the quality were analyzed.

FINDINGS

From the initial search results (MEDLINE n = 3,860, EMBASE n = 3,113 articles), 159 RCTs published by neurosurgeons were extracted for final evaluation. Of the RCTs, 62% have been published since 1995; 52% came from the USA, UK, and Germany. The median RCT sample size was 78 patients and the median follow-up 35.7 weeks. Fifty-two percent of all RCTs were of good, 37% of moderate, and 11% of bad quality, with an improvement over time. RCTs with financial funding and RCTs with a sample size of >78 patients were of significantly better quality. There were no major differences in the rating of the studies between the two investigators.

CONCLUSIONS

Only a fraction of neurosurgically relevant literature consists of RCTs, but the quality is satisfying and has significantly improved over the last years. An adequate sample size and sufficient financial support seem to be of substantial importance with regard to the quality of the study. Our data also show that by using a standardized checklist, the quality of trials can be reliably assessed by observers of different experience and educational levels.

A systematic review of trends in the methodological quality of randomized controlled trials in various research fields

OBJECTIVES

We sought to evaluate the trends in the methodological quality of randomized controlled trials in various medical fields.

STUDY DESIGN AND SETTING

Relevant studies were retrieved by the PubMed and the ISI Web of science databases.

RESULTS

Thirty-five out of 457 retrieved studies met the inclusion criteria. Twenty-one out of 35 selected studies reported significant improvement in at least one methodological quality factor. Overall quality scores were increased in 13 out of 26 studies providing relevant data. The most commonly separately examined key quality factors were allocation concealment and blinding in 13 out of 21 studies that reported relevant data. Allocation concealment was the quality characteristic most commonly reported as significantly improving during the reviewed period (in five out of eight studies reporting relevant comparative data).

CONCLUSION

Certain aspects of methodological quality have improved significantly over time, but others remain stagnant. Further efforts to improve study design, conduct, and reporting of randomized controlled trials are warranted.

Quality of reporting of key methodological items of randomized controlled trials in clinical ophthalmic journals

PURPOSE

To evaluate the reporting quality of key methodological items in randomized controlled trials (RCTs) in four general clinical ophthalmology journals.

METHODS

The reporting of 11 key methodological items in RCTs published in American Journal of Ophthalmology, Archives of Ophthalmology, British Journal of Ophthalmology and Ophthalmology in the year 2005 was assessed.

RESULTS

Sixty-seven eligible RCTs were assessed and the mean number of items reported was 6.3 per RCT. No significant difference in the mean number of items reported was found between the four journals (P=0.20). The most frequently reported item was ethics approval and informed consent (97.0%), followed by masking status (85.1%), description of withdrawals (76.1%), adverse events (73.1%), and intention-to-treat analysis (71.6%). Details on sequence generation, randomization restriction, allocation concealment, allocation implementation, patient flow diagrams, and sample size calculation were reported in <50% of the RCTs assessed. Both sample size and page length of the RCTs correlated with the number of methodological items reported (P=0.024 and P=0.008, respectively).

CONCLUSIONS

Similar to other specialties, rooms for improvement exist in the reporting of key methodological items of RCTs in clinical ophthalmic journals. Stricter adoption of the CONSORT statement might enhance the reporting quality of RCTs in ophthalmic journals.

Quality of randomized clinical trials in juvenile idiopathic arthritis

OBJECTIVES

We evaluated the quality of randomized clinical trials (RCTs) of therapy for juvenile idiopathic arthritis (JIA) using an individual component approach and assessed temporal changes.

METHODS

A systematic review of the literature was performed to identify all RCTs involving exclusively JIA patients. Two investigators independently assessed the identified articles for six quality indicators: generation of allocation sequence, allocation concealment, masking, intention-to-treat (ITT) analysis, dropout rates and clearly stated primary outcome.

RESULTS

Fifty-two RCTs involving JIA patients were assessed. Generation of allocation sequence was unclear in 79% of the studies. Reporting of allocation concealment was adequate in only one-third of the studies. Masking was adequate in 73%, inadequate in 19% and unclear in 8% of the reports. ITT analysis was employed in 37% of the reports. Per-protocol analysis was used in 40% and in 23% the method was unclear. Most of the reports (67%) had dropout rates < or = 20%. About half of the reports (n = 25) failed to show a significant effect of the experimental treatment. No significant associations were found between the study results and quality indicators. With the exception of adequate masking and dropout rate, all quality indicators showed a trend of improvement over the decades.

CONCLUSIONS

The quality of RCTs in JIA based on the selected indicators was poor. Although there were some positive changes over time, the reporting and methodological quality of trials should be improved. New, more powerful and acceptable RCT designs should be developed in this patient population.

Primer: assessing the efficacy and safety of nonpharmacologic treatments for chronic rheumatic diseases

Nonpharmacologic treatments (including surgery, technical operation, rehabilitation, physiotherapy, education or the use of external technical devices) represent a wide range of treatments for chronic rheumatic disease. Randomized, controlled trials (RCTs) are recognized as the best method for avoiding bias in assessing nonpharmacologic treatments. Designs such as nonrandomized studies, cluster-randomized trials, patient-preference trials, modified Zelen-design trials, tracker trials and expertise-based RCTs could, however, be used to assess such treatments. Assessing nonpharmacologic treatments involves methodologic issues linked to difficulties associated with blinding, duration of the study, main outcomes of the study, difficulties associated with standardizing the intervention and the influence of health-care providers. Hence, these treatments cannot be assessed according to the standards used for pharmacologic treatments. As well, specific instruments such as A CheckList to Evaluate A Report of a NonPharmacological Trial (CLEAR NPT) should be used to assess the quality of reports in this field. Important reporting guidelines that take an evidence-based approach to improve the quality of reports from RCTs, such as the Consolidated Standards of Reporting Trials (CONSORT) statement, should be extended to take these issues into account.

Does the CONSORT checklist improve the quality of reports of randomised controlled trials? A systematic review

OBJECTIVE

To determine whether the adoption of the CONSORT checklist is associated with improvement in the quality of reporting of randomised controlled trials (RCTs).

DATA SOURCES

MEDLINE, EMBASE, Cochrane CENTRAL, and reference lists of included studies and of experts were searched to identify eligible studies published between 1996 and 2005.

STUDY SELECTION

Studies were eligible if they (a) compared CONSORT-adopting and non-adopting journals after the publication of CONSORT, (b) compared CONSORT adopters before and after publication of CONSORT, or (c) a combination of (a) and (b). Outcomes examined included reports for any of the 22 items on the CONSORT checklist or overall trial quality.

DATA SYNTHESIS

1128 studies were retrieved, of which 248 were considered possibly relevant. Eight studies were included in the review. CONSORT adopters had significantly better reporting of the method of sequence generation (risk ratio [RR], 1.67; 95% CI, 1.19-2.33), allocation concealment (RR, 1.66; 95% CI, 1.37-2.00) and overall number of CONSORT items than non-adopters (standardised mean difference, 0.83; 95% CI, 0.46-1.19). CONSORT adoption had less effect on reporting of participant flow (RR, 1.14; 95% CI, 0.89-1.46) and blinding of participants (RR, 1.09; 95% CI, 0.84-1.43) or data analysts (RR, 5.44; 95% CI, 0.73-36.87). In studies examining CONSORT-adopting journals before and after the publication of CONSORT, description of the method of sequence generation (RR, 2.78; 95% CI, 1.78-4.33), participant flow (RR, 8.06; 95% CI, 4.10-15.83), and total CONSORT items (standardised mean difference, 3.67 items; 95% CI, 2.09-5.25) were improved after adoption of CONSORT by the journal.

CONCLUSIONS

Journal adoption of CONSORT is associated with improved reporting of RCTs.

Les paramètres de validité d'un essai thérapeutique et leur influence sur l'élaboration d'une médecine fondée sur les preuves : revue de la littérature

INTRODUCTION

The purpose of this study was to review the validity of clinical trials and checklists used to evaluate them in systematic reviews and meta-analyses.

METHODS

We researched Medline and bibliographies of relevant articles with use of keywords and author names.

RESULTS

Methodological choices that have a demonstrated influence on the results of clinical trials include collecting informed consent, comparison of publication with protocol, selection of patients, randomization procedure, number of subjects to include, number lost to follow-up, blinding of patients and evaluation. The type of comparison (with placebo and classic treatment, adjustment of level of significance to the number of measures and number of criteria) and intent-to-treat analysis has a demonstrated influence. The level of this influence is not always precisely determined. In some other fields we didn't find any scientific demonstration even if they have a potential effect.

CONCLUSION

The validity of quantitative checklists is low because of lack of scientific demonstration of their relevance.

Violation of the intent-to-treat principle and rate of missing data in superiority trials assessing structural outcomes in rheumatic diseases

OBJECTIVE

To evaluate the methodologic quality of and identify methodologic issues in superiority trials assessing structural outcomes in rheumatic diseases.

METHODS

We searched Medline and the Cochrane Central Register of Controlled Trials for reports of randomized controlled trials assessing structural outcomes in osteoarthritis (OA), rheumatoid arthritis (RA), and osteoporosis (OP) published between January 1994 and December 2003 in high-impact factor general medical and specialty journals. One reader extracted data (quality assessment, intent-to-treat analysis [ITT analysis], rate of missing data, and methods of handling missing data), using a standardized form.

RESULTS

A total of 81 reports were included in the analysis (37 on OP, 34 on RA, and 10 on OA). The mean +/- SD methodologic quality scores on the Jadad scale (possible range 0-5) and the Delphi list (possible range 0-9) were 2.9 +/- 1.2 and 6.4 +/- 1.3, respectively. Although it was reported in 54 articles (66.7%) that the analysis was done on an ITT basis, full ITT analysis was performed in only 6 of the studies (7.4%), modified ITT analysis in 11 (13.6%), and case-complete analysis in 48 (59.3%); the analysis was unclear in 16 articles (19.8%). The rate of missing data on structural outcomes could be determined in only 63 articles (77.8%) and was >20% in approximately one-third of these reports. Methods for handling missing data on structural outcomes were described in 19 articles (23.5%) and were, in general, inappropriate.

CONCLUSION

Lack of ITT analysis and a high rate of missing data in superiority trials assessing structural outcomes may bias results from such trials. Our recommendations for improving these shortcomings may help researchers plan, analyze, and report the results of such trials.

The quality of reporting and outcome measures in randomized clinical trials related to upper-extremity disorders

PURPOSE

Randomized clinical trials can provide strong evidence regarding effective treatment options. The quality of reporting and the type of outcome measures used are important when judging whether results justify change in clinical practice. The aim of this study was to assess the quality of reporting of randomized clinical trials related to treatment of upper-extremity disorders, published in 4 hand surgical and orthopedic journals during an 11-year period, and assess the type of outcome measures used in the trials.

METHODS

Eligible articles were identified by reviewing all abstracts published in the 4 journals from 1992 through 2002. The quality of reporting was assessed by a modified Jadad scale that consisted of 3 items (randomization, blinding, and withdrawals/dropouts). A higher score (0-5) indicated higher quality. The outcome measures were classified according to the International Classification of Functioning, Disability and Health into the levels of body function and structure, activity, and participation.

RESULTS

Of 92 articles reporting randomized clinical trials, 40 articles described appropriate randomization method that implied they were truly randomized studies, 31 articles did not describe the randomization method, and 21 articles (23%) described inappropriate randomization methods. Double or single blinding was reported in 33 articles. Absence or description of withdrawals/dropouts was shown in 77 articles. The median quality score calculated for all 92 articles was 2 (range, 0-5) points. The median score for the 28 articles published 1992 through 1996 was 1 (range, 0-5) points and for the 64 articles published from 1997 through 2002 was 3 (range, 0-5) points. All trials used outcome measures on body function and structure level; 41% used measures of activity and/or participation.

CONCLUSIONS

There is a need to improve the quality of reporting of upper-extremity randomized clinical trials and to increase the use of outcome measures covering different aspects of disability.

Randomized Trials of Neurosurgical Interventions: A Systematic Appraisal

OBJECTIVE

To systematically appraise the study design and quality of reporting of randomized controlled trials (RCTs) on neurosurgical procedures and to identify potential defects and biases.

METHODS

Randomized controlled trials with at least five patients comparing any neurosurgical procedure against another procedure, nonsurgical treatment, or no treatment were retrieved from MEDLINE, EMBASE, and the Cochrane Library. We analyzed study design, quality of reporting, and trial results.

RESULTS

The median sample size in the 108 eligible reports was 68 patients. Ninety-nine trials (91.7%) reported inclusion and exclusion criteria, 55 (50.9%) mentioned the randomization mode, and 87 (80.6%) adequately described withdrawals, but only 31 (28.7%) described allocation concealment, only 23 (21.3%) gave power calculations, and only 20 (18.5%) were adequately powered. Significant efficacy or trend for efficacy was claimed in 46 reports (42.6%), and no difference between the compared procedures was found in 60 trials (55.6%). Trials with a larger sample size were more likely to report withdrawals (P = 0.02) and power calculations (P = 0.006). Only 14 trials (13.6%) were double-blind, and this was less frequent in longer trials (P = 0.02). Among quality criteria, only the reporting of randomization mode improved significantly over time (P = 0.015).

CONCLUSION

Several aspects of the design and reporting of randomized controlled trials on neurosurgical procedures can be improved. Larger, adequately powered, and accurately reported trials are needed.

Quality of Abstracts Describing Randomized Trials in the Proceedings of American Society of Clinical Oncology Meetings: Guidelines for Improved Reporting

Purpose

To evaluate the quality of reporting in abstracts describing randomized controlled trials (RCTs) included in the Proceedings of American Society of Clinical Oncology (ASCO) meetings and to propose reporting guidelines for abstracts that are submitted to future meetings.

Methods

Guidelines for reporting of RCTs in abstracts were developed by extracting key elements from published guidelines for full reports of RCTs, and modified based on an expert survey. Abstracts presenting results of RCTs with sample size ≥ 200 were identified from the ASCO Proceedings for the years 1989 to 1998. Information regarding the quality of each abstract was extracted, and a quality score (possible range, 0 to 10) was assigned based on adherence to the guidelines.

Results

Brief description of the intervention, explicit identification of the primary end point, and presentation of results accompanied by statistical tests were regarded by experts as the most important items to include in an abstract, whereas presentation of secondary and subgroup analyses was the least important. Deficiencies in reporting were present in almost all of the 510 abstracts; for example, only 22% of the abstracts provided explicit identification of the primary end point. The median quality score was 5.5 (range, 2.0 to 8.5); the quality score improved with time (P < .0001) and was better for oral or plenary presentations (P = .0003).

Conclusion

The quality of reporting of RCTs in abstracts submitted to Annual Meetings of ASCO is suboptimal. Although space precludes the inclusion of details required in the final report, abstracts could be improved through the use of explicit minimal guidelines, which are suggested in this article.

Citrullination of synovial proteins in murine models of rheumatoid arthritis

OBJECTIVE

Antibodies directed to citrulline-containing proteins are highly specific for rheumatoid arthritis (RA) and can be detected in up to 80% of patients with RA. Citrulline is a nonstandard amino acid that can be incorporated into proteins only by posttranslational modification of arginine by peptidylarginine deiminase (PAD) enzymes. The objective of this study was to investigate the presence of anticitrulline antibodies, PAD enzymes, and citrullinated antigens in mouse models of both acute and chronic destructive arthritis: streptococcal cell wall (SCW)-induced arthritis and collagen-induced arthritis (CIA), respectively.

METHODS

Synovial tissue biopsy specimens were obtained from naive mice, mice with CIA, and mice with SCW-induced arthritis. The expression of messenger RNA (mRNA) for PAD enzymes was analyzed by reverse transcriptase-polymerase chain reaction; the presence of PAD proteins and their products (citrullinated proteins) was analyzed by Western blotting and by immunolocalization. The presence of anticitrullinated protein antibodies was investigated by an anti-cyclic citrullinated peptide (anti-CCP) enzyme-linked immunosorbent assay (ELISA) and an ELISA using in vitro citrullinated fibrinogen.

RESULTS

In both mouse models, PAD type 2 (PAD2) mRNA was present in the synovium but was not translated into PAD2 protein. In contrast, PAD4 mRNA, although absent from healthy synovium, was readily transcribed and translated by polymorphonuclear neutrophils infiltrating the synovial tissue during inflammation. As a consequence, several synovial proteins were subjected to citrullination. One of these proteins was identified as fibrin, which has been reported to be citrullinated also in synovium of patients with RA. Although generation of citrullinated antigens during synovial inflammation in the mice was eminent, no anti-CCP antibodies could be detected.

CONCLUSION

Citrullination of synovial antigens is an active process during joint inflammation in both mice and humans, but the induction of autoantibodies directed to these proteins is a more specific phenomenon, detectable only in human RA patients.

Nearly half of dental randomized controlled trials published in German are not included in Medline

Randomized controlled trials (RCTs) are considered the most reliable type of clinical intervention studies. However, not all reports of RCTs are accessible in Medline. This can impede the validity of the results of systematic reviews. Ten German-language dental journals were manually searched to locate reports of controlled clinical trials published between 1970 and 2000. The publication type was determined and compared with Medline. Of the 15 777 articles, 210 reports of RCTs and 410 articles of non-randomized controlled clinical trials (CCTs) were identified. Only 56% of the RCTs and 75% of the CCTs are available in Medline. Of the 118 reports of RCTs registered in Medline. 15 are indexed with the correct Publication Type term. Our data suggest that (a) hand-searching plays a valuable role in identifying reports of clinical dental trials, and (b) a literature search in Medline is likely to yield incomplete results.

Discrepancy between published report and actual conduct of randomized clinical trials

Randomized clinical trial (RCT) publications with inappropriate random-sequence generation, lack of allocation concealment, or imperfect blinding yield inflated estimates of effect compared to those in which adequate methods are described. RCTs that do not state methods used yield similar effect estimates, suggesting that inadequate methods were used. We compared RCT publications with investigator reports of actual practice for 40 rheumatology RCTs published in 1997/1998. In RCTs in which these methods were not described in the trial reports and would thus have been characterized as "inadequate," investigators reported using methods of random-sequence generation and allocation concealment that would be considered adequate in 77.4 and 78.1% of RCTs, respectively. This suggests that, in contrast to previous reports, inadequate random-sequence generation and allocation concealment, per se, may not be a major problem in RCTs. Characterizing RCTs as "good" or "poor" quality based on the published report is likely to be inappropriate.