Assessing the Quality of Abstracts in Randomized Controlled Trials Published in High Impact Cardiovascular Journals

Characteristics and Impact of Randomized Trials on Drugs or Devices in Cardiovascular Medicine

INTRODUCTION

Clinical trials, essential for medical advancement, vary significantly in methodology and regulatory pathways depending on the type of therapeutic intervention (i.e., drugs or devices). This study aimed to determine whether the drug or device intervention types influence the impact of randomized trials in cardiovascular medicine.

METHODS

We analyzed late-breaking randomized controlled trials presented at major cardiology conferences from 2015 to 2021. The primary endpoint was the total number of citations obtained. Secondary endpoints included the number of citations at 1 and 2 years, number of total and 1-year mentions, and several metrics of study conduct and publication. Statistical analysis included tests for comparisons of continuous or categorical variables, based on their distribution, as appropriate. To adjust the results for potential confounders, univariable and multivariable regression models were utilized. Additionally, sensitivity analyses were conducted to explore both the effect of neutral or positive study outcomes on the comparative impact of drug versus device trials and the impact of the coronavirus disease 2019 (COVID-19) pandemic on the primary endpoint.

RESULTS

Of 382 eligible randomized trials, 227 (59.4%) were trials of drugs and 155 (40.6%) were trials of devices. Drug trials had a higher median number of total citations compared to device studies (93 [interquartile range {IQR} 48-137] vs. 82 [IQR 39-192]; p = 0.025). This difference was consistent at 1 and 2 years and was also observed in the number of total mentions and mentions at 1 year. All the metrics of study conduct and publication were similar, except for drug studies being more often stopped prematurely (8.8 vs. 1.9%; p = 0.006). After adjusting for multiple potential confounders, the difference in citations and mentions was no longer statistically significant. However, drug trials remained more likely to be stopped prematurely (adjusted odds ratio = 1.15; 95% confidence interval 1.03-1.28; p = 0.009). Positive study outcomes significantly influenced the number of citations and the likelihood of a trial being stopped prematurely.

CONCLUSIONS

Drug trials are often stopped early and receive more citations and mentions than device trials. However, these differences are mainly due to factors other than the treatment itself. Studies published simultaneously tend to get more attention, and drug trials with positive results are cited more often than those with neutral results.

Prepublication abstract-only reports compared with full-text manuscripts for randomised controlled trials in inflammatory bowel disease: a systematic review

INTRODUCTION

Randomised controlled trials (RCTs) of key therapies in inflammatory bowel disease (IBD) are often presented and available as abstracts for significant periods of time prior to full publication, often being employed to make strategic and clinical prescribing decisions. We compared the concordance of prepublication abstract-only reports and their respective full-text manuscripts.

METHODS

Pairs of full-text manuscripts and their respective prepublication abstract-only reports for the same RCT outcomes, at the same time point of analysis were included. The RCTs were on treatments for IBD with full-text manuscripts published between 2010 and 2023.

RESULTS

We found 77 pairs of full-text manuscripts and their prepublication abstract-only reports. There were significant mismatches in the reporting of stated planned outcomes (65/77 matched, p<0.001) and primary outcomes reported in their results sections (67/77, p<0.001); trial registrations (34/65, p<0.001); the number of randomised participants (49/77, p=0.18); participants reaching end of study (21/71, p<0.001) and primary outcome data (40/73, p<0.001). Authors conclusions matched (75/77, p=0.157). Authors did not provide explicit or implied justifications for the absence or non-concordance for any of the above items.

CONCLUSIONS

Abstract-only reports have consistent issues with both limited reporting of key information and significant differences in data when compared with their later full-text publications. These are not related to further recruitment of patients or word count limitations and are never explained. As abstracts are often used in guidelines, reviews and stakeholder decision-making on prescribing, caution in their use is strongly suggested. Further work is needed to enhance minimum reporting standards in abstract-only works and ensure consistency with final published papers.

Effects of interventions targeting the systemic inflammatory response to cardiac surgery on clinical outcomes in adults

BACKGROUND

Organ injury is a common and severe complication of cardiac surgery that contributes to the majority of deaths. There are no effective treatment or prevention strategies. It has been suggested that innate immune system activation may have a causal role in organ injury. A wide range of organ protection interventions targeting the innate immune response have been evaluated in randomised controlled trials (RCTs) in adult cardiac surgery patients, with inconsistent results in terms of effectiveness.

OBJECTIVES

The aim of the review was to summarise the results of RCTs of organ protection interventions targeting the innate immune response in adult cardiac surgery. The review considered whether the interventions had a treatment effect on inflammation, important clinical outcomes, or both.

SEARCH METHODS

CENTRAL, MEDLINE, Embase, conference proceedings and two trial registers were searched on October 2022 together with reference checking to identify additional studies.

SELECTION CRITERIA

RCTs comparing organ protection interventions targeting the innate immune response versus placebo or no treatment in adult patients undergoing cardiac surgery where the treatment effect on innate immune activation and on clinical outcomes of interest were reported.

DATA COLLECTION AND ANALYSIS

Searches, study selection, quality assessment, and data extractions were performed independently by pairs of authors. The primary inflammation outcomes were peak IL-6 and IL-8 concentrations in blood post-surgery. The primary clinical outcome was in-hospital or 30-day mortality. Treatment effects were expressed as risk ratios (RR) and standardised mean difference (SMD) with 95% confidence intervals (CI). Meta-analyses were performed using random effects models, and heterogeneity was assessed using I2.

MAIN RESULTS

A total of 40,255 participants from 328 RCTs were included in the synthesis. The effects of treatments on IL-6 (SMD -0.77, 95% CI -0.97 to -0.58, I2 = 92%) and IL-8 (SMD -0.92, 95% CI -1.20 to -0.65, I2 = 91%) were unclear due to heterogeneity. Heterogeneity for inflammation outcomes persisted across multiple sensitivity and moderator analyses. The pooled treatment effect for in-hospital or 30-day mortality was RR 0.78, 95% CI 0.68 to 0.91, I2 = 0%, suggesting a significant clinical benefit. There was little or no treatment effect on mortality when analyses were restricted to studies at low risk of bias. Post hoc analyses failed to demonstrate consistent treatment effects on inflammation and clinical outcomes. Levels of certainty for pooled treatment effects on the primary outcomes were very low.

AUTHORS' CONCLUSIONS

A systematic review of RCTs of organ protection interventions targeting innate immune system activation did not resolve uncertainty as to the effectiveness of these treatments, or the role of innate immunity in organ injury following cardiac surgery.

Reporting quality of abstracts from randomised controlled trials published in leading critical care nursing journals: a methodological quality review

OBJECTIVES

To evaluate the methodological quality of randomised controlled trial (RCT) abstracts in leading critical care nursing journals against the Consolidated Standards of Reporting Trials-Abstracts (CONSORT-A) checklist and to identify variables related with abstract reporting quality.

DESIGN

Descriptive methodological quality review.

DATA SOURCE

We searched the PubMed database and the websites of each included journal.

ELIGIBILITY CRITERIA

We included RCT abstracts published between 2011 and 2021 in the first 11 Scopus-ranking (2021) critical care nursing journals that reported the results of RCTs in English that referred to the care of adult patients with acute/critical illness or their relatives and/or caregivers and conducted in ICUs.

DATA EXTRACTION AND SYNTHESIS

Two independent investigators extracted the data using a prespecified 17-item checklist directly derived from CONSORT-A. For the primary outcome, each item was evaluated whether it was adequately reported or not and descriptive statistics were reported. An overall score was calculated by summing the results of all items and multivariate linear regression was conducted to detect potential predictors.

RESULTS

Seventy-eight RCT abstracts were included in this review. The items with the highest CONSORT-A adherence were authors, objective, conclusion, participants, interventions and outcome. The randomisation item had the lowest CONSORT-A adherence, followed by trial registration, funding source, harms or side effects, recruitment, blinding and outcome results. The average CONSORT-A score was 8.5±1.5 points (of the maximum 17.5 points). Multivariate analysis indicated that the categorised word score and publication date were positively associated with the overall CONSORT-A score, while the first author's country being in Asia was negatively associated with the overall CONSORT-A score.

CONCLUSIONS

The 78 RCT abstracts showed poor overall adherence to CONSORT-A. The results indicate that the methodological reporting quality of RCT abstracts in critical care nursing journals requires improvement to facilitate assessment of the applicability and relevance of the results reported.

Abstracts of randomized controlled trials in the field of dentofacial trauma: Reporting quality and spin

BACKGROUND/AIM

Abstracts of randomized controlled trials (RCTs) provide a summary of the entire trial report. Their transparent, detailed, and accurate reporting is essential for clinical decision-making and evidence-based dental practice. The aim of this study was to assess the reporting quality and prevalence of spin in abstracts of RCTs in the field of dentofacial trauma.

MATERIALS AND METHODS

The PubMed database was searched to identify RCT reports published between 2017 and 2021 in the field of dentofacial trauma. The reporting quality of abstracts was assessed according to the 16-item CONSORT for Abstracts checklist and measured with an overall quality score (OQS, range: 0-16). Linear regression analyses were used to identify factors associated with reporting quality. Among the included RCTs, parallel-group RCTs with non-significant primary outcomes were selected for spin assessment using predefined spin strategies.

RESULTS

One hundred and twelve eligible abstracts were identified and included. The mean OQS was 4.51 (SD, 1.35; 95% CI, 4.26-4.76). Abstracts with more than 250 words (p = .004) and a structured format (p = .032) had significantly better reporting quality. Of the 30 abstracts that were eligible for spin assessment, spin was identified in 23 (76.7%). Among these, spin was observed in the Conclusions sections of 22 abstracts (73.3%) and the Results sections of 9 abstracts (30.0%).

CONCLUSIONS

Among RCT abstracts in the field of dentofacial trauma, the reporting quality was sub-optimal and the prevalence of spin was relatively high. Strict adherence to the CONSORT for Abstracts guidelines is needed to ensure complete and transparent reporting. Relevant stakeholders need to make concerted efforts to avoid spin.

Evaluation of reporting quality of abstracts of randomized controlled trials regarding patients with COVID-19 using the CONSORT statement for abstracts

OBJECTIVE

To evaluate the reporting quality of randomized controlled trial (RCT) abstracts regarding patients with coronavirus disease 2019 (COVID-19) and to analyze the factors influencing the quality.

METHODS

The PubMed, Embase, Web of Science, and Cochrane Library databases were searched to collect RCTs on patients with COVID-19. The retrieval time was from inception to December 1, 2020. The CONSORT statement for abstracts was used to evaluate the reporting quality of RCT abstracts.

RESULTS

A total of 53 RCT abstracts were included. The CONSORT statement for abstracts showed that the average reporting rate of all items was 50.2%. The items with a lower reporting quality were mainly the trial design and the details of randomization and blinding (<10%). The mean overall adherence score across all studies was 8.68 ± 2.69 (range 4-13.5). Multivariate linear regression analysis showed that the higher reporting scores were associated with higher journal impact factor (P < 0.01), international collaboration (P = 0.04), and structured abstract format (P < 0.01).

CONCLUSIONS

Although many RCTs on patients with COVID-19 have been published in different journals, the overall quality of reporting in the included RCT abstracts was suboptimal, thus diminishing their potential usefulness, and this may mislead clinical decision-making. In order to improve the reporting quality, it is necessary to promote and actively apply the CONSORT statement for abstracts.

Assessment of reporting quality in randomised controlled clinical trial abstracts of dental implantology published from 2014 to 2016

OBJECTIVES

Access to full texts of randomised controlled clinical trials (RCTs) is often limited, so brief summaries of studies play a pivotal role. In 2008, a checklist was provided to ensure the transparency and completeness of abstracts. The aim of this investigation was to estimate adherence to the reporting guidelines of the Consolidated Standards of Reporting Trials (CONSORT) criteria for abstracts (CONSORT-A) in RCT publications.

PRIMARY ENDPOINT

Assessment according to the percentage of compliance with the 16 CONSORT-A criteria per study.

MATERIALS AND METHODS

This study is based on a full survey (212 RCT abstracts in dental implantology, PubMed search, publication period 2014-2016, 45 journals, median impact factor: 2.328). In addition to merely documenting 'adherence' to criteria, the authors also assessed the 'complete implementation' of the requested information where possible. The collection of data was performed independently by two dentists, and a final consensus was reached. The primary endpoint was evaluated by medians and quartiles. Additionally, a Poisson regression was conducted to detect influencing factors.

RESULTS

A median of 50% (Q1-Q3: 44%-63%) was documented for the 16 criteria listed in the CONSORT-A statement. Nine of the 16 criteria were considered in fewer than 50% of the abstracts. 'Correct implementation' was achieved for a median of 43% (Q1-Q3: 31%-50%) of the criteria. An additional application of Poisson regression revealed that the number of words used had a locally significant impact on the number of reported CONSORT criteria for abstracts (incidence rate ratio 1.001, 95% CI 1.001 to 1.002).

CONCLUSION

Transparent and complete reporting in abstracts appears problematic. A limited word count seems to result in a reduction in necessary information. As current scientific knowledge is often not readily available in the form of publications, abstracts constitute the primary basis for decision making in clinical practice and research. This is why journals should refrain from limiting the number of words too strictly in order to facilitate comprehensive reporting in abstracts.

A multiyear systematic survey of the quality of reporting for randomised trials in dentistry, neurology and geriatrics published in journals of Spain and Latin America

BACKGROUND

The Iberoamerican Cochrane Network is currently developing an extensive project to identify Spanish-language journals that publish original clinical research in Spain and Latin America. The project is called BADERI (Database of Iberoamerican Essays and Journal) and feeds the research articles, mainly randomised clinical trials (RCTs), into CENTRAL (Cochrane Collaboration Central Register of Controlled Trials). This study aims to assess the quality of reporting of RCTs published in Spanish and Latin American journals for three clinical fields and assess changes over time.

METHODS

We did a systematic survey with time trend analysis of RCTs for dentistry, geriatrics, and neurology. These fields were chosen for pragmatic reasons as they had not yet been completed in BADERI. After screening RCTs from 1990 to 2018 for randomised or quasi-randomised clinical trials, we extracted data for 23 CONSORT items. The primary outcome was the total score of the 23 predefined CONSORT 2010 items for each RCT (score range from 0 to 34). The secondary outcome measure was the score for each one of these 23 items.

RESULTS

A total of 392 articles from 1990 to 2018 were included as follows: dentistry (282), neurology (80), and geriatrics (30). We found that the overall compliance score for the CONSORT items included in this study for all 392 RCTs analysed was 12.6 on a scale with a maximum score of 34. With time, the quality of reporting improved slightly for all RCTs. None of the articles achieved the complete individual CONSORT item compliance score. The lowest overall compliance percentage was for item 10 (Randomisation implementation) and item 24 (Protocol registration), with a dismal 1% compliance across all included RCTs, regardless of country.

CONCLUSIONS

CONSORT compliance is very poor in the 392 analysed RCTs. The impact of the CONSORT statement on improving the completeness of RCT reporting in Latin America and Spain is not clear. Iberoamerican journals should become more involved in endorsing and enforcing adherence to the CONSORT guidelines.

Abstracts for reports of randomised trials of COVID-19 interventions had low quality and high spin

OBJECTIVES

To assess the reporting quality of abstracts for published randomized controlled trials (RCTs) of interventions for coronavirus disease 2019 (COVID-19), including the use of spin strategies and the level of spin for RCTs with statistically non-significant primary outcomes, and to explore potential predictors for reporting quality and the severity of spin.

STUDY DESIGN AND SETTING

PubMed was searched to find RCTs that tested interventions for COVID-19, and the reporting quality and spin in the abstracts were assessed. Linear regression analyses were used to identify potential predictors.

RESULTS

Forty RCT abstracts were included in our assessment of reporting quality, and a higher word count in the abstract was significantly correlated with higher reporting scores (95% CI 0.044 to 0.658, P=0.026). Multiple spin strategies were identified. Our multivariate analyses showed that geographical origin was associated with severity of spin, with research from non-Asian regions containing fewer spin strategies (95% CI -0.760 to -0.099, P=0.013).

CONCLUSIONS

The reporting quality of abstracts of RCTs of interventions for COVID-19 is far from satisfactory. A relatively high proportion of the abstracts contained spin, and the findings reported in the results and conclusion sections of these abstracts need to be interpreted with caution.

Interventions to prevent and treat sarcopenia in a surgical population: a systematic review and meta-analysis

BACKGROUND

The aim of this systematic review was to summarize the results of trials evaluating interventions for the reduction of sarcopenia in patients undergoing surgery.

METHODS

Searches were conducted using the Cochrane Central Register of Controlled Trials, MEDLINE and Embase. RCTs evaluating exercise, dietary or pharmacological interventions to address sarcopenia in the perioperative period were included. Treatment effect estimates were expressed as standardized mean differences (MDs) with confidence intervals, and heterogeneity was expressed as I2 values.

RESULTS

Seventy trials including 3402 participants were selected for the data synthesis. Exercise interventions significantly increased muscle mass (MD 0.62, 95 per cent c.i. 0.34 to 0.90; P < 0.001), muscle strength (MD 0.55, 0.39 to 0.71; P < 0.001), measures of gait speed (MD 0.42, 0.05 to 0.79; P = 0.03), and reduced time for completion of set exercises (MD -0.76, -1.12 to -0.40; P < 0.001) compared with controls. Subgroup analysis showed that interventions in the early postoperative period were more likely to have a positive effect on muscle mass (MD 0.71, 0.35 to 1.07; P < 0.001) and timed tests (MD -0.70, -1.10 to -0.30; P = 0.005) than preoperative interventions. Treatment effects on muscle mass (MD 0.09, -0.31 to 0.49; P = 0.66) and strength (MD 0.46, -0.01 to 0.92; P = 0.05) were attenuated by the presence of cancer. Results of analyses restricted to nine trials at low risk of allocation concealment bias and fourteen trials at low risk of attrition bias were comparable to those of the primary analysis. Risk-of-bias assessment showed that most trials were at high risk of incomplete outcome and attrition bias, thus reducing the estimate of certainty of the evidence according to the GRADE assessment tool.

CONCLUSION

Exercise interventions appear beneficial in reducing the impact of sarcopenia. Because of the high risk of bias and low certainty of the current evidence, large RCTs using standardized measures of muscle mass should be undertaken.

Reporting of methodological studies in health research: a protocol for the development of the MethodologIcal STudy reportIng Checklist (MISTIC)

INTRODUCTION

Methodological studies (ie, studies that evaluate the design, conduct, analysis or reporting of other studies in health research) address various facets of health research including, for instance, data collection techniques, differences in approaches to analyses, reporting quality, adherence to guidelines or publication bias. As a result, methodological studies can help to identify knowledge gaps in the methodology of health research and strategies for improvement in research practices. Differences in methodological study names and a lack of reporting guidance contribute to lack of comparability across studies and difficulties in identifying relevant previous methodological studies. This paper outlines the methods we will use to develop an evidence-based tool-the MethodologIcal STudy reportIng Checklist-to harmonise naming conventions and improve the reporting of methodological studies.

METHODS AND ANALYSIS

We will search for methodological studies in the Cumulative Index to Nursing and Allied Health Literature, Cochrane Library, Embase, MEDLINE, Web of Science, check reference lists and contact experts in the field. We will extract and summarise data on the study names, design and reporting features of the included methodological studies. Consensus on study terms and recommended reporting items will be achieved via video conference meetings with a panel of experts including researchers who have published methodological studies.

ETHICS AND DISSEMINATION

The consensus study has been exempt from ethics review by the Hamilton Integrated Research Ethics Board. The results of the review and the reporting guideline will be disseminated in stakeholder meetings, conferences, peer-reviewed publications, in requests to journal editors (to endorse or make the guideline a requirement for authors), and on the Enhancing the QUAlity and Transparency Of health Research (EQUATOR) Network and reporting guideline websites.

REGISTRATION

We have registered the development of the reporting guideline with the EQUATOR Network and publicly posted this project on the Open Science Framework (www.osf.io/9hgbq).

A tutorial on methodological studies: the what, when, how and why

BACKGROUND

Methodological studies - studies that evaluate the design, analysis or reporting of other research-related reports - play an important role in health research. They help to highlight issues in the conduct of research with the aim of improving health research methodology, and ultimately reducing research waste.

MAIN BODY

We provide an overview of some of the key aspects of methodological studies such as what they are, and when, how and why they are done. We adopt a "frequently asked questions" format to facilitate reading this paper and provide multiple examples to help guide researchers interested in conducting methodological studies. Some of the topics addressed include: is it necessary to publish a study protocol? How to select relevant research reports and databases for a methodological study? What approaches to data extraction and statistical analysis should be considered when conducting a methodological study? What are potential threats to validity and is there a way to appraise the quality of methodological studies?

CONCLUSION

Appropriate reflection and application of basic principles of epidemiology and biostatistics are required in the design and analysis of methodological studies. This paper provides an introduction for further discussion about the conduct of methodological studies.

Quality of reporting for randomised clinical trials published in Latin American and Spanish journals: A protocol for a systematic survey of three clinical specialities

INTRODUCTION

Quality of reporting refers to how published articles communicate how the research was done and what was found. Gaps and imprecisions of reporting hamper the assessment of the methodological quality and internal and external validity. The CONsolidated Standards of Reporting Trials (CONSORT) are a set of evidence-based recommendations of the minimum elements to be included in the reporting of randomised controlled trials (RCTs) to ensure a complete and transparent account of what was done, how it was done and what was found. Few studies have been conducted on the impact of CONSORT on RCTs published in Latin American and Spanish journals. We aim to assess the reporting quality of RCTs of three clinical specialities published in Spanish and Latin American journals, as well as to assess changes over time and associations of quality with journal and country indicators.

METHODS AND ANALYSIS

We will conduct a systematic survey of all RCTs published in Spanish-language journals in three clinical fields (dentistry, neurology and geriatrics) from 1990 to 2018. We will include RCTs from previous work that has identified all RCTs on these medical fields published in Spain and Latin America. We will update this work via handsearching of relevant journals. Assessment of quality of reporting will be conducted independently and in duplicate using the CONSORT 2010 Statement. We will also extract journal and country indicators. We will conduct descriptive statistics and secondary analyses considering the year, country, and journal of publication, among others.

ETHICS AND DISSEMINATION

The Universidad de Santiago de Chile's ethics committee approved the protocol. We will disseminate the results of this work in peer-reviewed scientific journals and conference proceedings. We expect to raise awareness among researchers, journal editors and funders on the importance of training in reporting guidelines and using them from the inception of RCT protocols.