Reducing waste from incomplete or unusable reports of biomedical research

Research waste among randomized controlled trials in preterm infants: a Cross-sectional study

OBJECTIVE

Randomized controlled trials (RCTs) are the gold standard for evaluating efficacy; however, they may contribute to research waste. This study examined the extent of research waste in RCTs involving preterm infants over the past two decades.

METHODS

This cross-sectional study searched ClinicalTrials.gov between 2001 and 2020 to identify RCTs involving preterm infants. Research waste was defined as the occurrence of any of the following: non-publication, poor reporting, or avoidable design deficiencies. We searched PubMed, Embase, and Google Scholar databases to determine publication status. The CONSORT checklist was used to evaluate the reporting adequacy. Design deficiency was identified based on the risk of bias, evaluated using the Cochrane tool, and the presence of a relevant systematic review.

RESULTS

A total of 100 RCTs were eligible for inclusion. The primary research focus was pulmonary diseases (28%), followed by nutritional (15%) and ophthalmological diseases. Seventy-eight of the 100 RCTs were published and these were likelier to have an enrollment size greater than 300 (26% vs. 5%, p = .038). Inadequate reporting was observed in 25 published RCTs, while 47 had design deficiencies. Overall, 69 of the 100 RCTs exhibited at least one feature of research waste. Having a primary investigator from North America or Europe (odds ratio [OR] 0.168, 95% confidence interval [CI] 0.040-0.711, p = .015) and an enrollment size greater than 300 (OR 0.074, 95% CI 0.018-0.304, p < .001) were independently associated with reduced research waste.

CONCLUSION

Nearly 70% of RCTs involving preterm infants exhibited features of research waste. However, large-scale RCTs conducted in North America and Europe were less likely to contribute to this issue.

CONSORT 2025 Statement: Updated Guideline for Reporting Randomized Trials

Importance

Well-designed and properly executed randomized trials are considered the most reliable evidence on the benefits of health care interventions. However, there is overwhelming evidence that the quality of reporting is not optimal. The CONSORT (Consolidated Standards of Reporting Trials) statement was designed to improve the quality of reporting and provides a minimum set of items to be included in a report of a randomized trial. CONSORT was first published in 1996, then updated in 2001 and 2010. Herein, we present the updated CONSORT 2025 statement, which aims to account for recent methodological advancements and feedback from end users.

Observations

We conducted a scoping review of the literature and developed a project-specific database of empirical and theoretical evidence related to CONSORT to generate a list of potential changes to the checklist. The list was enriched with recommendations provided by the lead authors of existing CONSORT extensions (harms, outcomes, nonpharmacological treatment), other related reporting guidelines (Template for Intervention Description and Replication [TIDieR]), and recommendations from other sources (eg, personal communications). The list of potential changes to the checklist was assessed in a large, international, online, 3-round Delphi survey involving 317 participants and discussed at a 2-day online expert consensus meeting of 30 invited international experts. We have made substantive changes to the CONSORT checklist. We added 7 new checklist items, revised 3 items, deleted 1 item, and integrated several items from key CONSORT extensions. We also restructured the CONSORT checklist, with a new section on open science. The CONSORT 2025 statement consists of a 30-item checklist of essential items that should be included when reporting the results of a randomized trial and a diagram for documenting the flow of participants through the trial. To facilitate implementation of CONSORT 2025, we have also developed an expanded version of the CONSORT 2025 checklist, with bullet points eliciting critical elements of each item.

Conclusions and Relevance

Authors, editors, reviewers, and other potential users should use CONSORT 2025 when writing and evaluating manuscripts of randomized trials to ensure that trial reports are clear and transparent.

"If we ask, we must act": co-designing the implementation of the EQ-5D-Y-5L as a Paediatric Patient Peported Outcome Measure in Routine hospital Outpatient Care for Kids to meaningfully impact clinical visits (P-PROM ROCK Phase 2)

PURPOSE

To co-design use of the EQ-5D-Y-5L, a generic Paediatric Patient Reported Outcome Measure (P-PROM), in Routine Outpatient Care for Kids (ROCK), maximising its impact on patient-clinician visits.

METHODS

This Phase 2 co-design study was guided by the co-design framework for public service design and Double Diamond model. Data collection involved facilitated workshops (building on Phase 1), followed by feedback and optimisation sessions. Participants included service providers (doctors, nurses, allied health and medical record staff), adolescents, and caregivers with lived experience of providing or receiving outpatient care at a tertiary paediatric hospital in Australia.

RESULTS

Five co-design workshops, nine feedback, and two optimisation sessions were conducted with nine service providers, two adolescents, and three caregivers. Co-design participants created resources to introduce EQ-5D-Y-5L as a 'general health tracking questionnaire' and explain its purpose. EQ-5D-Y-5L responses were designed to be displayed by item. A display of results over time was also designed. A patient empowerment approach was taken with regards to flagging specific EQ-5D-Y-5L items for discussion with clinicians, whereby patients or caregivers control which items are flagged. To ensure clinical review and action of EQ-5D-Y-5L responses, resources, including clinician training, clinician decision support tool, and matching patient booklet and resource pathway, were co-designed. Combined, these design elements make up the P-PROM ROCK Program.

CONCLUSION

Consumer engagement produced important insights that would've otherwise been missed, ensuring the P-PROM ROCK Program empowers patients. For generic P-PROMs to meaningfully impact patient-clinician visits, supports and resources are required to ensure clinical review and action.

Node-RADS: a systematic review and meta-analysis of diagnostic performance, category-wise malignancy rates, and inter-observer reliability

OBJECTIVE

To perform a systematic review and meta-analysis to estimate diagnostic performance, category-wise malignancy rates, and inter-observer reliability of Node Reporting and Data System 1.0 (Node-RADS).

METHODS

Five electronic databases were systematically searched for primary studies on the use of Node-RADS to report the possibility of cancer involvement of lymph nodes on CT and MRI from January 1, 2021, until April 15, 2024. The study quality was assessed by modified Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) and Quality Appraisal of Diagnostic Reliability (QAREL) tools. The diagnostic accuracy was estimated with bivariate random-effects model, while the pooled category-wise malignancy rates were obtained with random-effects model.

RESULTS

Six Node-RADS-CT studies and three Node-RADS-MRI studies covering nine types of cancer were included. The study quality was mainly damaged by inappropriate index test and unknown timing according to QUADAS-2, and unclear blindness during the rating process according to QAREL. The area under hierarchical summary receiver operating characteristic curve (95% conventional interval) was 0.92 (0.89-0.94) for Node-RADS ≥ 3 as positive and 0.91 (0.88-0.93) for Node-RADS ≥ 4 as positive, respectively. The pooled malignancy rates (95% CIs) of Node-RADS 1 to 5 were 4% (0-10%), 31% (9-58%), 55% (34-75%), 89% (73-99%), and 100% (97-100%), respectively. The inter-observer reliability of five studies was interpreted as fair to substantial.

CONCLUSION

Node-RADS presented a promising diagnostic performance with an increasing probability of malignancy along higher category. However, the evidence for inter-observer reliability of Node-RADS is insufficient, and may hinder its implementation in clinical practice for lymph node assessment.

KEY POINTS

Question Node-RADS is designed for structured reporting of the possibility of cancer involvement of lymph nodes, but the evidence supporting its application has not been summarized. Findings Node-RADS presented diagnostic performance with AUC of 0.92, and malignancy rates for categories 1-5 ranged from 4% to 100%, while the inter-observer reliability was unclear. Clinical relevance Node-RADS is a useful tool for structured reporting of the possibility of cancer involvement of lymph nodes with high diagnostic performance and appropriate malignancy rate for each category, but unclear inter-observer reliability may hinder its implementation in clinical practice.

CONSORT 2025 statement: updated guideline for reporting randomized trials

Well-designed and properly executed randomized trials are considered the most reliable evidence on the benefits of healthcare interventions. However, there is overwhelming evidence that the quality of reporting is not optimal. The CONSORT (Consolidated Standards of Reporting Trials) statement was designed to improve the quality of reporting and provides a minimum set of items to be included in a report of a randomized trial. CONSORT was first published in 1996, then updated in 2001 and 2010. Here, we present the updated CONSORT 2025 statement, which aims to account for recent methodological advancements and feedback from end users. We conducted a scoping review of the literature and developed a project-specific database of empirical and theoretical evidence related to CONSORT, to generate a list of potential changes to the checklist. The list was enriched with recommendations provided by the lead authors of existing CONSORT extensions (Harms, Outcomes, Non-Pharmacological Treatment), other related reporting guidelines (TIDieR) and recommendations from other sources (such as personal communications). The list of potential changes to the checklist was assessed in a large, international, online, three-round Delphi survey involving 317 participants and discussed at a two-day online expert consensus meeting of 30 invited international experts. We have made substantive changes to the CONSORT checklist. We added seven new checklist items, revised three items, deleted one item, and integrated several items from key CONSORT extensions. We also restructured the CONSORT checklist, with a new section on open science. The CONSORT 2025 statement consists of a 30-item checklist of essential items that should be included when reporting the results of a randomized trial and a diagram for documenting the flow of participants through the trial. To facilitate implementation of CONSORT 2025, we have also developed an expanded version of the CONSORT 2025 checklist, with bullet points eliciting critical elements of each item. Authors, editors, reviewers, and other potential users should use CONSORT 2025 when writing and evaluating manuscripts of randomized trials to ensure that trial reports are clear and transparent.

CONSORT 2025 statement: updated guideline for reporting randomised trials

Well designed and properly executed randomised trials are considered the most reliable evidence on the benefits of healthcare interventions. However, there is overwhelming evidence that the quality of reporting is not optimal. The CONSORT (Consolidated Standards of Reporting Trials) statement was designed to improve the quality of reporting and provides a minimum set of items to be included in a report of a randomised trial. CONSORT was first published in 1996, then updated in 2001 and 2010. Here, we present the updated CONSORT 2025 statement, which aims to account for recent methodological advancements and feedback from end users. We conducted a scoping review of the literature and developed a project-specific database of empirical and theoretical evidence related to CONSORT, to generate a list of potential changes to the checklist. The list was enriched with recommendations provided by the lead authors of existing CONSORT extensions (Harms, Outcomes, Non-pharmacological Treatment), other related reporting guidelines (TIDieR) and recommendations from other sources (eg, personal communications). The list of potential changes to the checklist was assessed in a large, international, online, three-round Delphi survey involving 317 participants and discussed at a two-day online expert consensus meeting of 30 invited international experts. We have made substantive changes to the CONSORT checklist. We added seven new checklist items, revised three items, deleted one item, and integrated several items from key CONSORT extensions. We also restructured the CONSORT checklist, with a new section on open science. The CONSORT 2025 statement consists of a 30-item checklist of essential items that should be included when reporting the results of a randomised trial and a diagram for documenting the flow of participants through the trial. To facilitate implementation of CONSORT 2025, we have also developed an expanded version of the CONSORT 2025 checklist, with bullet points eliciting critical elements of each item. Authors, editors, reviewers, and other potential users should use CONSORT 2025 when writing and evaluating manuscripts of randomised trials to ensure that trial reports are clear and transparent.

CONSORT 2025 statement: updated guideline for reporting randomised trials

BACKGROUND

Well designed and properly executed randomised trials are considered the most reliable evidence on the benefits of healthcare interventions. However, there is overwhelming evidence that the quality of reporting is not optimal. The CONSORT (Consolidated Standards of Reporting Trials) statement was designed to improve the quality of reporting and provides a minimum set of items to be included in a report of a randomised trial. CONSORT was first published in 1996, then updated in 2001 and 2010. Here, we present the updated CONSORT 2025 statement, which aims to account for recent methodological advancements and feedback from end users.

METHODS

We conducted a scoping review of the literature and developed a project-specific database of empirical and theoretical evidence related to CONSORT, to generate a list of potential changes to the checklist. The list was enriched with recommendations provided by the lead authors of existing CONSORT extensions (Harms, Outcomes, Non-pharmacological Treatment), other related reporting guidelines (TIDieR) and recommendations from other sources (eg, personal communications). The list of potential changes to the checklist was assessed in a large, international, online, three-round Delphi survey involving 317 participants and discussed at a two-day online expert consensus meeting of 30 invited international experts.

RESULTS

We have made substantive changes to the CONSORT checklist. We added seven new checklist items, revised three items, deleted one item, and integrated several items from key CONSORT extensions. We also restructured the CONSORT checklist, with a new section on open science. The CONSORT 2025 statement consists of a 30-item checklist of essential items that should be included when reporting the results of a randomised trial and a diagram for documenting the flow of participants through the trial. To facilitate implementation of CONSORT 2025, we have also developed an expanded version of the CONSORT 2025 checklist, with bullet points eliciting critical elements of each item.

CONCLUSION

Authors, editors, reviewers, and other potential users should use CONSORT 2025 when writing and evaluating manuscripts of randomised trials to ensure that trial reports are clear and transparent.

Incomplete reporting and spin in acupuncture randomised controlled trials: a cross-sectional meta-epidemiological study

OBJECTIVES

To investigate the reporting, data sharing and spin (using reporting strategies to emphasise the benefit of non-significant results) in acupuncture randomised controlled trials (RCTs).

DESIGN

Cross-sectional meta-epidemiological study.

DATA SOURCES

Eligible studies indexed in MEDLINE, Embase, CENTRAL, CBM, CNKI, Wanfang Data and VIP Database between 1 January 2014 and 1 May 2024.

ELIGIBILITY CRITERIA

Peer-reviewed acupuncture RCTs used traditional medicine (TM), published in English or Chinese, two parallel arms for humans.

MAIN OUTCOME MEASURES

We assessed (1) the reporting of acupuncture RCTs by the Consolidated Standards for Reporting Trials (CONSORT) 2010 statement and STandards for Reporting Interventions in Clinical Trials of Acupuncture (STRICTA) checklist; (2) the data sharing level by the International Committee of Medical Journal Editors (ICMJE) data sharing statement; (3) spin frequency and level by the prespecified spin strategies.

RESULTS

This study evaluated 476 eligible studies, of which 166 (34.9%) explored the specific efficacy or safety of acupuncture in the nervous system, 68 (14.3%) in the motor system and 61 (12.8%) in the digestive system. 244 (57.7%) studies used conventional acupuncture, 296 (62.2%) used multicentre study design and 369 (77.5%) were supported by institutional funding. 312 (65.5%) eligible studies were poorly reported. The sufficiently reporting scores of the CONSORT 2010 statement and the STRICTA checklist differed from 0.63% to 97.5%, and 32 (59.3%) items were less than 50%. For the data sharing level of acupuncture RCTs, only 66 (17.2%) studies followed the ICMJE data sharing statement, but 49 (14.5%) need to require authors to obtain data, and only 5 (1.5%) provided data by open access. Spins were identified in 408 (85.7%) studies (average spin frequencies: 2.94). 59 (37.2%) studies with non-significant primary outcomes had spin levels.

CONCLUSIONS

This study found that the reporting of acupuncture RCTs was low compliance with the CONSORT 2010 statement, the STRICTA checklist and the ICMJE data sharing statement, and spin appeared frequently. Journal policies on using reporting guidelines, data sharing and equitable consideration of non-significant results might enhance the reporting of acupuncture RCTs.

TRIAL REGISTRATION NUMBER

This study was registered at the Open Science Framework (OSF): (https://doi.org/10.17605/OSF.IO/2WTE6, and https://doi.org/10.17605/OSF.IO/9XDN4,).

Reducing risk of bias in interventional studies during their design and conduct: a scoping review

BACKGROUND

Interventional studies are intended to provide robust evidence. Yet poorly designed or conducted studies may bias research results and skew resulting evidence. While there have been advances in the assessment of risk of bias, it is unclear how to intervene against risks of bias during study design and conduct.

OBJECTIVE

To identify interventions to reduce or predict risk of bias in interventional studies during their design and conduct.

SEARCH STRATEGY

For this scoping review, we searched three electronic bibliographic databases (MEDLINE, Embase, and Cochrane Library) and nine grey literature sources and Google from in September 2024. This was supplemented by a natural language processing fuzzy matching search of the top 2000 relevant publications in the electronic bibliographic databases. Publications were included if they described the implementation and effectiveness of an intervention during study design or conduct aimed at reducing risk of bias in interventional studies. The characteristics and effect of the interventions were recorded.

RESULT

We identified, and reviewed the title and abstracts of, a total of 41,793 publications, reports, documents and grey literature, with 24,677 from electronic bibliographic databases and 17,140 from grey literature sources. There were 67 publications from bibliographic databases and 24 items from grey literature that were considered potentially eligible for inclusion, and the full-text of these were reviewed. Only three studies met the inclusion criteria. The first intervention was offering education and training to researchers during study design. This training included the implementation of a more rigorous participant screening process and systematic participant tracking program that reduced loss to follow-up and missing data, particularly for long-term follow-up trials. The second intervention was introducing an independent clinical events committee during study conduct. This was intended to mitigate bias due to conflicts of interest affecting the analysis and interpretation of results. The third intervention was to provide participants with financial incentives in randomized controlled trials, so that participants could more actively accomplish the requirements of the trials.

CONCLUSION

Despite the major impact of risk of bias on study outcomes, there are few empirical interventions to address this during study design or conduct.

CONSORT 2025 statement: Updated guideline for reporting randomised trials

BACKGROUND

Well designed and properly executed randomised trials are considered the most reliable evidence on the benefits of healthcare interventions. However, there is overwhelming evidence that the quality of reporting is not optimal. The CONSORT (Consolidated Standards of Reporting Trials) statement was designed to improve the quality of reporting and provides a minimum set of items to be included in a report of a randomised trial. CONSORT was first published in 1996, then updated in 2001 and 2010. Here, we present the updated CONSORT 2025 statement, which aims to account for recent methodological advancements and feedback from end users.

METHODS

We conducted a scoping review of the literature and developed a project-specific database of empirical and theoretical evidence related to CONSORT, to generate a list of potential changes to the checklist. The list was enriched with recommendations provided by the lead authors of existing CONSORT extensions (Harms, Outcomes, Non-pharmacological Treatment), other related reporting guidelines (TIDieR) and recommendations from other sources (e.g., personal communications). The list of potential changes to the checklist was assessed in a large, international, online, three-round Delphi survey involving 317 participants and discussed at a two-day online expert consensus meeting of 30 invited international experts.

RESULTS

We have made substantive changes to the CONSORT checklist. We added seven new checklist items, revised three items, deleted one item, and integrated several items from key CONSORT extensions. We also restructured the CONSORT checklist, with a new section on open science. The CONSORT 2025 statement consists of a 30-item checklist of essential items that should be included when reporting the results of a randomised trial and a diagram for documenting the flow of participants through the trial. To facilitate implementation of CONSORT 2025, we have also developed an expanded version of the CONSORT 2025 checklist, with bullet points eliciting critical elements of each item.

CONCLUSIONS

Authors, editors, reviewers, and other potential users should use CONSORT 2025 when writing and evaluating manuscripts of randomised trials to ensure that trial reports are clear and transparent.

Reporting of Noninferiority Margins on ClinicalTrials.gov: A Systematic Review

Importance

The noninferiority margin is a key methodological parameter in noninferiority trials that affects both sample size calculation and interpretation of results.

Objective

To assess (1) the reporting of the noninferiority margin on ClinicalTrials.gov, including when it was reported; (2) the consistency of the noninferiority margin between registration and publication; and (3) the reporting of the noninferiority margin at registration in a sample of recent trials.

Evidence Review

This systematic review was conducted in 2 stages. Stage 1 involved identifying all noninferiority trials registered on ClinicalTrials.gov with a primary completion date of January 1, 2010, to January 1, 2015, and searching for corresponding publications. Stage 2 included all noninferiority trials registered between January 1, 2022, and June 30, 2023. Two reviewers independently and manually extracted methodological characteristics related to the noninferiority design for each trial at registration and, when available, in results posted on ClinicalTrials.gov and in publications. The time points for reporting the noninferiority margin were at registration, during the patient enrollment phase (between start date and primary completion date), after the primary completion date, or in results posted.

Findings

Among the 266 trials completed between 2010 and 2015, only 8 (3.0%) reported the noninferiority margin at registration. The noninferiority margin was reported after registration for 31 of 266 trials (11.7%), with 11 (4.1%) reporting it during the patient enrollment phase and 20 (7.5%) reporting it after the primary completion date. Of the 132 trials with results posted on ClinicalTrials.gov, 79 (59.8%) reported the noninferiority margin. A corresponding publication was found for 208 trials, with 196 (94.2%) reporting the noninferiority margin. For 5 trials reporting the margin at both registration and in publication, the noninferiority margin was consistent in both sources. Among the 327 noninferiority trials first posted between 2022 and 2023, 30 (9.2%) reported the noninferiority margin at registration.

Conclusions and Relevance

In this systematic review, the reporting of the noninferiority margin on ClinicalTrials.gov was low. Mandatory reporting of the design and the noninferiority margin at registration could enhance the transparency and favor more reliable results.

Epidemiology and reporting characteristics of systematic reviews of clinical prediction models: a scoping review

OBJECTIVES

This study aimed to explore research trends and areas of interest in systematic reviews (SRs) and meta-analysis of clinical prediction models (CPMs), while summarizing their conduct and reporting characteristics.

STUDY DESIGN AND SETTING

A scoping review was conducted, with searches performed in PubMed, Embase, and Cochrane Library from inception to January 7, 2023. Pairs of reviewers independently screened potentially eligible studies. Data on bibliographic and methodological characteristics were collected and analyzed descriptively.

RESULTS

A total of 1004 SRs published between 2001 and 2023 were included. The number of SRs increased significantly after 2020, with the majority originating from Europe (44.1%) and Asia (26.7%). Populations and outcomes were categorized into 19 and 34 classifications, respectively. The general population was the most frequently targeted (38.7%), and mortality was the most common outcome (18.9%). The prediction or diagnosis of neoplasms in the general population was the most prevalent focus (7.2%). Prognostic models were included only in 69.6% of SRs, while diagnostic models were included in 16.8%; 13.6% included both. The number of primary studies included in SRs ranged from 1 to 495, and the models ranged from 1 to 731. Most SRs lacked standardized reporting: 88.3% did not frame their review questions using established frameworks, and 79.8% did not follow standardized checklists for data extraction. Quality and risk of bias assessments were reported in 76.5% of SRs, with the Prediction model Risk of Bias Assessment Tool (27.9%) and the Quality Assessment of Diagnostic Accuracy Studies-2 tool (17.0%) being the most common. Narrative synthesis was the predominant method for evidence summarization (63.5%), while meta-analysis was conducted in 36.5%. Measures of model performance were summarized in 80.5% of SRs, with discrimination being the most frequently reported (67.7%). Only 5.2% assessed the certainty of evidence. Moreover, 42.2% of SRs published a protocol, 76.0% clearly stated support, and 91.1% stated competing interests.

CONCLUSION

The number of SRs of CPMs has grown substantially, with increasing diversity in populations and outcomes. However, significant variability in conduct and reporting was observed. Future SRs should strictly follow well-developed guidelines, and a dedicated study assessing the reporting quality and risk of bias in SRs of CPMs is warranted.

HOW TO REPORT OUTCOMES IN CLINICAL DENTAL RESEARCH

Outcomes, also known as endpoints, are a critical component in clinical research evaluating the effects of healthcare interventions. The validity of a clinical study depends on the appropriate selection and usage of outcomes. Therefore, complete, accurate, and transparent reporting of outcomes is essential for the critical appraisal of a study's methods and findings. However, empirical research has shown that the reporting of outcomes is often incomplete and selective in clinical dental research, hindering evidence synthesis and evidence-based dental practice. To improve and standardize outcome reporting, reporting guidelines that provide specific guidance for all types of outcomes, namely the SPIRIT-Outcomes 2022 and CONSORT-Outcomes 2022, have been developed and released recently. In addition, reporting guidelines for certain types of outcomes have also been published, including harms, patient-reported outcomes (PROs), and surrogate outcomes. The present article describes common classifications of outcomes, current issues in outcome reporting, and using reporting guidelines to standardize and improve outcome reporting in clinical dental research. The role of core outcome sets in outcome reporting is also discussed. This article aims to provide guidance and suggestions to help improve the completeness and transparency of outcome reporting and reduce relevant research waste in clinical dental research.

A call for transparency, improved reporting, and interpretation of trials using surrogate end points in cardiac electrophysiology

In this call for transparency, we aim to disseminate knowledge about recent CONSORT-Surrogate and SPIRIT-Surrogate checklists. SPIRIT-Surrogate is an extension of the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) checklist, developed as a consensus document and designed as a reporting guideline for randomized controlled trial (RCT) protocols using surrogate end points as the primary end points. CONSORT-Surrogate is an extension of the Consolidated Standards of Reporting Trials (CONSORT) checklist, a consensus-driven reporting guideline for RCTs using surrogate end points as the primary end points. We discuss the definition of surrogate and target/final end points in cardiac electrophysiology (EP) RCTs. We review historical examples of surrogate paradoxes in the cardiac EP field and consider the theoretical framework of a surrogate paradox. Furthermore, we consider the bioethics of transparent reporting of clinical trial protocols and results. A nonlethal cardiac arrhythmia and its burden (eg, atrial fibrillation, premature ventricular contractions, nonsustained ventricular tachycardia) is a surrogate end point unless justified otherwise. Therefore, clinical investigators in the cardiac EP field need to implement the SPIRIT-Surrogate and CONSORT-Surrogate extension checklists for transparent reporting.

Improving research transparency with individualized report cards: A feasibility study in clinical trials at a large university medical center

BACKGROUND

Research transparency is crucial for ensuring the relevance, integrity, and reliability of scientific findings. However, previous work indicates room for improvement across transparency practices. The primary objective of this study was to develop an extensible tool to provide individualized feedback and guidance for improved transparency across phases of a study. Our secondary objective was to assess the feasibility of implementing this tool to improve transparency in clinical trials.

METHODS

We developed study-level "report cards" that combine tailored feedback and guidance to investigators across several transparency practices, including prospective registration, availability of summary results, and open access publication. The report cards were generated through an automated pipeline for scalability. We also developed an infosheet to summarize relevant laws, guidelines, and resources relating to transparency. To assess the feasibility of using these tools to improve transparency, we conducted a single-arm intervention study at Berlin's university medical center, the Charité - Universitätsmedizin Berlin. Investigators (n = 92) of 155 clinical trials were sent individualized report cards and the infosheet, and surveyed to assess their perceived usefulness. We also evaluated included trials for improvements in transparency following the intervention.

RESULTS

Survey responses indicated general appreciation for the report cards and infosheet, with a majority of participants finding them helpful to build awareness of the transparency of their trial and transparency requirements. However, improvement on transparency practices was minimal and largely limited to linking publications in registries. Investigators also commented on various challenges associated with implementing transparency, including a lack of clarity around best practices and institutional hurdles.

CONCLUSIONS

This study demonstrates the potential of developing and using tools, such as report cards, to provide individualized feedback at scale to investigators on the transparency of their study. While these tools were positively received by investigators, the limited improvement in transparency practices suggests that awareness alone is likely not sufficient to drive improvement. Future research and implementation efforts may adapt the tools to further practices or research areas, and explore integrated approaches that combine the report cards with incentives and institutional support to effectively strengthen transparency in research.

The Power of Prehabilitation, the Reporting of Power Calculations in Randomized Clinical Trials Evaluating Prehabilitation in Cancer Surgery: A Systematic Review and Meta-research Study

OBJECTIVE

To assess sample size calculation reporting in randomized controlled trials (RCTs) investigating prehabilitation interventions in oncological surgery patients.

DATA SOURCES

A systematic literature search was performed in multiple medical databases from inception to April 2023, including MEDLINE, Embase, The Cochrane Library, CINHAL, AMED, and PsychINFO.

STUDY SELECTION

The inclusion criteria used were RCTs evaluating effectiveness of exercise, nutrition, and/or psychological interventions on postoperative outcomes of adult patients undergoing oncological surgery.

DATA EXTRACTION

Two authors (DS and SV) extracted information on the sample size calculation parameters, including type I error (α), power (1-β), mean (or mean difference between randomization arms), and variance (eg, standard deviation) for continuous outcomes, and event rates or event rate difference between randomization arms for dichotomous outcomes. When possible, we recalculated the sample size required using the collected data, given a 10% margin of error.

DATA SYNTHESIS

Of the 59 included publications (58 RCTs), 26 (44%) reported sufficient information to complete sample size recalculation. Of those that provided sufficient information allowing us to recalculate the required sample size, 11 (42%) were within a 10% margin of the reported sample size, whereas 9 (35%) were >10% higher than reported sample size and 6 (23%) were >10% lower than reported sample size.

CONCLUSIONS

Over half of the published RCTs in this field exhibit poor sample size calculation reporting. Most RCTs that report sufficient sample size information were underpowered. More stringent reporting requirements are necessary.

The case for development of a core outcome set (COS) and supplemental reporting guidelines for influenza vaccine challenge trial research in swine

Previously, we systematically reviewed more than 20 years of influenza vaccine challenge trial research in pigs to answer the question, "does vaccinating sows protect offspring?" Overall, most studies were well designed but clinical heterogeneity made between-study comparisons challenging. Studies varied by samples, outcomes, and assays selected for measurement. Additionally, data essential for inclusion of findings in meta-analyses were often insufficiently reported and as a result, summary effect measures were either not derived or were not meaningful. Clinical heterogeneity and reporting issues complicate and limit what can be learned cumulatively from research and both represent two types of avoidable research waste. Here, we illustrate each concern using data collected tangentially during the systematic review and propose two corrective strategies, both of which have broad applicability across veterinary intervention research; (i) develop a Core Outcome Set (COS) to reduce unnecessary clinical heterogeneity in future research and (ii) encourage funders and journal editors to require submitted research protocols and manuscripts adhere to established reporting guidelines. As a reporting corollary, we developed a supplemental checklist specific to influenza vaccine challenge trial research in swine and propose that it is completed by researchers and included with all study protocol and manuscript submissions. The checklist serves two purposes: as a reminder of details essential to report for inclusion of findings in meta-analyses and sub-group meta-analyses (e.g., antigenic or genomic descriptions of influenza vaccine and challenge viruses), and as an aid to help synthesis researchers fully characterize and comprehensively include studies in reviews.

Poor hypotheses and research waste in biology: learning from a theory crisis in psychology

While psychologists have extensively discussed the notion of a "theory crisis" arising from vague and incorrect hypotheses, there has been no debate about such a crisis in biology. However, biologists have long discussed communication failures between theoreticians and empiricists. We argue such failure is one aspect of a theory crisis because misapplied and misunderstood theories lead to poor hypotheses and research waste. We review its solutions and compare them with methodology-focused solutions proposed for replication crises. We conclude by discussing how promoting inclusion, diversity, equity, and accessibility (IDEA) in theoretical biology could contribute to ameliorating breakdowns in the theory-empirical cycle.

An Appraisal of the Quality of Development and Reporting of Predictive Models in Neurosurgery: A Systematic Review

BACKGROUND AND OBJECTIVES

Significant evidence has indicated that the reporting quality of novel predictive models is poor because of confounding by small data sets, inappropriate statistical analyses, and a lack of validation and reproducibility. The Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD) statement was developed to increase the generalizability of predictive models. This study evaluated the quality of predictive models reported in neurosurgical literature through their compliance with the TRIPOD guidelines.

METHODS

Articles reporting prediction models published in the top 5 neurosurgery journals by SCImago Journal Rank-2 (Neurosurgery, Journal of Neurosurgery, Journal of Neurosurgery: Spine, Journal of NeuroInterventional Surgery, and Journal of Neurology, Neurosurgery, and Psychiatry) between January 1st, 2018, and January 1st, 2023, were identified through a PubMed search strategy that combined terms related to machine learning and prediction modeling. These original research articles were analyzed against the TRIPOD criteria.

RESULTS

A total of 110 articles were assessed with the TRIPOD checklist. The median compliance was 57.4% (IQR: 50.0%-66.7%). Models using machine learning-based models exhibited lower compliance on average compared with conventional learning-based models (57.1%, 50.0%-66.7% vs 68.1%, 50.2%-68.1%, P = .472). Among the TRIPOD criteria, the lowest compliance was observed in blinding the assessment of predictors and outcomes (n = 7, 12.7% and n = 10, 16.9%, respectively), including an informative title (n = 17, 15.6%) and reporting model performance measures such as confidence intervals (n = 27, 24.8%). Few studies provided sufficient information to allow for the external validation of results (n = 26, 25.7%).

CONCLUSION

Published predictive models in neurosurgery commonly fall short of meeting the established guidelines laid out by TRIPOD for optimal development, validation, and reporting. This lack of compliance may represent the minor extent to which these models have been subjected to external validation or adopted into routine clinical practice in neurosurgery.

Is CONSORT a Risk of Bias Tool for Experimental Studies: A Big Misunderstanding

Introduction

In order to avoid bias in systematic reviews (SRs), the accuracy in selection of risk of bias (RoB) assessment tools is essential to obtain highest level of evidence for evidence-based decision making in health care. We aimed to review how 'CONSORT statement', as a reporting quality of randomized controlled trials, has been misused in recent SRs as a ROB tool.

Methods

A mini-review was performed in international databases including PubMed, Google Scholar and hand-searches for published and unpublished literature from 2000 to 2021 and written in English. The following keywords: risk of bias, "Consolidated Standards of Reporting Trials", CONSORT, "systematic review" were used. Citations were screened and those meeting our inclusion criteria were retained.

Results

A total of 11 SRs were identified that misused CONSORT as a ROB tool, four of which were used only CONSORT as ROB tool. Different authentic magazines from various countries were recognized.

Conclusion

The CONSORT statement aims to increase clarity and consistency of transparency of reporting in randomized controlled trials. It is quite essential to draw the attention of SR researchers, journal editors/reviewers as well as the reading audience to the fact that CONSORT statement CONSORT statement is not a ROB tool.

Home-based physical symptom management for family caregivers: systematic review and meta-analysis

BACKGROUND

Patients with life-limiting conditions are often cared for at home by family, typically without adequate training to carry out the challenging tasks performed. This systematic review assessed the efficacy of interventions designed to help family caregivers manage pain and other symptoms in adults and children with life-limiting conditions at home.

METHODS

A systematic search was performed on seven databases. A narrative synthesis was conducted, along with a meta-analysis comparing outcomes in those who received an intervention to those who did not, or to preintervention scores.

RESULTS

84 eligible studies were identified. Significant improvements in pain and fatigue in patients with cancer were found compared with patients in the control group and baseline. Caregivers of patients with cancer receiving an intervention, compared with the control group caregivers, showed significant improvements in self-efficacy and active coping and lower avoidant coping. This group also showed significant improvements in burden, self-efficacy, anxiety and depression, and decreases in avoidant coping pre- to post intervention. Patients with dementia whose caregivers received an intervention showed significantly reduced pain intensity and improvements in quality of life pre- to post intervention. Caregivers of patients with dementia showed significantly reduced distress pre- to post intervention. No beneficial effects were found for caregivers of patients with Parkinson's disease or heart failure, although only limited analyses could be performed.

CONCLUSIONS

Interventions targeting family caregivers can improve both patient symptoms and caregiver outcomes, as demonstrated in cancer and dementia care. Future mixed-methods research should collect data from caregiver and patient dyads, identifying key intervention components. There is also need for more studies on caregivers of paediatric patients.

Does drug dispensing influence patients' medication knowledge and medication adherence? A systematic review and meta-analysis

BACKGROUND

Inadequate medication knowledge and medication nonadherence by patients are considered an issue in healthcare, as they can lead to negative outcomes, such as therapeutic failures and hospitalization. Even though drug dispensing, which has pharmacist counseling as a core element, is a service traditionally performed by pharmacists, there is still no evidence about the influence of this service on these health outcomes.

OBJECTIVE

To evaluate the influence of drug dispensing on patients' medication knowledge and medication adherence.

METHODS

A systematic review was conducted in which a literature search was performed in the PubMed/Medline, Biblioteca Virtual da Saúde, Web of Science, and Embase databases, as well as in gray literature. Two reviewers read the titles, abstracts and complete texts according to the eligibility criteria and extracted the data from the included articles. Original studies-of any design-evaluating the influence of drug dispensing on patients' medication knowledge and/or adherence in community pharmacies were included. The methodological quality was assessed through the tools provided by the JBI Institute. The data was analyzed through qualitative synthesis and a meta-analysis was conducted for randomized controlled trials which used the outcome of medication adherence using the RStudio version 4.3.3 program.

RESULTS

A total of 7,590 studies were identified in the initial search, of which 11 articles met the eligibility criteria and were included in this systematic review. The studies were published in Africa, Latin America, Asia, Europe and Australia. Most of the studies were interventional (n = 7). Four studies evaluated the influence of drug dispensing on the patient's medication knowledge, and all showed that knowledge increased after dispensing. Eight studies evaluated the influence of dispensing on medication adherence. Three studies were included in the meta-analysis, which showed moderate heterogeneity (I2 = 44%, p = 0.17). The results indicated that there was no statistically significant difference in medication adherence post-dispensing (RR: 1.19; 95%CI 0.99 to 1.43, p = 0.07). Six studies met more than 70% of the quality assessment criteria.

CONCLUSION

This systematic review demonstrated that patient's medication knowledge can be increased through drug dispensing. However, the meta-analysis indicated that drug dispensing does not have an impact on medication adherence. Our findings can support evidence-based decisions, guiding the planning and development of public policies and interventions which improve drug dispensing for patients, families, and communities.

The Songdo consensus: Development of minimum reporting standards for studies of intervention in idiopathic anal fistula using a modified nominal group technique

AIM

Cryptoglandular anal fistulas carry a substantial burden to quality of life. Surgery is the only effective curative treatment but requires balancing fistula healing against pain, wounds and continence impairment. Sphincter-preserving procedures do exist but demonstrate variable rates of success. A lack of consistency and precision in outcome reporting and methodological quality hinders effective evidence-based decision-making. We aimed to establish a series of minimum reporting standards for interventional studies in idiopathic anal fistula, to eradicate low-quality studies, thus providing a consistent baseline of useful evidence.

METHODS

An international group of 16 experts participated in a modified nominal group technique consensus. The nominal question was: 'What should be the minimum set of reporting standards for studies of intervention in idiopathic anal fistula?' The process was conducted between May and June 2023, culminating in a hybrid in-person/virtual meeting that took place at the Songdo International Proctology Symposium in June 2023.

RESULTS

Initial idea generation resulted in 37 statements within the first round. Themes included variable reporting of follow-up and incontinence. Participants indicated their agreement via a 9-point Likert scale. Any statement achieving >70% consensus was retained. Subsequent group discussion condensed the list to 11 statements for further voting and a final minimum set of 12 reporting standards was created.

CONCLUSION

To date, this is the first study dedicated to developing minimum reporting standards for interventional studies in idiopathic anal fistula using a modified nominal group technique. These standards will instruct researchers in producing meticulous, high-quality studies that are accurate, transparent and reproducible.

Deep Learning-Based Image Restoration and Super-Resolution for Fluorescence Microscopy: Overview and Resources

Fluorescence microscopy is a key method for the visualization of cellular, subcellular, and molecular live-cell dynamics, enabling access to novel insights into mechanisms of health and disease. However, effects like phototoxicity, the fugitive nature of signals, photo bleaching, and method-inherent noise can degrade the achievable signal-to-noise ratio and image resolution. In recent years, deep learning (DL) approaches have been increasingly applied to remove these degradations. In this review, we give a brief overview over existing classical and DL approaches for denoising, deconvolution, and computational super-resolution of fluorescence microscopy data. We summarize existing open-source databases within these fields as well as code repositories related to corresponding publications and further contribute an example project for DL-based image denoising, which provides a low barrier entry into DL coding and respective applications. In summary, we supply interested researchers with tools to apply or develop DL applications in live-cell imaging and foster research participation in this field.

Developing a mobile-based brief intervention to reduce cannabis-impaired driving among youth: An intervention mapping approach

Behaviour change interventions delivered via smartphones have the potential to reduce youth cannabis use and driving under the influence of cannabis (DUIC). Countless smartphone applications (either downloadable or web-based) are available to help reduce substance use and impaired driving. However, most are developed without evidence-based content and theory, and many have poor user engagement. This study aims to: (1) describe the systematic development and theoretical foundations of a youth DUIC smartphone intervention, and (2) describe the pre-testing among a sample of youth and adult cannabis educators (prior to efficacy testing). A 6-step Intervention Mapping approach was utilized to combine theory, evidence, and user feedback to develop and implement the 'High Alert' intervention. This evidence-based and iterative process entailed: (1) conducting a needs assessment, (2) identifying intervention objectives, which map on the following DUIC determinants: knowledge, attitudes, risk perception, perceived norms, and self-efficacy, (3) selecting intervention theory and design, (4) developing of the intervention, (5) implementation, and (6) evaluation. Application of Intervention Mapping resulted in a smartphone web-based application that could support reductions in cannabis use and DUIC. The 'High Alert' intervention was created to include four modules with contents focusing on educating youth on the dangers and legal risks of DUIC, limiting risky situations, avoiding riding with an impaired driver, planning a safe ride home, and promoting safer cannabis use. Future research will test the efficacy of the intervention in reducing risky cannabis use and DUIC among youth.

Evaluating research waste and traits among randomized controlled trials of scars over the past 20 years: a cross-sectional study

PURPOSE

To analyze the changes in the characteristics of randomized controlled trials (RCTs) in the field of scarring over the last two decades, unveil the components of research waste (RW) within these RCTs, and identify targets for improvement.

METHODS

A search was conducted on ClinicalTrials.gov for RCTs registered from January 2000 to December 2023, using "scar" as the keyword. The search was carried out in January 2024.

RESULTS

391 RCTs were included in this analysis. The global registration of RCTs in scarring has exhibited a consistent increase annually, with the proportion in Asia gradually rising, while the shares in North America and Europe have demonstrated a declining trend. In the analysis of RW, 232 RCTs were included, of which 96 (41.4%) have been published. Among the published RCTs, 56 (58.3%) were evaluated to have sufficient reporting, while 47 RCTs (48.9%) were identified as having avoidable design flaws. Ultimately, 183 RCTs (78.9%) exhibited at least one form of RW. Multicenter design (OR: 3.324, 95%CI: 1.385-7.975, P = 0.018), non-pharmacological interventions (OR: 2.61, 95%CI: 1.253-5.435, P = 0.010), the absence of external funding (OR: 0.325, 95%CI: 0.144-0.732, P = 0.031), and participant numbers exceeding 50 (OR: 3.269, 95%CI: 1.573-6.794, P = 0.002) were identified as independent protective factors against waste.

CONCLUSIONS

This study delineates the changes in the characteristics of scar RCTs globally over the past two decades, uncovering a substantial burden of RW in scarring research. It provides an evidential reference for more rational planning of future scar-related RCTs and for minimizing RW.

Several methods for assessing research waste in reviews with a systematic search: a scoping review

Background

Research waste is present in all study designs and can have significant consequences for science, including reducing the reliability of research findings and contributing to the inefficient use of resources. Estimates suggest that as much as 85% of all biomedical research is wasted. However, it is uncertain how avoidable research waste is assessed in specific types of study designs and what methods could be used to examine different aspects of research waste. We aimed to investigate which methods, systematic reviews, scoping reviews, and overviews of reviews discussing research waste, have used to assess avoidable research waste.

Materials and Methods

We published a protocol in the Open Science Framework prospectively (https://osf.io/2fbp4). We searched PubMed and Embase with a 30-year limit (January 1993-August 2023). The concept examined was how research waste and related synonyms (e.g., unnecessary, redundant, duplicate, etc.) were assessed in reviews with a systematic search: systematic, scoping, or overviews of reviews. We extracted data on the method used in the review to examine for research waste and for which study design this method was applied.

Results

The search identified 4,285 records of which 93 reviews with systematic searches were included. The reviews examined a median of 90 (range 10-6,781) studies, where the study designs most commonly included were randomized controlled trials (48%) and systematic reviews (33%). In the last ten years, the number of reports assessing research waste has increased. More than 50% of examined reviews reported evaluating methodological research waste among included studies, typically using tools such as one of Cochrane Risk of Bias tools (n = 8) for randomized controlled trials or AMSTAR 1 or 2 (n = 12) for systematic reviews. One fourth of reviews assessed reporting guideline adherence to e.g., CONSORT (n = 4) for randomized controlled trials or PRISMA (n = 6) for systematic reviews.

Conclusion

Reviews with systematic searches focus on methodological quality and reporting guideline adherence when examining research waste. However, this scoping review revealed that a wide range of tools are used, which may pose difficulties in comparing examinations and performing meta-research. This review aids researchers in selecting methodologies and contributes to the ongoing discourse on optimizing research efficiency.

The BSSO Foundry: A community of practice for ontologies in the behavioural and social sciences

There has been a rapid expansion in the quantity and complexity of data, information and knowledge created in the behavioural and social sciences, yet the field is not advancing understanding, practice or policy to the extent that the insights warrant. One challenge is that research often progresses in disciplinary silos and is reported using inconsistent and ambiguous terminology. This makes it difficult to integrate and aggregate findings to produce cumulative bodies of knowledge that can be translated to applied settings. Ontologies can address these challenges; their development and use have the potential to accelerate the behavioural and social sciences. Ontologies can facilitate communication through precise specification and dissemination of terms, and enable efficient data integration, sharing, comparison and analysis. The widespread use of ontologies in the biomedical and biological sciences has led to multiple successes. It is time now for the behavioural and social sciences to follow that lead. In recent years, a number of ontologies have been developed within the behavioural and social sciences; however, efforts have tended to be isolated, with limited resources to support developers and those who work (or would like to work) with and use ontologies. There is a need for coordination and exchange to reduce duplication of work and leverage the value of a community to support the interoperability of these ontologies (linking of entities across domains and datasets). We have therefore initiated the Behavioural and Social Sciences Ontology (BSSO) Foundry, a community of practice and online repository for the development, adoption and use of ontologies in the behavioural and social sciences. The BSSO Foundry aligns with and builds upon the model provided by the Open Biological and Biomedical Ontology Foundry. We describe this new initiative and how to join and contribute to the community of interoperable ontologies for the behavioural and social sciences.

Checking the inventory: Illustrating different methods for individual participant data meta-analytic structural equation modeling

Researchers may have at their disposal the raw data of the studies they wish to meta-analyze. The goal of this study is to identify, illustrate, and compare a range of possible analysis options for researchers to whom raw data are available, wanting to fit a structural equation model (SEM) to these data. This study illustrates techniques that directly analyze the raw data, such as multilevel and multigroup SEM, and techniques based on summary statistics, such as correlation-based meta-analytical structural equation modeling (MASEM), discussing differences in procedures, capabilities, and outcomes. This is done by analyzing a previously published collection of datasets using open source software. A path model reflecting the theory of planned behavior is fitted to these datasets using different techniques involving SEM. Apart from differences in handling of missing data, the ability to include study-level moderators, and conceptualization of heterogeneity, results show differences in parameter estimates and standard errors across methods. Further research is needed to properly formulate guidelines for applied researchers looking to conduct individual participant data MASEM.

The trinity of good research: Distinguishing between research integrity, ethics, and governance

The words integrity, ethics, and governance are used interchangeably in relation to research. This masks important differences that must be understood when trying to address concerns regarding research culture. While progress has been made in identifying negative aspects of research culture (such as inequalities in hiring/promotion, perverse incentives, etc.) and practical issues that lead to research waste (outcome reporting bias, reproducibility, etc.), the responsibility for addressing these problems can be unclear due to the complexity of the research environment. One solution is to provide a clearer distinction between the perspectives of "Research Integrity," "Research Ethics," and "Research Governance." Here, it is proposed that Research Integrity should be understood as focused on the character of researchers, and consequently the responsibility for promoting it lies primarily with researchers themselves. This is a different perspective from Research Ethics, which is focused on judgments on the ethical acceptability of research, and should primarily be the responsibility of research ethics committees, often including input from the public as well as the research community. Finally, Research Governance focuses on legal and policy requirements, and although complementary to research integrity and ethics, is primarily the responsibility of expert research support officers with the skills and experience to address technical compliance.

The delicate nature of a constructive peer review: pearls from the editorial board

Peer review stands as a cornerstone of academic publishing, especially in the era of evidence-based neurosurgery - the scientific literature relies on proficient peer reviewers. Providing a constructive peer review is an art and learned skill that requires knowledge of study design and expertise in the neurosurgical subspeciality. Peer reviewers guard against arbitrary decision-making and are essential in ensuring that published manuscripts are of the highest quality. However, there remains a scarcity in the formal training relating to the peer review process. The objective of this article is therefore to shed light on this process through the lens of the Editorial Board. We encourage our invited peer reviewers to make use of this guide when appraising potential manuscripts.

Recommendations for prediction models in clinical practice guidelines for cardiovascular diseases are over-optimistic: a global survey utilizing a systematic literature search

Background

This study aimed to synthesize the recommendations for prediction models in cardiovascular clinical practice guidelines (CPGs) and assess the methodological quality of the relevant primary modeling studies.

Methods

We performed a systematic literature search of all available cardiovascular CPGs published between 2018 and 2023 that presented specific recommendations (whether in support or non-support) for at least one multivariable clinical prediction model. For the guideline-recommended models, the assessment of the methodological quality of their primary modeling studies was conducted using the Prediction model Risk Of Bias ASsessment Tool (PROBAST).

Results

In total, 46 qualified cardiovascular CPGs were included, with 69 prediction models and 80 specific recommendations. Of the 80 specific recommendations, 74 supported 57 models (53 were fully recommended and 4 were conditionally recommended) in cardiovascular practice with moderate to strong strength. Most of the guideline-recommended models were focused on predicting prognosis outcomes (53/57, 93%) in primary and tertiary prevention, focusing primarily on long-term risk stratification and prognosis management. A total of 10 conditions and 7 types of target population were involved in the 57 models, while heart failure (14/57, 25%) and a general population with or without cardiovascular risk factor(s) (12/57, 21%) received the most attention from the guidelines. The assessment of the methodological quality of 57 primary studies on the development of the guideline-recommended models revealed that only 40% of the modeling studies had a low risk of bias (ROB). The causes of high ROB were mainly in the analysis and participant domains.

Conclusions

Global cardiovascular CPGs presented an unduly positive appraisal of the existing prediction models in terms of ROB, leading to stronger recommendations than were warranted. Future cardiovascular practice may benefit from well-established clinical prediction models with better methodological quality and extensive external validation.

Rethinking our future: Describing and enhancing the impacts of dissemination and implementation science for cancer prevention and control

Background

Researchers generally do an excellent job tracking the scientific impacts of their scholarship in ways that are relevant for academia (e.g., publications, grants) but too often neglect to focus on broader impacts on population health and equity. The National Cancer Institute's Implementation Science Centers in Cancer Control (ISC3) includes 7 P50 Centers that are interested in broad measures of impact. We provide an overview of the approach underway within the ISC3 consortium to identify health and social impacts.

Methods

ISC3 adapted and applied the Translational Science Benefits Model (TSBM) to identify the impact on the discipline of D&I science and to consider dissemination and implementation (D&I) impacts in the four original TSBM domains: (1) clinical; (2) community; (3) economic; and (4) policy. To collect data from all Centers, we: (1) co-developed a set of detailed impact indicators with examples; (2) created a data collection template; and (3) summarized the impact data from each center.

Results

Based on data from 48 ISC3 pilot studies, cores, or consortium activities, we identified 84 distinct benefits. The most common impacts were shown for implementation science (43%), community (28%), and clinical (18%). Frequent audiences included primary care providers, public health practitioners, and community partners. ISC3 members highlighted the need for product feedback, and storytelling assistance to advance impact.

Conclusions

The ISC3 consortium is using a participatory approach to successfully apply the TSBM, thus seeking to maximize the real-world impacts of D&I science. The D&I field needs to prioritize ways to more fully document and communicate societal impacts.

Evaluating Compliance of Randomized Controlled Trial Abstracts in Plastic Surgery Journals with CONSORT Guidelines Using GPT-4 AI

Background

The quality of reporting in randomized controlled trials (RCTs) is crucial for accurate interpretation and synthesis of evidence. The Consolidated Standards of Reporting Trials (CONSORT) guidelines provide a standardized framework for reporting RCT abstracts. This study aimed to evaluate the adherence of RCT abstracts published in three major plastic surgery journals to the CONSORT tool guideline for reporting abstracts, utilizing Generative Pre-trained Transformer 4 artificial intelligence (GPT-4 AI) technology.

Methods

Abstracts of RCTs published between 2010 and 2023 were collected. The GPT-4 AI model was utilized to assess the abstracts based on the CONSORT criteria. Descriptive statistics were used to report the compliance scores and identify areas where abstracts lacked compliance.

Results

Of the initially identified 500 abstracts, a total of 371 RCT abstracts met the inclusion criteria and were analyzed. The mean CONSORT score was 10.05 (±2.22), with a median score of 10.72. Specific areas where abstracts lacked compliance included trial design (39.6%), participant details (28.8%), intervention descriptions (15.6%), randomization process (25.3%), and the number of participants analyzed (33.4%). Trial registration (18.3%) and funding information (15.1%) were also frequently missing.

Conclusions

Our study's innovative use of the GPT-4 AI model for analysis demonstrated the potential of AI technology in streamlining and enhancing the evaluation of research compliance. We advocate for heightened awareness and more rigorous application of CONSORT guidelines among authors, reviewers, and journal editors. Emphasizing the role of AI technology in the evaluative process can further improve the reporting quality of future RCTs in plastic surgery, contributing to more reliable and transparent research in the field.

Is orthodontic research falling prey to obscure and predatory journals? A bibliometric study

AIMS

To evaluate where orthodontic research papers are published and to explore potential relationships between the journal of publication and the characteristics of the research study and authorship.

METHODS

An online literature search of seven research databases was undertaken to identify orthodontic articles published in English language over a 12-month period (1 January-31 December 2022) (last search: 12 June 2023). Data extracted included journal, article, and author characteristics. Journal legitimacy was assessed using a ternary classification scheme including available blacklists and whitelists, cross-checking of indexing claims and history of sending unsolicited emails. The level of evidence (LOE) of all included studies was assessed using a modified Oxford LOE classification scale. Univariable and multivariable ordinal logistic regression analyses were performed to examine possible associations between the level of evidence, journal discipline, and authorship characteristics.

RESULTS

A total of 753 studies, published by 246 unique journal titles, were included and further assessed. Nearly two-thirds of orthodontic papers were published in non-orthodontic journals (62.8%) and over half (55.6%) of the articles were published in open-access policy journals. About a fifth of the articles (21.2%) were published either in presumed predatory journals or in journals of uncertain legitimacy. Journal discipline was significantly associated with the level of evidence. Higher-quality orthodontic studies were more likely published in established orthodontic journals (likelihood ratio test P < .001).

LIMITATIONS

The identification and classification of predatory journals are challenging due to their covert nature.

CONCLUSIONS

The majority of orthodontic articles were published in non-orthodontic journals. In addition, approximately one in five orthodontic studies were published in presumed predatory journals or in journals of uncertain legitimacy. Studies with higher levels of evidence were more likely to be published in established orthodontic journals.

Development and validation of observational and qualitative study protocol reporting checklists for novice researchers (ObsQual checklist)

INTRODUCTION

Currently, no reporting guidelines exist for observational and qualitative study protocols. In an effort to enhance the quality of research protocols, we introduce two study protocol reporting checklists that we have developed.

MAIN RECOMMENDATIONS

These checklists include educational components and examples intended to assist novice researchers. Through the analysis of 333 study protocols submitted for ethical review, our checklists have been developed and validated, demonstrating their applicability across various observational and qualitative study designs.

CHANGES IN MANAGEMENT

We provide insights into the systematic implementation of these checklists alongside complementary elements that support their effectiveness. We recommend longitudinal monitoring and evaluation of checklist utilization.

"Luck of the draw really": a qualitative exploration of Australian trainee doctors' experiences of mandatory research

BACKGROUND

Many medical trainees, prior to achieving specialist status, are required to complete a mandatory research project, the usefulness of which has been debated. The aim of this study was to gain an in-depth understanding of trainees' experiences and satisfaction of conducting such research projects in Australia.

METHODS

A qualitative descriptive approach was used. Semi-structured interviews with trainees were undertaken between May 2021 and June 2022. Australian medical trainees who had completed a research project as part of specialty training within the past five years were invited to participate. The purposive sample was drawn from participants in a survey on the same topic who had indicated interest in participating in an interview. Interviews explored trainees' overall experience of and satisfaction with conducting research projects, as well as their perceptions of research training, support, barriers, enablers, and perceived benefits. Interviews were transcribed verbatim and thematically analysed.

RESULTS

Sixteen medical doctors from seven medical colleges were interviewed. Trainee experience and satisfaction was highly variable between participants and was shaped by four factors: 1) trainees entered their specialty training with their own perspectives on the value and purpose of the research project, informed by their previous experiences with research and perceived importance of research in their planned career path; 2) in conducting the project, enablers including protected time, supervisor support and institutional structures, were vital to shaping their experience; 3) trainees' access to these enablers was variable, mediated by a combination of luck, and the trainees' own drive and research skill; and 4) project outcomes, in terms of research merit, learning, career benefits and impacts on patient care.

CONCLUSIONS

Trainee experiences of doing research were mixed, with positive experiences often attributed to chance rather than an intentionally structured learning experience. We believe alternatives to mandatory trainee research projects must be explored, including recognising other forms of research learning activities, and directing scarce resources to supporting the few trainees who plan to pursue clinician researcher careers.

Text classification models for assessing the completeness of randomized controlled trial publications based on CONSORT reporting guidelines

Complete and transparent reporting of randomized controlled trial publications (RCTs) is essential for assessing their credibility. We aimed to develop text classification models for determining whether RCT publications report CONSORT checklist items. Using a corpus annotated with 37 fine-grained CONSORT items, we trained sentence classification models (PubMedBERT fine-tuning, BioGPT fine-tuning, and in-context learning with GPT-4) and compared their performance. We assessed the impact of data augmentation methods (Easy Data Augmentation (EDA), UMLS-EDA, text generation and rephrasing with GPT-4) on model performance. We also fine-tuned section-specific PubMedBERT models (e.g., Methods) to evaluate whether they could improve performance compared to the single full model. We performed 5-fold cross-validation and report precision, recall, F1 score, and area under curve (AUC). Fine-tuned PubMedBERT model that uses the sentence along with the surrounding sentences and section headers yielded the best overall performance (sentence level: 0.71 micro-F1, 0.67 macro-F1; article-level: 0.90 micro-F1, 0.84 macro-F1). Data augmentation had limited positive effect. BioGPT fine-tuning and GPT-4 in-context learning exhibited suboptimal results. Methods-specific model improved recognition of methodology items, other section-specific models did not have significant impact. Most CONSORT checklist items can be recognized reasonably well with the fine-tuned PubMedBERT model but there is room for improvement. Improved models can underpin the journal editorial workflows and CONSORT adherence checks.

Evaluation of research co-design in health: a systematic overview of reviews and development of a framework

BACKGROUND

Co-design with consumers and healthcare professionals is widely used in applied health research. While this approach appears to be ethically the right thing to do, a rigorous evaluation of its process and impact is frequently missing. Evaluation of research co-design is important to identify areas of improvement in the methods and processes, as well as to determine whether research co-design leads to better outcomes. We aimed to build on current literature to develop a framework to assist researchers with the evaluation of co-design processes and impacts.

METHODS

A multifaceted, iterative approach, including three steps, was undertaken to develop a Co-design Evaluation Framework: 1) A systematic overview of reviews; 2) Stakeholder panel meetings to discuss and debate findings from the overview of reviews and 3) Consensus meeting with stakeholder panel. The systematic overview of reviews included relevant papers published between 2000 and 2022. OVID (Medline, Embase, PsycINFO), EBSCOhost (Cinahl) and the Cochrane Database of Systematic reviews were searched for papers that reported co-design evaluation or outcomes in health research. Extracted data was inductively analysed and evaluation themes were identified. Review findings were presented to a stakeholder panel, including consumers, healthcare professionals and researchers, to interpret and critique. A consensus meeting, including a nominal group technique, was applied to agree upon the Co-design Evaluation Framework.

RESULTS

A total of 51 reviews were included in the systematic overview of reviews. Fifteen evaluation themes were identified and grouped into the following seven clusters: People (within co-design group), group processes, research processes, co-design context, people (outside co-design group), system and sustainment. If evaluation methods were mentioned, they mainly included qualitative data, informal consumer feedback and researchers' reflections. The Co-Design Evaluation Framework used a tree metaphor to represent the processes and people in the co-design group (below-ground), underpinning system- and people-level outcomes beyond the co-design group (above-ground). To evaluate research co-design, researchers may wish to consider any or all components in the tree.

CONCLUSIONS

The Co-Design Evaluation Framework has been collaboratively developed with various stakeholders to be used prospectively (planning for evaluation), concurrently (making adjustments during the co-design process) and retrospectively (reviewing past co-design efforts to inform future activities).

Meta-research on reporting guidelines for artificial intelligence: are authors and reviewers encouraged enough in radiology, nuclear medicine, and medical imaging journals?

PURPOSE

To determine how radiology, nuclear medicine, and medical imaging journals encourage and mandate the use of reporting guidelines for artificial intelligence (AI) in their author and reviewer instructions.

METHODS

The primary source of journal information and associated citation data used was the Journal Citation Reports (June 2023 release for 2022 citation data; Clarivate Analytics, UK). The first- and second-quartile journals indexed in the Science Citation Index Expanded and the Emerging Sources Citation Index were included. The author and reviewer instructions were evaluated by two independent readers, followed by an additional reader for consensus, with the assistance of automatic annotation. Encouragement and submission requirements were systematically analyzed. The reporting guidelines were grouped as AI-specific, related to modeling, and unrelated to modeling.

RESULTS

Out of 102 journals, 98 were included in this study, and all of them had author instructions. Only five journals (5%) encouraged the authors to follow AI-specific reporting guidelines. Among these, three required a filled-out checklist. Reviewer instructions were found in 16 journals (16%), among which one journal (6%) encouraged the reviewers to follow AI-specific reporting guidelines without submission requirements. The proportions of author and reviewer encouragement for AI-specific reporting guidelines were statistically significantly lower compared with those for other types of guidelines (P < 0.05 for all).

CONCLUSION

The findings indicate that AI-specific guidelines are not commonly encouraged and mandated (i.e., requiring a filled-out checklist) by these journals, compared with guidelines related to modeling and unrelated to modeling, leaving vast space for improvement. This meta-research study hopes to contribute to the awareness of the imaging community for AI reporting guidelines and ignite large-scale group efforts by all stakeholders, making AI research less wasteful.

CLINICAL SIGNIFICANCE

This meta-research highlights the need for improved encouragement of AI-specific guidelines in radiology, nuclear medicine, and medical imaging journals. This can potentially foster greater awareness among the AI community and motivate various stakeholders to collaborate to promote more efficient and responsible AI research reporting practices.

Major mistakes or errors in the use of trial sequential analysis in systematic reviews or meta-analyses - the METSA systematic review

BACKGROUND

Systematic reviews and data synthesis of randomised clinical trials play a crucial role in clinical practice, research, and health policy. Trial sequential analysis can be used in systematic reviews to control type I and type II errors, but methodological errors including lack of protocols and transparency are cause for concern. We assessed the reporting of trial sequential analysis.

METHODS

We searched Medline and the Cochrane Database of Systematic Reviews from 1 January 2018 to 31 December 2021 for systematic reviews and meta-analysis reports that include a trial sequential analysis. Only studies with at least two randomised clinical trials analysed in a forest plot and a trial sequential analysis were included. Two independent investigators assessed the studies. We evaluated protocolisation, reporting, and interpretation of the analyses, including their effect on any GRADE evaluation of imprecision.

RESULTS

We included 270 systematic reviews and 274 meta-analysis reports and extracted data from 624 trial sequential analyses. Only 134/270 (50%) systematic reviews planned the trial sequential analysis in the protocol. For analyses on dichotomous outcomes, the proportion of events in the control group was missing in 181/439 (41%), relative risk reduction in 105/439 (24%), alpha in 30/439 (7%), beta in 128/439 (29%), and heterogeneity in 232/439 (53%). For analyses on continuous outcomes, the minimally relevant difference was missing in 125/185 (68%), variance (or standard deviation) in 144/185 (78%), alpha in 23/185 (12%), beta in 63/185 (34%), and heterogeneity in 105/185 (57%). Graphical illustration of the trial sequential analysis was present in 93% of the analyses, however, the Z-curve was wrongly displayed in 135/624 (22%) and 227/624 (36%) did not include futility boundaries. The overall transparency of all 624 analyses was very poor in 236 (38%) and poor in 173 (28%).

CONCLUSIONS

The majority of trial sequential analyses are not transparent when preparing or presenting the required parameters, partly due to missing or poorly conducted protocols. This hampers interpretation, reproducibility, and validity.

STUDY REGISTRATION

PROSPERO CRD42021273811.

Physical functioning in the lumbar spinal surgery population: A systematic review and narrative synthesis of outcome measures and measurement properties of the physical measures

BACKGROUND

International agreement supports physical functioning as a key domain to measure interventions effectiveness for low back pain. Patient reported outcome measures (PROMs) are commonly used in the lumbar spinal surgery population but physical functioning is multidimensional and necessitates evaluation also with physical measures.

OBJECTIVE

1) To identify outcome measures (PROMs and physical) used to evaluate physical functioning in the lumbar spinal surgery population. 2) To assess measurement properties and describe the feasibility and interpretability of physical measures of physical functioning in this population.

STUDY DESIGN

Two-staged systematic review and narrative synthesis.

METHODS

This systematic review was conducted according to a registered and published protocol. Two stages of searching were conducted in MEDLINE, EMBASE, Health & Psychosocial Instruments, CINAHL, Web of Science, PEDro and ProQuest Dissertations & Theses. Stage one included studies to identify physical functioning outcome measures (PROMs and physical) in the lumbar spinal surgery population. Stage two (inception to 10 July 2023) included studies assessing measurement properties of stage one physical measures. Two independent reviewers determined study eligibility, extracted data and assessed risk of bias (RoB) according to COSMIN guidelines. Measurement properties were rated according to COSMIN criteria. Level of evidence was determined using a modified GRADE approach.

RESULTS

Stage one included 1,101 reports using PROMs (n = 70 established in literature, n = 67 developed by study authors) and physical measures (n = 134). Stage two included 43 articles assessing measurement properties of 34 physical measures. Moderate-level evidence supported sufficient responsiveness of 1-minute stair climb and 50-foot walk tests, insufficient responsiveness of 5-minute walk and sufficient reliability of distance walked during the 6-minute walk. Very low/low-level evidence limits further understanding.

CONCLUSIONS

Many physical measures of physical functioning are used in lumbar spinal surgery populations. Few have investigations of measurement properties. Strongest evidence supports responsiveness of 1-minute stair climb and 50-foot walk tests and reliability of distance walked during the 6-minute walk. Further recommendations cannot be made because of very low/low-level evidence. Results highlight promise for a range of measures, but prospective, low RoB studies are required.