Reducing waste from incomplete or unusable reports of biomedical research

[Developing, Evaluating and Implementing Digital Health Interventions Part II - Discussion Paper of the Digital Health Working Group of the German Network for Health Services Research (DNVF)]

The methodological challenges of evaluating digital interventions (DI) for health services research are omnipresent. The Digital Health Working Group of the German Network for Health Services Research (DNVF) presented and discussed these challenges in a two-part discussion paper. The first part addressed challenges in definition, development and evaluation of DI. In this paper, which represents the second part, the definition of outcomes, reporting of results, synthesis of evidence, and implementation are addressed as methodological challenges of DI. Potential solutions are presented and the need to address these challenges in future research are discussed.

Reducing bias, increasing transparency and calibrating confidence with preregistration

Flexibility in the design, analysis and interpretation of scientific studies creates a multiplicity of possible research outcomes. Scientists are granted considerable latitude to selectively use and report the hypotheses, variables and analyses that create the most positive, coherent and attractive story while suppressing those that are negative or inconvenient. This creates a risk of bias that can lead to scientists fooling themselves and fooling others. Preregistration involves declaring a research plan (for example, hypotheses, design and statistical analyses) in a public registry before the research outcomes are known. Preregistration (1) reduces the risk of bias by encouraging outcome-independent decision-making and (2) increases transparency, enabling others to assess the risk of bias and calibrate their confidence in research outcomes. In this Perspective, we briefly review the historical evolution of preregistration in medicine, psychology and other domains, clarify its pragmatic functions, discuss relevant meta-research, and provide recommendations for scientists and journal editors.

Evaluating the online impact of reporting guidelines for randomised trial reports and protocols: a cross-sectional web-based data analysis of CONSORT and SPIRIT initiatives

Reporting guidelines are tools to help improve the transparency, completeness, and clarity of published articles in health research. Specifically, the CONSORT (Consolidated Standards of Reporting Trials) and SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials) statements provide evidence-based guidance on what to include in randomised trial articles and protocols to guarantee the efficacy of interventions. These guidelines are subsequently described and discussed in journal articles and used to produce checklists. Determining the online impact (i.e., number and type of links received) of these articles can provide insights into the dissemination of reporting guidelines in broader environments (web-at-large) than simply that of the scientific publications that cite them. To address the technical limitations of link analysis, here the Debug-Validate-Access-Find (DVAF) method is designed and implemented to measure different facets of the guidelines' online impact. A total of 65 articles related to 38 reporting guidelines are taken as a baseline, providing 240,128 URL citations, which are then refined, analysed, and categorised using the DVAF method. A total of 15,582 links to journal articles related to the CONSORT and SPIRIT initiatives were identified. CONSORT 2010 and SPIRIT 2013 were the reporting guidelines that received most links (URL citations) from other online objects (5328 and 2190, respectively). Overall, the online impact obtained is scattered (URL citations are received by different article URL IDs, mainly from link-based DOIs), narrow (limited number of linking domain names, half of articles are linked from fewer than 29 domain names), concentrated (links come from just a few academic publishers, around 60% from publishers), non-reputed (84% of links come from dubious websites and fake domain names) and highly decayed (89% of linking domain names were not accessible at the time of the analysis). In light of these results, it is concluded that the online impact of these guidelines could be improved, and a set of recommendations are proposed to this end.

Supplementary Information

The online version contains supplementary material available at 10.1007/s11192-022-04542-z.

Guidelines for Reporting Outcomes in Trial Protocols: The SPIRIT-Outcomes 2022 Extension

IMPORTANCE

Complete information in a trial protocol regarding study outcomes is crucial for obtaining regulatory approvals, ensuring standardized trial conduct, reducing research waste, and providing transparency of methods to facilitate trial replication, critical appraisal, accurate reporting and interpretation of trial results, and knowledge synthesis. However, recommendations on what outcome-specific information should be included are diverse and inconsistent. To improve reporting practices promoting transparent and reproducible outcome selection, assessment, and analysis, a need for specific and harmonized guidance as to what outcome-specific information should be addressed in clinical trial protocols exists.

OBJECTIVE

To develop harmonized, evidence- and consensus-based standards for describing outcomes in clinical trial protocols through integration with the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) 2013 statement.

EVIDENCE REVIEW

Using the Enhancing the Quality and Transparency of Health Research (EQUATOR) methodological framework, the SPIRIT-Outcomes 2022 extension of the SPIRIT 2013 statement was developed by (1) generation and evaluation of candidate outcome reporting items via consultation with experts and a scoping review of existing guidance for reporting trial outcomes (published within the 10 years prior to March 19, 2018) identified through expert solicitation, electronic database searches of MEDLINE and the Cochrane Methodology Register, gray literature searches, and reference list searches; (2) a 3-round international Delphi voting process (November 2018-February 2019) completed by 124 panelists from 22 countries to rate and identify additional items; and (3) an in-person consensus meeting (April 9-10, 2019) attended by 25 panelists to identify essential items for outcome-specific reporting to be addressed in clinical trial protocols.

FINDINGS

The scoping review and consultation with experts identified 108 recommendations relevant to outcome-specific reporting to be addressed in trial protocols, the majority (72%) of which were not included in the SPIRIT 2013 statement. All recommendations were consolidated into 56 items for Delphi voting; after the Delphi survey process, 19 items met criteria for further evaluation at the consensus meeting and possible inclusion in the SPIRIT-Outcomes 2022 extension. The discussions during and after the consensus meeting yielded 9 items that elaborate on the SPIRIT 2013 statement checklist items and are related to completely defining and justifying the choice of primary, secondary, and other outcomes (SPIRIT 2013 statement checklist item 12) prospectively in the trial protocol, defining and justifying the target difference between treatment groups for the primary outcome used in the sample size calculations (SPIRIT 2013 statement checklist item 14), describing the responsiveness of the study instruments used to assess the outcome and providing details on the outcome assessors (SPIRIT 2013 statement checklist item 18a), and describing any planned methods to account for multiplicity relating to the analyses or interpretation of the results (SPIRIT 2013 statement checklist item 20a).

CONCLUSIONS AND RELEVANCE

This SPIRIT-Outcomes 2022 extension of the SPIRIT 2013 statement provides 9 outcome-specific items that should be addressed in all trial protocols and may help increase trial utility, replicability, and transparency and may minimize the risk of selective nonreporting of trial results.

Key items for reports of primary care research: an international Delphi study

OBJECTIVE

Reporting guidelines can improve dissemination and application of findings and help avoid research waste. Recent studies reveal opportunities to improve primary care (PC) reporting. Despite increasing numbers of guidelines, none exists for PC research. This study aims to prioritise candidate reporting items to inform a reporting guideline for PC research.

DESIGN

Delphi study conducted by the Consensus Reporting Items for Studies in Primary Care (CRISP) Working Group.

SETTING

International online survey.

PARTICIPANTS

Interdisciplinary PC researchers and research users.

MAIN OUTCOME MEASURES

We drew potential reporting items from literature review and a series of international, interdisciplinary surveys. Using an anonymous, online survey, we asked participants to vote on and whether each candidate item should be included, required or recommended in a PC research reporting guideline. Items advanced to the next Delphi round if they received>50% votes to include. Analysis used descriptive statistics plus synthesis of free-text responses.

RESULTS

98/116 respondents completed round 1 (84% response rate) and 89/98 completed round 2 (91%). Respondents included a variety of healthcare professions, research roles, levels of experience and all five world regions. Round 1 presented 29 potential items, and 25 moved into round 2 after rewording and combining items and adding 2 new items. A majority of round 2 respondents voted to include 23 items (90%-100% for 11 items, 80%-89% for 3 items, 70%-79% for 3 items, 60%-69% for 3 items and 50%-59% for 3 items).

CONCLUSION

Our Delphi study identified items to guide the reporting of PC research that has broad endorsement from the community of producers and users of PC research. We will now use these results to inform the final development of the CRISP guidance for reporting PC research.

Guidelines for Reporting Outcomes in Trial Reports: The CONSORT-Outcomes 2022 Extension

IMPORTANCE

Clinicians, patients, and policy makers rely on published results from clinical trials to help make evidence-informed decisions. To critically evaluate and use trial results, readers require complete and transparent information regarding what was planned, done, and found. Specific and harmonized guidance as to what outcome-specific information should be reported in publications of clinical trials is needed to reduce deficient reporting practices that obscure issues with outcome selection, assessment, and analysis.

OBJECTIVE

To develop harmonized, evidence- and consensus-based standards for reporting outcomes in clinical trial reports through integration with the Consolidated Standards of Reporting Trials (CONSORT) 2010 statement.

EVIDENCE REVIEW

Using the Enhancing the Quality and Transparency of Health Research (EQUATOR) methodological framework, the CONSORT-Outcomes 2022 extension of the CONSORT 2010 statement was developed by (1) generation and evaluation of candidate outcome reporting items via consultation with experts and a scoping review of existing guidance for reporting trial outcomes (published within the 10 years prior to March 19, 2018) identified through expert solicitation, electronic database searches of MEDLINE and the Cochrane Methodology Register, gray literature searches, and reference list searches; (2) a 3-round international Delphi voting process (November 2018-February 2019) completed by 124 panelists from 22 countries to rate and identify additional items; and (3) an in-person consensus meeting (April 9-10, 2019) attended by 25 panelists to identify essential items for the reporting of outcomes in clinical trial reports.

FINDINGS

The scoping review and consultation with experts identified 128 recommendations relevant to reporting outcomes in trial reports, the majority (83%) of which were not included in the CONSORT 2010 statement. All recommendations were consolidated into 64 items for Delphi voting; after the Delphi survey process, 30 items met criteria for further evaluation at the consensus meeting and possible inclusion in the CONSORT-Outcomes 2022 extension. The discussions during and after the consensus meeting yielded 17 items that elaborate on the CONSORT 2010 statement checklist items and are related to completely defining and justifying the trial outcomes, including how and when they were assessed (CONSORT 2010 statement checklist item 6a), defining and justifying the target difference between treatment groups during sample size calculations (CONSORT 2010 statement checklist item 7a), describing the statistical methods used to compare groups for the primary and secondary outcomes (CONSORT 2010 statement checklist item 12a), and describing the prespecified analyses and any outcome analyses not prespecified (CONSORT 2010 statement checklist item 18).

CONCLUSIONS AND RELEVANCE

This CONSORT-Outcomes 2022 extension of the CONSORT 2010 statement provides 17 outcome-specific items that should be addressed in all published clinical trial reports and may help increase trial utility, replicability, and transparency and may minimize the risk of selective nonreporting of trial results.

Evidence synthesis, digital scribes, and translational challenges for artificial intelligence in healthcare

Healthcare has well-known challenges with safety, quality, and effectiveness, and many see artificial intelligence (AI) as essential to any solution. Emerging applications include the automated synthesis of best-practice research evidence including systematic reviews, which would ultimately see all clinical trial data published in a computational form for immediate synthesis. Digital scribes embed themselves in the process of care to detect, record, and summarize events and conversations for the electronic record. However, three persistent translational challenges must be addressed before AI is widely deployed. First, little effort is spent replicating AI trials, exposing patients to risks of methodological error and biases. Next, there is little reporting of patient harms from trials. Finally, AI built using machine learning may perform less effectively in different clinical settings.

Self-correction in science: The effect of retraction on the frequency of citations

We investigate the citation frequency of retracted scientific papers in science. For the period of five years before and after retraction, we counted the citations to papers in a sample of over 3,000 retracted, and a matched sample of another 3,000 non-retracted papers. Retraction led to a decrease in average annual citation frequency from about 5 before, to 2 citations after retraction. In contrast, for non-retracted control papers the citation counts were 4, and 5, respectively. Put differently, we found only a limited effect of retraction: retraction decreased citation frequency only by about 60%, as compared to non-retracted papers. Thus, retracted papers often live on. For effective self-correction the scientific enterprise needs to be more effective in removing retracted papers from the scientific record. We discuss recent proposals to do so.

Identifying common core outcome domains from core outcome sets of musculoskeletal conditions: protocol for a systematic review

BACKGROUND

Core outcome sets (COSs) aim to reduce outcome heterogeneity in clinical practice and research by suggesting a minimum number of agreed-upon outcomes in clinical trials. Most COSs in the musculoskeletal field are developed for specific conditions. We propose that there are likely to be common core domains within existing musculoskeletal COSs that may be used as a starting point in the development of future COSs. We aim to identify common core domains from existing COSs and to facilitate the development of new COSs for musculoskeletal conditions. As a secondary aim, we will assess the development quality of these COSs.

METHODS

A systematic review including musculoskeletal COSs. We will search Core Outcome Measures in Effectiveness Trials (COMET) database, MEDLINE, EMBASE, Scopus, Cochrane Methodology Register and International Consortium for Health Outcome Measurement (ICHOM). Studies will be included if related to the development of a COS in adults with musculoskeletal conditions and for any type of intervention. Quality will be assessed using the Core Outcome Set-Standards for Development (COS-STAD) recommendations. Data extracted will include scope of the COS, health condition, interventions and outcome domains. Primary outcomes will be all core domains recommended within each COS. We define a common core outcome domain as one present in at least 67% of all COSs. All findings will be summarized and presented using descriptive statistics.

DISCUSSION

This systematic review of COSs will describe the core domains recommended within each musculoskeletal COS. Common domains found may be used in the initial stages of development of future musculoskeletal COSs.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO CRD42021239141.

A qualitative study of the barriers to using blinding in in vivo experiments and suggestions for improvement

In animal experiments, blinding (also known as masking) is a methodological strategy to reduce the risk that scientists, animal care staff, or other staff involved in the research may consciously or subconsciously influence the outcome. Lack of masking has been shown to correlate with an overestimation of treatment efficacy and false positive findings. We conducted exploratory interviews across academic and a commercial setting to discuss the implementation of masking at four stages of the experiment: during allocation and intervention, during the conduct of the experiment, during the outcome assessment, and during the data analysis. The objective was to explore the awareness, engagement, perceptions, and the barriers to implementing masking in animal experiments. We conducted multiple interviews, to explore 30 different experiments, and found examples of excellent practice but also areas where masking was rarely implemented. Significant barriers arose from the operational and informatic systems implemented. These systems have prioritised the management of welfare without considering how to allow researchers to use masking in their experiments. For some experiments, there was a conflict between the management of welfare for an individual animal versus delivering a robust experiment where all animals are treated in the same manner. We identified other challenges related to the level of knowledge on the purpose of masking or the implementation and the work culture. The exploration of these issues provides insight into how we, as a community, can identify the most significant barriers in a given research environment. Here, we offer practical solutions to enable researchers to implement masking as standard. To move forward, we need both the individual scientists to embrace the use of masking and the facility managers and institutes to engage and provide a framework that supports the scientists.

Reporting quality in preclinical animal experimental research in 2009 and 2018: A nationwide systematic investigation

Lack of translation and irreproducibility challenge preclinical animal research. Insufficient reporting methodologies to safeguard study quality is part of the reason. This nationwide study investigates the reporting prevalence of these methodologies and scrutinizes the reported information’s level of detail. Publications were from two time periods to convey any reporting progress and had at least one author affiliated to a Danish University. We retrieved all relevant animal experimental studies using a predefined research protocol and a systematic search. A random sampling of 250 studies from 2009 and 2018 led to 500 publications in total. Reporting of measures known to impact study results estimates were assessed. Part I discloses a simplified two-level scoring “yes/no” to identify the presence of reporting. Part II demonstrates an additional three-level scoring to analyze the reported information’s level of detail. Overall reporting prevalence is low, although minor improvements are noted. Reporting of randomization increased from 24.0% in 2009 to 40.8% in 2018, blinded experiment conduct from 2.4% to 4.4%, blinded outcome assessment from 23.6% to 38.0%, and sample size calculation from 3.2% to 14.0%. Poor reporting of details is striking with reporting of the random allocation method to groups being only 1.2% in 2009 and 6.0% in 2018. Reporting of sample size calculation method was 2.4% in 2009 and 7.6% in 2018. Only conflict-of-interest statements reporting increased from 37.6% in 2009 to 90.4%. Measures safeguarding study quality are poorly reported in publications affiliated with Danish research institutions. Only a modest improvement was noted during the period 2009–2018, and the lack of details urgently prompts institutional strategies to accelerate this. We suggest thorough teaching in designing, conducting and reporting animal studies. Education in systematic review methodology should be implemented in this training and will increase motivation and behavior working towards quality improvements in science.

Linguistic spin in randomized controlled trials about age-related macular degeneration

Objective

To evaluate the prevalence, type and extent of linguistic spin in randomized controlled trials (RCTs) exploring interventions in patients with age-related macular degeneration (AMD), as well as to investigate whether study variables were correlated with linguistic spin.

Study design and setting

PubMed was searched from 2011 to 2020 to identify RCTs including patients with AMD. Two authors independently assessed a total of 96 RCTs. Linear regression analyses were performed to investigate whether linguistic spin was correlated with predefined study variables.

Results

Linguistic spin was found in 61 of 96 abstracts (63.5%) and in 90 of 96 main texts (93.8%). Use of words pointing out the beneficial effect of a treatment and the use of '(statistically) significant/significance' without reporting a P-value or a 95% confidence interval (CI) were the most frequently identified categories of linguistic spin. Sample size was significantly correlated with the total linguistic spin score (95% CI 0.38-5.23, P = 0.02).

Conclusion

A high prevalence and extent of linguistic spin in RCTs about AMD was found. We highlighted the importance of objective reporting and awareness of linguistic spin among ophthalmologists and other readers.

Establishing reporting standards for participant characteristics in post-stroke aphasia research: An international e-Delphi exercise and consensus meeting

Objective

To establish international, multidisciplinary expert consensus on minimum participant characteristic reporting standards in aphasia research (DESCRIBE project).

Methods

An international, three-round e-Delphi exercise and consensus meeting, involving multidisciplinary researchers, clinicians and journal editors working academically or clinically in the field of aphasia.

Results

Round 1 of the DESCRIBE e-Delphi exercise ( n = 156) generated 113 items, 20 of which reached consensus by round 3. The final consensus meeting ( n = 19 participants) established DESCRIBE's 14 participant characteristics that should be reported in aphasia studies: age; years of education; biological sex; language of treatment/testing; primary language; languages used; history of condition(s) known to impact communication/cognition; history of previous stroke; lesion hemisphere; time since onset of aphasia; conditions arising from the neurological event; and, for communication partner participants, age, biological sex and relationship to person with aphasia. Each characteristic has been defined and matched with standard response options to enable consistent reporting.

Conclusion

Aphasia research studies should report the 14 DESCRIBE participant characteristics as a minimum. Consistent adherence to the DESCRIBE minimum reporting standard will reduce research wastage and facilitate evidence-based aphasia management by enabling replication and collation of research findings, and translation of evidence into practice.

Reducing waste in collection of quality-of-life data through better reporting: a case study

PURPOSE

This study describes the reporting of the preference-based health-related quality-of-life (HRQOL) instrument, the EQ-5D, and proposes strategies to improve reporting and reduce research waste. The EQ-5D is a validated instrument widely used for health economic evaluation and is useful for informing health policy.

METHODS

As part of a systematic review of papers reporting EQ-5D utility weights in patients with coronary artery disease, we noted the reasons data from some papers could not be reused in a meta-analysis, including whether health utility weights and sufficient statistical details were reported. Research waste was quantified using: (1) the percentage of papers and sample size excluded, and (2) researcher time and cost reviewing poorly reported papers.

RESULTS

Our search strategy found 5942 papers. At title and abstract screening 93% were excluded. Of the 379 full text papers screened, 130 papers reported using EQ-5D. Only 46% (60/130) of those studies reported utility weights and/or statistical properties enabling meta-analysis. Only 67% of included papers had reported EQ-5D in the title or abstract. A total sample size of 133,298 was excluded because of poor reporting. The cost of researcher time wasted estimated to be between $3816 and $13,279 for our review.

CONCLUSIONS

Poor reporting of EQ-5D data creates research waste where potentially useful data are excluded from meta-analyses and economic evaluations. Poor reporting of HRQOL instruments also creates waste due to additional time spent reviewing papers for systematic reviews that are subsequently excluded.

RECOMMENDATIONS

Studies using the EQ-5D should report utility weights with appropriate summary statistics to enable reuse in meta-analysis and more robust evidence for health policy. We recommend authors report the HRQOL instrument in the title or abstract in line with current reporting guidelines (CONSORT-PRO and SPIRIT-PRO Extensions) to make it easier for other researchers to find. Validated instruments should also be listed in the Medical Subject Headings (MeSH) to improve cataloguing and retrieval of previous research.

Transparency and reporting characteristics of COVID-19 randomized controlled trials

BACKGROUND

In the context of the COVID-19 pandemic, randomized controlled trials (RCTs) are essential to support clinical decision-making. We aimed (1) to assess and compare the reporting characteristics of RCTs between preprints and peer-reviewed publications and (2) to assess whether reporting improves after the peer review process for all preprints subsequently published in peer-reviewed journals.

METHODS

We searched the Cochrane COVID-19 Study Register and L·OVE COVID-19 platform to identify all reports of RCTs assessing pharmacological treatments of COVID-19, up to May 2021. We extracted indicators of transparency (e.g., trial registration, data sharing intentions) and assessed the completeness of reporting (i.e., some important CONSORT items, conflict of interest, ethical approval) using a standardized data extraction form. We also identified paired reports published in preprint and peer-reviewed publications.

RESULTS

We identified 251 trial reports: 121 (48%) were first published in peer-reviewed journals, and 130 (52%) were first published as preprints. Transparency was poor. About half of trials were prospectively registered (n = 140, 56%); 38% (n = 95) made their full protocols available, and 29% (n = 72) provided access to their statistical analysis plan report. A data sharing statement was reported in 68% (n = 170) of the reports of which 91% stated their willingness to share. Completeness of reporting was low: only 32% (n = 81) of trials completely defined the pre-specified primary outcome measures; 57% (n = 143) reported the process of allocation concealment. Overall, 51% (n = 127) adequately reported the results for the primary outcomes while only 14% (n = 36) of trials adequately described harms. Primary outcome(s) reported in trial registries and published reports were inconsistent in 49% (n = 104) of trials; of them, only 15% (n = 16) disclosed outcome switching in the report. There were no major differences between preprints and peer-reviewed publications. Of the 130 RCTs published as preprints, 78 were subsequently published in a peer-reviewed journal. There was no major improvement after the journal peer review process for most items.

CONCLUSIONS

Transparency, completeness, and consistency of reporting of COVID-19 clinical trials were insufficient both in preprints and peer-reviewed publications. A comparison of paired reports published in preprint and peer-reviewed publication did not indicate major improvement.

Systematic reviews are rarely used to contextualise new results-a systematic review and meta-analysis of meta-research studies

BACKGROUND

Results of new studies should be interpreted in the context of what is already known to compare results and build the state of the science. This systematic review and meta-analysis aimed to identify and synthesise results from meta-research studies examining if original studies within health use systematic reviews to place their results in the context of earlier, similar studies.

METHODS

We searched MEDLINE (OVID), EMBASE (OVID), and the Cochrane Methodology Register for meta-research studies reporting the use of systematic reviews to place results of original clinical studies in the context of existing studies. The primary outcome was the percentage of original studies included in the meta-research studies using systematic reviews or meta-analyses placing new results in the context of existing studies. Two reviewers independently performed screening and data extraction. Data were synthesised using narrative synthesis and a random-effects meta-analysis was performed to estimate the mean proportion of original studies placing their results in the context of earlier studies. The protocol was registered in Open Science Framework.

RESULTS

We included 15 meta-research studies, representing 1724 original studies. The mean percentage of original studies within these meta-research studies placing their results in the context of existing studies was 30.7% (95% CI [23.8%, 37.6%], I2=87.4%). Only one of the meta-research studies integrated results in a meta-analysis, while four integrated their results within a systematic review; the remaining cited or referred to a systematic review. The results of this systematic review are characterised by a high degree of heterogeneity and should be interpreted cautiously.

CONCLUSION

Our systematic review demonstrates a low rate of and great variability in using systematic reviews to place new results in the context of existing studies. On average, one third of the original studies contextualised their results. Improvement is still needed in researchers' use of prior research systematically and transparently-also known as the use of an evidence-based research approach, to contribute to the accumulation of new evidence on which future studies should be based.

SYSTEMATIC REVIEW REGISTRATION

Open Science registration number https://osf.io/8gkzu/.

The proportion of randomized controlled trials that inform clinical practice

Prior studies suggest that clinical trials are often hampered by problems in design, conduct and reporting that limit their uptake in clinical practice. We have described 'informativeness' as the ability of a trial to guide clinical, policy or research decisions. Little is known about the proportion of initiated trials that inform clinical practice. We created a cohort of randomized interventional clinical trials in three disease areas (ischemic heart disease, diabetes mellitus and lung cancer), that were initiated between 1 January 2009 and 31 December 2010 using ClinicalTrials.gov. We restricted inclusion to trials aimed at answering a clinical question related to the treatment or prevention of disease. Our primary outcome was the proportion of clinical trials fulfilling four conditions of informativeness: importance of the clinical question, trial design, feasibility, and reporting of results. Our study included 125 clinical trials. The proportion meeting four conditions for informativeness was 26.4% (95% CI 18.9 - 35.0). Sixty-seven percent of participants were enrolled in informative trials. The proportion of informative trials did not differ significantly between our three disease areas. Our results suggest that the majority of clinical trials designed to guide clinical practice possess features that may compromise their ability to do so. This highlights opportunities to improve the scientific vetting of clinical research.

Developing a Core Outcome Set for the Evaluation of Antibiotic Use in Prelabor Rupture of Membranes: A Systematic Review and Semi-Structured Interview

Background: Prelabor rupture of membranes (PROM) is associated with maternal and neonatal infections. Although guidelines suggest prophylactic antibiotics for pregnant women with PROM, the optimal antibiotic regimen remains controversial. Synthesizing the data from different studies is challenging due to variations in reported outcomes. Objective: This study aimed to form the initial list of outcomes for the core outcome set (COS) that evaluates antibiotic use in PROM by identifying all existing outcomes and patients' views. Methods: Relevant studies were identified by searching PubMed, EMBASE, Cochrane Library, Chinese National Knowledge Infrastructure, Wanfang, and VIP databases. We also screened the references of the included studies as a supplementary search. We extracted basic information from the articles and the outcomes. Two reviewers independently selected the studies, extracted the data, extracted the outcomes, and grouped them into domains. Then, semi-structured interviews based on the potential factors collected by the systematic review were conducted at West China Second Hospital of Sichuan University. Pregnant women who met the diagnostic criteria for PROM were enrolled. Participants reported their concerns about the outcomes. Two researchers identified the pregnant women's concerns. Results: A total of 90 studies were enrolled in this systematic review. The median outcomes in the included studies was 7 (1-31), and 109 different unique outcomes were identified. Pre-term PROM (PPROM) had 97 outcomes, and term PROM (TPROM) had 70 outcomes. The classification and order of the core outcome domains of PPROM and TPROM were consistent. The physiological domain was the most common for PPROM and TPROM outcomes. Furthermore, 35.1 and 57.1% outcomes were only reported once in PPROM and TPROM studies, respectively. Thirty pregnant women participated in the semi-structured interviews; 10 outcomes were extracted after normalized, and the outcomes were reported in the systematic review. However, studies rarely reported pregnant women's concerns. Conclusion: There was considerable inconsistency in outcomes selection and reporting in studies about antibiotics in PROM. An initial core outcomes set for antibiotics in PROM was formed.

Paper 4: a review of reporting and disseminating approaches for rapid reviews in health policy and systems research

BACKGROUND

Transparent reporting of rapid reviews enables appropriate use of research findings and dissemination strategies can strengthen uptake and impact for the targeted knowledge users, including policy-makers and health system managers. The aim of this literature review was to understand reporting and dissemination approaches for rapid reviews and provide an overview in the context of health policy and systems research.

METHODS

A literature review and descriptive summary of the reporting and disseminating approaches for rapid reviews was conducted, focusing on available guidance and methods, considerations for engagement with knowledge users, and optimizing dissemination. MEDLINE, PubMed, Google scholar, as well as relevant websites and reference lists were searched from January 2017 to March 2021 to identify the relevant literature with no language restrictions. Content was abstracted and charted.

RESULTS

The literature review found limited guidance specific to rapid reviews. Building on the barriers and facilitators to systematic review use, we provide practical recommendations on different approaches and methods for reporting and disseminating expedited knowledge synthesis considering the needs of health policy and systems knowledge users. Reporting should balance comprehensive accounting of the research process and findings with what is "good enough" or sufficient to meet the requirements of the knowledge users, while considering the time and resources available to conduct a review. Typical approaches may be used when planning the dissemination of rapid review findings; such as peer-reviewed publications or symposia and clear and ongoing engagement with knowledge users in crafting the messages is essential so they are appropriately tailored to the target audience. Consideration should be given to providing different products for different audiences. Dissemination measures and bibliometrics are also useful to gauge impact and reach.

CONCLUSIONS

Limited guidance specific to the reporting and dissemination of rapid reviews is available. Although approaches to expedited synthesis for health policy and systems research vary, considerations for the reporting and dissemination of findings are pertinent to all.

Rapid reviews for health policy and systems decision-making: more important than ever before

BACKGROUND

Due to the explosion in rapid reviews in the literature during COVID-19, their utility in universal health coverage and in other routine situations, there is now a need to document and further advance the application of rapid review methods, particularly in low-resource settings where a scarcity of resources may preclude the production of a full systematic review. This is the introductory article for a series of articles to further the discussion of rapid reviews for health policy and systems decision-making.

MAIN BODY

The series of papers builds on a practical guide on the conduct and reporting of rapid reviews that was published in 2019. The first paper provides an evaluation of a rapid review platform that was implemented in four centers in low-resource settings, the second paper presents approaches to tailor the methods for decision-makers through rapid reviews, the third paper focuses on selecting different types of rapid review products, and the fourth pertains to reporting the results from a rapid review.

CONCLUSION

Rapid reviews have a great potential to inform universal health coverage and global health security interventions, moving forward, including preparedness and response plans to future pandemics. This series of articles will be useful for both researchers leading rapid reviews, as well as decision-makers using the results from rapid reviews.

Quantifying research waste in ecology

Research inefficiencies can generate huge waste: evidence from biomedical research has shown that most research is avoidably wasted and steps have been taken to tackle this costly problem. Although other scientific fields could also benefit from identifying and quantifying waste and acting to reduce it, no other estimates of research waste are available. Given that ecological issues interweave most of the United Nations Sustainable Development Goals, we argue that tackling research waste in ecology should be prioritized. Our study leads the way. We estimate components of waste in ecological research based on a literature review and a meta-analysis. Shockingly, our results suggest only 11-18% of conducted ecological research reaches its full informative value. All actors within the research system-including academic institutions, policymakers, funders and publishers-have a duty towards science, the environment, study organisms and the public, to urgently act and reduce this considerable yet preventable loss. We discuss potential ways forward and call for two major actions: (1) further research into waste in ecology (and beyond); (2) focused development and implementation of solutions to reduce unused potential of ecological research.

Trial registration of abstracts from the American Society of Anesthesiologists Meetings 2010–2016: A review of prospective trial registration and selective outcome reporting

Mandatory prospective trial registration was introduced in 2005 to reduce publication bias and selective outcome reporting. In this study, we measured the proportion of prospective trial registration in randomized controlled trials in the anesthesia literature after this introduction, discrepancies between these trial protocols and subsequent publications, the association between being prospectively registered and reporting positive or negative results, and between being prospectively registered and achieving publication. We reviewed all abstracts from the American Society of Anesthesiologists annual meetings between 2010–2016 and included randomized controlled trials in humans. The abstract conclusions were scored as positive or negative according to predetermined definitions. We conducted a systematic search for trial registration and subsequent publication. Of the 9789 abstracts reviewed, 1070 abstracts were included. 222 (21%) of these abstracts had undergone prospective trial registration. 168/222 (76%) had a corresponding journal publication. 81(48%) had a major discrepancy between registration and publication. 149 (67%) of the abstracts with registration had positive outcomes compared with 616 (73%) of those without (Odds Ratio 0.77; 95% CI: 0.56 to 1.06; P = 0.105). Abstracts that had been registered were more likely to proceed to publication than those that had not (Odds Ratio 3.82; 95% CI 2.73 to 5.35; P < 0.001). The proportion of randomized controlled trials being prospectively registered in anesthesia remains low. Discrepancies between registry entries and corresponding journal publications are common. There was no association between prospective trial registration and subsequent positive outcomes. There was a strong association between prospective trial registration and the likelihood of progression to journal publication.

Serious reporting deficiencies exist in minimal important difference studies: Current state and suggestions for improvement

OBJECTIVE

To evaluate reporting of minimal important difference (MID) estimates using anchor-based methods for patient-reported outcome measures (PROMs), and the association with reporting deficiencies on their credibility.

STUDY DESIGN AND SETTING

Systematic survey of primary studies empirically estimating MIDs. We searched Medline, EMBASE, PsycINFO, and the Patient-Reported Outcome and Quality of Life Instruments Database until October 2018. We evaluated study reporting, focusing on participants' demographics, intervention(s), characteristics of PROMs and anchors, and MID estimation method(s). We assessed the impact of reporting issues on credibility of MID estimates.

RESULTS

In 585 studies reporting on 5,324 MID estimates for 526 distinct PROMs, authors frequently failed to adequately report key characteristics of PROMs and MIDs, including minimum and maximum values of PROM scale, measure of variability accompanying the MID estimate and number of participants included in the MID calculation. Across MID estimates (n=5,324), the most serious reporting issues impacting credibility included infrequent reporting of the correlation between the anchor and PROM (66%), inadequate details to judge precision of MID point estimate (13%), and insufficient information about the threshold used to ascertain MIDs (16%).

CONCLUSION

Serious issues of incomplete reporting in the MID literature threaten the optimal use of MID estimates to inform the magnitude of effects of interventions on PROMs.

Mapping Chilean clinical research: a protocol for a scoping review and multiple evidence gap maps

INTRODUCTION

Clinical research broadly aims to influence decision-making in order to promote appropriate healthcare. Funding agencies should prioritise research projects according to needed research topics, methodological and cost-effectiveness considerations, and expected social value. In Chile, there is no local diagnosis regarding recent clinical research that might inform prioritisation for future research funding. This research aims to comprehensively identify and classify Chilean health research studies, elaborating evidence gap maps for the most burdensome local conditions.

METHODS AND ANALYSIS

We will search in electronic databases (MEDLINE, Embase, PsycINFO, CINAHL, LILACS and WoS) and perform hand searches to retrieve, identify and classify health research studies conducted in Chile or by authors whose affiliations are based in Chile, from 2000 onwards. We will elaborate evidence matrices for the 20 conditions with the highest burden in Chile (according to the Global Burden of Disease 2019) selected from those defined under the General Regime of the Health Guarantees Act. To elaborate the evidence gap maps, we will consider prioritised interventions and core outcome sets. To identify knowledge gaps and estimate redundant research, we will contrast these gap maps with the available international evidence of high or moderate certainty of evidence, for each specific clinical question. For this purpose, we will search systematic reviews using the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) approach.

ETHICS AND DISSEMINATION

No ethical approval is required to conduct this project. We will submit our results in both peer-reviewed journals and scientific conferences. We will aim to disseminate our findings through different academic platforms, social media, local press, among others. The final results will be communicated to local funding agencies and government stakeholders.

DISCUSSION

We aim to provide an accurate and up-to-date picture of the research gaps-to be filled by new future findings-and the identification of redundant research, which will constitute relevant information for local decision-makers.

Describing and Characterizing the Literature Regarding Umbilical Health in Intensively Raised Cattle: A Scoping Review

The objective of this scoping review was to describe and characterize the existing literature regarding umbilical health and identify gaps in knowledge. Six databases were searched for studies examining umbilical health in an intensively raised cattle population. There were 4249 articles initially identified; from these, 723 full text articles were then screened, with 150 articles included in the review. Studies were conducted in the USA (n = 41), Brazil (n = 24), Canada (n = 13), UK (n = 10), and 37 additional countries. Seventeen were classified as descriptive, 24 were clinical trials, and 109 were analytical observational studies. Umbilical outcomes evaluated in descriptive studies were infection (n = 11), parasitic infection (n = 5), and hernias (n = 2). Of the clinical trials, only one examined treatment of navel infections; the remainder evaluated preventative management factors for navel health outcomes (including infections (n = 17), myiasis (n = 3), measurements (n = 5), hernias (n = 1), and edema (n = 1)). Analytical observational studies examined risk factors for umbilical health (n = 60) and umbilical health as a risk factor (n = 60). Studies examining risk factors for umbilical health included navel health outcomes of infections (n = 28; 11 of which were not further defined), hernias (n = 8), scoring the navel sheath/flap size (n = 16), myiasis (n = 2), and measurements (n = 6). Studies examining umbilical health as a risk factor defined these risk factors as infection (n = 39; of which 13 were not further defined), hernias (n = 8; of which 4 were not further defined), navel dipping (n = 12), navel/sheath scores as part of conformation classification for breeding (n = 2), measurements (n = 3), and umbilical cord drying times (n = 2). This review highlights the areas in need of future umbilical health research such as clinical trials evaluating the efficacy of different treatments for umbilical infection. It also emphasizes the importance for future studies to clearly define umbilical health outcomes of interest, and consider standardization of these measures, including time at risk.

Publication bias in randomized controlled trials in dentistry. What factors affect statistical significance of outcomes?

OBJECTIVES

To record the proportion of Randomized Controlled Trials (RCTs) reporting significant (versus non- significant) primary outcomes, published across 12 high impact journals in Dentistry, covering 6 specialty domains. Associations with certain journal, publication and outcome characteristics were examined.

METHODS

We identified and included all RCTs published from January 1st, 2017 to December 31st, 2021 in the two journals with the highest impact factors (Clarivate Analytics, 2020) from each of the following domains: Periodontology, Endodontics, Restorative Dentistry/ Prosthodontics, Orthodontics, Paediatric Dentistry, Oral and Maxillofacial Surgery. The primary outcome was the proportion of significant/ non- significant findings reported for the primary outcomes under study, while a range of characteristics such as: journal, year of publication, impact factor, funding, registration and others, were tested for associations.

RESULTS

A total of 474 RCTs were identified and included, with the majority reporting statistically significant outcomes (321/474; 67.7%). The multivariable model revealed significant effects of predictors related to specialty domain (p = 0.01), continent (p = 0.003) and registration (p = 0.004). Compared to Periodontology, RCTs published in Endodontics (OR= 0.40; 95%CIs: 0.22, 0.76) and Orthodontics (OR= 0.41; 95%CIs: 0.23, 0.74) were less likely to present statistically significant effects. There was strong evidence that registered trials presented lower odds of reporting statistically significant findings (OR= 0.52; 95%CIs: 0.34, 0.81).

CONCLUSIONS

The entirety of dentistry domains demonstrated preferential publication practices of outcomes considered as "successful" and statistically significant, with domains such as Orthodontics and Endodontics being more balanced. Trial non- registration is still prevalent and associated with reporting of statistically significant effects.

CLINICAL SIGNIFICANCE

The findings of this empirical report bring attention to the interpretation of Systematic Reviews (SRs) conclusions. These largely depend on the availability and nature of outcomes of randomized controlled trials (RCTs) on a topic, which may impact on the synthesized estimate of a pooled effect and its direction.

How to increase value and reduce waste in research: initial experiences of applying Lean thinking and visual management in research leadership

Waste in research has been well documented, but initiatives to reduce it are scarce. Here, we share our initial experiences of implementing Lean thinking and visual management into hospital research units in the Region of Southern Denmark. A Transformation Guiding Team (TGT) anchored in the top management was established with participation from leaders, researchers and patient representatives. The role of the TGT was to implement Lean methods, considering patients as primary end-users of the research results. This is in line with an explicit decision on setting patient values first in clinical settings at participating hospitals. The leaders of the research units were instructed in Lean thinking and Lean methods during a five-module course focusing on increasing value and reducing waste in research production. Initial experiences were that Lean tools could create a patient-centred vision that through visual management could identify waste in work processes. Concerns were lack of evidence for using Lean methods in research leadership and that the model itself could be a time consumer. Some lessons learnt were that adding Lean tools in research leadership should not just provide increased research productivity, but also improve other important key performance indicators such as quality of research and patient-relevant results. We intend to evaluate the value of the initiative by follow-up research and publish the outcome of key behavioural and key performance indicators.

Writing up your clinical trial report for a scientific journal: the REPORT trial guide for effective and transparent research reporting without spin

The REPORT guide is a 'How to' guide to help you report your clinical research in an effective and transparent way. It is intended to supplement established first choice reporting tools, such as Consolidated Standards of Reporting Trials (CONSORT), by adding tacit knowledge (ie, learnt, informal or implicit knowledge) about reporting topics that we have struggled with as authors or see others struggle with as journal reviewers or editors. We focus on the randomised controlled trial, but the guide also applies to other study designs. Topics included in the REPORT guide cover reporting checklists, trial report structure, choice of title, writing style, trial registry and reporting consistency, spin or reporting bias, transparent data presentation (figures), open access considerations, data sharing and more. Preprint (open access): https://doi.org/10.31219/osf.io/qsxdz.

Choosing Wisely: An idea worth sustaining

OBJECTIVES

To evaluate the sustainability potential of Choosing Wisely (CW) to address unnecessary medical care at Ontario community hospitals.

DATA SOURCES/STUDY SETTING

Ontario community hospitals and their affiliated family health teams (FHTs).

STUDY DESIGN

A mixed-methods study involving the administration of a validated sustainability survey to CW implementation teams followed by their participation in focus groups.

DATA COLLECTION/EXTRACTION METHODS

Survey data were collected using an Excel file with an embedded, automated scoring system. We collated individual survey scores and generated aggregate team scores. We also performed descriptive statistics for quantitative data (frequencies, means). Qualitative data were triangulated with quantitative assessments to support data interpretations using the meta-matrix method.

PRINCIPAL FINDINGS

Fifteen CW implementation teams across four Ontario community hospitals and six affiliated primary care FHTs participated. CW priority areas investigated were de-prescribing of proton pump inhibitors (PPIs) and reducing Pre-Op testing and BUN/Urea lab testing. Survey results showed steady improvements in sustainability scores from baseline to final follow-up among most implementation teams: 10% increase for PPI de-prescribing (six FHTs) and 2% increase (three hospital teams); 18% increase in BUN/Urea lab testing (three hospital teams). Regardless of site or CW priority area, common facilitators were fit with existing processes and workflows, leadership support, and optimized team communication; common challenges were lack of awareness and buy-in, leadership engagement or a champion, and lack of fit with existing workflow and culture. All teams identified at least one challenge for which they co-designed and implemented a plan to maximize the sustainability potential of their CW initiative.

CONCLUSIONS

Evaluating the sustainability potential of an innovation such as Choosing Wisely is critical to ensuring that they have the best potential for impact. Our work highlights that implementation teams can be empowered to influence implementation efforts and to realize positive outcomes for their health care services and patients.

Reporting quality of sham needles used as controls in acupuncture trials: a methodological evaluation

OBJECTIVE

The description of controls is important in acupuncture clinical trials to interpret its effectiveness without fallacy. This paper aims to evaluate the reporting quality of acupuncture studies on the characteristics of sham needles.

STUDY DESIGN AND SETTING

Using a checklist developed from previously published reporting guidelines, the distribution of reported items and changes of reporting rates over time were investigated. Two-way ANOVA and linear regression were conducted.

RESULTS

Original articles of RCTs of any design involving sham needles as controls were eligible for assessment. 117 trials from three 2-year time periods between 2009 and 2018 were included. Seven items out of 25 were reported in more than 50% of the studies. While significant differences of reporting scores among categories were observed, there were no significant differences among time periods; no significant improvement was observed over time.

CONCLUSIONS

Low reporting qualities of sham needles used in acupuncture studies may influence how researchers understand the effectiveness of acupuncture. This study evaluated previous publications from 2009 to 2018 and found that reporting qualities on sham needles did not improve over time. Further studies are required to validate the items used in this study to endorse better reporting of controls in acupuncture trials.

Structured reporting to improve transparency of analyses in prognostic marker studies

BACKGROUND

Factors contributing to the lack of understanding of research studies include poor reporting practices, such as selective reporting of statistically significant findings or insufficient methodological details. Systematic reviews have shown that prognostic factor studies continue to be poorly reported, even for important aspects, such as the effective sample size. The REMARK reporting guidelines support researchers in reporting key aspects of tumor marker prognostic studies. The REMARK profile was proposed to augment these guidelines to aid in structured reporting with an emphasis on including all aspects of analyses conducted.

METHODS

A systematic search of prognostic factor studies was conducted, and fifteen studies published in 2015 were selected, three from each of five oncology journals. A paper was eligible for selection if it included survival outcomes and multivariable models were used in the statistical analyses. For each study, we summarized the key information in a REMARK profile consisting of details about the patient population with available variables and follow-up data, and a list of all analyses conducted.

RESULTS

Structured profiles allow an easy assessment if reporting of a study only has weaknesses or if it is poor because many relevant details are missing. Studies had incomplete reporting of exclusion of patients, missing information about the number of events, or lacked details about statistical analyses, e.g., subgroup analyses in small populations without any information about the number of events. Profiles exhibit severe weaknesses in the reporting of more than 50% of the studies. The quality of analyses was not assessed, but some profiles exhibit several deficits at a glance.

CONCLUSIONS

A substantial part of prognostic factor studies is poorly reported and analyzed, with severe consequences for related systematic reviews and meta-analyses. We consider inadequate reporting of single studies as one of the most important reasons that the clinical relevance of most markers is still unclear after years of research and dozens of publications. We conclude that structured reporting is an important step to improve the quality of prognostic marker research and discuss its role in the context of selective reporting, meta-analysis, study registration, predefined statistical analysis plans, and improvement of marker research.

Identifying Predatory Journals in Plastic Surgery: A Prospective Study

Background

Predatory journals promise high acceptance rates and quick publication in exchange for a processing fee. As these journals aim to maximize profits, they neglect traditional mechanisms used to ensure a high-quality publication. Unsolicited email invitations are a characteristic of predatory journals that often inundate the inboxes of surgeons. The objective of this study is to use these emails to identify potentially predatory journals in the area of surgery and plastic surgery.

Methods

Unsolicited email requests from surgery-related journals were collected over a 3-month period. Journals were evaluated using a modified version of the Rohrich and Weinstein checklist. The average number of "predatory" criteria met by these potentially predatory journals (PPJs) was compared to that of the top open-access plastic surgery journals which were assumed to be non-predatory for the purposes of this study.

Results

In total, 437 unsolicited email requests were received. Of these, 92 emails, representing 57 PPJs, were eligible for inclusion. On average, the PPJs met 5 of the 12 "predatory" criteria, compared to less than 1 in the comparison group. Approximately 96% of these emails, or the respective websites, contained obvious spelling or grammatical mistakes; 98% of these emails came from journals not listed on Scopus, Directory of Open Access Journals (DOAJ), and/or Web of Science.

Conclusions

Of the journals that sent unsolicited emails, 98% met 2 or more criteria and were deemed to be predatory. If a journal contains grammatical mistakes and is not listed on Scopus, DOAJ, and/or Web of Science, authors should be cautious.

Normative Approaches for Oral Health: Standards, Specifications, and Guidelines

Normative approaches have been developed with the aim of providing high-quality methods and strict criteria that, when applied correctly, lead to reliable results. Standards, specifications, and guidelines are needed to facilitate exchange of goods or information and secure comparability of data derived from different laboratories and sources. They are available along the whole flow from study development to test selection, study conduct, and reporting and are widely used for the evaluation of medical devices, market approval, and harmonization of terms and devices. Standards are developed by specific national and international organizations or by dedicated interest groups, mainly scientists in their respective fields. ISO (International Organization for Standardization) standards are developed following stringent regulations, and groups of experts formulate such standards. They should come from different areas (multistakeholder approach) to have as much and as broad input as possible and to avoid single-interest dominance. However, the presence of academia in such groups has been comparatively low. There is a clear need and responsibility of the oral health community to participate in the development of normative documents to provide methodological knowledge and experience, balance the interests of other stakeholders, and finally improve oral health. This will help to ensure that rapidly advancing fields of research, such as the oral health impacts of COVID-19 or the application of artificial intelligence in dentistry, benefit from standardization of approaches and reporting.

Do German university medical centres promote robust and transparent research? A cross-sectional study of institutional policies

BACKGROUND

In light of replication and translational failures, biomedical research practices have recently come under scrutiny. Experts have pointed out that the current incentive structures at research institutions do not sufficiently incentivise researchers to invest in robustness and transparency and instead incentivise them to optimize their fitness in the struggle for publications and grants. This cross-sectional study aimed to describe whether and how relevant policies of university medical centres in Germany support the robust and transparent conduct of research and how prevalent traditional metrics are.

METHODS

For 38 German university medical centres, we searched for institutional policies for academic degrees and academic appointments as well as websites for their core facilities and research in general between December 2020 and February 2021. We screened the documents for mentions of indicators of robust and transparent research (study registration; reporting of results; sharing of research data, code and protocols; open access; and measures to increase robustness) and for mentions of more traditional metrics of career progression (number of publications; number and value of awarded grants; impact factors; and authorship order).

RESULTS

While open access was mentioned in 16% of PhD regulations, other indicators of robust and transparent research were mentioned in less than 10% of institutional policies for academic degrees and academic appointments. These indicators were more frequently mentioned on the core facility and general research websites. Institutional policies for academic degrees and academic appointments had frequent mentions of traditional metrics.

CONCLUSIONS

References to robust and transparent research practices are, with a few exceptions, generally uncommon in institutional policies at German university medical centres, while traditional criteria for academic promotion and tenure still prevail.

Review of dietary assessment studies conducted among Khmer populations living in Cambodia

Background

Despite economic growth, Cambodia continues to have high rates of malnutrition, anaemia and nutrition‐related deficiencies. Government policies promote nutrition strategies, although dietary intake data is limited. A detailed synthesis of existing intake data is needed to inform nutrition policy and practice change. This review aims to characterise and assess quality of dietary assessment methods and outcomes from individual‐level ‘whole diet’ studies of Khmer people living in Cambodia.

Methods

Searches were conducted using PRISMA‐ScR guidelines. Included papers reported dietary intake at an individual level for ‘whole diet’. Studies using secondary data or lacking dietary assessment details were excluded. Extracted data included dietary assessment features, nutrient/food group intakes and database.

Results

Nineteen publications (15 studies) were included, with nine carried out among children under 5 years and six among women. Eleven studies reported intake by food groups and four by nutrients, prominently energy, protein, vitamin A, iron, calcium and zinc. Inconsistent intakes, food groupings and reporting of study characteristics limited data synthesis. All but one study used 24‐h recalls. Trained local fieldworkers used traditional interview‐administered data collection and varied portion estimation tools. Food composition databases for analysis were not tailored to the Cambodian diet. Overall quality was rated as ‘good’.

Conclusions

We recommend the development of a best‐practice protocol for conducting dietary assessment, a Cambodia‐specific food composition database and a competent trained workforce of nutrition professionals, with global support of expertise and funding for future dietary assessment studies conducted in Cambodia.

Fifteen studies with highly variable intake data included in the review.

The food composition databases used were not specific to Cambodian diet.

Minimum reporting standards and best practice protocols recommended, including in‐country nutrition training.

Lack of whole population dietary intake data indicates the need for a national survey.

Adherence in leading medical journals to the CONSORT 2010 statement for reporting of binary outcomes in randomised controlled trials: cross-sectional analysis

Clinicians and lay people tend to overestimate the effectiveness of a treatment when only the relative effect is presented, particularly if the relative effect is large, but the absolute effect is small. In recognition of this problem, item 17b of The Consolidated Standards of Reporting Trials (CONSORT) 2010 statement stipulates authors present both absolute and relative effects for binary outcomes in randomised controlled trials (RCTs). Adherence to item 17b and the effect of differing levels of CONSORT endorsement by journals on adherence is not well known. We assessed the extent to which item 17b is adhered to in 258 RCTs published in five leading medical journals (Annals of Internal Medicine, BMJ, JAMA, The Lancet and The New England Journal of Medicine) between January and December 2019 that all endorsed the CONSORT statement to varying degrees. Only 53 of 258 (20.5%; 95% CI 15.8% to 26.0%) included studies adhered fully to item 17b. Proportional adherence was higher in journals that endorsed the statement more strictly (BMJ and JAMA, 47.4% [34.0% to 61.0%]) compared with journals less strict in their endorsement (NEJM and Ann Intern Med, 12.2% [7.0% to 19.3%]; The Lancet, 14.1% [7.3% to 23.8%]). Journals that only recommend author adherence to CONSORT had a greater proportion of studies reporting only relative effects in the main results section (62.6%) and abstract (64.2%) compared with journals that require authors to submit a completed checklist (24.6% and 29.8%, respectively). The majority of RCTs (79.5%) with binary primary outcomes published in five leading medical journals during 2019 do not report both absolute and relative effect estimates as per item 17b of the CONSORT guideline despite its universal endorsement. Differences in adherence were observed between journals that endorsed the CONSORT statement to differing extents.

Anesthesia and neurotoxicity study design, execution, and reporting in the nonhuman primate: A systematic review

BACKGROUND

Concern for a role of anesthesia in neurotoxicity in children originated from neonatal rodent and nonhuman primate (NHP) models, yet prospective clinical studies have largely not supported this concern. The goal of this study was to conduct an objective assessment of published NHP study rigor in design, execution, and reporting.

METHODS

A MEDLINE search from 2005 to December 2021 was performed. Inclusion criteria included full-length original studies published in English under peer-reviewed journals. We documented experimental parameters on anesthetic dosing, monitoring, vitals, and experimental outcomes.

RESULTS

Twenty-three manuscripts were included. Critical issues identified in study design included: lack of blinding in data acquisition (57%) and analysis (100%), supratherapeutic (4-12 fold) maintenance dosing in 22% of studies, lack of sample size justification (91%) resulting in a mean (SD) sample size of 6 (3) animals per group. Critical items identified in the conduct and reporting of studies included: documentation of anesthesia provider (0%), electrocardiogram monitoring (35%), arterial monitoring (4%), spontaneous ventilation employed (35%), failed intubations resulting in comingling ventilated and unventilated animals in data analysis, inaccurate reporting of failed intubation, and only 50% reporting on survival. Inconsistencies were noted in drug-related induction of neuroapoptosis and region of occurrence. Further, 67%-100% of behavior outcomes were not significantly different from controls.

CONCLUSIONS

Important deficits in study design, execution, and reporting were identified in neonatal NHP studies. These results raise concern for the validity and reliability of these studies and may explain in part the divergence from results obtained in human neonates.

Knowledge translation concerns for the CONSORT-PRO extension reporting guidance: a review of reviews

This review of reviews aimed to appraise the use of the CONSORT-PRO Extension as an evaluation tool for assessing the reporting of patient-reported outcome (PROs) in publications, and to describe the reporting of PRO research across reviews. We also outlined how variation in such evaluations impacts knowledge translation and may lead to potential misuse of the CONSORT-PRO Extension. We systematically searched Medline, Pubmed and CINAHL from 2013 to 2025 March 2021 for reviews of the completeness of reporting of PRO endpoints according to CONSORT-PRO criteria. Two reviewers extracted details of each review, the percentage of included studies that addressed each CONSORT-PRO item, and key recommendations from each review. Fourteen reviews met inclusion criteria, and only six of these used the full CONSORT-PRO checklist with minimal justified modifications. The remaining eight studies made significant or unjustified adjustments to the CONSORT-PRO Extension. Review studies also varied in how they scored multi-component CONSORT-PRO items. CONSORT-PRO items were often unreported in trial reports, and certain CONSORT-PRO items were reported less often than others. The reporting of statistical approaches to dealing with missing PRO data were poor in RCTs included in all 14 review articles. Studies reviewing PRO publications often omitted recommended CONSORT-PRO items from their evaluations, which may cause confusion among readers regarding how best to report their PRO research according to the CONSORT-PRO extension. Many trials published since CONSORT-PRO's release did not report recommended CONSORT-PRO items, which may lead to misinterpretation and consequently to research waste.

Reducing research wastage by starting off on the right foot: optimally framing the research question

PURPOSE

Strongly framed research questions are clear as to the population (P), the exposures or interventions (E/I), comparison groups (C), outcomes (O), time when relevant (T), and what the investigator wants to know. A solid framework sets up the measurement model, analysis, and anticipated results. The purpose of this study was to estimate the extent to which research questions in journals that focused on patient-reported outcome measures (PROM) and quality of life (QOL) are clear.

METHODS

All 440 research articles published in four PROM journals in 2020. excluding reviews, psychometric, and qualitative papers, were reviewed. Research questions were classified as: (i) adequately framed (ii) poorly framed; or (iii) unframed based on clarity criteria. Examples from each journal were presented and reframed to match results in the article.

RESULTS

Of 440 articles, 195 (44.3%) were classified as adequately framed; 230 (52.2%) as poorly framed; and 15 (3.4%) as unframed. There was heterogeneity across journals (Chi-square: 20.8; 6 df; p = 0.002). Only 29% were framed according to what the investigators wanted to know; 72% were framed like a "to do" list; and 6% were framed as a research agenda.

CONCLUSION

Almost half of the questions were poorly framed or unframed a practice that could contribute to research wastage. Even "adequately framed" questions rarely stated what they wanted to know a priori, increasing the risk of biased reporting. Researchers, reviewers, and editors should encourage the use established frameworks for research questions.

Assessment of transparency and selective reporting of interventional trials studying colorectal cancer

Background

Colorectal cancer (CRC) is currently one of the most frequently diagnosed cancers. Our aim was to evaluate transparency and selective reporting in interventional trials studying CRC.

Methods

First, we assessed indicators of transparency with completeness of reporting, according to the CONSORT statement, and data sharing. We evaluated a selection of reporting items for a sample of randomized controlled trials (RCTs) studying CRC with published full-text articles between 2021–03-22 and 2018–03-22. Selected items were issued from the previously published CONSORT based peer-review tool (COBPeer tool). Then, we evaluated selective reporting through retrospective registration and primary outcome(s) switching between registration and publication. Finally, we determined if primary outcome(s) switching favored significant outcomes.

Results

We evaluated 101 RCTs with published full-text articles between 2021–03-22 and 2018–03-22. Five trials (5%) reported all selected CONSORT items completely. Seventy-four (73%), 53 (52%) and 13 (13%) trials reported the primary outcome(s), the allocation concealment process and harms completely. Twenty-five (25%) trials were willing to share data. In our sample, 49 (49%) trials were retrospectively registered and 23 (23%) trials had primary outcome(s) switching. The influence of primary outcome(s) switching could be evaluated in 16 (16/23 = 70%) trials, with 6 (6/16 = 38%) trials showing a discrepancy that favored statistically significant results.

Conclusions

Our results highlight a lack of transparency as well as frequent selective reporting in interventional trials studying CRC.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12885-022-09334-5.

Challenges and opportunities for using population health data to investigate cancer survivors' quality of life in Australia

There is a recognised need for reported national data that inform health policy, health professions, and consumers about the wellbeing of Australians with cancer and other chronic conditions. International initiatives have demonstrated the viability and benefits of utilising population-based cancer registries to monitor the prevalence and trajectory of health-related quality of life (HRQOL) outcomes among people with cancer. Establishing a similar level of monitoring in Australia would require timely access to health data collected by publicly funded, population-based cancer registries, and the capacity to link this information across jurisdictions. Combining information from different sources via data linkage is an efficient and cost-effective way to maximise how data are used to inform population health and policy development. However, linking health datasets has historically been highly restricted, resource-intensive, and costly in Australia due to complex and outdated legislative requirements, duplicative approval processes, and differing policy frameworks in each state and territory. This has resulted in significant research waste due to underutilisation of existing data, duplication of research efforts and resources, and data not being translated into decision-making. Recognising these challenges, from 2015 to 2017 the Productivity Commission investigated options for improving data availability and use in Australia, considering factors such as privacy, security, and intellectual property. The inquiry report recommended significant reforms for Australian legislation, including the creation of a data sharing and release structure to improve access to data for research and policy development purposes. This paper discusses (1) opportunities in HRQOL research enabled by data linkage, (2) barriers to data access and use in Australia and the implications for waste in HRQOL research, and (3) proposed legislative reforms for improving data availability and use in Australia.

Transparent, Open, and Reproducible Prevention Science

The field of prevention science aims to understand societal problems, identify effective interventions, and translate scientific evidence into policy and practice. There is growing interest among prevention scientists in the potential for transparency, openness, and reproducibility to facilitate this mission by providing opportunities to align scientific practice with scientific ideals, accelerate scientific discovery, and broaden access to scientific knowledge. The overarching goal of this manuscript is to serve as a primer introducing and providing an overview of open science for prevention researchers. In this paper, we discuss factors motivating interest in transparency and reproducibility, research practices associated with open science, and stakeholders engaged in and impacted by open science reform efforts. In addition, we discuss how and why different types of prevention research could incorporate open science practices, as well as ways that prevention science tools and methods could be leveraged to advance the wider open science movement. To promote further discussion, we conclude with potential reservations and challenges for the field of prevention science to address as it transitions to greater transparency, openness, and reproducibility. Throughout, we identify activities that aim to strengthen the reliability and efficiency of prevention science, facilitate access to its products and outputs, and promote collaborative and inclusive participation in research activities. By embracing principles of transparency, openness, and reproducibility, prevention science can better achieve its mission to advance evidence-based solutions to promote individual and collective well-being.