Increasing value and reducing waste in biomedical research regulation and management

Participant recruitment and retention in randomised controlled trials of melanoma surveillance: A scoping review

Background

This scoping review aims to collate and describe data on recruitment, retention, and strategies used to improve these, in randomised controlled trials of melanoma surveillance.

Methods

We searched MEDLINE, EMBASE, CINAHL and CENTRAL databases from inception until October 23, 2023. Two reviewers screened titles and abstracts, and full-texts, and one reviewer extracted data (convenience sample (n = 5) checked by a second). Eligibility criteria included: (i) RCT design, (ii) clinical setting, (iii) participants at increased risk of melanoma, (iv) interventions for early melanoma detection, and (v) early detection outcomes or surrogates such as improved skin self-examination. We calculated summary statistics and undertook qualitative data synthesis.

Results

From 1746 records, 21 trials (reported in 28 papers) were included. Recruitment sources included dermatology clinics, general practice sites, and hospital databases or registries. Trials reported proportions of those screened who were eligible (mean 75 %, range 24-100 %), proportions of those eligible who were randomised (mean 63 %, range 24-95 %), numbers randomised per month (mean 25 participants, range 2-74), and proportion of those randomised who completed outcome measurements (mean 85 %, range 59-100 %) for self-report questionnaires at primary timepoints). Recruitment strategies included targeted participant identification and flexible consent processes. Retention strategies included setting narrow eligibility criteria, reminders, and financial incentives. Reporting on strategies was limited and there were no reports on effectiveness. Few studies reported recruiter facing initiatives or public and patient involvement.

Discussion

More consistent and detailed reporting of recruitment and retention strategies in RCTs is needed, alongside evaluations of their effectiveness.

Barriers and Enablers of Primary Healthcare Professionals in Health Research Engagement: A Systematic Review of Qualitative Studies

Health professional engagement ensures relevant, clinically focused research that informs evidence-based care. Research shows health professionals may not engage optimally in research. Understanding barriers and enablers influencing participation is necessary to enhance engagement. This systematic review explores these factors among primary healthcare professionals. We searched peer-reviewed studies using CINAHL, Medline, and SCOPUS in February 2023, updated in June 2024. The review followed PRISMA and the ENTREQ checklist. Studies included those published in English, reporting factors influencing engagement among primary healthcare professionals. Qualitative data were extracted and thematically synthesized. Methodological quality was assessed using the Joanna Briggs Institute Critical Appraisal Checklist. Nineteen studies were included. Enablers include individual positive attitudes and a scholarly environment. Barriers include negative attitudes, an unconducive environment, and system constraints. Primary health professionals view research engagement positively, recognizing its potential to enhance health outcomes, professional growth, and business performance. Balancing clinical responsibilities, workload and research is challenging. Targeted strategies promoting partnerships and stakeholder involvement can foster a scholarly environment and empower research engagement.

The case for development of a core outcome set (COS) and supplemental reporting guidelines for influenza vaccine challenge trial research in swine

Previously, we systematically reviewed more than 20 years of influenza vaccine challenge trial research in pigs to answer the question, "does vaccinating sows protect offspring?" Overall, most studies were well designed but clinical heterogeneity made between-study comparisons challenging. Studies varied by samples, outcomes, and assays selected for measurement. Additionally, data essential for inclusion of findings in meta-analyses were often insufficiently reported and as a result, summary effect measures were either not derived or were not meaningful. Clinical heterogeneity and reporting issues complicate and limit what can be learned cumulatively from research and both represent two types of avoidable research waste. Here, we illustrate each concern using data collected tangentially during the systematic review and propose two corrective strategies, both of which have broad applicability across veterinary intervention research; (i) develop a Core Outcome Set (COS) to reduce unnecessary clinical heterogeneity in future research and (ii) encourage funders and journal editors to require submitted research protocols and manuscripts adhere to established reporting guidelines. As a reporting corollary, we developed a supplemental checklist specific to influenza vaccine challenge trial research in swine and propose that it is completed by researchers and included with all study protocol and manuscript submissions. The checklist serves two purposes: as a reminder of details essential to report for inclusion of findings in meta-analyses and sub-group meta-analyses (e.g., antigenic or genomic descriptions of influenza vaccine and challenge viruses), and as an aid to help synthesis researchers fully characterize and comprehensively include studies in reviews.

Poor hypotheses and research waste in biology: learning from a theory crisis in psychology

While psychologists have extensively discussed the notion of a "theory crisis" arising from vague and incorrect hypotheses, there has been no debate about such a crisis in biology. However, biologists have long discussed communication failures between theoreticians and empiricists. We argue such failure is one aspect of a theory crisis because misapplied and misunderstood theories lead to poor hypotheses and research waste. We review its solutions and compare them with methodology-focused solutions proposed for replication crises. We conclude by discussing how promoting inclusion, diversity, equity, and accessibility (IDEA) in theoretical biology could contribute to ameliorating breakdowns in the theory-empirical cycle.

How will we prepare for an uncertain future? The value of open data and code for unborn generations facing climate change

As the impacts of climate change continue to intensify, humans face new challenges to long-term survival. Humans will likely be battling these problems long after 2100, when many climate projections currently end. A more forward-thinking view on our science and its direction may help better prepare for the future of our species. Researchers may consider datasets the basic units of knowledge, whose preservation is arguably more important than the articles that are written about them. Storing data and code in long-term repositories offers insurance against our uncertain future. To ensure open data are useful, data must be FAIR (Findable, Accessible, Interoperable and Reusable) and be complete with all appropriate metadata. By embracing open science practices, contemporary scientists give the future of humanity the information to make better decisions, save time and other valuable resources, and increase global equity as access to information is made free. This, in turn, could enable and inspire a diversity of solutions, to the benefit of many. Imagine the collective science conducted, the models built, and the questions answered if all of the data researchers have collectively gathered were organized and immediately accessible and usable by everyone. Investing in open science today may ensure a brighter future for unborn generations.

Trial Waste in Hyaluronic Acid-Related Randomised Controlled Trials

BACKGROUND

A notable obstacle in applying the findings of hyaluronic acid (HA)-related randomised controlled trials (RCTs) to real-world patient treatment is trial waste (TW). To date, the extent of TW in RCTs for HA is not clear.

OBJECTIVES

To analyse the extents of TW within HA-RCTs and identify protective factors against TW.

METHODS

In July 2024, we searched the ClinicalTrials database using the 'hyaluronic acid' as keyword. We documented the data available and then explored PubMed and Scopus for the publication status. Reporting adequacy was evaluated using the CONSORT checklist. Design limitations were analysed based on bias risk and whether the article referenced a relevant systematic review. Subsequently, we evaluated extent of TW (unpublished studies, insufficient reporting and design flaws).

RESULTS

One hundred and eighty-four RCTs met the inclusion criteria. The analysis of TW excluded 53 RCTs completed after June 2020 that remained unpublished. Among the remaining 131 RCTs, 72 were published, 47 had adequate reporting and 19 had design limitations. Taken together, 96 RCTs (73.3%) exhibited at least one characteristic of TW. Characteristics of these RCTs included early registration (p < 0.001) and the absence of a multi-blind approach (p = 0.007). Registration prior to 2014 (p < 0.001) and the open-label or single-blinding design (p = 0.003) emerged as independent risk factor for TW.

CONCLUSION

We delineated the features of 184 HA-related RCTs. 73.3% of the RCTs included in TW analysis exhibited TW. The diverse traits of the different TW indicators identified could serve as valuable insights for conducting future HA-RCTs more rationally and efficiently.

Which variables are associated with recruitment failure? A nationwide review on obstetrical and gynaecological multicentre RCTs (2003-2023)

OBJECTIVE

We aim to assess which variables are associated with recruitment failure of obstetrical and gynaecological randomised controlled trials (RCTs), leading to an extension of the study period.

DESIGN

Nationwide study.

SETTING

A cohort of RCTs supported by the trial centre of the Dutch Consortium of Obstetrics and Gynaecology.

POPULATION

We included 83 RCTs that recruited patients between 1 March 2003 and 1 December 2023.

MAIN OUTCOME MEASURES

Main outcome was recruitment target not achieved within 6 months after the preplanned recruitment period. Secondary outcomes were recruitment target not achieved within an extension period of at least 12 months and premature termination of the trial. In all RCTs, we collected information on variables with a potential effect on recruitment failure, recorded at five levels; patient, doctor, participating centre, study organisation and study design.

RESULTS

In total, 46 of 83 RCTs (55%) did not achieve their targeted recruitment within the preplanned study period with a maximal extension period of 6 months. The most relevant variables for recruitment failure in multivariable risk prediction modelling were presence of a no-treatment arm (where treatment is standard clinical practice), a compensation fee of less than €200 per included patient, funding of less than €350 000, while a preceding pilot study lowered this risk.

CONCLUSIONS

We identified that the presence of a no-treatment arm, low funding and a low compensation fee per included patient were the most relevant risk factors for recruitment failure within the preplanned period, while a preceding pilot study lowered this risk. Awareness of these variables is important when designing future studies.

Discontinuation and non-publication of randomized controlled trials on cervical cancer or precancer

BACKGROUND

Research waste is a considerable problem in clinical trials, with nonpublication being a significant contributor. We aimed to determine the prevalence of discontinuation and nonpublication of randomized controlled trials (RCTs) on cervical cancer or precancer.

METHODS

We searched ClinicalTrials.gov for registered RCTs investigating cervical cancer or precancer that started between January 2000 and December 2020. The primary and secondary outcomes were trial nonpublication and premature discontinuation, respectively. Publication status was determined by systematic searches of peer-reviewed journals using the PubMed and Scopus databases.

RESULTS

A total of 113 RCTs met the inclusion criteria. Among the 85 trials completed before December 2020, 44 (51.8%) were prematurely discontinued and 40 (47.1%) were unpublished. A single-center design (61.4% vs. 34.1%, P = .012) and lack of external funding (59.1% vs. 36.6%, P = .038) were significantly associated with trial discontinuation. Large-scale (target sample size >400; 46.7% vs. 17.5%, P = .004) and externally funded trials (66.7% vs. 35.0%, P = .004) were more likely to be published. Multivariate logistic analysis revealed that a large sample size [odd ratio (OR): 4.125, 95% confidence interval (CI): 1.511-11.259, P = .006] and presence of external funding (OR: 3.714, 95% CI: 1.513-9.117, P = .004) were independent positive factors for trial publication.

CONCLUSION

A significant proportion of RCTs related to cervical cancer or precancer were discontinued early or remain unpublished, resulting in a waste of research resources.

The trinity of good research: Distinguishing between research integrity, ethics, and governance

The words integrity, ethics, and governance are used interchangeably in relation to research. This masks important differences that must be understood when trying to address concerns regarding research culture. While progress has been made in identifying negative aspects of research culture (such as inequalities in hiring/promotion, perverse incentives, etc.) and practical issues that lead to research waste (outcome reporting bias, reproducibility, etc.), the responsibility for addressing these problems can be unclear due to the complexity of the research environment. One solution is to provide a clearer distinction between the perspectives of "Research Integrity," "Research Ethics," and "Research Governance." Here, it is proposed that Research Integrity should be understood as focused on the character of researchers, and consequently the responsibility for promoting it lies primarily with researchers themselves. This is a different perspective from Research Ethics, which is focused on judgments on the ethical acceptability of research, and should primarily be the responsibility of research ethics committees, often including input from the public as well as the research community. Finally, Research Governance focuses on legal and policy requirements, and although complementary to research integrity and ethics, is primarily the responsibility of expert research support officers with the skills and experience to address technical compliance.

Advancing randomized controlled trials of vascular anomalies: an analysis of trial waste

A series of randomized controlled trials (RCTs) have advanced the therapeutic approaches for vascular anomalies (VA). However, a notable obstacle in applying the findings of these trials to real-world patient care is trial waste (TW). To date, the extent of TW in RCTs for VA is not clear. In June 2024, we searched the ClinicalTrials database using the entity names defined by ISSVA classification as search terms. We documented the data available and then explored PubMed and Scopus for the publication status. Reporting adequacy was evaluated using the CONSORT checklist. Design limitations were analyzed based on bias risk and whether the article referenced a relevant systematic review. One hundred fifty-nine RCTs met the inclusion criteria. The majority of RCTs focused on benign VA (81.1%) and utilized pharmacotherapy (79.9%). Over 90% of these trials were conducted in North America, Europe, and Asia as single-center studies (68.6%), with funding primarily from official institutions (83.7%). The analysis of TW excluded 61 RCTs completed after June 2020 that remained unpublished. Among the remaining 98 RCTs, 53 were published, 41 had adequate reporting, and 16 had design limitations. In total, 67 RCTs exhibited at least one characteristic of TW. The 31 RCTs without waste tended to enroll more participants (P = 0.014) and conduct studies across multiple centers (P < 0.001) and countries (P = 0.022). Multicenter participation (P = 0.028) emerged as an independent protective factor against TW.

CONCLUSION

We delineated the features of 159 VA RCTs and revealed that 68.4% of them exhibited TW. The diverse traits of the different TW indicators identified could serve as valuable insights for conducting future VA RCTs more rationally and efficiently.

WHAT IS KNOWN

• Currently, a number of RCTs have been conducted on vascular anomalies (VA). However, there has been no study analyzing the situation of trial waste in VA-related RCTs.

WHAT IS NEW

• This study is the first to describe the characteristics of VA-related RCTs globally over the past 20 years and has identified a high burden of trial waste in this field. Multicenter participation was an independent protective factor against trial waste.

Development and validation of observational and qualitative study protocol reporting checklists for novice researchers (ObsQual checklist)

INTRODUCTION

Currently, no reporting guidelines exist for observational and qualitative study protocols. In an effort to enhance the quality of research protocols, we introduce two study protocol reporting checklists that we have developed.

MAIN RECOMMENDATIONS

These checklists include educational components and examples intended to assist novice researchers. Through the analysis of 333 study protocols submitted for ethical review, our checklists have been developed and validated, demonstrating their applicability across various observational and qualitative study designs.

CHANGES IN MANAGEMENT

We provide insights into the systematic implementation of these checklists alongside complementary elements that support their effectiveness. We recommend longitudinal monitoring and evaluation of checklist utilization.

"Luck of the draw really": a qualitative exploration of Australian trainee doctors' experiences of mandatory research

BACKGROUND

Many medical trainees, prior to achieving specialist status, are required to complete a mandatory research project, the usefulness of which has been debated. The aim of this study was to gain an in-depth understanding of trainees' experiences and satisfaction of conducting such research projects in Australia.

METHODS

A qualitative descriptive approach was used. Semi-structured interviews with trainees were undertaken between May 2021 and June 2022. Australian medical trainees who had completed a research project as part of specialty training within the past five years were invited to participate. The purposive sample was drawn from participants in a survey on the same topic who had indicated interest in participating in an interview. Interviews explored trainees' overall experience of and satisfaction with conducting research projects, as well as their perceptions of research training, support, barriers, enablers, and perceived benefits. Interviews were transcribed verbatim and thematically analysed.

RESULTS

Sixteen medical doctors from seven medical colleges were interviewed. Trainee experience and satisfaction was highly variable between participants and was shaped by four factors: 1) trainees entered their specialty training with their own perspectives on the value and purpose of the research project, informed by their previous experiences with research and perceived importance of research in their planned career path; 2) in conducting the project, enablers including protected time, supervisor support and institutional structures, were vital to shaping their experience; 3) trainees' access to these enablers was variable, mediated by a combination of luck, and the trainees' own drive and research skill; and 4) project outcomes, in terms of research merit, learning, career benefits and impacts on patient care.

CONCLUSIONS

Trainee experiences of doing research were mixed, with positive experiences often attributed to chance rather than an intentionally structured learning experience. We believe alternatives to mandatory trainee research projects must be explored, including recognising other forms of research learning activities, and directing scarce resources to supporting the few trainees who plan to pursue clinician researcher careers.

Major mistakes or errors in the use of trial sequential analysis in systematic reviews or meta-analyses - the METSA systematic review

BACKGROUND

Systematic reviews and data synthesis of randomised clinical trials play a crucial role in clinical practice, research, and health policy. Trial sequential analysis can be used in systematic reviews to control type I and type II errors, but methodological errors including lack of protocols and transparency are cause for concern. We assessed the reporting of trial sequential analysis.

METHODS

We searched Medline and the Cochrane Database of Systematic Reviews from 1 January 2018 to 31 December 2021 for systematic reviews and meta-analysis reports that include a trial sequential analysis. Only studies with at least two randomised clinical trials analysed in a forest plot and a trial sequential analysis were included. Two independent investigators assessed the studies. We evaluated protocolisation, reporting, and interpretation of the analyses, including their effect on any GRADE evaluation of imprecision.

RESULTS

We included 270 systematic reviews and 274 meta-analysis reports and extracted data from 624 trial sequential analyses. Only 134/270 (50%) systematic reviews planned the trial sequential analysis in the protocol. For analyses on dichotomous outcomes, the proportion of events in the control group was missing in 181/439 (41%), relative risk reduction in 105/439 (24%), alpha in 30/439 (7%), beta in 128/439 (29%), and heterogeneity in 232/439 (53%). For analyses on continuous outcomes, the minimally relevant difference was missing in 125/185 (68%), variance (or standard deviation) in 144/185 (78%), alpha in 23/185 (12%), beta in 63/185 (34%), and heterogeneity in 105/185 (57%). Graphical illustration of the trial sequential analysis was present in 93% of the analyses, however, the Z-curve was wrongly displayed in 135/624 (22%) and 227/624 (36%) did not include futility boundaries. The overall transparency of all 624 analyses was very poor in 236 (38%) and poor in 173 (28%).

CONCLUSIONS

The majority of trial sequential analyses are not transparent when preparing or presenting the required parameters, partly due to missing or poorly conducted protocols. This hampers interpretation, reproducibility, and validity.

STUDY REGISTRATION

PROSPERO CRD42021273811.

Supporting study registration to reduce research waste

An estimated 82-89% of ecological research and 85% of medical research has limited or no value to the end user because of various inefficiencies. We argue that registration and registered reports can enhance the quality and impact of ecological research. Drawing on evidence from other fields, chiefly medicine, we support our claim that registration can reduce research waste. However, increasing registration rates, quality and impact will be very slow without coordinated effort of funders, publishers and research institutions. We therefore call on them to facilitate the adoption of registration by providing adequate support. We outline several aspects to be considered when designing a registration system that would best serve the field of ecology. To further inform the development of such a system, we call for more research to identify the causes of low registration rates in ecology. We suggest short- and long-term actions to bolster registration and reduce research waste.

Adherence to the Guideline for Reporting Evidence-based practice Educational interventions and Teaching (GREET) of studies on evidence-based healthcare e-learning: a cross-sectional study

OBJECTIVES

The objectives of this study are to assess reporting of evidence-based healthcare (EBHC) e-learning interventions using the Guideline for Reporting Evidence-based practice Educational interventions and Teaching (GREET) checklist and explore factors associated with compliant reporting.

DESIGN

Methodological cross-sectional study.

METHODS

Based on the criteria used in an earlier systematic review, we included studies comparing EBHC e-learning and any other form of EBHC training or no EBHC training. We searched Medline, Embase, ERIC, CINAHL, CENTRAL, SCOPUS, Web of Knowledge, PsycInfo, ProQuest and Best Evidence Medical Education up to 4 January 2023. Screening of titles, abstracts, full-text articles and data extraction was done independently by two authors. For each study, we assessed adherence to each of the 17 GREET items and extracted information on possible predictors. Adequacy of reporting for each item of the GREET checklist was judged with yes (provided complete information), no (provided no information), unclear (when insufficient information was provided), or not applicable, when the item was clearly of no relevance to the intervention described (such as for item 8-details about the instructors-in the studies which used electronic, self-paced intervention, without any tutoring). Studies' adherence to the GREET checklist was presented as percentages and absolute numbers. We performed univariate analysis to assess the association of potential adherence predictors with the GREET checklist. We summarised results descriptively.

RESULTS

We included 40 studies, the majority of which assessed e-learning or blended learning and mostly involved medical and other healthcare students. None of the studies fully reported all the GREET items. Overall, the median number of GREET items met (received yes) per study was 8 and third quartile (Q3) of GREET items met per study was 9 (min. 4 max. 14). When we used Q3 of the number of items met as cut-off point, adherence to the GREET reporting checklist was poor with 7 out of 40 studies (17.5%) reporting items of the checklist on acceptable level (adhered to at least 10 items out of 17). None of the studies reported on all 17 GREET items. For 3 items, 80% of included studies well reported information (received yes for these items): item 1 (brief description of intervention), item 4 (evidence-based practice content) and item 6 (educational strategies). Items for which 50% of included studies reported complete information (received yes for these items) included: item 9 (modes of delivery), item 11 (schedule) and 12 (time spent on learning). The items for which 70% or more of included studies did not provide information (received no for these items) included: item 7 (incentives) and item 13 (adaptations; for both items 70% of studies received no for them), item 14 (modifications of educational interventions-95% of studies received no for this item), item 16 (any processes to determine whether the materials and the educational strategies used in the educational intervention were delivered as originally planned-93% of studies received no for this item) and 17 (intervention delivery according to schedule-100% of studies received no for this item). Studies published after September 2016 showed slight improvements in nine reporting items. In the logistic regression models, using the cut-off point of Q3 (10 points or above) the odds of acceptable adherence to GREET guidelines were 7.5 times higher if adherence to other guideline (Consolidated Standards of Reporting Trials, Strengthening the Reporting of Observational Studies in Epidemiology, etc) was reported for a given study type (p=0.039), also higher number of study authors increased the odds of adherence to GREET guidance by 18% (p=0.037).

CONCLUSIONS

Studies assessing educational interventions on EBHC e-learning still poorly adhere to the GREET checklist. Using other reporting guidelines increased the odds of better GREET reporting. Journals should call for the use of appropriate use of reporting guidelines of future studies on teaching EBHC to increase transparency of reporting, decrease unnecessary research duplication and facilitate uptake of research evidence or result.

STUDY REGISTRATION NUMBER

The Open Science Framework (https://doi.org/10.17605/OSF.IO/V86FR).

Research waste among randomized controlled trials in ovarian cancer: A cross-sectional study

OBJECTIVE

To examine the association between trial characteristics and research waste in randomized controlled trials (RCTs) on ovarian cancer over the past two decades.

METHODS

ClinicalTrials.gov was searched for RCTs registered between 2000 and 2020 using the keyword ovarian cancer. Publication status of RCTs was determined through systematic searches of the PubMed and Google Scholar databases. Reporting adequacy was evaluated using the CONSORT checklist. Design limitations were assessed based on the risk of bias and whether a relevant systematic review was cited in the manuscript. The primary outcome was research waste, defined as an RCT that was unpublished, inadequately reported, or had avoidable design limitations.

RESULTS

Among the 117 RCTs evaluated, 89 (76.1 %) were published as of February 14, 2024. Published RCTs were more likely to be pharmacological, conducted in North America or Europe, have a multicenter or multinational design, have a larger sample size (over 200 participants), and receive external funding (P < 0.05). Among the published RCTs, 73 (82.0 %) and 24 (27.0 %) were considered adequately reported and free from design limitations, respectively. Overall, 96 of the 117 RCTs (82.1 %) were associated with research waste. Factors independently associated with research waste were an open-label design and smaller sample size (P < 0.05).

CONCLUSION

Over 80 % of the RCTs on ovarian cancer demonstrated at least one feature of research waste. Future efforts should focus on minimizing the potential waste in unblinded small-scale RCTs.

The response of Canada's clinical health research ecosystem to the COVID-19 pandemic

BACKGROUND

The response of Canada's research community to the COVID-19 pandemic provides a unique opportunity to examine the country's clinical health research ecosystem. We sought to describe patterns of enrolment across Canadian Institutes of Health Research (CIHR)-funded studies on COVID-19.

METHODS

We identified COVID-19 studies funded by the CIHR and that enrolled participants from Canadian acute care hospitals between January 2020 and April 2023. We collected information on study-and site-level variables from study leads, site investigators, and public domain sources. We described and evaluated factors associated with cumulative enrolment.

RESULTS

We obtained information for 23 out of 26 (88%) eligible CIHR-funded studies (16 randomized controlled trials [RCTs] and 7 cohort studies). The 23 studies were managed by 12 Canadian and 3 international coordinating centres. Of 419 Canadian hospitals, 97 (23%) enrolled a total of 28 973 participants - 3876 in RCTs across 78 hospitals (median cumulative enrolment per hospital 30, interquartile range [IQR] 10-61), and 25 097 in cohort studies across 62 hospitals (median cumulative enrolment per hospital 158, IQR 6-348). Of 78 hospitals recruiting participants in RCTs, 13 (17%) enrolled 50% of all RCT participants, whereas 6 of 62 hospitals (9.7%) recruited 54% of participants in cohort studies.

INTERPRETATION

A minority of Canadian hospitals enrolled the majority of participants in CIHR-funded studies on COVID-19. This analysis sheds light on the Canadian health research ecosystem and provides information for multiple key partners to consider ways to realize the full research potential of Canada's health systems.

Retention strategies are routinely communicated to potential trial participants but often differ from what was planned in the trial protocol: an analysis of adult participant information leaflets and their corresponding protocols

BACKGROUND

Retaining participants in randomised controlled trials (RCTs) is challenging and trial teams are often required to use strategies to ensure retention or improve it. Other than monetary incentives, there is no requirement to disclose the use of retention strategies to the participant. Additionally, not all retention strategies are developed at the planning stage, i.e. post-funding during protocol development, but some protocols include strategies for participant retention as retention is considered and planned for early in the trial planning stage. It is yet unknown if these plans are communicated in the corresponding participant information leaflets (PILs). The purpose of our study was to determine if PILs communicate plans to promote participant retention and, if so, are these outlined in the corresponding trial protocol.

METHODS

Ninety-two adult PILs and their 90 corresponding protocols from Clinical Trial Units (CTUs) in the UK were analysed. Directed (deductive) content analysis was used to analyse the participant retention text from the PILs. Data were presented using a narrative summary and frequencies where appropriate.

RESULTS

Plans to promote participant retention were communicated in 81.5% (n = 75/92) of PILs. Fifty-seven percent (n = 43/75) of PILs communicated plans to use "combined strategies" to promote participant retention. The most common individual retention strategy was telling the participants that data collection for the trial would be scheduled during routine care visits (16%; n = 12/75 PILs). The importance of retention and the impact that missing or deleted data (deleting data collected prior to withdrawal) has on the ability to answer the research question were explained in 6.5% (n = 6/92) and 5.4% (n = 5/92) of PILs respectively. Out of the 59 PILs and 58 matching protocols that both communicated plans to use strategies to promote participant retention, 18.6% (n = 11/59) communicated the same information, the remaining 81.4% (n = 48/59) of PILs either only partially communicated (45.8%; n = 27/59) the same information or did not communicate the same information (35.6%; n = 21/59) as the protocol with regard to the retention strategy(ies).

CONCLUSION

Retention strategies are frequently communicated to potential trial participants in PILs; however, the information provided often differs from the content in the corresponding protocol. Participant retention considerations are best done at the planning stage of the trial and we encourage trial teams to be consistent in the communication of these strategies in both the protocol and PIL.

Five aspects of research waste in biomedicine: A scoping review

BACKGROUND

The number of published journal articles has grown exponentially during the last 30 years, which may have led to some wasteful research. However, the terminology associated with research waste remains unclear. To address this, we aimed to identify, define, and categorize the aspects of research waste in published biomedical reports.

METHODS

In this scoping review, we systematically searched for biomedical literature reports from 1993 to 2023 in two databases, focusing on those addressing and defining research waste. Through data charting, we analyzed and categorized the aspects of research waste.

RESULTS

Based on 4285 initial records in the searches, a total of 832 reports were included in the analysis. The included reports were primarily narrative reviews (26%) and original reports (21%). We categorized research waste into five aspects: methodological, invisible, negligible, underreported, and structural (MINUS) research waste. More than half of the reports (56%) covered methodological research waste concerning flaws in study design, study conduct, or analysis. Invisible research waste covered nonpublication, discontinuation, and lack of data-sharing. Negligible research waste primarily concerned unnecessary repetition, for example, stemming from the absence of preceding a trial with a systematic review of the literature. Underreported research waste mainly included poor reporting, resulting in a lack of transparency. Structural research waste comprised inadequate management, collaboration, prioritization, implementation, and dissemination.

CONCLUSION

MINUS encapsulates the five main aspects of research waste. Recognizing these aspects of research waste is important for addressing and preventing further research waste and thereby ensuring efficient resource allocation and scientific integrity.

The use and application of Lean Management methods to research processes-a scoping review protocol

INTRODUCTION

Waste in medical research is a relatively well-known issue. However, only a few initiatives exist to address this issue. Lean Management methods (Lean) were developed in industrial manufacturing and later applied within healthcare improvement. Overall, the results from studies of the application of Lean to healthcare appear to be positive in terms of greater efficiency regarding treatment outcomes and patient care. Nevertheless, the application of Lean to improve research processes is not well studied and, given that research alongside clinical practice and experiential knowledge provides the foundation for the treatment and care of patients, it is paramount to identify approaches and review the degree to which they increase efficiency within research procedures. Therefore, this review will scope the landscape of studies that investigated Lean and how to implement Lean in research processes, particularly regarding healthcare research.

METHODS AND ANALYSIS

Our approach follows the methodological framework of Arksey and O'Malley for conducting scoping reviews (PRISMA-ScR). The search strategy for this scoping review was developed using the PCC model. We will identify the relevant literature by searching four search databases: Scopus, Web of Science, Academic Search Premier and Business Source Complete. Next, we will use citation pearl growing to identify all relevant published literature. The data charting process will follow the PRISMA-ScR checklist and will be organised using NVivo. We will generate qualitative and quantitative assessments of the extracted data by using NVivo, RStudio and Excel. We will follow the PRISMA-ScR guideline when reporting the results.

ETHICS AND DISSEMINATION

The review will comprise existing published studies and no primary data will be collected. Our findings will be shared through open access peer-reviewed journals, national and international conferences and emails to all relevant collaborative relationships. We plan to disseminate our findings via academic social media platforms, newspaper articles and blogposts.

Waste not, want not: call to action for spinal manipulative therapy researchers

BACKGROUND

Research waste is defined as research outcomes with no or minimal societal benefits. It is a widespread problem in the healthcare field. Four primary sources of research waste have been defined: (1) irrelevant or low priority research questions, (2) poor design or methodology, (3) lack of publication, and (4) biased or inadequate reporting. This commentary, which was developed by a multidisciplinary group of researchers with spinal manipulative therapy (SMT) research expertise, discusses waste in SMT research and provides suggestions to improve future research.

MAIN TEXT

This commentary examines common sources of waste in SMT research, focusing on design and methodological issues, by drawing on prior research and examples from clinical and mechanistic SMT studies. Clinical research is dominated by small studies and studies with a high risk of bias. This problem is compounded by systematic reviews that pool heterogenous data from varying populations, settings, and application of SMT. Research focusing on the mechanisms of SMT often fails to address the clinical relevance of mechanisms, relies on very short follow-up periods, and has inadequate control for contextual factors.

CONCLUSIONS

This call to action is directed to researchers in the field of SMT. It is critical that the SMT research community act to improve the way research is designed, conducted, and disseminated. We present specific key action points and resources, which should enhance the quality and usefulness of future SMT research.

Endorsement of open science practices by dental journals: A meta-research study

OBJECTIVES

This study evaluates the endorsement of open science practices by dental journals.

MATERIALS AND METHODS

This was a meta-research study that included journals listed in the 2021 Journal Citation Reports under Dentistry. A comprehensive evaluation was performed by accessing journal websites to ascertain the availability of publicly accessible instructions to authors in Portuguese, English, or Spanish. A researcher extracted information from the "Instructions for Authors" section, encompassing the journal's impact factor, mention of any reporting guidelines, details on data sharing, acceptance of articles in preprint format, and information regarding study protocol registration. Descriptive data analysis was conducted using the Stata 14.0 program, and an Open Science Score (OSS) (ranging from 0 to 100 %) was calculated for each journal by considering five open science practices. Pearson's correlation test was conducted to determine the relationship between the OSS score and journal impact factor.

RESULTS

Ninety journals were included in the study. Most journals (70 %) indicated the mandatory use of reporting guidelines, while 60 % recommended data sharing. Conversely, 46.7 % did not provide information on study protocol registration, and 44.4 % stipulated them as mandatory for authors. Regarding preprints, 50 % of the journals did not provide any information, but 46.7 % confirmed their acceptance. The mean OSS was 52.9 % (standard deviation 26.2). There was a weak correlation (Pearson's correlation coefficient of 0.221) between the journal impact factor and OSS (P-value=0.036).

CONCLUSION

This study found varying degrees of endorsement of open science practices among dental journals.

CLINICAL SIGNIFICANCE

Dental practitioners rely on high-quality, evidence-based research for informed decision-making. By assessing the endorsement of open science practices, our study contributes to improving the quality and reliability of dental research, ultimately enhancing the evidence base for clinical practice.

Recommendations for optimising pilot and feasibility work in surgery

BACKGROUND

Surgical trials are recognised as inherently challenging. Pilot and feasibility studies (PAFS) are increasingly acknowledged as a key method to optimise the design and conduct of randomised trials but remain limited in surgery. We used a mixed methods approach to develop recommendations for how surgical PAFS could be optimised.

METHODS

The findings from a quantitative analysis of funded surgical PAFS over a 10-year period and in-depth qualitative interviews with surgeons, methodologists and funders were triangulated and synthesised with available methodological guidance on PAFS.

RESULTS

The synthesis informed the development of an explanatory model describing root causes and compounding challenges that contribute to how and why surgical PAFS is not currently optimised. The four root causes identified include issues relating to (i) understanding the full scope of PAFS; (ii) design and conduct of PAFS; (iii) reporting of PAFS; and (iv) lack of appreciation of the value of PAFS by all stakeholder groups. Compounding challenges relate to both cultural issues and access to and interpretation of available methodological PAFS guidance. The study findings and explanatory model were used to inform the development of a practical guidance tool for surgeons and study teams to improve research practice.

CONCLUSIONS

Optimisation of PAFS in surgery requires a cultural shift in research practice amongst funders, academic institutions, regulatory bodies and journal editors, as well as amongst surgeons. Our 'Top Tips' guidance tool offers an accessible framework for surgeons designing PAFS. Adoption and utilisation of these recommendations will optimise surgical PAFS, facilitating successful and efficient future surgical trials.

Towards transparency: adoption of WHO best practices in clinical trial registration and reporting among top medical research funders in the USA

OBJECTIVE

To assess to what extent the clinical trial policies of the largest public and philanthropic funders of clinical research in the United States meet WHO best practices in trial registration and reporting.

METHODS

Public and philanthropic funders of clinical trials in the USA with >US$50 million annual spend were selected. The funders were assessed using an 11-item scoring tool based on WHO Joint Statement benchmarks. These 11 items fell into 4 categories, namely: trial registration, academic publication, monitoring and sanctions. An additional item captured whether and how funders referred to Consolidated Standards of Reporting Trials (CONSORT) within their trial policies. Each funder was independently assessed by two or three researchers. Funders were contacted to flag possible errors and omissions. Ambiguous or difficult-to-score items were settled by an independent adjudicator.

RESULTS

Fourteen funders were assessed. Our cross-sectional study found that, on average, funders have only implemented 4.1/11 (37%) of WHO best practices in clinical trial transparency. The most frequently adopted requirement was open access publishing (14/14 funders). The least frequently adopted were (1) requiring trial ID to appear in all publications (2/14 funders, 14%) and (2) making compliance reports public (2/14 funders, 14%). Public funders, on average, adopted more policy elements (5.2/11 items, 47%) than philanthropic funders (2.8/11 items, 25%). Only one funder's policy documents mentioned the CONSORT statement.

CONCLUSIONS

There is a significant variation between the number of best practice policy items adopted by medical research funders in the USA. Many funders fell significantly short of WHO Joint Statement benchmarks. Each funder could benefit from policy revision and strengthening.

Increasing Value and Reducing Waste of Research on Neurofeedback Effects in Post-traumatic Stress Disorder: A State-of-the-Art-Review

Post-Traumatic Stress Disorder (PTSD) is often considered challenging to treat due to factors that contribute to its complexity. In the last decade, more attention has been paid to non-pharmacological or non-psychological therapies for PTSD, including neurofeedback (NFB). NFB is a promising non-invasive technique targeting specific brainwave patterns associated with psychiatric symptomatology. By learning to regulate brain activity in a closed-loop paradigm, individuals can improve their functionality while reducing symptom severity. However, owing to its lax regulation and heterogeneous legal status across different countries, the degree to which it has scientific support as a psychiatric treatment remains controversial. In this state-of-the-art review, we searched PubMed, Cochrane Central, Web of Science, Scopus, and MEDLINE and identified meta-analyses and systematic reviews exploring the efficacy of NFB for PTSD. We included seven systematic reviews, out of which three included meta-analyses (32 studies and 669 participants) that targeted NFB as an intervention while addressing a single condition-PTSD. We used the MeaSurement Tool to Assess systematic Reviews (AMSTAR) 2 and the criteria described by Cristea and Naudet (Behav Res Therapy 123:103479, 2019, https://doi.org/10.1016/j.brat.2019.103479 ) to identify sources of research waste and increasing value in biomedical research. The seven assessed reviews had an overall extremely poor quality score (5 critically low, one low, one moderate, and none high) and multiple sources of waste while opening opportunities for increasing value in the NFB literature. Our research shows that it remains unclear whether NFB training is significantly beneficial in treating PTSD. The quality of the investigated literature is low and maintains a persistent uncertainty over numerous points, which are highly important for deciding whether an intervention has clinical efficacy. Just as importantly, none of the reviews we appraised explored the statistical power, referred to open data of the included studies, or adjusted their pooled effect sizes for publication bias and risk of bias. Based on the obtained results, we identified some recurrent sources of waste (such as a lack of research decisions based on sound questions or using an appropriate methodology in a fully transparent, unbiased, and useable manner) and proposed some directions for increasing value (homogeneity and consensus) in designing and reporting research on NFB interventions in PTSD.

Data-driven, cross-disciplinary collaboration: lessons learned at the largest academic health center in Latin America during the COVID-19 pandemic

Introduction

The COVID-19 pandemic has prompted global research efforts to reduce infection impact, highlighting the potential of cross-disciplinary collaboration to enhance research quality and efficiency.

Methods

At the FMUSP-HC academic health system, we implemented innovative flow management routines for collecting, organizing and analyzing demographic data, COVID-related data and biological materials from over 4,500 patients with confirmed SARS-CoV-2 infection hospitalized from 2020 to 2022. This strategy was mainly planned in three areas: organizing a database with data from the hospitalizations; setting-up a multidisciplinary taskforce to conduct follow-up assessments after discharge; and organizing a biobank. Additionally, a COVID-19 curated collection was created within the institutional digital library of academic papers to map the research output.

Results

Over the course of the experience, the possible benefits and challenges of this type of research support approach were identified and discussed, leading to a set of recommended strategies to enhance collaboration within the research institution. Demographic and clinical data from COVID-19 hospitalizations were compiled in a database including adults and a minority of children and adolescents with laboratory confirmed COVID-19, covering 2020-2022, with approximately 350 fields per patient. To date, this database has been used in 16 published studies. Additionally, we assessed 700 adults 6 to 11 months after hospitalization through comprehensive, multidisciplinary in-person evaluations; this database, comprising around 2000 fields per subject, was used in 15 publications. Furthermore, thousands of blood samples collected during the acute phase and follow-up assessments remain stored for future investigations. To date, more than 3,700 aliquots have been used in ongoing research investigating various aspects of COVID-19. Lastly, the mapping of the overall research output revealed that between 2020 and 2022 our academic system produced 1,394 scientific articles on COVID-19.

Discussion

Research is a crucial component of an effective epidemic response, and the preparation process should include a well-defined plan for organizing and sharing resources. The initiatives described in the present paper were successful in our aim to foster large-scale research in our institution. Although a single model may not be appropriate for all contexts, cross-disciplinary collaboration and open data sharing should make health research systems more efficient to generate the best evidence.

Applying a modified systematic review and integrated assessment framework (SYRINA) - a case study on triphenyl phosphate

This work presents a case study in applying a systematic review framework (SYRINA) to the identification of chemicals as endocrine disruptors. The suitability and performance of the framework is tested with regard to the widely accepted World Health Organization definition of an endocrine disruptor (ED). The endocrine disrupting potential of triphenyl phosphate (TPP), a well-studied flame retardant reported to exhibit various endocrine related effects was assessed. We followed the 7 steps of the SYRINA framework, articulating the research objective via Populations, Exposures, Comparators, Outcomes (PECO) statements, performed literature search and screening, conducted study evaluation, performed data extraction and summarized and integrated the evidence. Overall, 66 studies, consisting of in vivo, in vitro and epidemiological data, were included. We concluded that triphenyl phosphate could be identified as an ED based on metabolic disruption and reproductive function. We found that the tools used in this case study and the optimizations performed on the framework were suitable to assess properties of EDs. A number of challenges and areas for methodological development in systematic appraisal of evidence relating to endocrine disrupting potential were identified; significant time and effort were needed for the analysis of in vitro mechanistic data in this case study, thus increasing the workload and time needed to perform the systematic review process. Further research and development of this framework with regards to grey literature (non-peer-reviewed literature) search, harmonization of study evaluation methods, more consistent evidence integration approaches and a pre-defined method to assess links between adverse effect and endocrine activity are recommended. It would also be advantageous to conduct more case studies for a chemical with less data than TPP.

A Decade of Efforts to Add Value to Child Health Research Practices

OBJECTIVE

To identify practices that add value to improve the design, conduct, and reporting of child health research and reduce research waste.

STUDY DESIGN

In order to categorize the contributions of members of Standards for Research (StaR) in Child Health network, we developed a novel Child Health Improving Research Practices (CHIRP) framework comprised of 5 domains meant to counteract avoidable child health research waste and improve quality: 1) address research questions relevant to children, their families, clinicians, and researchers; 2) apply appropriate research design, conduct and analysis; 3) ensure efficient research oversight and regulation; 4) Provide accessible research protocols and reports; and 5) develop unbiased and usable research reports, including 17 responsible research practice recommendations. All child health research relevant publications by the 48 original StaR standards' authors over the last decade were identified, and main topic areas were categorized using this framework.

RESULTS

A total of 247 publications were included in the final sample: 100 publications (41%) in domain 1 (3 recommendations), 77 publications (31%) in domain 2 (3), 35 publications (14%) in domain 3 (4), 20 publications (8%) in domain 4 (4), and 15 publications (6%) in domain 5 (3). We identified readily implementable "responsible" research practices to counter child health research waste and improve quality, especially in the areas of patients and families' engagement throughout the research process, developing Core Outcome Sets, and addressing ethics and regulatory oversight issues.

CONCLUSION

While most of the practices are readily implementable, increased awareness of methodological issues and wider guideline uptake is needed to improve child health research. The CHIRP Framework can be used to guide responsible research practices that add value to child health research.

Prognostic Biomarkers in Kidney Transplantation: A Systematic Review and Critical Appraisal

SIGNIFICANCE STATEMENT

Why are there so few biomarkers accepted by health authorities and implemented in clinical practice, despite the high and growing number of biomaker studies in medical research ? In this meta-epidemiological study, including 804 studies that were critically appraised by expert reviewers, the authors have identified all prognostic kidney transplant biomarkers and showed overall suboptimal study designs, methods, results, interpretation, reproducible research standards, and transparency. The authors also demonstrated for the first time that the limited number of studies challenged the added value of their candidate biomarkers against standard-of-care routine patient monitoring parameters. Most biomarker studies tended to be single-center, retrospective studies with a small number of patients and clinical events. Less than 5% of the studies performed an external validation. The authors also showed the poor transparency reporting and identified a data beautification phenomenon. These findings suggest that there is much wasted research effort in transplant biomarker medical research and highlight the need to produce more rigorous studies so that more biomarkers may be validated and successfully implemented in clinical practice.

BACKGROUND

Despite the increasing number of biomarker studies published in the transplant literature over the past 20 years, demonstrations of their clinical benefit and their implementation in routine clinical practice are lacking. We hypothesized that suboptimal design, data, methodology, and reporting might contribute to this phenomenon.

METHODS

We formed a consortium of experts in systematic reviews, nephrologists, methodologists, and epidemiologists. A systematic literature search was performed in PubMed, Embase, Scopus, Web of Science, and Cochrane Library between January 1, 2005, and November 12, 2022 (PROSPERO ID: CRD42020154747). All English language, original studies investigating the association between a biomarker and kidney allograft outcome were included. The final set of publications was assessed by expert reviewers. After data collection, two independent reviewers randomly evaluated the inconsistencies for 30% of the references for each reviewer. If more than 5% of inconsistencies were observed for one given reviewer, a re-evaluation was conducted for all the references of the reviewer. The biomarkers were categorized according to their type and the biological milieu from which they were measured. The study characteristics related to the design, methods, results, and their interpretation were assessed, as well as reproducible research practices and transparency indicators.

RESULTS

A total of 7372 publications were screened and 804 studies met the inclusion criteria. A total of 1143 biomarkers were assessed among the included studies from blood ( n =821, 71.8%), intragraft ( n =169, 14.8%), or urine ( n =81, 7.1%) compartments. The number of studies significantly increased, with a median, yearly number of 31.5 studies (interquartile range [IQR], 23.8-35.5) between 2005 and 2012 and 57.5 (IQR, 53.3-59.8) between 2013 and 2022 ( P < 0.001). A total of 655 studies (81.5%) were retrospective, while 595 (74.0%) used data from a single center. The median number of patients included was 232 (IQR, 96-629) with a median follow-up post-transplant of 4.8 years (IQR, 3.0-6.2). Only 4.7% of studies were externally validated. A total of 346 studies (43.0%) did not adjust their biomarker for key prognostic factors, while only 3.1% of studies adjusted the biomarker for standard-of-care patient monitoring factors. Data sharing, code sharing, and registration occurred in 8.8%, 1.1%, and 4.6% of studies, respectively. A total of 158 studies (20.0%) emphasized the clinical relevance of the biomarker, despite the reported nonsignificant association of the biomarker with the outcome measure. A total of 288 studies assessed rejection as an outcome. We showed that these rejection studies shared the same characteristics as other studies.

CONCLUSIONS

Biomarker studies in kidney transplantation lack validation, rigorous design and methodology, accurate interpretation, and transparency. Higher standards are needed in biomarker research to prove the clinical utility and support clinical use.

Ensuring the affordable becomes accessible-lessons from ketamine, a new treatment for severe depression

In this paper, the case study of ketamine as a new treatment for severe depression is used to outline the challenges of repurposing established medicines and we suggest potential solutions. The antidepressant effects of generic racemic ketamine were identified over 20 years ago, but there were insufficient incentives for commercial entities to pursue its registration, or support for non-commercial entities to fill this gap. As a result, the evaluation of generic ketamine was delayed, piecemeal, uncoordinated, and insufficient to gain approval. Meanwhile, substantial commercial investment enabled the widespread registration of a patented, intranasal s-enantiomeric ketamine formulation (Spravato®) for depression. However, Spravato is priced at $600-$900/dose compared to ~$5/dose for generic ketamine, and the ~AUD$100 million annual government investment requested in Australia (to cover drug costs alone) has been rejected twice, leaving this treatment largely inaccessible for Australian patients 2 years after Therapeutic Goods Administration approval. Moreover, emerging evidence indicates that generic racemic ketamine is at least as effective as Spravato, but no comparative trials were required for regulatory approval and have not been conducted. Without action, this story will repeat regularly in the next decade with a new wave of psychedelic-assisted psychotherapy treatments, for which the original off-patent molecules could be available at low-cost and reduce the overall cost of treatment. Several systemic reforms are required to ensure that affordable, effective options become accessible; these include commercial incentives, public and public-private funding schemes, reduced regulatory barriers and more coordinated international public funding schemes to support translational research.

Undertaking Studies Within A Trial to evaluate recruitment and retention strategies for randomised controlled trials: lessons learnt from the PROMETHEUS research programme

Background

Randomised controlled trials ('trials') are susceptible to poor participant recruitment and retention. Studies Within A Trial are the strongest methods for testing the effectiveness of strategies to improve recruitment and retention. However, relatively few of these have been conducted.

Objectives

PROMoting THE Use of Studies Within A Trial aimed to facilitate at least 25 Studies Within A Trial evaluating recruitment or retention strategies. We share our experience of delivering the PROMoting THE Use of Studies Within A Trial programme, and the lessons learnt for undertaking randomised Studies Within A Trial.

Design

A network of 10 Clinical Trials Units and 1 primary care research centre committed to conducting randomised controlled Studies Within A Trial of recruitment and/or retention strategies was established. Promising recruitment and retention strategies were identified from various sources including Cochrane systematic reviews, the Study Within A Trial Repository, and existing prioritisation exercises, which were reviewed by patient and public members to create an initial priority list of seven recruitment and eight retention interventions. Host trial teams could apply for funding and receive support from the PROMoting THE Use of Studies Within A Trial team to undertake Studies Within A Trial. We also tested the feasibility of undertaking co-ordinated Studies Within A Trial, across multiple host trials simultaneously.

Setting

Clinical trials unit-based trials recruiting or following up participants in any setting in the United Kingdom were eligible.

Participants

Clinical trials unit-based teams undertaking trials in any clinical context in the United Kingdom.

Interventions

Funding of up to £5000 and support from the PROMoting THE Use of Studies Within A Trial team to design, implement and report Studies Within A Trial.

Main outcome measures

Number of host trials funded.

Results

Forty-two Studies Within A Trial were funded (31 host trials), across 12 Clinical Trials Units. The mean cost of a Study Within A Trial was £3535. Twelve Studies Within A Trial tested the same strategy across multiple host trials using a co-ordinated Study Within A Trial design, and four used a factorial design. Two recruitment and five retention strategies were evaluated in more than one host trial. PROMoting THE Use of Studies Within A Trial will add 18% more Studies Within A Trial to the Cochrane systematic review of recruitment strategies, and 79% more Studies Within A Trial to the Cochrane review of retention strategies. For retention, we found that pre-notifying participants by card, letter or e-mail before sending questionnaires was effective, as was the use of pens, and sending personalised text messages to improve questionnaire response. We highlight key lessons learnt to guide others planning Studies Within A Trial, including involving patient and public involvement partners; prioritising and selecting strategies to evaluate and elements to consider when designing a Study Within A Trial; obtaining governance approvals; implementing Studies Within A Trial, including individual and co-ordinated Studies Within A Trials; and reporting Study Within A Trials.

Limitations

The COVID-19 pandemic negatively impacted five Studies Within A Trial, being either delayed (n = 2) or prematurely terminated (n = 3).

Conclusions

PROMoting THE Use of Studies Within A Trial significantly increased the evidence base for recruitment and retention strategies. When provided with both funding and practical support, host trial teams successfully implemented Studies Within A Trial.

Future work

Future research should identify and target gaps in the evidence base, including widening Study Within A Trial uptake, undertaking more complex Studies Within A Trial and translating Study Within A Trial evidence into practice.

Study registration

All Studies Within A Trial in the PROMoting THE Use of Studies Within A Trial programme had to be registered with the Northern Ireland Network for Trials Methodology Research Study Within A Trial Repository.

Funding

This award was funded by the National Institute for Health and Care Research (NIHR) Health Technology Assessment programme (NIHR award ref: 13/55/80) and is published in full in Health Technology Assessment; Vol. 28, No. 2. See the NIHR Funding and Awards website for further award information.

How do trial teams plan for retention during the design stage of the trial? A scoping review

BACKGROUND

Retention to trials is important to ensure the results of the trial are valid and reliable. The SPIRIT guidelines (18b) require "plans to promote participant retention and complete follow-up, including list of any outcome data to be collected for participants who discontinue or deviate from intervention protocols" be included in trial protocols. It is unknown how often protocols report this retention information. The purpose of our scoping review is to establish if, and how, trial teams report plans for retention during the design stage of the trial.

MATERIALS AND METHODS

A scoping review with searches in key databases (PubMed, Scopus, EMBASE, CINAHL (EBSCO), and Web of Science from 2014 to 2019 inclusive) to identify randomised controlled trial protocols. We produced descriptive statistics on the characteristics of the trial protocols and also on those adhering to SPIRIT item 18b. A narrative synthesis of the retention strategies was also conducted.

RESULTS

Eight-hundred and twenty-four protocols met our inclusion criteria. RCTs (n = 722) and pilot and feasibility trial protocols (n = 102) reported using the SPIRIT guidelines during protocol development 35% and 34.3% of the time respectively. Of these protocols, only 9.5% and 11.4% respectively reported all aspects of SPIRIT item 18b "plans to promote participant retention and to complete follow-up, including list of any outcome data for participants who discontinue or deviate from intervention protocols". Of the RCT protocols, 36.8% included proactive "plans to promote participant retention" regardless of whether they reported using SPIRIT guidelines or not. Most protocols planned "combined strategies" (48.1%). Of these, the joint most commonly reported were "reminders and data collection location and method" and "reminders and monetary incentives". The most popular individual retention strategy was "reminders" (14.7%) followed by "monetary incentives- conditional" (10.2%). Of the pilot and feasibility protocols, 40.2% included proactive "plans to promote participant retention" with the use of "combined strategies" being most frequent (46.3%). The use of "monetary incentives - conditional" (22%) was the most popular individual reported retention strategy.

CONCLUSION

There is a lack of reporting of plans to promote participant retention in trial protocols. Proactive planning of retention strategies during the trial design stage is preferable to the reactive implementation of retention strategies. Prospective retention planning and clear communication in protocols may inform more suitable choice, costing and implementation of retention strategies and improve transparency in trial conduct.

Gynecological trials frequently exclude people based on their symptoms rather than their condition: a systematic review of Cochrane reviews and their component trials

OBJECTIVES

To identify strategies used in recent randomized controlled trials (RCTs) and their associated Cochrane Reviews where patients with the same gynecological condition present with different symptoms but would plausibly benefit from a common intervention.

STUDY DESIGN AND SETTING

We searched the Cochrane library (February 2022) for reviews in polycystic ovarian syndrome (PCOS) and endometriosis. Reviews were included if the intervention was intended to treat all condition-specific symptoms. For each trial we recorded the strategy used and the number of potentially eligible participants excluded as a direct result of the chosen strategy. For each review we recorded the numbers of RCTs and participants excluded on the basis of symptoms experienced.

RESULTS

There were 89 distinct PCOS trials in 13 reviews, and 13 Endometriosis trials in 11 reviews. Most trials restricted their eligibility to participants with specific symptoms (55% PCOS, 46% endometriosis). The second most common strategy was to measure and analyze clinical outcomes that were not relevant to all participants (38% PCOS, 31% endometriosis). Reviews excluded 27% of trials in participants evaluating the same intervention in participants experiencing the same condition based on the outcomes measured in the trials.

CONCLUSION

Most gynecological trials exclude patients who could benefit from treatment or measure outcomes not relevant to all participants. We introduce a taxonomy to describe trial design strategies for conditions with heterogeneous symptoms.

The post-award effort of managing and reporting on funded research: a scoping review

INTRODUCTION

Reporting is a mechanism for funding organisations to monitor and manage the progress, outputs, outcomes, and impacts of the research they fund. Inconsistent approaches to reporting and post-award management, and a growing demand for research information, can lead to perception of unnecessary administrative effort that impacts on decision-making and research activity. Identifying this effort, and what stakeholders see as unmet need for improvement, is crucial if funders and Higher Education Institutions (HEIs) are to streamline their practices and provide better support with reporting activities. In this review, we summarise the processes in post-award management, compare current practices, and explore the purpose of collecting information on funded research. We also identify areas where unnecessary effort is perceived and improvement is needed, using previously reported solutions to inform recommendations for funders and HEIs.

METHODS

We conducted a scoping review of the relevant research and grey literature. Electronic searches of databases, and manual searches of journals and funder websites, resulted in inclusion of 52 records and 11 websites. Information on HEI and funder post-award management processes was extracted, catalogued, and summarised to inform discussion.

RESULTS

Post-award management is a complex process that serves many purposes but requires considerable effort, particularly in the set up and reporting of research. Perceptions of unnecessary effort stem from inefficiencies in compliance, data management and reporting approaches, and there is evidence of needed improvement in mechanisms of administrative support, research impact assessment, monitoring, and evaluation. Solutions should focus on integrating digital systems to reduce duplication, streamlining reporting methods, and improving administrative resources in HEIs.

CONCLUSIONS

Funders and HEIs should work together to support a more efficient post-award management process. The value of research information, and how it is collected and used, can be improved by aligning practices and addressing the specific issues highlighted in this review.

The post-award effort of managing and reporting on funded research: a scoping review

INTRODUCTION

Reporting is a mechanism for funding organisations to monitor and manage the progress, outputs, outcomes, and impacts of the research they fund. Inconsistent approaches to reporting and post-award management, and a growing demand for research information, can lead to perception of unnecessary administrative effort that impacts on decision-making and research activity. Identifying this effort, and what stakeholders see as unmet need for improvement, is crucial if funders and Higher Education Institutions (HEIs) are to streamline their practices and provide better support with reporting activities. In this review, we summarise the processes in post-award management, compare current practices, and explore the purpose of collecting information on funded research. We also identify areas where unnecessary effort is perceived and improvement is needed, using previously reported solutions to inform recommendations for funders and HEIs.

METHODS

We conducted a scoping review of the relevant research and grey literature. Electronic searches of databases, and manual searches of journals and funder websites, resulted in inclusion of 52 records and 11 websites. Information on HEI and funder post-award management processes was extracted, catalogued, and summarised to inform discussion.

RESULTS

Post-award management is a complex process that serves many purposes but requires considerable effort, particularly in the set up and reporting of research. Perceptions of unnecessary effort stem from inefficiencies in compliance, data management and reporting approaches, and there is evidence of needed improvement in mechanisms of administrative support, research impact assessment, monitoring, and evaluation. Solutions should focus on integrating digital systems to reduce duplication, streamlining reporting methods, and improving administrative resources in HEIs.

CONCLUSIONS

Funders and HEIs should work together to support a more efficient post-award management process. The value of research information, and how it is collected and used, can be improved by aligning practices and addressing the specific issues highlighted in this review.

Influence of research evidence on the use of cardiovascular clinical prediction rules in primary care: an exploratory qualitative interview study

BACKGROUND

Cardiovascular clinical prediction rules (CPRs) are widely used in primary care. They accumulate research evidence through derivation, external validation, and impact studies. However, existing knowledge about the influence of research evidence on the use of CPRs is limited. Therefore, we explored how primary care clinicians' perceptions of and experiences with research influence their use of cardiovascular CPRs.

METHODS

We conducted an exploratory qualitative interview study with thematic analysis. Primary care clinicians were recruited from the WWAMI (Washington, Wyoming, Alaska, Montana and Idaho) region Practice and Research Network (WPRN). We used purposeful sampling to ensure maximum variation within the participant group. Data were collected by conducting semi-structured online interviews. We analyzed data using inductive thematic analysis to identify commonalities and differences within themes.

RESULTS

Of 29 primary care clinicians who completed the questionnaire, 15 participated in the interview. We identified two main themes relating to the influence of clinicians' perceptions of and experiences with cardiovascular CPR research on their decisions about using cardiovascular CPRs: "Seek and judge" and "be acquainted and assume." When clinicians are familiar with, trust, and feel confident in using research evidence, they might actively search and assess the evidence, which may then influence their decisions about using cardiovascular CPRs. However, clinicians, who are unfamiliar with, distrust, or find it challenging to use research evidence, might be passively acquainted with evidence but do not make their own judgment on the trustworthiness of such evidence. Therefore, these clinicians might not rely on research evidence when making decisions about using cardiovascular CPRs.

CONCLUSIONS

Clinicians' perceptions and experiences could influence how they use research evidence in decisions about using cardiovascular CPRs. This implies, when promoting evidence-based decisions, it might be useful to target clinicians' unfamiliarity, distrust, and challenges regarding the use of research evidence rather than focusing only on their knowledge and skills. Further, because clinicians often rely on evidence-unrelated factors, guideline developers and policymakers should recommend cardiovascular CPRs supported by high-quality evidence.

What is the role of randomised trials in implementation science?

BACKGROUND

There is a consistent demand for implementation science to inform global efforts to close the gap between evidence and practice. Key evaluation questions for any given implementation strategy concern the assessment and understanding of effects. Randomised trials are generally accepted as offering the most trustworthy design for establishing effectiveness but may be underused in implementation science.

MAIN BODY

There is a continuing debate about the primacy of the place of randomised trials in evaluating implementation strategies, especially given the evolution of more rigorous quasi-experimental designs. Further critiques of trials for implementation science highlight that they cannot provide 'real world' evidence, address urgent and important questions, explain complex interventions nor understand contextual influences. We respond to these critiques of trials and highlight opportunities to enhance their timeliness and relevance through innovative designs, embedding within large-scale improvement programmes and harnessing routine data. Our suggestions for optimising the conditions for randomised trials of implementation strategies include strengthening partnerships with policy-makers and clinical leaders to realise the long-term value of rigorous evaluation and accelerating ethical approvals and decluttering governance procedures for lower risk studies.

CONCLUSION

Policy-makers and researchers should avoid prematurely discarding trial designs when evaluating implementation strategies and work to enhance the conditions for their conduct.

Assessment of redundancy, methodological and reporting quality, and potential discrepancies of results of systematic reviews of early mobilisation of critically ill adults: a meta-research protocol

INTRODUCTION

Several systematic reviews (SRs) have been conducted to determine the effectiveness of early mobilisation in critically ill adults with heterogeneous methodology and results. Redundancy in conducting SRs, unclear justification when leading new SRs or updating, and discordant results of SRs on the same research question may generate research waste that makes it difficult for clinicians to keep up to date with the best available evidence. This meta-research aims to assess the redundancy, methodological and reporting quality, and potential reasons for discordance in the results reported by SRs conducted to determine the effectiveness of early mobilisation in critically ill adult patients.

METHODS AND ANALYSIS

A meta-research of early mobilisation SRs in critically ill adult patients will be conducted. A search of MEDLINE (Ovid), Embase (Ovid), CINAHL (EBSCOhost), Cochrane Library, Epistemonikos and other search resources will be conducted. Two independent reviewers will perform study selection, data extraction and quality appraisal. Discrepancies will be resolved by consensus or a third reviewer. The redundancy of SRs will be assessed by the degree of overlap of primary studies. In addition, the justification for conducting new SRs will be evaluated with the 'Evidence-Based Research' framework. The methodological quality of the SRs will be assessed with the A MeaSurement Tool to Assess systematic Reviews 2 tool, and the quality of the reports through compliance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses statement. To assess the potential reasons for discordance in the results of the SRs considering divergence in results and their interpretation.

ETHICS AND DISSEMINATION

As meta-research, this study does not involve the participation of people whose rights may be violated. However, this overview will be developed rigorously and systematically to achieve valid and reliable results. The findings of this meta-research study will be presented at conferences and published in a peer-reviewed journal related to rehabilitation, critical care or research methodology.

TRIAL REGISTRATION NUMBER

osf.io/kxwq9.

Evidence from UK Research Ethics Committee members on what makes a good research ethics review, and what can be improved

The rapid development of vaccines and other innovative medical technologies in response to the COVID-19 pandemic required streamlined and efficient ethics and governance processes. In the UK the Health Research Authority (HRA) oversees and coordinates a number of the relevant research governance processes including the independent ethics review of research projects. The HRA was instrumental in facilitating the rapid review and approval of COVID-19 projects, and following the end of the pandemic, have been keen to integrate new ways of working into the UK Health Departments' Research Ethics Service. In January 2022 the HRA commissioned a public consultation that identified strong public support for alternative ethics review processes. Here we report feedback from 151 current research ethics committee members conducted at three annual training events, where we asked members to critically reflect on their ethics review activities, and to share new ideas or ways of working. The results showed a high regard for good quality discussion among members with diverse experience. Good chairing, organisation, feedback and the opportunity for reflection on ways of working were considered key. Areas for improvement included the consistency of information provided to committees by researchers, and better structuring of discussions by allowing signposting of the key issues that ethics committee members might need to consider.

Effectiveness of interventions by non-professional community-level workers or family caregivers to improve outcomes for physical impairments or disabilities in low resource settings: systematic review of task-sharing strategies

BACKGROUND

In low-resource settings, access to basic rehabilitation could be supplemented by community-level interventions provided by community health workers, health volunteers, or family caregivers. Yet, it is unclear whether basic physical rehabilitation interventions delivered to adults by non-professional alternative resources in the community, under task-shifting or task-sharing approaches, are effective as those delivered by skilled rehabilitation professionals. We aim to synthesize evidence on the effectiveness of community-level rehabilitation interventions delivered by non-professional community-level workers or informal caregivers to improve health outcomes for persons with physical impairments or disabilities.

METHODS

We performed a systematic review with a PROSPERO registration. Eight databases were searched for (PubMed, CINAHL, Global Health, PDQ Evidence, Scopus, ProQuest, CENTRAL, and Web of Science), supplemented by snowballing and key-informant recommendations, with no time restrictions, applied. Controlled and non-controlled experiments were included if reporting the effects of interventions on mobility, activities of daily living (ADLs), quality of life, or social participation outcomes. Two independent investigators performed the eligibility decisions, data extraction, risk of bias, and assessed the quality of the evidence using the GRADE approach.

RESULTS

Ten studies (five randomized controlled trials [RCTs]) involving 2149 participants were included. Most common targeted stroke survivors (n = 8); family caregivers were most frequently used to deliver the intervention (n = 4); and the intervention was usually provided in homes (n = 7), with training initiated in the hospital (n = 4). Of the four RCTs delivered by family caregivers, one demonstrated a statistically significant improvement in mobility (effect size: 0.3; confidence interval [CI] 121.81-122.19; [p = 0.04]) and another one in ADLs (effect size: 0.4; CI 25.92-35.08; [p = 0.03]). Of the five non-RCT studies by community health workers or volunteers, one demonstrated a statistically significant improvement in mobility (effect size: 0.3; CI 10.143-16.857; [p < 0.05]), while two demonstrated improved statistically significant improvement in ADLs (effect size: 0.2; CI 180.202-184.789 [p = 0.001]; 0.4; CI - 7.643-18.643; [p = 0.026]). However, the quality of evidence, based on GRADE criteria, was rated as low to very low.

CONCLUSIONS

While task-sharing is a possible strategy to meet basic rehabilitation needs in low-resource settings, the current evidence on the effectiveness of delivering rehabilitation interventions by non-professional community-level workers and informal caregivers is inconclusive. We can use the data and experiences from existing studies to better design studies and improve the implementation of interventions. Trial registration PROSPERO registration number: CRD42022319130.

Heterogeneity in the definition of delirium in ICUs and association with the intervention effect in randomized controlled trials: a meta-epidemiological study

PURPOSE

To evaluate the heterogeneity in the definition of delirium in randomized controlled trials (RCTs) included in meta-analyses of delirium in intensive care units (ICUs) and to explore whether intervention effect depends on the definition used.

METHODS

We searched PubMed for meta-analyses including RCTs evaluating prevention or treatment strategies of delirium in ICU. The definition of delirium was collected from RCTs and classified as validated (DSM criteria, CAM-ICU, ICDSC, NEECHAM, DRS-R98) or non-validated (non-validated scales, set of symptoms, physician appreciation or not reported). We conducted a meta-epidemiological analysis to compare intervention effects between trials using or not a validated definition by a two-step method as primary analysis and a multilevel model as secondary analysis. A ratio of odds ratios (ROR) < 1 indicated larger intervention effects in trials using a non-validated definition.

RESULTS

Of 149 RCTs (41 meta-analyses), 109 (73.1%) used a validated definition and 40 (26.8%) did not (including 31 [20.8%] not reporting the definition). The primary analysis of 7 meta-analyses (30 RCTs) found no significant difference in intervention effects between trials using a validated definition and the others (ROR = 0.54, 95% CI 0.27-1.08), whereas the secondary multilevel analysis including 12 meta-analyses (67 RCTs) found significantly larger effects for trials using a non-validated versus a validated definition (ROR = 0.36, 95% CI 0.21-0.62).

CONCLUSION

The definition of delirium was heterogeneous across RCTs, with one-fifth not reporting how they evaluated delirium. We did not find a significant association with intervention effect in the primary analysis. The secondary analysis including more studies revealed significantly larger intervention effects in trials using a non-validated versus a validated definition.

Patient and public involvement to inform the protocol of a clinical trial comparing total hip arthroplasty with exercise: an exploratory qualitative case study

OBJECTIVE

To explore patient, clinician and decision-maker perceptions on a clinical trial evaluating the effectiveness of total hip arthroplasty (THA) compared with exercise to inform the trial protocol.

DESIGN

This is an exploratory qualitative case study using a constructivist paradigm.

SETTING AND PARTICIPANTS

Participants were enrolled into three key stakeholder groups: patients eligible for THA, clinicians, and decision makers. Focus group interviews were conducted in undisturbed conference rooms at two hospitals in Denmark, according to group status using semi-structured interview guides.

ANALYSIS

Interviews were recorded, transcribed verbatim and thematic analysed using an inductive approach.

RESULTS

We conducted 4 focus group interviews with 14 patients, 1 focus group interview with 4 clinicians (2 orthopaedic surgeons and 2 physiotherapists) and 1 focus group interview with 4 decision-makers. Two main themes were generated. 'Treatment expectations and beliefs impact management choices' covered three supporting codes: Treatment without surgery is unlikely to lead to recovery; Clinician authority impacts the management narrative; The 'surgery vs exercise' debate. 'Factors influencing clinical trial integrity and feasibility' highlighted three supporting codes: Who is considered eligible for surgery?; Facilitators and barriers for surgery and exercise in a clinical trial context; Improvements in hip pain and hip function are the most important outcomes.

CONCLUSIONS

Based on key stakeholder treatment expectations and beliefs, we implemented three main strategies to improve the methodological rigorousness of our trial protocol. First, we added an observational study investigating the generalisability to address a potential low enrolment rate. Second, we developed an enrolment procedure using generic guidance and balanced narrative conveyed by an independent clinician to facilitate communication of clinical equipoise. Third, we adopted change in hip pain and function as the primary outcome. These findings highlight the value of patient and public involvement in the development of trial protocols to reduce bias in comparative clinical trials evaluating surgical and non-surgical management.

TRIAL REGISTRATION NUMBER

NCT04070027 (pre-results).

Development of a risk-tailored approach and dashboard for efficient management and monitoring of investigator-initiated trials

BACKGROUND

Most randomized controlled trials (RCTs) in the academic setting have limited resources for clinical trial management and monitoring. Inefficient conduct of trials was identified as an important source of waste even in well-designed studies. Thoroughly identifying trial-specific risks to enable focussing of monitoring and management efforts on these critical areas during trial conduct may allow for the timely initiation of corrective action and to improve the efficiency of trial conduct. We developed a risk-tailored approach with an initial risk assessment of an individual trial that informs the compilation of monitoring and management procedures in a trial dashboard.

METHODS

We performed a literature review to identify risk indicators and trial monitoring approaches followed by a contextual analysis involving local, national and international stakeholders. Based on this work we developed a risk-tailored management approach with integrated monitoring for RCTs and including a visualizing trial dashboard. We piloted the approach and refined it in an iterative process based on feedback from stakeholders and performed formal user testing with investigators and staff of two clinical trials.

RESULTS

The developed risk assessment comprises four domains (patient safety and rights, overall trial management, intervention management, trial data). An accompanying manual provides rationales and detailed instructions for the risk assessment. We programmed two trial dashboards tailored to one medical and one surgical RCT to manage identified trial risks based on daily exports of accumulating trial data. We made the code for a generic dashboard available on GitHub that can be adapted to individual trials.

CONCLUSIONS

The presented trial management approach with integrated monitoring enables user-friendly, continuous checking of critical elements of trial conduct to support trial teams in the academic setting. Further work is needed in order to show effectiveness of the dashboard in terms of safe trial conduct and successful completion of clinical trials.

The role of patient advocates and sarcoma community initiatives in musculoskeletal oncology. Moving towards Evidence-Based Advocacy to empower Evidence-Based Medicine

Musculoskeletal sarcomas are rare cancers that as the whole family of sarcomas pose several challenges at different levels, ranging from medical knowledge to clinical research and policymaking. Addressing these challenges, necessarily calls for the inclusion of patient perspective inside the decision-making processes of every area that contributes to treatment improvement, from the provision of high-quality services by healthcare organisations to research issues. Without patient-provided inputs to inform decisions, the current paradigm of patient-centred care makes no sense and sounds at the least irrational if not unethical. Putting PROMs on "centre stage" in cancer research and care, could allow to build a truly Evidence Based Advocacy (EBA) and therefore to empower Evidence Based Medicine (EBM).

Pilot and feasibility studies: extending the conceptual framework

In 2016, we published a conceptual framework outlining the conclusions of our work in defining pilot and feasibility studies. Since then, the CONSORT extension to randomised pilot and feasibility trials has been published and there have been further developments in the pilot study landscape. In this paper, we revisit and extend our framework to incorporate the various feasibility pathways open to researchers, which include internal pilot studies. We consider, with examples, when different approaches to feasibility and pilot studies are more effective and efficient, taking into account the pragmatic decisions that may need to be made. The ethical issues involved in pilot studies are discussed. We end with a consideration of the funders' perspective in making difficult resource decisions to include feasibility work and the policy implications of these; throughout, we provide examples of the uncertainties and compromises that researchers have to navigate to make progress in the most efficient way.

Reproducibility and Scientific Integrity of Big Data Research in Urban Public Health and Digital Epidemiology: A Call to Action

The emergence of big data science presents a unique opportunity to improve public-health research practices. Because working with big data is inherently complex, big data research must be clear and transparent to avoid reproducibility issues and positively impact population health. Timely implementation of solution-focused approaches is critical as new data sources and methods take root in public-health research, including urban public health and digital epidemiology. This commentary highlights methodological and analytic approaches that can reduce research waste and improve the reproducibility and replicability of big data research in public health. The recommendations described in this commentary, including a focus on practices, publication norms, and education, are neither exhaustive nor unique to big data, but, nonetheless, implementing them can broadly improve public-health research. Clearly defined and openly shared guidelines will not only improve the quality of current research practices but also initiate change at multiple levels: the individual level, the institutional level, and the international level.

Assessment of an electronic system for research ethics committees document management: An observational study

Since 1 January 2020, the Central Research Ethics Committee of the Health Ministry implemented PRIISA.BA, an in-house developed electronic system for online submission of health research applications to the 63 public and private research ethics committees (RECs) of Buenos Aires City, Argentina. This study though to compare the times to first review and the time to approval among applications submitted prior to PRIISA.BA and thereafter, across public RECs. All public RECs of the city were invited to participate. Overall, 453 applications from 10 RECs (242 pre- and 211 post-PRIISA.BA) were available for the analyses. There was a decrease in the time to first review and an increase in the time to approval after PRIISA.BA implementation. The increase in time to approval was transient and limited to the first three months. The results were consistent with analyses limited to non-COVID applications. Our results show an increase in the times to approval after the implementation of an electronic system for online submission of health research applications that, although transient, was significant. These data could be relevant to other RECs implementing this technology since it emphasizes the need of monitoring potential unnecessary delays in reviews during the critical initial period.

Low Research-Data Availability in Educational-Psychology Journals: No Indication of Effective Research-Data Policies

Research-data availability contributes to the transparency of the research process and the credibility of educational-psychology research and science in general. Recently, there have been many initiatives to increase the availability and quality of research data. Many research institutions have adopted research-data policies. This increased awareness might have raised the sharing of research data in empirical articles. To test this idea, we coded 1,242 publications from six educational-psychology journals and the psychological journal Cognition (as a baseline) published in 2018 and 2020. Research-data availability was low (3.85% compared with 62.74% in Cognition) but has increased from 0.32% (2018) to 7.16% (2020). However, neither the data-transparency level of the journal nor the existence of an official research-data policy on the level of the corresponding author’s institution was related to research-data availability. We discuss the consequences of these findings for institutional research-data-management processes.

Centre for Statistical and Methodological Excellence (CESAME): A Consortium Initiative for Improving Methodology in Randomised Clinical Trials

When conducting randomised clinical trials, the choice of methodology and statistical analyses will influence the results. If the planned methodology is not of optimal quality and predefined in detail, there is a risk of biased trial results and interpretation. Even though clinical trial methodology is already at a very high standard, there are many trials that deliver biased results due to the implementation of inadequate methodology, poor data quality and erroneous or biased analyses. To increase the internal and external validity of randomised clinical trial results, several international institutions within clinical intervention research have formed The Centre for Statistical and Methodological Excellence (CESAME). Based on international consensus, the CESAME initiative will develop recommendations for the proper methodological planning, conduct and analysis of clinical intervention research. CESAME aims to increase the validity of randomised clinical trial results which will ultimately benefit patients worldwide across medical specialities. The work of CESAME will be performed within 3 closely interconnected pillars: (1) planning randomised clinical trials; (2) conducting randomised clinical trials; and (3) analysing randomised clinical trials.