Reporting transparency and completeness in trials: Paper 4 - reporting of randomised controlled trials conducted using routinely collected electronic records - room for improvement

Randomised trials conducted using cohorts: a scoping review

INTRODUCTION

Cohort studies generate and collect longitudinal data for a variety of research purposes. Randomised controlled trials (RCTs) increasingly use cohort studies as data infrastructures to help identify and recruit trial participants and assess outcomes.

OBJECTIVE

To examine the extent, range and nature of research using cohorts for RCTs and describe the varied definitions and conceptual boundaries for RCTs using cohorts.

DESIGN

Scoping review.

DATA SOURCES

Searches were undertaken in January 2021 in MEDLINE (Ovid) and EBM Reviews-Cochrane Methodology Registry (Final issue, third Quarter 2012).

ELIGIBILITY CRITERIA

Reports published between January 2007 and December 2021 of (a) cohorts used or planned to be used, to conduct RCTs, or (b) RCTs which use cohorts to recruit participants and/or collect trial outcomes, or (c) methodological studies discussing the use of cohorts for RCTs.

DATA EXTRACTION AND SYNTHESIS

Data were extracted on the condition being studied, age group, setting, country/continent, intervention(s) and comparators planned or received, unit of randomisation, timing of randomisation, approach to informed consent, study design and terminology.

RESULTS

A total of 175 full-text articles were assessed for eligibility. We identified 61 protocols, 9 descriptions of stand-alone cohorts intended to be used for future RCTs, 39 RCTs using cohorts and 34 methodological papers.The use and scope of this approach is growing. The thematics of study are far-ranging, including population health, oncology, mental and behavioural disorders, and musculoskeletal conditions.Authors reported that this approach can lead to more efficient recruitment, more representative samples, and lessen disappointment bias and crossovers.

CONCLUSION

This review outlines the development of cohorts to conduct RCTs including the range of use and innovative changes and adaptations. Inconsistencies in the use of terminology and concepts are highlighted. Guidance now needs to be developed to support the design and reporting of RCTs conducted using cohorts.

Use of routine healthcare data in randomised implementation trials: a methodological mixed-methods systematic review

BACKGROUND

Routine data are increasingly used in randomised controlled trials evaluating healthcare interventions. They can aid participant identification, outcome assessment, and intervention delivery. Randomised implementation trials evaluate the effect of implementation strategies on implementation outcomes. Implementation strategies, such as reminders, are used to increase the uptake of evidence-based interventions into practice, while implementation outcomes, such as adoption, are key measures of the implementation process. The use of routine data in effectiveness trials has been explored; however, there are no reviews on implementation trials. We therefore aimed to describe how routine data have been used in randomised implementation trials and the design characteristics of these trials.

METHODS

We searched MEDLINE (Ovid) and Cochrane Central Register of Controlled Trials from Jan 2000 to Dec 2021 and manually searched protocols from trial registers. We included implementation trials and type II and type III hybrid effectiveness-implementation trials conducted using routine data. We extracted quantitative and qualitative data and narratively synthesised findings.

RESULTS

From 4206 titles, we included 80 trials, of which 22.5% targeted implementation of evidence-based clinical guidelines. Multicomponent implementation strategies were more commonly evaluated (70.0%) than single strategies. Most trials assessed adoption as the primary outcome (65.0%). The majority of trials extracted data from electronic health records (EHRs) (62.5%), and 91.3% used routine data for outcome ascertainment. Reported reasons for using routine data were increasing efficiency, assessing outcomes, reducing research burden, improving quality of care, identifying study samples, confirming findings, and assessing representativeness. Data quality, the EHR system, research governance, and external factors such as government policy could act either as facilitators or barriers.

CONCLUSIONS

Adherence to guidance on designing and reporting implementation studies, and specifically to harmonise the language used in describing implementation strategies and implementation outcomes, would aid identification of studies and data extraction. Routine healthcare data are widely used for participant identification, outcome assessment and intervention delivery. Researchers should familiarise themselves with the barriers and facilitators to using routine data, and efforts could be made to improve data quality to overcome some of the barriers.

REGISTRATION

PROSPERO CRD42022292321.

Paper II: thematic framework analysis of registry-based randomized controlled trials provided insights for designing trial ready registries

OBJECTIVES

Registry-based randomized controlled trials (RRCTs) are increasingly used, promising to address challenges associated with traditional randomized controlled trials. We identified strengths and limitations reported in planned and completed RRCTs to inform future RRCTs.

STUDY DESIGN AND SETTING

We conducted an environmental scan of literature discussing conceptual or methodological strengths and limitations of using registries for trial design and conduct (n = 12), followed by an analysis of RRCT protocols (n = 13) and reports (n = 77) identified from a scoping review. Using framework analysis, we developed and refined a conceptual framework of RRCT-specific strengths and limitations. We mapped and interpreted strengths and limitations discussed by authors of RRCT articles using framework codes and quantified the frequencies at which these were mentioned.

RESULTS

Our conceptual framework identified six main RRCT strengths and four main RRCT limitations. Considering implications for RRCT conduct and design, we formulated ten recommendations for registry designers, administrators, and trialists planning future RRCTs.

CONCLUSION

Consideration and application of empirically underpinned recommendations for future registry design and trial conduct may help trialists utilize registries and RRCTs to their full potential.

Paper I: Heterogeneous use of registry data for participant identification and primary outcome ascertainment is found in registry-based randomized controlled trials: A scoping review

OBJECTIVES

Registry-based randomized controlled trials (RRCTs) have potential to address limitations of traditional clinical trials. To describe their current use, information on planned and published RRCTs was identified and synthesized.

STUDY DESIGN AND SETTING

A scoping review of published RRCT protocols and reports was conducted. Articles published between 2010 and 2021 identified from electronic database searching, a recent review of RRCTs, and targeted searching for recent RRCT protocols (2018-2021) were screened. Data on trial data sources, types of primary outcomes, and how these primary outcomes were described, selected, and reported were extracted.

RESULTS

Ninety RRCT articles (77 reports; 13 protocols) were included. Forty nine (54%) used or planned to rely on registry data for their trial, 26 (29%) used both registry and additional data, and 15 (17%) used the registry solely for recruitment. Primary outcomes were routinely collected from the registry for 66 articles (73%). Only 28 articles (31%) described any methods to promote outcome data quality during or after data collection. Core outcome sets were not used in any of the trials.

CONCLUSION

With improvements in registry design, outcome selection, measurement, and reporting, future RRCTs may deliver on promises of efficient, high-quality trials that address clinically relevant questions.

Generalizability and reach of a randomized controlled trial to improve oral health among home care recipients: comparing participants and nonparticipants at baseline and during follow-up

Background

The generalizability of randomized controlled trials (RCTs) with a low response can be limited by systematic differences between participants and nonparticipants. This participation bias, however, is rarely investigated because data on nonparticipants is usually not available. The purpose of this article is to compare all participants and nonparticipants of a RCT to improve oral health among home care recipients at baseline and during follow-up using claims data.

Methods

Seven German statutory health and long-term care insurance funds invited 9656 home care recipients to participate in the RCT MundPflege. Claims data for all participants (n = 527, 5.5% response) and nonparticipants (n = 9129) were analyzed. Associations between trial participation and sex, age, care dependency, number of Elixhauser diseases, and dementia, as well as nursing, medical, and dental care utilization at baseline, were investigated using multivariable logistic regression. Associations between trial participation and the probability of (a) moving into a nursing home, (b) being hospitalized, and (c) death during 1 year of follow-up were examined via Cox proportional hazards regressions, controlling for baseline variables.

Results

At baseline, trial participation was positively associated with male sex (odds ratio 1.29 [95% confidence interval 1.08–1.54]), high (vs. low 1.46 [1.15–1.86]) care dependency, receiving occasional in-kind benefits to relieve caring relatives (1.45 [1.15–1.84]), having a referral by a general practitioner to a medical specialist (1.62 [1.21–2.18]), and dental care utilization (2.02 [1.67–2.45]). It was negatively associated with being 75–84 (vs. < 60 0.67 [0.50–0.90]) and 85 + (0.50 [0.37–0.69]) years old. For morbidity, hospitalizations, and formal, respite, short-term, and day or night care, no associations were found. During follow-up, participants were less likely to move into a nursing home than nonparticipants (hazard ratio 0.50 [0.32–0.79]). For hospitalizations and mortality, no associations were found.

Conclusions

For half of the comparisons, differences between participants and nonparticipants were observed. The RCT’s generalizability is limited, but to a smaller extent than one would expect because of the low response. Routine data provide a valuable source for investigating potential differences between trial participants and nonparticipants, which might be used by future RCTs to evaluate the generalizability of their findings.

Trial registration

German Clinical Trials Register DRKS00013517. Retrospectively registered on June 11, 2018.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13063-022-06470-y.

Success and failure factors of patient recruitment for industry-sponsored clinical trials and the role of the electronic health records-a qualitative interview study in the Nordic countries

Background

Patient recruitment for clinical trials is challenging—only approximately one third of all trials recruit their participants as planned. The pharmaceutical industry’s views on recruitment success have not been comprehensively investigated, although the industry globally conducts almost one third of all clinical drug trials. This study explored patient recruitment success and failure factors and the role of electronic health records (EHR) in the recruitment of trial participants in the Nordic countries.

Methods

A descriptive qualitative interview study was conducted with 21 representatives of the pharmaceutical industry or contract research organizations operating in Finland, Sweden, Denmark, and Norway. The interviews covered 34 clinical pre-market drug trials. Qualitative data were analyzed using inductive content analysis.

Results

Four main categories were derived to represent both success and failure factors, whereas a fifth category represented only failure factors: (1) sponsor-related (protocol and trial preparation and feasibility evaluations), (2) site/investigator-related (access to patients, motivation, commitment and resources), (3) patient-related recruitment factors (medical need, patients’ role in their care and attitudes towards trials), (4) Sponsor—sites—patients collaboration factors, and (5) start-up related factors. EHR was the most important source of recruitment, utilized in 29 out of 34 trials discussed. Revision of the legislation regulating the secondary use of EHR was highlighted as the most effective measure to facilitate the use of EHR in recruitment of trial participants.

Conclusions

The industry representatives recognized quite well their own role in contributing to the success or failure of the recruitment: to facilitate recruitment of trial participants, many obstacles can be avoided with better trial preparation and proper feasibility evaluations. As access to patients represents one of the key success or failure factors of recruitment, and as the EHR is regarded the main source of searching for and finding patients, the development of EHR utilization appears to represent a powerful tool to improve patient recruitment.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13063-022-06144-9.

Reporting transparency and completeness in Trials: Paper 2 - reporting of randomised trials using registries was often inadequate and hindered the interpretation of results

OBJECTIVE

Registries are important data sources for randomized controlled trials (RCTs), but reporting of how they are used may be inadequate. The objective was to describe the current adequacy of reporting of RCTs using registries.

STUDY DESIGN AND SETTING

We used a database of trials using registries from a scoping review supporting the development of the 2021 CONSORT extension for Trials Conducted Using Cohorts and Routinely Collected Data (CONSORT-ROUTINE). Reporting completeness of 13 CONSORT-ROUTINE items was assessed.

RESULTS

We assessed reports of 47 RCTs that used a registry, published between 2011 and 2018. Of the 13 CONSORT-ROUTINE items, 6 were adequately reported in at least half of reports (2 in at least 80%). The 7 other items were related to routinely collected data source eligibility (32% adequate), data linkage (8% adequate), validation and completeness of data used for outcome assessment (8% adequate), validation and completeness of data used for participant recruitment (0% adequate), participant flow (9% adequate), registry funding (6% adequate) and interpretation of results in consideration of registry use (25% adequate).

CONCLUSION

Reporting of trials using registries was often poor, particularly details on data linkage and quality. Better reporting is needed for appropriate interpretation of the results of these trials.