Rationale, Methodological Quality, and Reporting of Cluster-Randomized Controlled Trials in Critical Care Medicine: A Systematic Review

Recruitment characteristics of randomised trials in critical care: A systematic review

BACKGROUND/AIMS

To summarise the temporal trends of recruitment and methodological characteristics of critical care randomised controlled trials with the primary outcome of mortality.

METHODS

PubMed was searched for articles meeting inclusion and exclusion criteria. Randomised controlled trials, with primary outcome of mortality, of adult and paediatric critical care patients treated in an intensive care unit, were included. Neonatal intensive care unit trials, non-English publications and conference proceedings or abstracts without full-length publications were excluded. Duplicate literature search, article selection and quality assessment were performed by two reviewers with disputes resolved through discussion. Data were extracted into a custom-designed Research Electronic Data Capture database.

RESULTS

The search identified 67,199 records of which 230 were included. The annual number of critical care randomised controlled trials published increased gradually over a 30-year period from 0 in 1990 to 19 in 2014 with stabilisation at 8-11 between 2015 and 2020. Twenty-seven percent of randomised controlled trials were low risk in all categories using the Cochrane Risk of Bias tool. Methodological characteristics such as registration on clinical trials registries and data safety monitoring committee presence significantly (p < 0.001) increased over time. The median recruitment was 376 patients (interquartile range 125-895) with significant increase (p = 0.002) from 62 (interquartile range: 33-486) in 1991 to 725 (interquartile range: 537-2600) in 2020. This was accompanied by an increase in recruitment times. Thus overall, recruitment rates did not increase. Early cessation occurred in 23% (54/230) of randomised controlled trials with no temporal trend.

CONCLUSION

The number, size and some methodological qualities of critical randomised controlled trials with primary outcome of mortality have increased over time, but rates of recruitment and early cessation have been unchanged.

Influential methods reports for group-randomized trials and related designs

BACKGROUND

This article identifies the most influential methods reports for group-randomized trials and related designs published through 2020. Many interventions are delivered to participants in real or virtual groups or in groups defined by a shared interventionist so that there is an expectation for positive correlation among observations taken on participants in the same group. These interventions are typically evaluated using a group- or cluster-randomized trial, an individually randomized group treatment trial, or a stepped wedge group- or cluster-randomized trial. These trials face methodological issues beyond those encountered in the more familiar individually randomized controlled trial.

METHODS

PubMed was searched to identify candidate methods reports; that search was supplemented by reports known to the author. Candidate reports were reviewed by the author to include only those focused on the designs of interest. Citation counts and the relative citation ratio, a new bibliometric tool developed at the National Institutes of Health, were used to identify influential reports. The relative citation ratio measures influence at the article level by comparing the citation rate of the reference article to the citation rates of the articles cited by other articles that also cite the reference article.

RESULTS

In total, 1043 reports were identified that were published through 2020. However, 55 were deemed to be the most influential based on their relative citation ratio or their citation count using criteria specific to each of the three designs, with 32 group-randomized trial reports, 7 individually randomized group treatment trial reports, and 16 stepped wedge group-randomized trial reports. Many of the influential reports were early publications that drew attention to the issues that distinguish these designs from the more familiar individually randomized controlled trial. Others were textbooks that covered a wide range of issues for these designs. Others were "first reports" on analytic methods appropriate for a specific type of data (e.g. binary data, ordinal data), for features commonly encountered in these studies (e.g. unequal cluster size, attrition), or for important variations in study design (e.g. repeated measures, cohort versus cross-section). Many presented methods for sample size calculations. Others described how these designs could be applied to a new area (e.g. dissemination and implementation research). Among the reports with the highest relative citation ratios were the CONSORT statements for each design.

CONCLUSIONS

Collectively, the influential reports address topics of great interest to investigators who might consider using one of these designs and need guidance on selecting the most appropriate design for their research question and on the best methods for design, analysis, and sample size.

Cluster randomized trials of individual-level interventions were at high risk of bias

OBJECTIVES

To describe the prevalence of risks of bias in cluster-randomized trials of individual-level interventions, according to the Cochrane Risk of Bias tool.

STUDY DESIGN AND SETTING

Review undertaken in duplicate of a random sample of 40 primary reports of cluster-randomized trials of individual-level interventions.

RESULTS

The most common reported reasons for adopting cluster randomization were the need to avoid contamination (17, 42.5%) and practical considerations (14, 35%). Of the 40 trials all but one was assessed as being at risk of bias. A majority (27, 67.5%) were assessed as at risk due to the timing of identification and recruitment of participants; many (21, 52.5%) due to an apparent lack of adequate allocation concealment; and many due to selectively reported results (22, 55%), arising from a mixture of reasons including lack of documentation of primary outcome. Other risks mostly occurred infrequently.

CONCLUSION

Many cluster-randomized trials evaluating individual-level interventions appear to be at risk of bias, mostly due to identification and recruitment biases. We recommend that investigators carefully consider the need for cluster randomization; follow recommended procedures to mitigate risks of identification and recruitment bias; and adhere to good reporting practices including clear documentation of primary outcome and allocation concealment methods.