Prevalence and reporting of recruitment, randomisation and treatment errors in clinical trials: A systematic review

Categorisation of continuous covariates for stratified randomisation: How should we adjust?

To obtain valid inference following stratified randomisation, treatment effects should be estimated with adjustment for stratification variables. Stratification sometimes requires categorisation of a continuous prognostic variable (eg, age), which raises the question: should adjustment be based on randomisation categories or underlying continuous values? In practice, adjustment for randomisation categories is more common. We reviewed trials published in general medical journals and found none of the 32 trials that stratified randomisation based on a continuous variable adjusted for continuous values in the primary analysis. Using data simulation, this article evaluates the performance of different adjustment strategies for continuous and binary outcomes where the covariate-outcome relationship (via the link function) was either linear or non-linear. Given the utility of covariate adjustment for addressing missing data, we also considered settings with complete or missing outcome data. Analysis methods included linear or logistic regression with no adjustment for the stratification variable, adjustment for randomisation categories, or adjustment for continuous values assuming a linear covariate-outcome relationship or allowing for non-linearity using fractional polynomials or restricted cubic splines. Unadjusted analysis performed poorly throughout. Adjustment approaches that misspecified the underlying covariate-outcome relationship were less powerful and, alarmingly, biased in settings where the stratification variable predicted missing outcome data. Adjustment for randomisation categories tends to involve the highest degree of misspecification, and so should be avoided in practice. To guard against misspecification, we recommend use of flexible approaches such as fractional polynomials and restricted cubic splines when adjusting for continuous stratification variables in randomised trials.

Problem-solving therapy for pregnant women experiencing depressive symptoms and intimate partner violence: A randomised, controlled feasibility trial in rural Ethiopia

Evidence for the feasibility of brief psychological interventions for pregnant women experiencing intimate partner violence (IPV) in rural, low-income country settings is scarce. In rural Ethiopia, the prevalence of antenatal depressive symptoms and lifetime IPV are 29% and 61%, respectively. We aimed to assess the feasibility and related implementation outcomes of brief problem-solving therapy (PST) adapted for pregnant women experiencing IPV (PST-IPV) in rural Ethiopia, and of a randomised, controlled feasibility study design. We recruited 52 pregnant women experiencing depressive symptoms and past-year IPV from two antenatal care (ANC) services. Consenting women were randomised to PST-IPV (n = 25), 'standard' PST (not adapted for women experiencing IPV; n = 12) or enhanced usual care (information about sources of support; n = 15). Masked data collectors conducted outcome assessments nine weeks post-enrolment. Addis Ababa University (#032/19/CDT) and King's College London (#HR-18/19-9230) approved the study. Fidelity to randomisation was impeded by strong cultural norms about what constituted IPV. However, recruitment was feasible (recruitment rate: 1.5 per day; 37% of women screened were eligible). The intervention and trial were acceptable to women (4% declined initial screening, none declined to participate, and 76% attended all four sessions of either active intervention). PST-IPV was acceptable to ANC providers: none dropped out. Sessions lasting up to a mean 52 minutes raised questions about the appropriateness of the model to this context. Competence assessments recommended supplementary communication skills training. Fidelity assessments indicated high adherence, quality, and responsiveness but assessing risks and social networks, and discussing confidentiality needed improvement. Adjustments to optimise a future, fully powered, randomised controlled trial include staggering recruitment in line with therapist availability, more training on the types of IPV and how to discuss them, automating randomisation, a supervision cascade model, and conducting post-intervention outcome assessments immediately and three months postpartum. Registration: Pan African Clinical Trials Registry #PACTR202002513482084 (13/12/2019): https://pactr.samrc.ac.za/TrialDisplay.aspx?TrialID=9601.

Research integrity in randomized clinical trials: A scoping umbrella review

BACKGROUND

Randomized clinical trials (RCTs) are experiencing a crisis of confidence in their trustworthiness. Although a comprehensive literature search yielded several reviews on RCT integrity, an overarching overview is lacking.

OBJECTIVES

The authors undertook a scoping umbrella review of the research integrity literature concerning RCTs.

SEARCH STRATEGY AND SELECTION CRITERIA

Following prospective registration (https://osf.io/3ursn), two reviewers independently searched PubMed, Scopus, The Cochrane Library, and Google Scholar, without language or time restrictions, until November 2021. The authors included systematic reviews covering any aspect of research integrity throughout the RCT lifecycle.

DATA COLLECTION AND ANALYSIS

The authors assessed methodological quality using a modified AMSTAR 2 (A Measurement Tool to Assess Systematic Reviews) tool and collated the main findings.

MAIN RESULTS

A total of 55 relevant reviews, summarizing 6001 studies (median per review, 63; range, 8-1106) from 1964 to 2021, had an overall critically low quality of 96% (53 reviews). Topics covered included general aspects (15%), design and approval (22%), conduct and monitoring (11%), reporting (38%), postpublication concerns (2%), and future research (13%). The most common integrity issues covered were ethics (18%) and transparency (18%).

CONCLUSIONS

Low-quality reviews identified various integrity issues across the RCT lifecycle, emphasizing the importance of high ethical standards and professionalism while highlighting gaps in the integrity landscape. Multistakeholder consensus is needed to develop specific RCT integrity standards.

Handling misclassified stratification variables in the analysis of randomised trials with continuous outcomes

Many trials use stratified randomisation, where participants are randomised within strata defined by one or more baseline covariates. While it is important to adjust for stratification variables in the analysis, the appropriate method of adjustment is unclear when stratification variables are affected by misclassification and hence some participants are randomised in the incorrect stratum. We conducted a simulation study to compare methods of adjusting for stratification variables affected by misclassification in the analysis of continuous outcomes when all or only some stratification errors are discovered, and when the treatment effect or treatment-by-covariate interaction effect is of interest. The data were analysed using linear regression with no adjustment, adjustment for the strata used to perform the randomisation (randomisation strata), adjustment for the strata if all errors are corrected (true strata), and adjustment for the strata after some errors are discovered and corrected (updated strata). The unadjusted model performed poorly in all settings. Adjusting for the true strata was optimal, while the relative performance of adjusting for the randomisation strata or the updated strata varied depending on the setting. As the true strata are unlikely to be known with certainty in practice, we recommend using the updated strata for adjustment and performing subgroup analyses, provided the discovery of errors is unlikely to depend on treatment group, as expected in blinded trials. Greater transparency is needed in the reporting of stratification errors and how they were addressed in the analysis.

Clinical trial recruiters' experiences working with trial eligibility criteria: results of an exploratory, cross-sectional, online survey in the UK

BACKGROUND

Eligibility criteria are a fundamental element of clinical trial design, defining who can and who should not participate in a trial. Problems with the design or application of criteria are known to occur and pose risks to participants' safety and trial integrity, sometimes also negatively impacting on trial recruitment and generalisability. We conducted a short, exploratory survey to gather evidence on UK recruiters' experiences interpreting and applying eligibility criteria and their views on how criteria are communicated and developed.

METHODS

Our survey included topics informed by a wider programme of work at the Clinical Trials Research Unit, University of Leeds, on assuring eligibility criteria quality. Respondents were asked to answer based on all their trial experience, not only on experiences with our trials. The survey was disseminated to recruiters collaborating on trials run at our trials unit, and via other mailing lists and social media. The quantitative responses were descriptively analysed, with inductive analysis of free-text responses to identify themes.

RESULTS

A total of 823 eligible respondents participated. In total, 79% of respondents reported finding problems with eligibility criteria in some trials, and 9% in most trials. The main themes in the types of problems experienced were criteria clarity (67% of comments), feasibility (34%), and suitability (14%). In total, 27% of those reporting some level of problem said these problems had led to patients being incorrectly included in trials; 40% said they had led to incorrect exclusions. Most respondents (56%) reported accessing eligibility criteria mainly in the trial protocol. Most respondents (74%) supported the idea of recruiter review of eligibility criteria earlier in the protocol development process.

CONCLUSIONS

Our survey corroborates other evidence about the existence of suboptimal trial eligibility criteria. Problems with clarity were the most often reported, but the number of comments on feasibility and suitability suggest some recruiters feel eligibility criteria and associated assessments can hinder recruitment to trials. Our proposal for more recruiter involvement in protocol development has strong support and some potential benefits, but questions remain about how best to implement this. We invite other trialists to consider our other suggestions for how to assure quality in trial eligibility criteria.

Adherence to participant flow diagrams in trials on postoperative pain management after total hip and knee arthroplasty: a methodological review

BACKGROUND

The Consolidated Standards of Reporting Trials (CONSORT) statement aims to improve transparent reporting of randomised clinical trials. It comprises a participant flow diagram with the reporting of essential numbers for enrolment, allocation and analyses. We aimed to quantify the use of participant flow diagrams in randomised clinical trials on postoperative pain management after total hip and knee arthroplasty.

METHODS

We searched PubMed, Embase and CENTRAL up till January 2020. The primary outcome was the proportion of trials with adequate reporting of participant flow diagrams, defined as reporting of number of participants screened for eligibility, randomised and included in the primary analysis. Secondary outcomes were recruitment (randomised:screened) and retention (analysed:randomised) rates, reporting of a statistical strategy, reasons for exclusion from the primary analysis and handling of missing outcome data. Trends over time were assessed with statistical process control.

RESULTS

Of the 570 included trials, we found adequate reporting in 240 (42%). Reporting with participant flow diagram increased significantly over time. Median recruitment was 73% (IQR 44-91%), and retention was 97% (IQR 93-100%). These rates did not change over time. Trials with adequate reporting of participant flow were more likely to report a statistical strategy (41% vs 8%), reasons for post-randomisation exclusions (100% vs 55%) and handling of missing outcome data (14% vs 6%).

CONCLUSIONS

Adherence to participant flow diagrams for RCTs has increased significantly over time. Still, there is room for improvement of adequate reporting of flow diagrams, to increase transparency of trials details.

Assessing the potential for prevention or earlier detection of on-site monitoring findings from randomised controlled trials: Further analyses of findings from the prospective TEMPER triggered monitoring study

BACKGROUND/AIMS

Clinical trials should be designed and managed to minimise important errors with potential to compromise patient safety or data integrity, employ monitoring practices that detect and correct important errors quickly, and take robust action to prevent repetition. Regulators highlight the use of risk-based monitoring, making greater use of centralised monitoring and reducing reliance on centre visits. The TEMPER study was a prospective evaluation of triggered monitoring (a risk-based monitoring method), whereby centres are prioritised for visits based on central monitoring results. Conducted in three UK-based randomised cancer treatment trials of investigational medicine products with time-to-event outcomes, it found high levels of serious findings at triggered centre visits but also at visits to matched control centres that, based on central monitoring, were not of concern. Here, we report a detailed review of the serious findings from TEMPER centre visits. We sought to identify feasible, centralised processes which might detect or prevent these findings without a centre visit.

METHODS

The primary outcome of this study was the proportion of all 'major' and 'critical' TEMPER centre visit findings theoretically detectable or preventable through a feasible, centralised process. To devise processes, we considered a representative example of each finding type through an internal consensus exercise. This involved (a) agreeing the potential, by some described process, for each finding type to be centrally detected or prevented and (b) agreeing a proposed feasibility score for each proposed process. To further assess feasibility, we ran a consultation exercise, whereby the proposed processes were reviewed and rated for feasibility by invited external trialists.

RESULTS

In TEMPER, 312 major or critical findings were identified at 94 visits. These findings comprised 120 distinct issues, for which we proposed 56 different centralised processes. Following independent review of the feasibility of the proposed processes by 87 consultation respondents across eight different trial stakeholder groups, we conclude that 306/312 (98%) findings could theoretically be prevented or identified centrally. Of the processes deemed feasible, those relating to informed consent could have the most impact. Of processes not currently deemed feasible, those involving use of electronic health records are among those with the largest potential benefit.

CONCLUSIONS

This work presents a best-case scenario, where a large majority of monitoring findings were deemed theoretically preventable or detectable by central processes. Caveats include the cost of applying all necessary methods, and the resource implications of enhanced central monitoring for both centre and trials unit staff. Our results will inform future monitoring plans and emphasise the importance of continued critical review of monitoring processes and outcomes to ensure they remain appropriate.

The interval between progression and therapy initiation is the key prognostic parameter in relapsing diffuse large B cell lymphoma: analysis from the Czech Lymphoma Study Group database (NIHIL)

Relapsing diffuse large B cell lymphomas (rDLBCL) represent a heterogeneous disease. This heterogeneity should be recognized and reflected, because it can deform the interpretation of clinical trial results. DLBCL patients with the first relapse and without CNS involvement were identified in the Czech Lymphoma Study Group (CLSG) database. Interval-to-therapy (ITT) was defined as the time between the first manifestation of rDLBCL and the start of any treatment. The overall survival (OS) of different ITT cohorts (< 7 vs. 7-21 vs. > 21 days) was compared. In total, 587 rDLBCLs (51.8% males) progressed with a median of 12.8 months (range 1.6 to 152.3) since the initial diagnosis (2000-2017). At the time of relapse, the median age was 67 years (range 22-95). First-line therapy was administered in 99.3% of the patients; CHOP and anti-CD20 were given to 69.2% and 84.7% of the patients, respectively. The salvage immune/chemotherapy was administered in 88.1% of the patients (39.2% platinum-based regimen). The median ITT was 20 days (range 1-851), but 23.2% of patients initiated therapy within 7 days. The 5-year OS was 17.4% (range 10-24.5%) vs. 20.5% (range 13.5-27.4%) vs. 42.2% (range 35.5-48.8%) for ITT < 7 vs. 7-21 vs. > 21 days (p < 0.001). ITT was associated with B symptoms (p 0.004), ECOG (p < 0.001), stage (p 0.002), bulky disease (p 0.005), elevated LDH (p < 0.001), and IPI (p < 0.001). The ITT mirrors the real clinical behavior of rDLBCL. There are patients (ITT < 7 days) with aggressive disease and a poor outcome. Conversely, there are rDLBCLs with ITT ≥ 21 days who survive for a long time.

Towards standardisation of evidence-based clinical care process specifications

There is a strong push towards standardisation of treatment approaches, care processes and documentation of clinical practice. However, confusion persists regarding terminology and description of many clinical care process specifications which this research seeks to resolve by developing a taxonomic characterisation of clinical care process specifications. Literature on clinical care process specifications was analysed, creating the starting point for identifying common characteristics and how each is constructed and used in the clinical setting. A taxonomy for clinical care process specifications is presented. The De Bleser approach to limited clinical care process specifications characterisation was extended and each clinical care process specification is successfully characterised in terms of purpose, core elements and relationship to the other clinical care process specification types. A case study on the diagnosis and treatment of Type 2 Diabetes in the United Kingdom was used to evaluate the taxonomy and demonstrate how the characterisation framework applies. Standardising clinical care process specifications ensures that the format and content are consistent with expectations, can be read more quickly and high-quality information can be recorded about the patient. Standardisation also enables computer interpretability, which is important in integrating Learning Health Systems into the modern clinical environment. The approach presented allows terminologies for clinical care process specifications that were widely used interchangeably to be easily distinguished, thus, eliminating the existing confusion.

A Network Analysis of Multiple Myeloma Related Gene Signatures

Multiple myeloma (MM) is the second most prevalent hematological cancer. MM is a complex and heterogeneous disease, and thus, it is essential to leverage omics data from large MM cohorts to understand the molecular mechanisms underlying MM tumorigenesis, progression, and drug responses, which may aid in the development of better treatments. In this study, we analyzed gene expression, copy number variation, and clinical data from the Multiple Myeloma Research Consortium (MMRC) dataset and constructed a multiple myeloma molecular causal network (M3CN). The M3CN was used to unify eight prognostic gene signatures in the literature that shared very few genes between them, resulting in a prognostic subnetwork of the M3CN, consisting of 178 genes that were enriched for genes involved in cell cycle (fold enrichment = 8.4, p value = 6.1 × 10⁻²⁶). The M3CN was further used to characterize immunomodulators and proteasome inhibitors for MM, demonstrating the pleiotropic effects of these drugs, with drug-response signature genes enriched across multiple M3CN subnetworks. Network analyses indicated potential links between these drug-response subnetworks and the prognostic subnetwork. To elucidate the structure of these important MM subnetworks, we identified putative key regulators predicted to modulate the state of these subnetworks. Finally, to assess the predictive power of our network-based models, we stratified MM patients in an independent cohort, the MMRF-CoMMpass study, based on the prognostic subnetwork, and compared the performance of this subnetwork against other signatures in the literature. We show that the M3CN-derived prognostic subnetwork achieved the best separation between different risk groups in terms of log-rank test p-values and hazard ratios. In summary, this work demonstrates the power of a probabilistic causal network approach to understanding molecular mechanisms underlying the different MM signatures.

Sample size calculation in multi-centre clinical trials

Background

Multi-centre randomized controlled clinical trials play an important role in modern evidence-based medicine. Advantages of collecting data from more than one site are numerous, including accelerated recruitment and increased generalisability of results. Mixed models can be applied to account for potential clustering in the data, in particular when many small centres contribute patients to the study. Previously proposed methods on sample size calculation for mixed models only considered balanced treatment allocations which is an unlikely outcome in practice if block randomisation with reasonable choices of block length is used.

Methods

We propose a sample size determination procedure for multi-centre trials comparing two treatment groups for a continuous outcome, modelling centre differences using random effects and allowing for arbitrary sample sizes. It is assumed that block randomisation with fixed block length is used at each study site for subject allocation. Simulations are used to assess operation characteristics such as power of the sample size approach. The proposed method is illustrated by an example in disease management systems.

Results

A sample size formula as well as a lower and upper boundary for the required overall sample size are given. We demonstrate the superiority of the new sample size formula over the conventional approach of ignoring the multi-centre structure and show the influence of parameters such as block length or centre heterogeneity. The application of the procedure on the example data shows that large blocks require larger sample sizes, if centre heterogeneity is present.

Conclusion

Unbalanced treatment allocation can result in substantial power loss when centre heterogeneity is present but not considered at the planning stage. When only few patients by centre will be recruited, one has to weigh the risk of imbalance between treatment groups due to large blocks and the risk of unblinding due to small blocks. The proposed approach should be considered when planning multi-centre trials.

Electronic supplementary material

The online version of this article (10.1186/s12874-018-0602-y) contains supplementary material, which is available to authorized users.