Impact of retrospective data verification to prepare the ICON6 trial for use in a marketing authorization application

Monitoring metrics over time: Why clinical trialists need to systematically collect site performance metrics

Background

Over the last decade, there has been an increasing interest in risk-based monitoring (RBM) in clinical trials, resulting in a number of guidelines from regulators and its inclusion in ICH GCP. However, there is a lack of detail on how to approach RBM from a practical perspective, and insufficient understanding of best practice.

Purpose

We present a method for clinical trials units to track their metrics within clinical trials using descriptive statistics and visualisations.

Research Design

We suggest descriptive statistics and visualisations within a SWAT methodology.

Study Sample

We illustrate this method using the metrics from TEMPER, a monitoring study carried out in three trials at the MRC Clinical Trials Unit at UCL.

Data Collection

The data collection for TEMPER is described in DOI: 10.1177/1740774518793379.

Results

We show the results and discuss a protocol for a Study-Within-A-Trial (SWAT 167) for those wishing to use the method.

Conclusions

The potential benefits metric tracking brings to clinical trials include enhanced assessment of sites for potential corrective action, improved evaluation and contextualisation of the influence of metrics and their thresholds, and the establishment of best practice in RBM. The standardisation of the collection of such monitoring data would benefit both individual trials and the clinical trials community.

Research monitoring practices in critical care research: a survey of current state and attitudes

Background/Aims

In 2016, international standards governing clinical research recommended that the approach to monitoring a research project should be undertaken based on risk, however it is unknown whether this approach has been adopted in Australia and New Zealand (ANZ) throughout critical care research. The aims of the project were to: 1) Gain an understanding of current research monitoring practices in academic-led clinical trials in the field of critical care research, 2) Describe the perceived barriers and enablers to undertaking research monitoring.

Methods

Electronic survey distributed to investigators, research co-ordinators and other research staff currently undertaking and supporting academic-led clinical trials in the field of critical care in ANZ.

Results

Of the 118 respondents, 70 were involved in the co-ordination of academic trials; the remaining results pertain to this sub-sample. Fifty-eight (83%) were working in research units associated with hospitals, 29 (41%) were experienced Research Coordinators and 19 (27%) Principal Investigators; 31 (44%) were primarily associated with paediatric research. Fifty-six (80%) develop monitoring plans with 33 (59%) of these undertaking a risk assessment; the most common barrier reported was lack of expertise. Nineteen (27%) indicated that centralised monitoring was used, noting that technology to support centralised monitoring (45/51; 88%) along with support from data managers and statisticians (45/52; 87%) were key enablers. Coronavirus disease-19 (COVID-19) impacted monitoring for 82% (45/55) by increasing remote (25/45; 56%) and reducing onsite (29/45; 64%) monitoring.

Conclusions

Contrary to Good Clinical Practice guidance, risk assessments to inform monitoring plans are not being consistently performed due to lack of experience and guidance. There is an urgent need to enhance risk assessment methodologies and develop technological solutions for centralised statistical monitoring.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-022-01551-7.

Monitoring strategies for clinical intervention studies

BACKGROUND

Trial monitoring is an important component of good clinical practice to ensure the safety and rights of study participants, confidentiality of personal information, and quality of data. However, the effectiveness of various existing monitoring approaches is unclear. Information to guide the choice of monitoring methods in clinical intervention studies may help trialists, support units, and monitors to effectively adjust their approaches to current knowledge and evidence.

OBJECTIVES

To evaluate the advantages and disadvantages of different monitoring strategies (including risk-based strategies and others) for clinical intervention studies examined in prospective comparative studies of monitoring interventions.

SEARCH METHODS

We systematically searched CENTRAL, PubMed, and Embase via Ovid for relevant published literature up to March 2021. We searched the online 'Studies within A Trial' (SWAT) repository, grey literature, and trial registries for ongoing or unpublished studies.

SELECTION CRITERIA

We included randomized or non-randomized prospective, empirical evaluation studies of different monitoring strategies in one or more clinical intervention studies. We applied no restrictions for language or date of publication.

DATA COLLECTION AND ANALYSIS

We extracted data on the evaluated monitoring methods, countries involved, study population, study setting, randomization method, and numbers and proportions in each intervention group. Our primary outcome was critical and major monitoring findings in prospective intervention studies. Monitoring findings were classified according to different error domains (e.g. major eligibility violations) and the primary outcome measure was a composite of these domains. Secondary outcomes were individual error domains, participant recruitment and follow-up, and resource use. If we identified more than one study for a comparison and outcome definitions were similar across identified studies, we quantitatively summarized effects in a meta-analysis using a random-effects model. Otherwise, we qualitatively summarized the results of eligible studies stratified by different comparisons of monitoring strategies. We used the GRADE approach to assess the certainty of the evidence for different groups of comparisons.

MAIN RESULTS

We identified eight eligible studies, which we grouped into five comparisons. 1. Risk-based versus extensive on-site monitoring: based on two large studies, we found moderate certainty of evidence for the combined primary outcome of major or critical findings that risk-based monitoring is not inferior to extensive on-site monitoring. Although the risk ratio was close to 'no difference' (1.03 with a 95% confidence interval [CI] of 0.81 to 1.33, below 1.0 in favor of the risk-based strategy), the high imprecision in one study and the small number of eligible studies resulted in a wide CI of the summary estimate. Low certainty of evidence suggested that monitoring strategies with extensive on-site monitoring were associated with considerably higher resource use and costs (up to a factor of 3.4). Data on recruitment or retention of trial participants were not available. 2. Central monitoring with triggered on-site visits versus regular on-site visits: combining the results of two eligible studies yielded low certainty of evidence with a risk ratio of 1.83 (95% CI 0.51 to 6.55) in favor of triggered monitoring intervention. Data on recruitment, retention, and resource use were not available. 3. Central statistical monitoring and local monitoring performed by site staff with annual on-site visits versus central statistical monitoring and local monitoring only: based on one study, there was moderate certainty of evidence that a small number of major and critical findings were missed with the central monitoring approach without on-site visits: 3.8% of participants in the group without on-site visits and 6.4% in the group with on-site visits had a major or critical monitoring finding (odds ratio 1.7, 95% CI 1.1 to 2.7; P = 0.03). The absolute number of monitoring findings was very low, probably because defined major and critical findings were very study specific and central monitoring was present in both intervention groups. Very low certainty of evidence did not suggest a relevant effect on participant retention, and very low certainty evidence indicated an extra cost for on-site visits of USD 2,035,392. There were no data on recruitment. 4. Traditional 100% source data verification (SDV) versus targeted or remote SDV: the two studies assessing targeted and remote SDV reported findings only related to source documents. Compared to the final database obtained using the full SDV monitoring process, only a small proportion of remaining errors on overall data were identified using the targeted SDV process in the MONITORING study (absolute difference 1.47%, 95% CI 1.41% to 1.53%). Targeted SDV was effective in the verification of source documents, but increased the workload on data management. The other included study was a pilot study, which compared traditional on-site SDV versus remote SDV and found little difference in monitoring findings and the ability to locate data values despite marked differences in remote access in two clinical trial networks. There were no data on recruitment or retention. 5. Systematic on-site initiation visit versus on-site initiation visit upon request: very low certainty of evidence suggested no difference in retention and recruitment between the two approaches. There were no data on critical and major findings or on resource use.

AUTHORS' CONCLUSIONS

The evidence base is limited in terms of quantity and quality. Ideally, for each of the five identified comparisons, more prospective, comparative monitoring studies nested in clinical trials and measuring effects on all outcomes specified in this review are necessary to draw more reliable conclusions. However, the results suggesting risk-based, targeted, and mainly central monitoring as an efficient strategy are promising. The development of reliable triggers for on-site visits is ongoing; different triggers might be used in different settings. More evidence on risk indicators that identify sites with problems or the prognostic value of triggers is needed to further optimize central monitoring strategies. In particular, approaches with an initial assessment of trial-specific risks that need to be closely monitored centrally during trial conduct with triggered on-site visits should be evaluated in future research.

Cediranib in addition to chemotherapy for women with relapsed platinum-sensitive ovarian cancer (ICON6): overall survival results of a phase III randomised trial

BACKGROUND

Cediranib, an oral anti-angiogenic VEGFR 1-3 inhibitor, was studied at a daily dose of 20 mg in combination with platinum-based chemotherapy and as maintenance in a randomised trial in patients with first relapse of 'platinum-sensitive' ovarian cancer and has been shown to improve progression-free survival (PFS).

PATIENTS AND METHODS

ICON6 (NCT00532194) was an international three-arm, double-blind, placebo-controlled randomised trial. Between December 2007 and December 2011, 456 women were randomised, using stratification, to receive either chemotherapy with placebo throughout (arm A, reference); chemotherapy with concurrent cediranib, followed by maintenance placebo (arm B, concurrent); or chemotherapy with concurrent cediranib, followed by maintenance cediranib (arm C, maintenance). Due to an enforced redesign of the trial in September 2011, the primary endpoint became PFS between arms A and C which we have previously published, and the overall survival (OS) was defined as a secondary endpoint, which is reported here.

RESULTS

After a median follow-up of 25.6 months, strong evidence of an effect of concurrent plus maintenance cediranib on PFS was observed [hazard ratio (HR) 0.56, 95% confidence interval (CI) 0.44-0.72, P < 0.0001]. In this final update of the survival analysis, 90% of patients have died. There was a 7.4-month difference in median survival and an HR of 0.86 (95% CI: 0.67-1.11, P = 0.24) in favour of arm C. There was strong evidence of a departure from the assumption of non-proportionality using the Grambsch-Therneau test (P = 0.0031), making the HR difficult to interpret. Consequently, the restricted mean survival time (RMST) was used and the estimated difference over 6 years by the RMST was 4.8 months (95% CI: -0.09 to 9.74 months).

CONCLUSIONS

Although a statistically significant difference in time to progression was seen, the enforced curtailment in recruitment meant that the secondary analysis of OS was underpowered. The relative reduction in the risk of death of 14% risk of death was not conventionally statistically significant, but this improvement and the increase in the mean survival time in this analysis suggest that cediranib may have worthwhile activity in the treatment of recurrent ovarian cancer and that further research should be undertaken.

Time-consuming and expensive data quality monitoring procedures persist in clinical trials: A national survey

INTRODUCTION

The Good Clinical Practice guideline identifies that data monitoring is an essential research activity. However, limited evidence exists on how to perform monitoring including the amount or frequency that is needed to ensure data quality. This study aims to explore the monitoring procedures that are implemented to ensure data quality in Australian clinical research studies.

MATERIAL AND METHODS

Clinical studies listed on the Australian and New Zealand Clinical Trials Registry were invited to participate in a national cross-sectional, mixed-mode, multi-contact (postal letter and e-mail) web-based survey. Information was gathered about the types of data quality monitoring procedures being implemented.

RESULTS

Of the 3689 clinical studies contacted, 589 (16.0%) responded, of which 441 (77.4%) completed the survey. Over half (55%) of the studies applied source data verification (SDV) compared to risk-based targeted and triggered monitoring (10-11%). Conducting 100% on-site monitoring was most common for those who implemented the traditional approach. Respondents who did not conduct 100% monitoring, included 1-25% of data points for SDV, centralized or on-site monitoring. The incidence of adverse events and protocol deviations were the most likely factors to trigger a site visit for risk-based triggered (63% and 44%) and centralized monitoring (48% and 44%), respectively.

CONCLUSION

Instead of using more optimal risk-based approaches, small single-site clinical studies are conducting traditional monitoring procedures which are time consuming and expensive. Formal guidelines need to be improved and provided to all researchers for 'new' risk-based monitoring approaches.

Monitoring in practice - How are UK academic clinical trials monitored? A survey

Background

Despite the US Food and Drug Administration (FDA) and European Medicines Agency (EMA) encouraging the use of risk-based monitoring for trials in 2013, there remains a lack of evidence-based guidelines on how to monitor. We surveyed the academic United Kingdom Clinical Research Collaboration (UKCRC) registered clinical trials units (CTUs) to find out their policy on monitoring of phase III randomised clinical trials of an investigational medicinal product (CTIMPs).

Methods

An online survey of monitoring policy with sections on the CTU, central monitoring and on-site monitoring was sent to all 50 UKCRC registered CTUs in November 2018.

Descriptive data analysis and tabulations are reported using the total number answering each question.

Results

A total of 43/50 (86%) of CTUs responded with 38 conducting phase III randomised CTIMP trials. Of these 38 CTUs, 34 finished the survey. Most CTUs (36/37, 97%) use a central monitoring process to guide, target or supplement site visits. More than half (19/36, 53%) of CTUs do not use an automated monitoring report when centrally monitoring trials and all units use trial team knowledge to make a final decision on whether an on-site visit is required.

A total of 31/34 (91%) CTUs used triggers to decide whether or not to conduct an on-site monitoring visit. On-site, a mixture of source data verification and checking of processes was carried out.

The CTUs overwhelmingly (27/34, 79%) selected optimising central monitoring as their most pressing concern.

Conclusion

The survey showed a wide variation in phase III randomised CTIMP trial monitoring practices by academic clinical trials units within a single research-active country. We urgently need to develop evidence-based regulator-agreed guidance for CTUs on best practice for both central and on-site monitoring and to develop tools for all CTUs to use.