Clinical trial monitoring effectiveness: Remote risk-based monitoring versus on-site monitoring with 100% source data verification

Factors Associated with Inclusion of Japan in Phase I Multiregional Clinical Trials in Oncology

BACKGROUND

Early inclusion of Japan in the global development program could be a key factor in reducing the drug lag, making participation in phase I multiregional clinical trials (Ph. I MRCTs) an important consideration for oncology drug development in Japan. We aimed to investigate the factors associated with the inclusion of Japan in Ph. I MRCTs in oncology.

METHODS

We compared the trial design, target population, type of primary tested drug, trial conduct profile, and sponsor profile for Ph. I MRCTs with or without Japan conducted by the top 20 companies in more than two countries and started between January 1, 2011, and December 31, 2020.

RESULTS

One hundred and ninety-seven Ph. I MRCTs included Japan, and 697 did not. Detailed features of the Ph. I MRCTs in oncology were summarized, and several factors (trial design, target population, trial conduct profile, and sponsor profile) associated with inclusion of Japan in the Ph. I MRCTs were identified.

CONCLUSIONS

It is important for Japanese subsidiaries within global pharmaceutical companies to closely communicate with the headquarters based on medical practice and unmet needs in Japan to join global development from an early stage. In addition, further efforts to attract emerging biopharmaceutical companies to Japan from the regulatory and/or political perspectives would be needed, thereby preventing drug lag in Japan.

Effectiveness of remote risk-based monitoring and potential benefits for combination with direct data capture

BACKGROUND

In recent years, alternative monitoring approaches, such as risk-based and remote monitoring techniques, have been recommended instead of traditional on-site monitoring to achieve more efficient monitoring. Remote risk-based monitoring (R2BM) is a monitoring technique that combines risk-based and remote monitoring and focuses on the detection of critical data and process errors. Direct data capture (DDC), which directly collects electronic source data, can facilitate R2BM by minimizing the extent of source documents that must be reviewed and reducing the additional workload on R2BM. In this study, we evaluated the effectiveness of R2BM and the synergistic effect of combining R2BM with DDC.

METHODS

R2BM was prospectively conducted with eight participants in a randomized clinical trial using a remote monitoring system that uploaded photographs of source documents to a cloud location. Critical data and processes were verified by R2BM, and later, all were confirmed by on-site monitoring to evaluate the ability of R2BM to detect critical data and process errors and the workload of uploading photographs for clinical trial staff. In addition, the reduction of the number of uploaded photographs was evaluated by assuming that the DDC was introduced for data collection.

RESULTS

Of the 4645 data points, 20.9% (n = 973, 95% confidence interval = 19.8-22.2) were identified as critical. All critical data errors corresponding to 5.4% (n = 53/973, 95% confidence interval = 4.1-7.1) of the critical data and critical process errors were detectable by R2BM. The mean number of uploaded photographs and the mean time to upload them per visit per participant were 34.4 ± 11.9 and 26.5 ± 11.8 min (mean ± standard deviation), respectively. When assuming that DDC was introduced for data collection, 45.0% (95% confidence interval = 42.2-47.9) of uploaded photographs for R2BM were reduced.

CONCLUSIONS

R2BM can detect 100% of the critical data and process errors without on-site monitoring. Combining R2BM with DDC reduces the workload of R2BM and further improves its efficiency.

Evaluating registry-based trial economics: Results from the STRESS clinical trial

Background

Registry-based trials have the potential to reduce randomized clinical trial (RCT) costs. However, observed cost differences also may be achieved through pragmatic trial designs. A systematic comparison of trial costs across different designs has not been previously performed.

Methods

We conducted a study to compare the current Steroids to Reduce Systemic inflammation after infant heart surgery (STRESS) registry-based RCT vs. two established designs: pragmatic RCT and explanatory RCT. The primary outcome was total RCT design costs. Secondary outcomes included: RCT duration and personnel hours. Costs were estimated using the Duke Clinical Research Institute's pricing model.

Results

The Registry-Based RCT estimated duration was 31.9 weeks greater than the other designs (259.5 vs. 227.6 weeks). This delay was caused by the Registry-Based design's periodic data harvesting that delayed site closing and statistical reporting. Total personnel hours were greatest for the Explanatory design followed by the Pragmatic design and the Registry-Based design (52,488 vs 29,763 vs. 24,480 h, respectively). Total costs were greatest for the Explanatory design followed by the Pragmatic design and the Registry-Based design ($10,140,263 vs. $4,164,863 vs. $3,268,504, respectively). Thus, Registry-Based total costs were 32 % of the Explanatory and 78 % of the Pragmatic design.

Conclusion

Total costs for the STRESS RCT with a registry-based design were less than those for a pragmatic design and much less than an explanatory design. Cost savings reflect design elements and leveraging of registry resources to improve cost efficiency, but delays to trial completion should be considered.

Opportunities and challenges for decentralised clinical trials in sub-Saharan Africa: a qualitative study

INTRODUCTION

Digital health has gained traction in research and development, and clinical decision support systems. The COVID-19 pandemic accelerated the adoption of decentralised clinical trials (DCTs) as a mitigation and efficiency improvement strategy. We assessed the opportunities and challenges of a digital transformation in clinical research in sub-Saharan Africa from different stakeholders' perspectives.

METHODS

A qualitative study, including 40 in-depth semi structured interviews, was conducted with investigators of three leading research institutions in sub-Saharan Africa and Switzerland, contract research organisations and sponsors managing clinical trials in sub-Saharan Africa. A thematic approach was used for the analysis.

RESULTS

Interviewees perceived DCTs as an opportunity for trial efficiency improvement, quality improvement and reducing the burden of people participating in clinical trials. However, to gain and maintain an optimal quality of clinical trials, a transition period is necessary to tackle contextual challenges before DCTs are being implemented. The main challenges are categorised into four themes: (1) usability and practicability of the technology; (2) paradigm shift and trial data quality; (3) ethical and regulatory hurdles and (4) contextual factors (site-specific research environment and sociocultural aspects).

CONCLUSION

The transformation from a site to a patient-centric model with an increased responsibility of participants should be context adapted. The transformation requires substantial investment, training of the various stakeholders and an efficient communication. Additionally, commitment of sponsors, investigators, ethics and regulatory authorities and the buy-in of the communities are essential for this change.

Monitoring metrics over time: Why clinical trialists need to systematically collect site performance metrics

Background

Over the last decade, there has been an increasing interest in risk-based monitoring (RBM) in clinical trials, resulting in a number of guidelines from regulators and its inclusion in ICH GCP. However, there is a lack of detail on how to approach RBM from a practical perspective, and insufficient understanding of best practice.

Purpose

We present a method for clinical trials units to track their metrics within clinical trials using descriptive statistics and visualisations.

Research Design

We suggest descriptive statistics and visualisations within a SWAT methodology.

Study Sample

We illustrate this method using the metrics from TEMPER, a monitoring study carried out in three trials at the MRC Clinical Trials Unit at UCL.

Data Collection

The data collection for TEMPER is described in DOI: 10.1177/1740774518793379.

Results

We show the results and discuss a protocol for a Study-Within-A-Trial (SWAT 167) for those wishing to use the method.

Conclusions

The potential benefits metric tracking brings to clinical trials include enhanced assessment of sites for potential corrective action, improved evaluation and contextualisation of the influence of metrics and their thresholds, and the establishment of best practice in RBM. The standardisation of the collection of such monitoring data would benefit both individual trials and the clinical trials community.

Lack of transparent reporting of trial monitoring approaches in randomised controlled trials: A systematic review of contemporary protocol papers

BACKGROUND

Monitoring is essential to ensure patient safety and data integrity in clinical trials as per Good Clinical Practice. The Standard Protocol Items: Recommendations for Interventional Trials Statement and its checklist guides authors to include monitoring in their protocols. We investigated how well monitoring was reported in published 'protocol papers' for contemporary randomised controlled trials.

METHODS

A systematic search was conducted in PubMed to identify eligible protocol papers published in selected journals between 1 January 2020 and 31 May 2020. Protocol papers were classified by whether they reported monitoring and, if so, by the details of monitoring. Data were summarised descriptively.

RESULTS

Of 811 protocol papers for randomised controlled trials, 386 (48%; 95% CI: 44%-51%) explicitly reported some monitoring information. Of these, 20% (77/386) reported monitoring information consistent with an on-site monitoring approach, and 39% (152/386) with central monitoring, 26% (101/386) with a mixed approach, while 14% (54/386) did not provide sufficient information to specify an approach. Only 8% (30/386) of randomised controlled trials reported complete details about all of scope, frequency and organisation of monitoring; frequency of monitoring was the least reported. However, 6% (25/386) of papers used the term 'audit' to describe 'monitoring'.

DISCUSSION

Monitoring information was reported in only approximately half of the protocol papers. Suboptimal reporting of monitoring hinders the clinical community from having the full information on which to judge the validity of a trial and jeopardises the value of protocol papers and the credibility of the trial itself. Greater efforts are needed to promote the transparent reporting of monitoring to journal editors and authors.

An approach to virtual clinical trial site visits: Lessons from the MeTeOR trial

Objective

To provide a framework for conducting clinical trial site visits virtually over videoconference, and to report on our experience doing so during the twelve-year follow-up of the Meniscal Tear in Osteoarthritis Research (MeTeOR) trial.

Design

Using published FDA guidance and prior literature, we created a structure for virtual site visits that prioritized monitoring for protocol compliance, safety, and data integrity. We conducted site visits in three stages: preparation for the visit, the virtual meeting itself, and follow-up. The preparation phase involved a review of relevant site-specific documents and a written report on the findings prior to the visit. The virtual visit itself was focused on any questions the site staff had about the pre-visit report, observing a mock study visit, touring physical spaces, and understanding the site staff's work environment. In the follow-up phase, we wrote a post-visit report summarizing the discussion during the visit and feedback given by the coordinating site.

Results

We found that the virtual site visits conducted as part of the MeTeOR trial follow-up ran smoothly. Although we could not directly compare in-person and virtual site visits, site staff unanimously appreciated the efficiency and effectiveness of the virtual site visits. We noted that displaying physical workspaces over videoconferencing was difficult, and a notable drawback to this method.

Conclusions

To our knowledge, this is the first published framework for conducting virtual clinical trial site visits. Conducting these visits virtually confer several advantages in terms of time, money, and efficiency.

Monitoring strategies for clinical intervention studies

BACKGROUND

Trial monitoring is an important component of good clinical practice to ensure the safety and rights of study participants, confidentiality of personal information, and quality of data. However, the effectiveness of various existing monitoring approaches is unclear. Information to guide the choice of monitoring methods in clinical intervention studies may help trialists, support units, and monitors to effectively adjust their approaches to current knowledge and evidence.

OBJECTIVES

To evaluate the advantages and disadvantages of different monitoring strategies (including risk-based strategies and others) for clinical intervention studies examined in prospective comparative studies of monitoring interventions.

SEARCH METHODS

We systematically searched CENTRAL, PubMed, and Embase via Ovid for relevant published literature up to March 2021. We searched the online 'Studies within A Trial' (SWAT) repository, grey literature, and trial registries for ongoing or unpublished studies.

SELECTION CRITERIA

We included randomized or non-randomized prospective, empirical evaluation studies of different monitoring strategies in one or more clinical intervention studies. We applied no restrictions for language or date of publication.

DATA COLLECTION AND ANALYSIS

We extracted data on the evaluated monitoring methods, countries involved, study population, study setting, randomization method, and numbers and proportions in each intervention group. Our primary outcome was critical and major monitoring findings in prospective intervention studies. Monitoring findings were classified according to different error domains (e.g. major eligibility violations) and the primary outcome measure was a composite of these domains. Secondary outcomes were individual error domains, participant recruitment and follow-up, and resource use. If we identified more than one study for a comparison and outcome definitions were similar across identified studies, we quantitatively summarized effects in a meta-analysis using a random-effects model. Otherwise, we qualitatively summarized the results of eligible studies stratified by different comparisons of monitoring strategies. We used the GRADE approach to assess the certainty of the evidence for different groups of comparisons.

MAIN RESULTS

We identified eight eligible studies, which we grouped into five comparisons. 1. Risk-based versus extensive on-site monitoring: based on two large studies, we found moderate certainty of evidence for the combined primary outcome of major or critical findings that risk-based monitoring is not inferior to extensive on-site monitoring. Although the risk ratio was close to 'no difference' (1.03 with a 95% confidence interval [CI] of 0.81 to 1.33, below 1.0 in favor of the risk-based strategy), the high imprecision in one study and the small number of eligible studies resulted in a wide CI of the summary estimate. Low certainty of evidence suggested that monitoring strategies with extensive on-site monitoring were associated with considerably higher resource use and costs (up to a factor of 3.4). Data on recruitment or retention of trial participants were not available. 2. Central monitoring with triggered on-site visits versus regular on-site visits: combining the results of two eligible studies yielded low certainty of evidence with a risk ratio of 1.83 (95% CI 0.51 to 6.55) in favor of triggered monitoring intervention. Data on recruitment, retention, and resource use were not available. 3. Central statistical monitoring and local monitoring performed by site staff with annual on-site visits versus central statistical monitoring and local monitoring only: based on one study, there was moderate certainty of evidence that a small number of major and critical findings were missed with the central monitoring approach without on-site visits: 3.8% of participants in the group without on-site visits and 6.4% in the group with on-site visits had a major or critical monitoring finding (odds ratio 1.7, 95% CI 1.1 to 2.7; P = 0.03). The absolute number of monitoring findings was very low, probably because defined major and critical findings were very study specific and central monitoring was present in both intervention groups. Very low certainty of evidence did not suggest a relevant effect on participant retention, and very low certainty evidence indicated an extra cost for on-site visits of USD 2,035,392. There were no data on recruitment. 4. Traditional 100% source data verification (SDV) versus targeted or remote SDV: the two studies assessing targeted and remote SDV reported findings only related to source documents. Compared to the final database obtained using the full SDV monitoring process, only a small proportion of remaining errors on overall data were identified using the targeted SDV process in the MONITORING study (absolute difference 1.47%, 95% CI 1.41% to 1.53%). Targeted SDV was effective in the verification of source documents, but increased the workload on data management. The other included study was a pilot study, which compared traditional on-site SDV versus remote SDV and found little difference in monitoring findings and the ability to locate data values despite marked differences in remote access in two clinical trial networks. There were no data on recruitment or retention. 5. Systematic on-site initiation visit versus on-site initiation visit upon request: very low certainty of evidence suggested no difference in retention and recruitment between the two approaches. There were no data on critical and major findings or on resource use.

AUTHORS' CONCLUSIONS

The evidence base is limited in terms of quantity and quality. Ideally, for each of the five identified comparisons, more prospective, comparative monitoring studies nested in clinical trials and measuring effects on all outcomes specified in this review are necessary to draw more reliable conclusions. However, the results suggesting risk-based, targeted, and mainly central monitoring as an efficient strategy are promising. The development of reliable triggers for on-site visits is ongoing; different triggers might be used in different settings. More evidence on risk indicators that identify sites with problems or the prognostic value of triggers is needed to further optimize central monitoring strategies. In particular, approaches with an initial assessment of trial-specific risks that need to be closely monitored centrally during trial conduct with triggered on-site visits should be evaluated in future research.