Concept and implementation of a study dashboard module for a continuous monitoring of trial recruitment and documentation

The future of clinical trials and drug development: 2050

A workshop held at the 18th Annual Conference of the Pharmaceutical Contract Management Group in Krakow on 9 September 2022 asked over 200 delegates what the clinical trial landscape would look like in 2050. Issues considered included who will be running the pharmaceutical industry in 2050; how 'health chips', wearables and diagnostics will impact on finding the right patients to study; how will artificial intelligence be designing and controlling clinical trials; and what will the role of the Clinical Research Associate, the critical observer, documenter and conductor of a clinical trial need to look like by 2050. The consensus was that, by 2050, if you are working in clinical trials, you will be a data scientist. We can expect to see an increasing role of new technologies and a new three-phase registration model for novel therapies. The first phase will involve an aspect of quality evaluation and biological proof-of-concept probably involving more preclinical modelling and engineered human cell lines and fewer animal studies than currently used. Once registered, new products will enter a period of adaptive clinical development (delivered as a single study) intended to establish safety. This phase will most likely take around 1-2 years and explore tailored options for administration. Investigations will most likely be conducted in patients, possibly in a 'patient-in-a-box' setting (hospital or healthcare centre, virtual or microsite). On completion of safety licencing, drugs will begin an assessment of efficacy in partnership with those responsible for reimbursement - testing will be performed in patients, possibly where individual patient involvement in safety testing will offer some reimbursement deal for future treatment. Change is coming, though its precise form will likely depend on the creativity and vision of sponsors, regulators and payers.

Development of a risk-tailored approach and dashboard for efficient management and monitoring of investigator-initiated trials

BACKGROUND

Most randomized controlled trials (RCTs) in the academic setting have limited resources for clinical trial management and monitoring. Inefficient conduct of trials was identified as an important source of waste even in well-designed studies. Thoroughly identifying trial-specific risks to enable focussing of monitoring and management efforts on these critical areas during trial conduct may allow for the timely initiation of corrective action and to improve the efficiency of trial conduct. We developed a risk-tailored approach with an initial risk assessment of an individual trial that informs the compilation of monitoring and management procedures in a trial dashboard.

METHODS

We performed a literature review to identify risk indicators and trial monitoring approaches followed by a contextual analysis involving local, national and international stakeholders. Based on this work we developed a risk-tailored management approach with integrated monitoring for RCTs and including a visualizing trial dashboard. We piloted the approach and refined it in an iterative process based on feedback from stakeholders and performed formal user testing with investigators and staff of two clinical trials.

RESULTS

The developed risk assessment comprises four domains (patient safety and rights, overall trial management, intervention management, trial data). An accompanying manual provides rationales and detailed instructions for the risk assessment. We programmed two trial dashboards tailored to one medical and one surgical RCT to manage identified trial risks based on daily exports of accumulating trial data. We made the code for a generic dashboard available on GitHub that can be adapted to individual trials.

CONCLUSIONS

The presented trial management approach with integrated monitoring enables user-friendly, continuous checking of critical elements of trial conduct to support trial teams in the academic setting. Further work is needed in order to show effectiveness of the dashboard in terms of safe trial conduct and successful completion of clinical trials.

Concept and development of an interactive tool for trial recruitment planning and management

Background

Predicting and monitoring recruitment in large, complex trials is essential to ensure appropriate resource management and budgeting. In a novel partnership between clinical trial investigators of the South African Medical Research Council and industrial engineers from the Stellenbosch University Health Systems Engineering and Innovation Hub, we developed a trial recruitment tool (TRT). The objective of the tool is to serve as a computerised decisions-support system to aid the planning and management phases of the trial recruitment process.

Method

The specific requirements of the TRT were determined in several workshops between the partners. A Poisson process simulation model was formulated and incorporated in the TRT to predict the recruitment duration. The assumptions underlying the model were made in consultation with the trial team at the start of the project and were deemed reasonable. Real-world data extracted from a current cluster trial, Project MIND, based in 24 sites in South Africa was used to verify the simulation model and to develop the monitoring component of the TRT.

Results

The TRT comprises a planning and monitoring component. The planning component generates different trial scenarios for predicted trial recruitment duration based on user inputs, e.g. number of sites, initiation delays. The monitoring component uses and analyses the data retrieved from the trial management information system to generate different levels of information, displayed visually on an interactive, user-friendly dashboard. Users can analyse the results at trial or site level, changing input parameters to see the resultant effect on the duration of trial recruitment.

Conclusion

This TRT is an easy-to-use tool that assists in the management of the trial recruitment process. The TRT has potential to expedite improved management of clinical trials by providing the appropriate information needed for the planning and monitoring of the trial recruitment phase. This TRT extends prior tools describing historic recruitment only to using historic data to predict future recruitment. The broader project demonstrates the value of collaboration between clinicians and engineers to optimise their respective skillsets.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13063-021-05112-z.

Utilizing open-source platforms to build and deploy interactive patient-reported quality of life tracking tools for monitoring protocol adherence

PURPOSE

Tracking patient-reported outcomes (PROs) and quality-of-life response rates is essential for clinical trials. Historically, rates are monitored through scheduled reports, which can require gathering, merging, and cleaning data from multiple databases. At the end of this process, if gaps are found, new data are entered and the cycle repeats, leaving a trail of reports that are not up-to-date or immediately accessible to the investigator. The financial and person-hour cost of utilizing clinical research staff for this purpose is impractical. Online dashboards are continuously updated to monitor data, providing on-demand access to promote successful research.

METHODS

Dashboard implementation utilizes R, an open-source statistical programming language, RMarkdown, a markup language, Flexdashboard, which creates structural elements, and Shiny, allowing investigators the ability to interact with data within the dashboard. By leveraging these four elements, powerful, cost-effective interactive dashboards can be built.

RESULTS

Numerous dashboards have been utilized to identify potentially missing data and increase protocol adherence. Immediate patient consultation can occur to retrieve protocol-related forms, reducing research staff and patient burden while improving trial effectiveness. Dashboards can monitor PROs, enrollment, demographics, toxicity, and biomarker data, clinical outcomes, and implemented predictive models, creating a single hub for on-demand clinical trial monitoring.

CONCLUSION

By employing a set of freely available tools, the burden of utilizing study staff to continuously monitor trials is greatly reduced. These tools allow users to rapidly build and deploy dynamic dashboards capable of meeting the research needs of any investigator while limiting missing data through simplified monitoring of protocol adherence.

Statistical models to predict recruitment in clinical trials were rarely used by statisticians in UK and European networks

OBJECTIVE

Identify the current practice for recruitment prediction and monitoring within clinical trials.

STUDY DESIGN AND SETTING

Chief investigators (CIs) were surveyed to identify data sources and adjustments made to support recruitment prediction. Statisticians were surveyed to determine methods and adjustments used when predicting and monitoring recruitment. Participants were identified from the National Institute for Health Research recently funded studies, the UK Clinical Research Collaboration registered Clinical Trial Units network or by the European Clinical Research Infrastructure Network.

RESULTS

A total of 51 CIs (UK = 32, ECRIN = 19) and 104 statisticians (UK = 51, ECRIN = 53) were contacted. Response rates varied (CIs UK = 53% ECRIN = 32%; statisticians UK = 98% ECRIN = 36%). Multiple data sources are used to support recruitment rates, most commonly audit data from multiple sites. Variation in individual site recruitment rates are frequently incorporated, but staggered site openings were featured more commonly among UK respondents. Simple prediction methods are preferred to rarely used statistical models. Lack of familiarity with statistical methods are barriers to their use with evidence needed to justify the time required to support their implementation.

CONCLUSION

Simplistic methods will continue as the mainstay of prediction; however, generation of evidence supporting the benefits of complex statistical models should promote their implementations. Multiple data sources to support recruitment prediction are being used, and further work on the quality of these data is needed. Pressure to be optimistic about recruitment rates for the trial to be attractive to funders was felt by a sizable minority.

Early warnings and repayment plans: novel trial management methods for monitoring and managing data return rates in a multi-centre phase III randomised controlled trial with paper Case Report Forms

Background

Monitoring and managing data returns in multi-centre randomised controlled trials is an important aspect of trial management. Maintaining consistently high data return rates has various benefits for trials, including enhancing oversight, improving reliability of central monitoring techniques and helping prepare for database lock and trial analyses. Despite this, there is little evidence to support best practice, and current standard methods may not be optimal.

Methods

We report novel methods from the Trial of Imaging and Schedule in Seminoma Testis (TRISST), a UK-based, multi-centre, phase III trial using paper Case Report Forms to collect data over a 6-year follow-up period for 669 patients. Using an automated database report which summarises the data return rate overall and per centre, we developed a Microsoft Excel-based tool to allow observation of per-centre trends in data return rate over time. The tool allowed us to distinguish between forms that can and cannot be completed retrospectively, to inform understanding of issues at individual centres. We reviewed these statistics at regular trials unit team meetings. We notified centres whose data return rate appeared to be falling, even if they had not yet crossed the pre-defined acceptability threshold of an 80% data return rate. We developed a set method for agreeing targets for gradual improvement with centres having persistent data return problems. We formalised a detailed escalation policy to manage centres who failed to meet agreed targets. We conducted a post-hoc, descriptive analysis of the effectiveness of the new processes.

Results

The new processes were used from April 2015 to September 2016. By May 2016, data return rates were higher than they had been at any time previously, and there were no centres with return rates below 80%, which had never been the case before. In total, 10 centres out of 35 were contacted regarding falling data return rates. Six out of these 10 showed improved rates within 6–8 weeks, and the remainder within 4 months.

Conclusions

Our results constitute preliminary effectiveness evidence for novel methods in monitoring and managing data return rates in randomised controlled trials. We encourage other researchers to work on generating better evidence-based methods in this area, whether through more robust evaluation of our methods or of others.

Assessing effects of behavioral intervention on treatment outcomes among patients initiating HIV care: Rationale and design of iENGAGE intervention trial

During the initial year of HIV diagnosis, while patients are often overwhelmed adjusting to this life changing diagnosis, they must develop self-care behaviors for attending regular medical care visits and antiretroviral therapy (ART) adherence to achieve and sustain viral suppression (VS). Maintaining "HIV adherence" and integrating it into one's daily life is required to sustain VS over time. The HIV care continuum or "treatment cascade," an epidemiological snapshot of the national epidemic in the United States (US), indicates that a minority of persons living with HIV (PLWH) have achieved VS. Little evidence exists regarding the effects of interventions focusing on PLWH newly initiating outpatient HIV care. An intervention that focuses on both retention in care and ART adherence skills delivered during the pivotal first year of HIV care is lacking. To address this, we developed a theory-based intervention evaluated in the Integrating Engagement and Adherence Goals upon Entry (iENGAGE) study, a National Institute of Allergy and Infectious Diseases (NIAID) funded randomized behavioral intervention trial. Here we present the study objectives, design and rationale, as well as the intervention components, targeting rapid and sustained VS through retention in HIV care and ART adherence during participants' first year of HIV care. The primary outcome of the study is 48-week VS (<200 c/mL). The secondary outcomes are retention in care, including HIV visit adherence and visit constancy, as well as ART adherence.

A Web-Based Data Collection Platform for Multisite Randomized Behavioral Intervention Trials: Development, Key Software Features, and Results of a User Survey

Background

Meticulous tracking of study data must begin early in the study recruitment phase and must account for regulatory compliance, minimize missing data, and provide high information integrity and/or reduction of errors. In behavioral intervention trials, participants typically complete several study procedures at different time points. Among HIV-infected patients, behavioral interventions can favorably affect health outcomes. In order to empower newly diagnosed HIV positive individuals to learn skills to enhance retention in HIV care, we developed the behavioral health intervention Integrating ENGagement and Adherence Goals upon Entry (iENGAGE) funded by the National Institute of Allergy and Infectious Diseases (NIAID), where we deployed an in-clinic behavioral health intervention in 4 urban HIV outpatient clinics in the United States. To scale our intervention strategy homogenously across sites, we developed software that would function as a behavioral sciences research platform.

Objective

This manuscript aimed to: (1) describe the design and implementation of a Web-based software application to facilitate deployment of a multisite behavioral science intervention; and (2) report on results of a survey to capture end-user perspectives of the impact of this platform on the conduct of a behavioral intervention trial.

Methods

In order to support the implementation of the NIAID-funded trial iENGAGE, we developed software to deploy a 4-site behavioral intervention for new clinic patients with HIV/AIDS. We integrated the study coordinator into the informatics team to participate in the software development process. Here, we report the key software features and the results of the 25-item survey to evaluate user perspectives on research and intervention activities specific to the iENGAGE trial (N=13).

Results

The key features addressed are study enrollment, participant randomization, real-time data collection, facilitation of longitudinal workflow, reporting, and reusability. We found 100% user agreement (13/13) that participation in the database design and/or testing phase made it easier to understand user roles and responsibilities and recommended participation of research teams in developing databases for future studies. Users acknowledged ease of use, color flags, longitudinal work flow, and data storage in one location as the most useful features of the software platform and issues related to saving participant forms, security restrictions, and worklist layout as least useful features.

Conclusions

The successful development of the iENGAGE behavioral science research platform validated an approach of early and continuous involvement of the study team in design development. In addition, we recommend post-hoc collection of data from users as this led to important insights on how to enhance future software and inform standard clinical practices.

Trial Registration

Clinicaltrials.gov NCT01900236; (https://clinicaltrials.gov/ct2/show/NCT01900236 (Archived by WebCite at http://www.webcitation.org/6qAa8ld7v)