Generating evidence on a risk-based monitoring approach in the academic setting - lessons learned

BACKGROUND

In spite of efforts to employ risk-based strategies to increase monitoring efficiency in the academic setting, empirical evidence on their effectiveness remains sparse. This mixed-methods study aimed to evaluate the risk-based on-site monitoring approach currently followed at our academic institution.

METHODS

We selected all studies monitored by the Clinical Trial Unit (CTU) according to Risk ADApted MONitoring (ADAMON) at the University Hospital Basel, Switzerland, between 01.01.2012 and 31.12.2014. We extracted study characteristics and monitoring information from the CTU Enterprise Resource Management system and from monitoring reports of all selected studies. We summarized the data descriptively. Additionally, we conducted semi-structured interviews with the three current CTU monitors.

RESULTS

During the observation period, a total of 214 monitoring visits were conducted in 43 studies resulting in 2961 documented monitoring findings. Our risk-based approach predominantly identified administrative (46.2%) and patient right findings (49.1%). We identified observational study design, high ADAMON risk category, industry sponsorship, the presence of an electronic database, experienced site staff, and inclusion of vulnerable study population to be factors associated with lower numbers of findings. The monitors understand the positive aspects of a risk-based approach but fear missing systematic errors due to the low frequency of visits.

CONCLUSIONS

We show that the factors mostly increasing the risk for on-site monitoring findings are underrepresented in the current risk analysis scheme. Our risk-based on-site approach should further be complemented by centralized data checks, allowing monitors to transform their role towards partners for overall trial quality, and success.

A systematic review of on-site monitoring methods for health-care randomised controlled trials

BACKGROUND

Monitoring the conduct of clinical trials is recommended by International Conference of Harmonisation Good Clinical Practice (ICH GCP) guidelines and is integral to trial quality assurance. On-site monitoring, that is, visiting trial sites, is one part of this process but little is known about the procedures that are performed in practice.

PURPOSE

To examine and summarise published on-site monitoring methods for health-care clinical trials, including evaluations of their benefits and costs to trials.

METHODS

A systematic literature review identified all articles reporting the methods and practices of on-site monitoring of randomised controlled trials (RCTs). Articles were categorised into (1) reports from research groups and organisations, (2) reports from individual RCTs, (3) randomised trials of on-site monitoring interventions, (4) cost simulations, or (5) surveys of trial staff and monitors. Data were extracted on the characteristics of the trials and groups reporting on-site monitoring (e.g., geographical origin, sponsor, and trial focus). Information from articles in categories (1)-(3) was summarised on the frequency and scope of site monitoring visits, monitoring team size and composition, activities during site visits, and reporting structures. Evaluations of the benefits and disadvantages of on-site monitoring were examined for all included articles.

RESULTS

In total, 57 articles were identified, comprising 21 articles about the on-site monitoring practices of 16 research groups, 30 articles from 26 RCTs, 1 on-site monitoring intervention RCT, 2 cost simulations, and 3 surveys. Publications in categories (1)-(3), mostly originated from the United States (33/52, 63%) or Europe (15/52, 29%), were predominantly describing non-commercial organisations or trials (45/52, 87%), with heart disease (9/26, 35%) or cancer (5/26, 19%) the commonest focus of individual RCTs. The frequency of visits ranged from every 6-8 weeks up to once every 3 years, with mostly all trial sites visited. The number of monitors visiting a site varied between 1 and 8. The most common on-site monitoring activity was verifying source data and consent forms, with a focus on data accuracy. Only six articles evaluated their on-site monitoring process, with improvements observed in recruitment rates and protocol adherence but with direct costs and staff time viewed as the major disadvantages. The on-site monitoring RCT ended prematurely so preventing full assessment.

LIMITATIONS

Trialists and organisations may utilise additional unpublished on-site monitoring systems. The varied terminology used to describe monitoring may have limited identification of some relevant articles.

CONCLUSIONS

This review demonstrated that on-site monitoring is utilised in trials worldwide but systems vary considerably with little evidence to support practice. These on-site monitoring practices need to be evaluated empirically, including costs, to provide robust evidence for the contribution of site visits to trial performance and quality.

Gynecologic oncology group strategies to improve timeliness of publication

BACKGROUND

The Gynecologic Oncology Group (GOG) is a multi-institution cooperative group funded by the National Cancer Institute to conduct clinical trials encompassing clinical and basic scientific research in gynecologic malignancies. These results are disseminated via publication in peer-reviewed journals. This process requires collaboration of numerous investigators located in diverse cancer research centers. Coordination of manuscript development is positioned within the Statistical and Data Center (SDC), thus allowing the SDC personnel to manage the process and refine strategies to promote earlier dissemination of results. A major initiative to improve timeliness utilizing the assignment, monitoring, and enforcement of deadlines for each phase of manuscript development is the focus of this investigation.

PURPOSE

Document improvement in timeliness via comparison of deadline compliance and time to journal submission due to expanded administrative and technologic initiatives implemented in 2006.

METHODS

Major steps in the publication process include generation of first draft by the First Author and submission to SDC, Co-author review, editorial review by Publications Subcommittee, response to journal critique, and revision. Associated with each step are responsibilities of First Author to write or revise, collaborating Biostatistician to perform analysis and interpretation, and assigned SDC Clinical Trials Editorial Associate to format/revise according to journal requirements. Upon the initiation of each step, a deadline for completion is assigned. In order to improve efficiency, a publications database was developed to track potential steps in manuscript development that enables the SDC Director of Administration and the Publications Subcommittee Chair to assign, monitor, and enforce deadlines. They, in turn, report progress to Group Leadership through the Operations Committee. The success of the strategies utilized to improve the GOG publication process was assessed by comparing the timeliness of each potential step in the development of primary Phase II manuscripts during 2003-2006 versus 2007-2010.

RESULTS

Improvement was noted in 10 of 11 identified steps resulting in a cumulative average improvement of 240 days from notification of data maturity to First Author through first submission to a journal. Moreover, the average time to journal acceptance has improved by an average of 346 days.

LIMITATIONS

The investigation is based on only Phase II trials to ensure comparability of manuscript complexity. Nonetheless, the procedures employed are applicable to the development of any clinical trials manuscript.

CONCLUSIONS

The assignment, monitoring, and enforcement of deadlines for all stages of manuscript development have resulted in increased efficiency and timeliness. The positioning and support of manuscript development within the SDC provide a valuable resource to authors in meeting assigned deadlines, accomplishing peer review, and complying with journal requirements.