Harnessing the Power of Quality Assurance Data: Can We Use Statistical Modeling for Quality Risk Assessment of Clinical Trials?

Assessment of the Relationship Between Protocol Adherence, Study Complexity and Personnel in Surgical Clinical Trials

BACKGROUND

In order to improve quality and efficiency of surgical trials, we assessed protocols complexity and examined whether it influenced the conduct, as measured by the number and types of deviations that occurred during the execution phase. Knowledge of these facts and performance of research team would allow to effectively mitigate the occurrence of deviations.

METHODS

Thirty-five research protocols were rated according to a previously established complexity scoring model. Statistical analyses were performed to examine associations between protocol complexity, number of protocol/informed consent amendments vs. number/types of protocol deviations; as well as correlations with phase of the study, type of investigational product, personnel changes/experience level were assessed.

RESULTS

Assessment of complexity score in Pearson's correlation test with the number of protocol deviations showed weak correlation, suggesting that other factors can influence protocol adherence. There was no correlation observed between number of deviations and type of study by investigational product category. In examining association between protocol deviations and number of subjects enrolled a trend was observed towards increased number of deviations once more subjects have entered the study. The higher number of protocol deviations was associated with increased number of protocol amendments (p = 0.0396), and there was no statistical significance observed between number of deviations and informed consent amendments (p = 0.5083). There was a moderate correlation detected between increased number of protocol deviations and total number of investigators on the study.

CONCLUSION

Protocol adherence can be improved with effective training and retention of research coordinators, investigators and frequent internal auditing to address discrepancies and effectively implement corrective actions. Upfront training of research personnel, with subsequent monitoring of performance metrics throughout the execution phase can reduce the total number of protocol deviations, ensure data integrity and improve quality of research conducted. Engagement of all stakeholders upfront, including clinical site personnel, can help develop well-designed clinical trial protocol, avoid time consuming and costly protocol and informed consent amendments at execution phase and ensure higher quality of research conducted, while allowing to meet objectives of the trial in a more efficient manner.

Statistical Modeling for Quality Risk Assessment of Clinical Trials: Follow-Up at the Era of Remote Auditing

BACKGROUND

As investigator site audits have largely been conducted remotely during the COVID-19 pandemic, remote quality monitoring has gained some momentum. To further facilitate the conduct of remote quality assurance (QA) activities for clinical trials, we developed new quality indicators, building on a previously published statistical modeling methodology.

METHODS

We modeled the risk of having an audit or inspection finding using historical audits and inspections data from 2011 to 2019. We used logistic regression to model finding risk for 4 clinical impact factor (CIF) categories: Safety Reporting, Data Integrity, Consent and Protecting Endpoints.

RESULTS

We could identify 15 interpretable factors influencing audit finding risk of 4 out of 5 CIF categories. They can be used to realistically predict differences in risk between 25 and 43% for different sites which suffice to rank sites by audit and inspection finding risk.

CONCLUSION

Continuous surveillance of the identified risk factors and resulting risk estimates could be used to complement remote QA strategies for clinical trials and help to manage audit targets and audit focus also in post-pandemic times.

Clinical Quality Considerations when Using Next-Generation Sequencing (NGS) in Clinical Drug Development

Next-generation sequencing (NGS) and decreased costs of genomic testing are changing the paradigm in precision medicine and continue to fuel innovation. Integration of NGS into clinical drug development has the potential to accelerate clinical trial conduct and ultimately will shape the landscape of clinical care by making it easier to identify patients who would benefit from particular therapy(ies) and to monitor treatment outcomes with less invasive tests. This has led to an increased use of NGS service providers by pharmaceutical sponsors: to screen patients for clinical trials eligibility and for patient stratification, expanded Companion Diagnostic (CDx) development for treatment recommendations and Comprehensive Genomic profiling (CGP). These changes are reshaping the face of clinical quality considerations for precision medicine. Although some clinical quality considerations do exist in Health Authorities (HA) guidances and regulations (e.g., International Conference of Harmonization Good Clinical Practices-GCP), there is currently no holistic GxP-like detailed framework for pharmaceutical sponsors using NGS service providers in clinical trials, or for the development of CDx and CGP. In this research, we identified existing and applicable regulations, guidelines and recommendations that could be translated into clinical quality considerations related to technology, data quality, patients and oversight. We propose these considerations as a basis for pharmaceutical sponsors using NGS service providers in clinical drug development to develop a set of guidelines for NGS clinical quality.

Bayesian Modeling for the Detection of Adverse Events Underreporting in Clinical Trials

Introduction

Safety underreporting is a recurrent issue in clinical trials that can impact patient safety and data integrity. Clinical quality assurance (QA) practices used to detect underreporting rely on on-site audits; however, adverse events (AEs) underreporting remains a recurrent issue. In a recent project, we developed a predictive model that enables oversight of AE reporting for clinical quality program leads (QPLs). However, there were limitations to using solely a machine learning model.

Objective

Our primary objective was to propose a robust method to compute the probability of AE underreporting that could complement our machine learning model. Our model was developed to enhance patients’ safety while reducing the need for on-site and manual QA activities in clinical trials.

Methods

We used a Bayesian hierarchical model to estimate the site reporting rates and assess the risk of underreporting. We designed the model with Project Data Sphere clinical trial data that are public and anonymized.

Results

We built a model that infers the site reporting behavior from patient-level observations and compares them across a study to enable a robust detection of outliers between clinical sites.

Conclusion

The new model will be integrated into the current dashboard designed for clinical QPLs. This approach reduces the need for on-site audits, shifting focus from source data verification to pre-identified, higher risk areas. It will enhance further QA activities for safety reporting from clinical trials and generate quality evidence during pre-approval inspections.

Supplementary Information

The online version contains supplementary material available at 10.1007/s40264-021-01094-8.

Cross-company collaboration to leverage analytics for clinical quality and accelerate drug development: The IMPALA industry group

Quality functions from pharmaceutical sponsor companies aim to increase the use of analytics in their oversight of Good Clinical Practices and Pharmacovigilance activities. To leverage and accelerate progress, several companies decided to establish a collaborative model. The goals of this collaboration span the sharing of knowledge and ideas, the sharing of analytics methods, discussion of talent upskilling and technology adoption strategies, and collaborative discussion on these potential changes with global Health Authorities.

Leveraging analytics to assure quality during the Covid-19 pandemic - The COVACTA clinical study example

The world has seen a shift in the ways of working during the Covid-19 pandemic. Routine activities performed at the clinical investigator sites (e.g. on-site audits) that are a part of Quality Assurance (QA) have not been feasible at this time. Analytics has played a huge role in contributing to our continued efforts of ensuring quality during the conduct of a clinical trial. Decisions driven through data, now more than ever, heavily contribute to the efficiency of QA activities. In this report, we share the approach we took to conduct QA activities for the COVACTA study (to treat Covid-19 pneumonia) by leveraging analytics.

Using Statistical Modeling for Enhanced and Flexible Pharmacovigilance Audit Risk Assessment and Planning

Background

The European Medicines Agency Good Pharmacovigilance Practices (GVP) guidelines provide a framework for pharmacovigilance (PV) audits, including limited guidance on risk assessment methods. Quality assurance (QA) teams of large and medium sized pharmaceutical companies generally conduct annual risk assessments of the PV system, based on retrospective review of data and pre-defined impact factors to plan for PV audits which require a high volume of manual work and resources. In addition, for companies of this size, auditing the entire “universe” of individual entities on an annual basis is generally prohibitive due to sheer volume. A risk assessment approach that enables efficient, temporal, and targeted PV audits is not currently available.

Methods

In this project, we developed a statistical model to enable holistic and efficient risk assessment of certain aspects of the PV system. We used findings from a curated data set from Roche operational and quality assurance PV data, covering a span of over 8 years (2011–2019) and we modeled the risk with a logistic regression on quality PV risk indicators defined as data stream statistics over sliding windows.

Results

We produced a model for each PV impact factor (e.g. 'Compliance to Individual Case Safety Report') for which we had enough features. For PV impact factors where modeling was not feasible, we used descriptive statistics. All the outputs were consolidated and displayed in a QA dashboard built on Spotfire^®.

Conclusion

The model has been deployed as a quality decisioning tool available to Roche Quality professionals. It is used, for example, to inform the decision on which affiliates (i.e. pharmaceutical company commercial entities) undergo audit for PV activities. The model will be continuously monitored and fine-tuned to ensure its reliability.