1
|
Ge L, Wang Z, Liu CC, Childress S, Wildfire J, Wu G. Assessing the performance of methods for central statistical monitoring of a binary or continuous outcome in multi-center trials: A simulation study. Contemp Clin Trials 2024; 143:107580. [PMID: 38796099 DOI: 10.1016/j.cct.2024.107580] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/03/2024] [Revised: 04/29/2024] [Accepted: 05/21/2024] [Indexed: 05/28/2024]
Abstract
BACKGROUND Quality study monitoring is fundamental to patient safety and data integrity. Regulators and industry consortia have increasingly advocated for risk-based monitoring (RBM) and central statistical monitoring (CSM) for more effective and efficient monitoring. Assessing which statistical methods underpin these approaches can best identify unusual data patterns in multi-center clinical trials that may be driven by potential systematic errors is important. METHODS We assessed various CSM techniques, including cross-tests, fixed-effects, mixed-effects, and finite mixture models, across scenarios with different sample sizes, contamination rates, and overdispersion via simulation. Our evaluation utilized threshold-independent metrics such as the area under the curve (AUC) and average precision (AP), offering a fuller picture of CSM performance. RESULTS All CSM methods showed consistent characteristics across center sizes or overdispersion. The adaptive finite mixture model outperformed others in AUC and AP, especially at 30% contamination, upholding high specificity unless converging to a single-component model due to low contamination or deviation. The mixed-effects model performed well at lower contamination rates. However, it became conservative in specificity and exhibited declined performance for binary outcomes under high deviation. Cross-tests and fixed-effects methods underperformed, especially when deviation increased. CONCLUSION Our evaluation explored the merits and drawbacks of multiple CSM methods, and found that relying on sensitivity and specificity alone is likely insufficient to fully measure predictive performance. The finite mixture method demonstrated more consistent performance across scenarios by mitigating the influence of outliers. In practice, considering the study-specific costs of false positives/negatives with available resources for monitoring is important.
Collapse
Affiliation(s)
- Li Ge
- Gilead Sciences, Foster City 94404, CA, USA; Department of Biostatistics and Medical Informatics, University of Wisconsin-Madison, Madison 53703, WI, USA
| | | | | | | | | | - George Wu
- Gilead Sciences, Foster City 94404, CA, USA.
| |
Collapse
|
2
|
Iflaifel M, Hall CL, Green HR, Willis A, Rennick-Egglestone S, Juszczak E, Townsend M, Martin J, Sprange K. Widening participation - recruitment methods in mental health randomised controlled trials: a qualitative study. BMC Med Res Methodol 2023; 23:211. [PMID: 37735627 PMCID: PMC10512591 DOI: 10.1186/s12874-023-02032-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/03/2023] [Accepted: 09/08/2023] [Indexed: 09/23/2023] Open
Abstract
BACKGROUND Barriers to mental health research participation are well documented including distrust of services and research; and stigma surrounding mental health. They can contribute to a lack of diversity amongst participants in mental health research, which threatens the generalisability of knowledge. Given the recent widespread use of the internet in medical research, this study aimed to explore the perspectives of key partners on the use of online (e.g. social media) and offline (e.g. in-person) recruitment as an approach to improving diversity in mental health randomised controlled trials (RCTs). METHODS Face-to-face and online interviews/focus groups with researchers working in mental health and Patient and Public Involvement partners in the United Kingdom. Recordings were transcribed and analysed using a combination of inductive and deductive thematic analysis. RESULTS Three focus groups and three interviews were conducted with a total N = 23 participants. Four overarching themes were identified: (1) recruitment reach; (2) Demographic factors that affect selection of recruitment method; (3) safety of technology, and; (4) practical challenges. Five main factors were identified that affect the choice of recruitment method: age, complexity of mental health problem and stigma, cultural and ethnicity differences and digital divide. The use of online methods was considered more accessible to people who may feel stigmatised by their mental health condition and with a benefit of reaching a wider population. However, a common view amongst participants was that online methods require closer data monitoring for quality of responders, are not fully secure and less trustworthy compared to offline methods that enable participants to build relationships with health providers. Funding, staff time and experience, organisational support, and technical issues such as spam or phishing emails were highlighted as practical challenges facing online recruitment. All participants agreed that using a hybrid approach tailored to the population under study is paramount. CONCLUSIONS This study highlighted the importance of offering a flexible and multifaceted recruitment approach by integrating online with offline methods to support inclusivity and widening participation in mental health research. The findings will be used to develop considerations for researchers designing RCTs to improve recruitment in mental health research.
Collapse
Affiliation(s)
- Mais Iflaifel
- Nottingham Clinical Trials Unit, University of Nottingham, Nottingham, UK
| | - Charlotte L Hall
- NIHR MindTech MedTech Co-operative, Institute of Mental Health, School of Medicine, University of Nottingham, Innovation Park, Triumph Road, Nottingham, UK
- NIHR Nottingham Biomedical Research Centre, Institute of Mental Health, University of Nottingham, Innovation Park, Triumph Road, Nottingham, UK
| | - Heidi R Green
- Previously: Health Services Research Unit, University of Aberdeen, Aberdeen, UK
- COUCH Health, Manchester, UK
| | - Andrew Willis
- Centre for Ethnic Health Research, Leicester/Diabetes Research Centre, University of Leicester, Leicester, UK
| | | | - Edmund Juszczak
- Nottingham Clinical Trials Unit, University of Nottingham, Nottingham, UK
| | - Mark Townsend
- NIHR Evaluation, Trials and Studies Coordinating Centre (NETSCC), Southampton, UK
| | - Jennifer Martin
- NIHR MindTech MedTech Co-operative, Institute of Mental Health, School of Medicine, University of Nottingham, Innovation Park, Triumph Road, Nottingham, UK
- NIHR Nottingham Biomedical Research Centre, Institute of Mental Health, University of Nottingham, Innovation Park, Triumph Road, Nottingham, UK
| | - Kirsty Sprange
- Nottingham Clinical Trials Unit, University of Nottingham, Nottingham, UK.
| |
Collapse
|
3
|
Nourani A, Ayatollahi H, Solaymani Dodaran M. Data management in diabetes clinical trials: a qualitative study. Trials 2022; 23:187. [PMID: 35241149 PMCID: PMC8895796 DOI: 10.1186/s13063-022-06110-5] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/29/2021] [Accepted: 02/15/2022] [Indexed: 11/16/2022] Open
Abstract
Background Clinical trials play an important role in expanding the knowledge of diabetes prevention, diagnosis, and treatment, and data management is one of the main issues in clinical trials. Lack of appropriate planning for data management in clinical trials may negatively influence achieving the desired results. The aim of this study was to explore data management processes in diabetes clinical trials in three research institutes in Iran. Method This was a qualitative study conducted in 2019. In this study, data were collected through in-depth semi-structured interviews with 16 researchers in three endocrinology and metabolism research institutes. To analyze data, the method of thematic analysis was used. Results The five themes that emerged from data analysis included (1) clinical trial data collection, (2) technologies used in data management, (3) data security and confidentiality management, (4) data quality management, and (5) data management standards. In general, the findings indicated that no clear and standard process was used for data management in diabetes clinical trials, and each research center executed its own methods and processes. Conclusion According to the results, the common methods of data management in diabetes clinical trials included a set of paper-based processes. It seems that using information technology can help facilitate data management processes in a variety of clinical trials, including diabetes clinical trials.
Collapse
Affiliation(s)
- Aynaz Nourani
- Department of Health Information Technology, Urmia University of Medical Sciences, Urmia, Iran
| | - Haleh Ayatollahi
- Health Management and Economics Research Center, Health Management Research Institute, Iran University of Medical Sciences, Tehran, Iran. .,Department of Health Information Management, School of Health Management and Information Sciences, Iran University of Medical Sciences, Tehran, Iran.
| | | |
Collapse
|
4
|
Bybee S, Cloyes K, Ellington L, Baucom B, Supiano K, Mooney K. Bots and nots: Safeguarding online survey research with underrepresented and diverse populations. PSYCHOLOGY & SEXUALITY 2022; 13:901-911. [PMID: 36439051 PMCID: PMC9697945 DOI: 10.1080/19419899.2021.1936617] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/03/2023]
Abstract
The COVID-19 pandemic has intensified the use of online recruitment and data collection for reaching historically underrepresented minorities (URMs) and other diverse groups. Preventing and detecting responses from automated accounts "bots" and those who misrepresent themselves is one challenge in utilizing online approaches. Through internet-mediated methods, interested LGBTQ+ and non-LGBTQ+ couples facing advanced cancer completed an interest form via REDCap®. Eligible participants received a direct link to electronic consent and surveys in REDCap®. Once responses to the interest form (N = 619) were received, the study PI: 1) assessed participants' entries and non-response survey data (time of completion, rate of recruitment, etc.), 2) temporarily postponed recruitment, 3) sent eligibility questionnaires, consent documents, and validated surveys to N= 10 couples and scrutinized these data for suspicious patterns or indications of untrustworthy data, 4) responded to potential participants via email, and 5) implemented additional strategies for detecting and preventing untrustworthy survey responses. Investigators must consider multi-step eligibility screening processes to detect and prevent the collection of untrustworthy data. Investigators' reliance on internet-mediated approaches for conducting research with diverse, hard-to-reach populations increases the importance of addressing threats to data validity. Ultimately, safeguarding internet-mediated research supports research accessibility and inclusion for URMs while also protecting participant data integrity.
Collapse
Affiliation(s)
- Sara Bybee
- University of Utah, College of Nursing, 10 S 2000 E, Salt Lake City, UT, 84112
| | | | | | | | | | | |
Collapse
|
5
|
Bradshaw MS, Payne SH. Detecting fabrication in large-scale molecular omics data. PLoS One 2021; 16:e0260395. [PMID: 34847169 PMCID: PMC8631639 DOI: 10.1371/journal.pone.0260395] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/18/2020] [Accepted: 11/09/2021] [Indexed: 01/22/2023] Open
Abstract
Fraud is a pervasive problem and can occur as fabrication, falsification, plagiarism, or theft. The scientific community is not exempt from this universal problem and several studies have recently been caught manipulating or fabricating data. Current measures to prevent and deter scientific misconduct come in the form of the peer-review process and on-site clinical trial auditors. As recent advances in high-throughput omics technologies have moved biology into the realm of big-data, fraud detection methods must be updated for sophisticated computational fraud. In the financial sector, machine learning and digit-frequencies are successfully used to detect fraud. Drawing from these sources, we develop methods of fabrication detection in biomedical research and show that machine learning can be used to detect fraud in large-scale omic experiments. Using the gene copy-number data as input, machine learning models correctly predicted fraud with 58-100% accuracy. With digit frequency as input features, the models detected fraud with 82%-100% accuracy. All of the data and analysis scripts used in this project are available at https://github.com/MSBradshaw/FakeData.
Collapse
Affiliation(s)
- Michael S. Bradshaw
- Computer Science Department, University of Colorado Boulder, Boulder, Colorado, United States of America
| | - Samuel H. Payne
- Biology Department, Brigham Young University, Provo, Utah, United States of America
| |
Collapse
|
6
|
Houston L, Martin A, Yu P, Probst Y. Time-consuming and expensive data quality monitoring procedures persist in clinical trials: A national survey. Contemp Clin Trials 2021; 103:106290. [PMID: 33503495 DOI: 10.1016/j.cct.2021.106290] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/19/2020] [Revised: 01/19/2021] [Accepted: 01/20/2021] [Indexed: 12/21/2022]
Abstract
INTRODUCTION The Good Clinical Practice guideline identifies that data monitoring is an essential research activity. However, limited evidence exists on how to perform monitoring including the amount or frequency that is needed to ensure data quality. This study aims to explore the monitoring procedures that are implemented to ensure data quality in Australian clinical research studies. MATERIAL AND METHODS Clinical studies listed on the Australian and New Zealand Clinical Trials Registry were invited to participate in a national cross-sectional, mixed-mode, multi-contact (postal letter and e-mail) web-based survey. Information was gathered about the types of data quality monitoring procedures being implemented. RESULTS Of the 3689 clinical studies contacted, 589 (16.0%) responded, of which 441 (77.4%) completed the survey. Over half (55%) of the studies applied source data verification (SDV) compared to risk-based targeted and triggered monitoring (10-11%). Conducting 100% on-site monitoring was most common for those who implemented the traditional approach. Respondents who did not conduct 100% monitoring, included 1-25% of data points for SDV, centralized or on-site monitoring. The incidence of adverse events and protocol deviations were the most likely factors to trigger a site visit for risk-based triggered (63% and 44%) and centralized monitoring (48% and 44%), respectively. CONCLUSION Instead of using more optimal risk-based approaches, small single-site clinical studies are conducting traditional monitoring procedures which are time consuming and expensive. Formal guidelines need to be improved and provided to all researchers for 'new' risk-based monitoring approaches.
Collapse
Affiliation(s)
- Lauren Houston
- School of Medicine, University of Wollongong, Australia; Illawarra Health and Medical Research Institute, Australia.
| | | | - Ping Yu
- Illawarra Health and Medical Research Institute, Australia; School of Computing and Information Technology, University of Wollongong, Australia
| | - Yasmine Probst
- School of Medicine, University of Wollongong, Australia; Illawarra Health and Medical Research Institute, Australia
| |
Collapse
|
7
|
Pradhan R, Singh S. Comparison of Data on Serious Adverse Events and Mortality in ClinicalTrials.gov, Corresponding Journal Articles, and FDA Medical Reviews: Cross-Sectional Analysis. Drug Saf 2018; 41:849-857. [PMID: 29644579 DOI: 10.1007/s40264-018-0666-y] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/25/2022]
Abstract
INTRODUCTION Inconsistencies in data on serious adverse events (SAEs) and mortality in ClinicalTrials.gov and corresponding journal articles pose a challenge to research transparency. OBJECTIVE The objective of this study was to compare data on SAEs and mortality from clinical trials reported in ClinicalTrials.gov and corresponding journal articles with US Food and Drug Administration (FDA) medical reviews. METHODS We conducted a cross-sectional study of a randomly selected sample of new molecular entities approved during the study period 1 January 2013 to 31 December 2015. We extracted data on SAEs and mortality from 15 pivotal trials from ClinicalTrials.gov and corresponding journal articles (the two index resources), and FDA medical reviews (reference standard). We estimated the magnitude of deviations in rates of SAEs and mortality between the index resources and the reference standard. RESULTS We found deviations in rates of SAEs (30% in ClinicalTrials.gov and 30% in corresponding journal articles) and mortality (72% in ClinicalTrials.gov and 53% in corresponding journal articles) when compared with the reference standard. The intra-class correlation coefficient between the three resources was 0.99 (95% confidence interval [CI] 0.98-0.99) for SAE rates and 0.99 (95% CI 0.97-0.99) for mortality rates. CONCLUSION There are differences in data on rates of SAEs and mortality in randomized clinical trials in both ClinicalTrials.gov and journal articles compared with FDA reviews. Further efforts should focus on decreasing existing discrepancies to enhance the transparency and reproducibility of data reporting in clinical trials.
Collapse
Affiliation(s)
- Richeek Pradhan
- Department of Quantitative Health Sciences, University of Massachusetts Medical School, 55 Lake Ave North, Worcester, MA, 01655-0002, USA
| | - Sonal Singh
- Department of Family Medicine and Community Health, Meyers Primary Care Institute, University of Massachusetts Medical School, 55 Lake Ave North, Worcester, MA, 01655-0002, USA.
| |
Collapse
|
8
|
Sudo T, Sato A. Investigation of the Factors Affecting Risk-Based Quality Management of Investigator-Initiated Investigational New-Drug Trials for Unapproved Anticancer Drugs in Japan. Ther Innov Regul Sci 2018; 51:589-596. [PMID: 30231689 DOI: 10.1177/2168479017705155] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
BACKGROUND With an increase in the complexity and cost of clinical trials and the advances in information technology, monitoring guidance issued by regulatory authorities recommends risk-adapted monitoring. To introduce the monitoring method for investigator-initiated investigational new drug (IND) trials using unapproved anticancer drugs, we performed exploratory retrospective analyses to identify risk factors for data quality. METHODS To select investigator-initiated IND trials using unapproved anticancer drugs, we set the trial selection criteria. Data collection was performed by using audit trails and monitoring reports. Collected data were analyzed by univariate and multivariate analyses to identify the independent risk factors related to error. RESULTS By trial selection criteria, 5 investigator-initiated IND trials using unapproved anticancer drugs were selected. The error rates of the total data, critical data, and noncritical data were 7.4%, 9.7%, and 5.9%, respectively. There was no difference between clinical research core hospitals certified by the Ministry of Health, Labour and Welfare and other hospitals in univariate analysis (odds ratio [OR], 1.00; 99% confidence interval [CI], 0.96-1.05; P = .9179). As the main independent risk factors related to error, critical data in the importance of data (OR, 1.28; 99% CI, 1.24-1.33; P < .0001) and groups with ≤3 patients after registration (OR, 1.12; 99% CI, 1.10-1.15; P < .0001) were significantly related to errors in multivariate analysis. CONCLUSIONS The results of this research suggest that the feasibility of risk-based monitoring and sampling source data verification was indicated for noncritical data and patients after the third case.
Collapse
Affiliation(s)
- Tomohisa Sudo
- 1 Division of Clinical Research Support, National Cancer Center Hospital East, Chiba, Japan.,2 Medical Science Program, Graduate School of Medicine, Keio University, Tokyo, Japan
| | - Akihiro Sato
- 1 Division of Clinical Research Support, National Cancer Center Hospital East, Chiba, Japan
| |
Collapse
|
9
|
McNamara C, Engelhardt N, Potter W, Yavorsky C, Masotti M, Di Clemente G. Risk-Based Data Monitoring: Quality Control in Central Nervous System (CNS) Clinical Trials. Ther Innov Regul Sci 2018; 53:176-182. [PMID: 29758992 DOI: 10.1177/2168479018774325] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
Monitoring the quality of clinical trial efficacy outcome data has received increased attention in the past decade, with regulatory guidance encouraging it to be conducted proactively, and remotely. However, the methods utilized to develop and implement risk-based data monitoring (RBDM) programs vary, and there is a dearth of published material to guide these processes in the context of central nervous system (CNS) trials. We reviewed regulatory guidance published within the past 6 years, generic white papers, and studies applying RBDM to data from CNS clinical trials. Methodologic considerations and system requirements necessary to establish an effective, real-time risk-based monitoring platform in CNS trials are presented. Key RBDM terms are defined in the context of CNS trial data, such as "critical data," "risk indicators," "noninformative data," and "mitigation of risk." Additionally, potential benefits of, and challenges associated with implementation of data quality monitoring are highlighted. Application of methodological and system requirement considerations to real-time monitoring of clinical ratings in CNS trials has the potential to minimize risk and enhance the quality of clinical trial data.
Collapse
Affiliation(s)
- Cynthia McNamara
- 1 Cronos Clinical Consulting Services, Inc, Lambertville, NJ, USA
| | - Nina Engelhardt
- 1 Cronos Clinical Consulting Services, Inc, Lambertville, NJ, USA
| | - William Potter
- 2 National Institute of Mental Health, Bethesda, MD, USA
| | | | - Matthew Masotti
- 1 Cronos Clinical Consulting Services, Inc, Lambertville, NJ, USA
| | | |
Collapse
|
10
|
Zink RC, Dmitrienko A, Dmitrienko A. Rethinking the Clinically Based Thresholds of TransCelerate BioPharma for Risk-Based Monitoring. Ther Innov Regul Sci 2018; 52:560-571. [PMID: 29714565 DOI: 10.1177/2168479017738981] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
BACKGROUND The quality of data from clinical trials has received a great deal of attention in recent years. Of central importance is the need to protect the well-being of study participants and maintain the integrity of final analysis results. However, traditional approaches to assess data quality have come under increased scrutiny as providing little benefit for the substantial cost. Numerous regulatory guidance documents and industry position papers have described risk-based approaches to identify quality and safety issues. In particular, the position paper of TransCelerate BioPharma recommends defining risk thresholds to assess safety and quality risks based on past clinical experience. This exercise can be extremely time-consuming, and the resulting thresholds may only be relevant to a particular therapeutic area, patient or clinical site population. In addition, predefined thresholds cannot account for safety or quality issues where the underlying rate of observing a particular problem may change over the course of a clinical trial, and often do not consider varying patient exposure. METHODS In this manuscript, we appropriate rules commonly utilized for funnel plots to define a traffic-light system for risk indicators based on statistical criteria that consider the duration of patient follow-up. Further, we describe how these methods can be adapted to assess changing risk over time. Finally, we illustrate numerous graphical approaches to summarize and communicate risk, and discuss hybrid clinical-statistical approaches to allow for the assessment of risk at sites with low patient enrollment. RESULTS We illustrate the aforementioned methodologies for a clinical trial in patients with schizophrenia. CONCLUSIONS Funnel plots are a flexible graphical technique that can form the basis for a risk-based strategy to assess data integrity, while considering site sample size, patient exposure, and changing risk across time.
Collapse
Affiliation(s)
- Richard C Zink
- 1 JMP Life Sciences, SAS Institute Inc, Cary, NC, USA.,2 Department of Biostatistics, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA
| | | | | |
Collapse
|
11
|
Brosteanu O, Schwarz G, Houben P, Paulus U, Strenge-Hesse A, Zettelmeyer U, Schneider A, Hasenclever D. Risk-adapted monitoring is not inferior to extensive on-site monitoring: Results of the ADAMON cluster-randomised study. Clin Trials 2017; 14:584-596. [PMID: 28786330 PMCID: PMC5718334 DOI: 10.1177/1740774517724165] [Citation(s) in RCA: 34] [Impact Index Per Article: 4.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
Background According to Good Clinical Practice, clinical trials must protect rights and
safety of patients and make sure that the trial results are valid and
interpretable. Monitoring on-site has an important role in achieving these
objectives; it controls trial conduct at trial sites and informs the sponsor
on systematic problems. In the past, extensive on-site monitoring with a
particular focus on formal source data verification often lost sight of
systematic problems in study procedures that endanger Good Clinical Practice
objectives. ADAMON is a prospective, stratified, cluster-randomised,
controlled study comparing extensive on-site monitoring with risk-adapted
monitoring according to a previously published approach. Methods In all, 213 sites from 11 academic trials were cluster-randomised between
extensive on-site monitoring (104) and risk-adapted monitoring (109).
Independent post-trial audits using structured manuals were performed to
determine the frequency of major Good Clinical Practice findings at the
patient level. The primary outcome measure is the proportion of audited
patients with at least one major audit finding. Analysis relies on logistic
regression incorporating trial and monitoring arm as fixed effects and site
as random effect. The hypothesis was that risk-adapted monitoring is
non-inferior to extensive on-site monitoring with a non-inferiority margin
of 0.60 (logit scale). Results Average number of monitoring visits and time spent on-site was 2.1 and 2.7
times higher in extensive on-site monitoring than in risk-adapted
monitoring, respectively. A total of 156 (extensive on-site monitoring: 76;
risk-adapted monitoring: 80) sites were audited. In 996 of 1618 audited
patients, a total of 2456 major audit findings were documented. Depending on
the trial, findings were identified in 18%–99% of the audited patients, with
no marked monitoring effect in any of the trials. The estimated monitoring
effect is −0.04 on the logit scale with two-sided 95% confidence interval
(−0.40; 0.33), demonstrating that risk-adapted monitoring is non-inferior to
extensive on-site monitoring. At most, extensive on-site monitoring could
reduce the frequency of major Good Clinical Practice findings by 8.2%
compared with risk-adapted monitoring. Conclusion Compared with risk-adapted monitoring, the potential benefit of extensive
on-site monitoring is small relative to overall finding rates, although
risk-adapted monitoring requires less than 50% of extensive on-site
monitoring resources. Clusters of findings within trials suggest that
complicated, overly specific or not properly justified protocol requirements
contributed to the overall frequency of findings. Risk-adapted monitoring in
only a sample of patients appears sufficient to identify systematic problems
in the conduct of clinical trials. Risk-adapted monitoring has a part to
play in quality control. However, no monitoring strategy can remedy defects
in quality of design. Monitoring should be embedded in a comprehensive
quality management approach covering the entire trial lifecycle.
Collapse
Affiliation(s)
- Oana Brosteanu
- 1 Clinical Trial Centre Leipzig, Leipzig University, Leipzig, Germany
| | - Gabriele Schwarz
- 2 Federal Institute for Drugs and Medical Devices, Bonn, Germany
| | - Peggy Houben
- 1 Clinical Trial Centre Leipzig, Leipzig University, Leipzig, Germany
| | - Ursula Paulus
- 3 Clinical Trials Centre Cologne, University of Cologne, Cologne, Germany
| | - Anke Strenge-Hesse
- 4 KKS-Network/National ECRIN Office, University of Cologne, Cologne, Germany
| | - Ulrike Zettelmeyer
- 3 Clinical Trials Centre Cologne, University of Cologne, Cologne, Germany
| | - Anja Schneider
- 1 Clinical Trial Centre Leipzig, Leipzig University, Leipzig, Germany
| | - Dirk Hasenclever
- 5 Institute for Medical Informatics, Statistics, and Epidemiology, Leipzig University, Leipzig, Germany
| |
Collapse
|
12
|
Statistical Monitoring in Clinical Trials: Best Practices for Detecting Data Anomalies Suggestive of Fabrication or Misconduct. Ther Innov Regul Sci 2016; 50:144-154. [DOI: 10.1177/2168479016630576] [Citation(s) in RCA: 14] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/20/2023]
|