Adaptive clinical trials: progress and challenges

A Contextual-Bandit-Based Approach for Informed Decision-Making in Clinical Trials

Clinical trials are conducted to evaluate the efficacy of new treatments. Clinical trials involving multiple treatments utilize the randomization of treatment assignments to enable the evaluation of treatment efficacies in an unbiased manner. Such evaluation is performed in post hoc studies that usually use supervised-learning methods that rely on large amounts of data collected in a randomized fashion. That approach often proves to be suboptimal in that some participants may suffer and even die as a result of having not received the most appropriate treatments during the trial. Reinforcement-learning methods improve the situation by making it possible to learn the treatment efficacies dynamically during the course of the trial, and to adapt treatment assignments accordingly. Recent efforts using multi-arm bandits, a type of reinforcement-learning method, have focused on maximizing clinical outcomes for a population that was assumed to be homogeneous. However, those approaches have failed to account for the variability among participants that is becoming increasingly evident as a result of recent clinical-trial-based studies. We present a contextual-bandit-based online treatment optimization algorithm that, in choosing treatments for new participants in the study, takes into account not only the maximization of the clinical outcomes as well as the patient characteristics. We evaluated our algorithm using a real clinical trial dataset from the International Stroke Trial. We simulated the online setting by sequentially going through the data of each participant admitted to the trial. Two bandits (one for each context) were created, with four choices of treatments. For a new participant in the trial, depending on the context, one of the bandits was selected. Then, we took three different approaches to choose a treatment: (a) a random choice (i.e., the strategy currently used in clinical trial settings), (b) a Thompson sampling-based approach, and (c) a UCB-based approach. Success probabilities of each context were calculated separately by considering the participants with the same context. Those estimated outcomes were used to update the prior distributions within the bandit corresponding to the context of each participant. We repeated that process through the end of the trial and recorded the outcomes and the chosen treatments for each approach. We also evaluated a context-free multi-arm-bandit-based approach, using the same dataset, to showcase the benefits of our approach. In the context-free case, we calculated the success probabilities for the Bernoulli sampler using the whole clinical trial dataset in a context-independent manner. The results of our retrospective analysis indicate that the proposed approach performs significantly better than either a random assignment of treatments (the current gold standard) or a multi-arm-bandit-based approach, providing substantial gains in the percentage of participants who are assigned the most suitable treatments. The contextual-bandit and multi-arm bandit approaches provide 72.63% and 64.34% gains, respectively, compared to a random assignment.

Longitudinal Speech Biomarkers for Automated Alzheimer's Detection

We introduce a novel audio processing architecture, the Open Voice Brain Model (OVBM), improving detection accuracy for Alzheimer's (AD) longitudinal discrimination from spontaneous speech. We also outline the OVBM design methodology leading us to such architecture, which in general can incorporate multimodal biomarkers and target simultaneously several diseases and other AI tasks. Key in our methodology is the use of multiple biomarkers complementing each other, and when two of them uniquely identify different subjects in a target disease we say they are orthogonal. We illustrate the OBVM design methodology by introducing sixteen biomarkers, three of which are orthogonal, demonstrating simultaneous above state-of-the-art discrimination for two apparently unrelated diseases such as AD and COVID-19. Depending on the context, throughout the paper we use OVBM indistinctly to refer to the specific architecture or to the broader design methodology. Inspired by research conducted at the MIT Center for Brain Minds and Machines (CBMM), OVBM combines biomarker implementations of the four modules of intelligence: The brain OS chunks and overlaps audio samples and aggregates biomarker features from the sensory stream and cognitive core creating a multi-modal graph neural network of symbolic compositional models for the target task. In this paper we apply the OVBM design methodology to the automated diagnostic of Alzheimer's Dementia (AD) patients, achieving above state-of-the-art accuracy of 93.8% using only raw audio, while extracting a personalized subject saliency map designed to longitudinally track relative disease progression using multiple biomarkers, 16 in the reported AD task. The ultimate aim is to help medical practice by detecting onset and treatment impact so that intervention options can be longitudinally tested. Using the OBVM design methodology, we introduce a novel lung and respiratory tract biomarker created using 200,000+ cough samples to pre-train a model discriminating cough cultural origin. Transfer Learning is subsequently used to incorporate features from this model into various other biomarker-based OVBM architectures. This biomarker yields consistent improvements in AD detection in all the starting OBVM biomarker architecture combinations we tried. This cough dataset sets a new benchmark as the largest audio health dataset with 30,000+ subjects participating in April 2020, demonstrating for the first time cough cultural bias.

Perinatal epidemiology: Issues, challenges, and potential solutions

Perinatal epidemiology research is concerned with identifying the effects of events during pregnancy on pregnancy outcomes that include maternal, fetal, and neonatal health outcomes. Randomized trials in perinatal research face many challenges, including randomization difficulties, ethical considerations, and inadequate statistical power due to the small number of subjects eligible for participation. For these reasons, most epidemiological studies conducted in this research field are observational and include different types of bias. This review describes the key methodological difficulties in the design and analysis of randomized and observational studies in perinatal epidemiology, and provides potential corrective approaches.

The adaptive designs CONSORT extension (ACE) statement: a checklist with explanation and elaboration guideline for reporting randomised trials that use an adaptive design

Adaptive designs (ADs) allow pre-planned changes to an ongoing trial without compromising the validity of conclusions and it is essential to distinguish pre-planned from unplanned changes that may also occur. The reporting of ADs in randomised trials is inconsistent and needs improving. Incompletely reported AD randomised trials are difficult to reproduce and are hard to interpret and synthesise. This consequently hampers their ability to inform practice as well as future research and contributes to research waste. Better transparency and adequate reporting will enable the potential benefits of ADs to be realised.This extension to the Consolidated Standards Of Reporting Trials (CONSORT) 2010 statement was developed to enhance the reporting of randomised AD clinical trials. We developed an Adaptive designs CONSORT Extension (ACE) guideline through a two-stage Delphi process with input from multidisciplinary key stakeholders in clinical trials research in the public and private sectors from 21 countries, followed by a consensus meeting. Members of the CONSORT Group were involved during the development process.The paper presents the ACE checklists for AD randomised trial reports and abstracts, as well as an explanation with examples to aid the application of the guideline. The ACE checklist comprises seven new items, nine modified items, six unchanged items for which additional explanatory text clarifies further considerations for ADs, and 20 unchanged items not requiring further explanatory text. The ACE abstract checklist has one new item, one modified item, one unchanged item with additional explanatory text for ADs, and 15 unchanged items not requiring further explanatory text.The intention is to enhance transparency and improve reporting of AD randomised trials to improve the interpretability of their results and reproducibility of their methods, results and inference. We also hope indirectly to facilitate the much-needed knowledge transfer of innovative trial designs to maximise their potential benefits. In order to encourage its wide dissemination this article is freely accessible on the BMJ and Trials journal websites."To maximise the benefit to society, you need to not just do research but do it well" Douglas G Altman.

The Adaptive designs CONSORT Extension (ACE) statement: a checklist with explanation and elaboration guideline for reporting randomised trials that use an adaptive design

Adaptive designs (ADs) allow pre-planned changes to an ongoing trial without compromising the validity of conclusions and it is essential to distinguish pre-planned from unplanned changes that may also occur. The reporting of ADs in randomised trials is inconsistent and needs improving. Incompletely reported AD randomised trials are difficult to reproduce and are hard to interpret and synthesise. This consequently hampers their ability to inform practice as well as future research and contributes to research waste. Better transparency and adequate reporting will enable the potential benefits of ADs to be realised.This extension to the Consolidated Standards Of Reporting Trials (CONSORT) 2010 statement was developed to enhance the reporting of randomised AD clinical trials. We developed an Adaptive designs CONSORT Extension (ACE) guideline through a two-stage Delphi process with input from multidisciplinary key stakeholders in clinical trials research in the public and private sectors from 21 countries, followed by a consensus meeting. Members of the CONSORT Group were involved during the development process.The paper presents the ACE checklists for AD randomised trial reports and abstracts, as well as an explanation with examples to aid the application of the guideline. The ACE checklist comprises seven new items, nine modified items, six unchanged items for which additional explanatory text clarifies further considerations for ADs, and 20 unchanged items not requiring further explanatory text. The ACE abstract checklist has one new item, one modified item, one unchanged item with additional explanatory text for ADs, and 15 unchanged items not requiring further explanatory text.The intention is to enhance transparency and improve reporting of AD randomised trials to improve the interpretability of their results and reproducibility of their methods, results and inference. We also hope indirectly to facilitate the much-needed knowledge transfer of innovative trial designs to maximise their potential benefits.

Development process of a consensus-driven CONSORT extension for randomised trials using an adaptive design

BACKGROUND

Adequate reporting of adaptive designs (ADs) maximises their potential benefits in the conduct of clinical trials. Transparent reporting can help address some obstacles and concerns relating to the use of ADs. Currently, there are deficiencies in the reporting of AD trials. To overcome this, we have developed a consensus-driven extension to the CONSORT statement for randomised trials using an AD. This paper describes the processes and methods used to develop this extension rather than detailed explanation of the guideline.

METHODS

We developed the guideline in seven overlapping stages: 1) Building on prior research to inform the need for a guideline; 2) A scoping literature review to inform future stages; 3) Drafting the first checklist version involving an External Expert Panel; 4) A two-round Delphi process involving international, multidisciplinary, and cross-sector key stakeholders; 5) A consensus meeting to advise which reporting items to retain through voting, and to discuss the structure of what to include in the supporting explanation and elaboration (E&E) document; 6) Refining and finalising the checklist; and 7) Writing-up and dissemination of the E&E document. The CONSORT Executive Group oversaw the entire development process.

RESULTS

Delphi survey response rates were 94/143 (66%), 114/156 (73%), and 79/143 (55%) in rounds 1, 2, and across both rounds, respectively. Twenty-seven delegates from Europe, the USA, and Asia attended the consensus meeting. The main checklist has seven new and nine modified items and six unchanged items with expanded E&E text to clarify further considerations for ADs. The abstract checklist has one new and one modified item together with an unchanged item with expanded E&E text. The E&E document will describe the scope of the guideline, the definition of an AD, and some types of ADs and trial adaptations and explain each reporting item in detail including case studies.

CONCLUSIONS

We hope that making the development processes, methods, and all supporting information that aided decision-making transparent will enhance the acceptability and quick uptake of the guideline. This will also help other groups when developing similar CONSORT extensions. The guideline is applicable to all randomised trials with an AD and contains minimum reporting requirements.

Adaptive designs in clinical trials: why use them, and how to run and report them

Adaptive designs can make clinical trials more flexible by utilising results accumulating in the trial to modify the trial's course in accordance with pre-specified rules. Trials with an adaptive design are often more efficient, informative and ethical than trials with a traditional fixed design since they often make better use of resources such as time and money, and might require fewer participants. Adaptive designs can be applied across all phases of clinical research, from early-phase dose escalation to confirmatory trials. The pace of the uptake of adaptive designs in clinical research, however, has remained well behind that of the statistical literature introducing new methods and highlighting their potential advantages. We speculate that one factor contributing to this is that the full range of adaptations available to trial designs, as well as their goals, advantages and limitations, remains unfamiliar to many parts of the clinical community. Additionally, the term adaptive design has been misleadingly used as an all-encompassing label to refer to certain methods that could be deemed controversial or that have been inadequately implemented.We believe that even if the planning and analysis of a trial is undertaken by an expert statistician, it is essential that the investigators understand the implications of using an adaptive design, for example, what the practical challenges are, what can (and cannot) be inferred from the results of such a trial, and how to report and communicate the results. This tutorial paper provides guidance on key aspects of adaptive designs that are relevant to clinical triallists. We explain the basic rationale behind adaptive designs, clarify ambiguous terminology and summarise the utility and pitfalls of adaptive designs. We discuss practical aspects around funding, ethical approval, treatment supply and communication with stakeholders and trial participants. Our focus, however, is on the interpretation and reporting of results from adaptive design trials, which we consider vital for anyone involved in medical research. We emphasise the general principles of transparency and reproducibility and suggest how best to put them into practice.

The immune system's role in sepsis progression, resolution, and long-term outcome

Sepsis occurs when an infection exceeds local tissue containment and induces a series of dysregulated physiologic responses that result in organ dysfunction. A subset of patients with sepsis progress to septic shock, defined by profound circulatory, cellular, and metabolic abnormalities, and associated with a greater mortality. Historically, sepsis-induced organ dysfunction and lethality were attributed to the complex interplay between the initial inflammatory and later anti-inflammatory responses. With advances in intensive care medicine and goal-directed interventions, early 30-day sepsis mortality has diminished, only to steadily escalate long after "recovery" from acute events. As so many sepsis survivors succumb later to persistent, recurrent, nosocomial, and secondary infections, many investigators have turned their attention to the long-term sepsis-induced alterations in cellular immune function. Sepsis clearly alters the innate and adaptive immune responses for sustained periods of time after clinical recovery, with immune suppression, chronic inflammation, and persistence of bacterial representing such alterations. Understanding that sepsis-associated immune cell defects correlate with long-term mortality, more investigations have centered on the potential for immune modulatory therapy to improve long-term patient outcomes. These efforts are focused on more clearly defining and effectively reversing the persistent immune cell dysfunction associated with long-term sepsis mortality.

CBER's Experience With Adaptive Design Clinical Trials

There is considerable interest among pharmaceutical and other medical product developers in adaptive clinical trials, in which knowledge learned during the course of a trial affects ongoing conduct or analysis of the trial. When the FDA released a draft Guidance document on adaptive design clinical trials in early 2010, expectations were high that it would lead to an increase in regulatory submissions involving adaptive design features, particularly for confirmatory trials. A 6-year (2008-2013) retrospective survey was performed within the Center for Biologics Evaluation and Research (CBER) at the FDA to gather information regarding the submission and evaluation of adaptive design trial proposals. We present an up-to-date summary of adaptive design proposals seen in CBER and provide an overview of our experiences. We share our concerns regarding the statistical issues and operational challenges raised during the review process for adaptive design trials. We also provide general recommendations for developing proposals for such trials. Our motivation in writing this paper was to encourage the best study design proposals to be submitted to CBER. Sometimes these can be adaptive, and sometimes a simpler design is most efficient.

Methodological issues in the design and analyses of neonatal research studies: Experience of the NICHD Neonatal Research Network

Impressive advances in neonatology have occurred over the 30 years of life of The Eunice Kennedy Shriver National Institute of Child Health and Human Development Neonatal Research Network (NRN). However, substantial room for improvement remains in investigating and further developing the evidence base for improving outcomes among the extremely premature. We discuss some of the specific methodological challenges in the statistical design and analysis of randomized trials and observational studies in this population. Challenges faced by the NRN include designing trials for unusual or rare outcomes, accounting for and explaining center variations, identifying other subgroup differences, and balancing safety and efficacy concerns between short-term hospital outcomes and longer-term neurodevelopmental outcomes. In conclusion, the constellation of unique patient characteristics in neonates calls for broad understanding and careful consideration of the issues identified in this article for conducting rigorous studies in this population.

Adaptive Clinical Trial Design: An Overview and Potential Applications in Dermatology

The challenges of drug development, including increasing costs, late-stage drug failures, and the decline in the number of drugs being approved by the US Food and Drug Administration over time, have generated interest in adaptive study designs that have the potential to address these problems. Adaptive trial designs use interim data analysis to amend trials, and have been recognized for more than a decade as a way to increase trial efficiency, partly by the increased probability of demonstrating a drug effect if one exists. In this article, we define adaptive trials; give examples of the most common types; highlight the pros, cons, and ethical considerations of these designs; and illustrate how these tools can be applied to drug development in dermatology.

Adaptive designs undertaken in clinical research: a review of registered clinical trials

Adaptive designs have the potential to improve efficiency in the evaluation of new medical treatments in comparison to traditional fixed sample size designs. However, they are still not widely used in practice in clinical research. Little research has been conducted to investigate what adaptive designs are being undertaken. This review highlights the current state of registered adaptive designs and their characteristics. The review looked at phase II, II/III and III trials registered on ClinicalTrials.gov from 29 February 2000 to 1 June 2014, supplemented with trials from the National Institute for Health Research register and known adaptive trials. A range of adaptive design search terms were applied to the trials extracted from each database. Characteristics of the adaptive designs were then recorded including funder, therapeutic area and type of adaptation. The results in the paper suggest that the use of adaptive designs has increased. They seem to be most often used in phase II trials and in oncology. In phase III trials, the most popular form of adaptation is the group sequential design. The review failed to capture all trials with adaptive designs, which suggests that the reporting of adaptive designs, such as in clinical trials registers, needs much improving. We recommend that clinical trial registers should contain sections dedicated to the type and scope of the adaptation and that the term 'adaptive design' should be included in the trial title or at least in the brief summary or design sections.

Biomarker-Guided Adaptive Trial Designs in Phase II and Phase III: A Methodological Review

BACKGROUND

Personalized medicine is a growing area of research which aims to tailor the treatment given to a patient according to one or more personal characteristics. These characteristics can be demographic such as age or gender, or biological such as a genetic or other biomarker. Prior to utilizing a patient's biomarker information in clinical practice, robust testing in terms of analytical validity, clinical validity and clinical utility is necessary. A number of clinical trial designs have been proposed for testing a biomarker's clinical utility, including Phase II and Phase III clinical trials which aim to test the effectiveness of a biomarker-guided approach to treatment; these designs can be broadly classified into adaptive and non-adaptive. While adaptive designs allow planned modifications based on accumulating information during a trial, non-adaptive designs are typically simpler but less flexible.

METHODS AND FINDINGS

We have undertaken a comprehensive review of biomarker-guided adaptive trial designs proposed in the past decade. We have identified eight distinct biomarker-guided adaptive designs and nine variations from 107 studies. Substantial variability has been observed in terms of how trial designs are described and particularly in the terminology used by different authors. We have graphically displayed the current biomarker-guided adaptive trial designs and summarised the characteristics of each design.

CONCLUSIONS

Our in-depth overview provides future researchers with clarity in definition, methodology and terminology for biomarker-guided adaptive trial designs.

The Potential of Adaptive Design in Animal Studies

Clinical trials are the backbone of medical research, and are often the last step in the development of new therapies for use in patients. Prior to human testing, however, preclinical studies using animal subjects are usually performed in order to provide initial data on the safety and effectiveness of prospective treatments. These studies can be costly and time consuming, and may also raise concerns about the ethical treatment of animals when potentially harmful procedures are involved. Adaptive design is a process by which the methods used in a study may be altered while it is being conducted in response to preliminary data or other new information. Adaptive design has been shown to be useful in reducing the time and costs associated with clinical trials, and may provide similar benefits in preclinical animal studies. The purpose of this review is to summarize various aspects of adaptive design and evaluate its potential for use in preclinical research.

Missing steps in a staircase: a qualitative study of the perspectives of key stakeholders on the use of adaptive designs in confirmatory trials

Background

Despite the promising benefits of adaptive designs (ADs), their routine use, especially in confirmatory trials, is lagging behind the prominence given to them in the statistical literature. Much of the previous research to understand barriers and potential facilitators to the use of ADs has been driven from a pharmaceutical drug development perspective, with little focus on trials in the public sector. In this paper, we explore key stakeholders’ experiences, perceptions and views on barriers and facilitators to the use of ADs in publicly funded confirmatory trials.

Methods

Semi-structured, in-depth interviews of key stakeholders in clinical trials research (CTU directors, funding board and panel members, statisticians, regulators, chief investigators, data monitoring committee members and health economists) were conducted through telephone or face-to-face sessions, predominantly in the UK. We purposively selected participants sequentially to optimise maximum variation in views and experiences. We employed the framework approach to analyse the qualitative data.

Results

We interviewed 27 participants. We found some of the perceived barriers to be: lack of knowledge and experience coupled with paucity of case studies, lack of applied training, degree of reluctance to use ADs, lack of bridge funding and time to support design work, lack of statistical expertise, some anxiety about the impact of early trial stopping on researchers’ employment contracts, lack of understanding of acceptable scope of ADs and when ADs are appropriate, and statistical and practical complexities. Reluctance to use ADs seemed to be influenced by: therapeutic area, unfamiliarity, concerns about their robustness in decision-making and acceptability of findings to change practice, perceived complexities and proposed type of AD, among others.

Conclusions

There are still considerable multifaceted, individual and organisational obstacles to be addressed to improve uptake, and successful implementation of ADs when appropriate. Nevertheless, inferred positive change in attitudes and receptiveness towards the appropriate use of ADs by public funders are supportive and are a stepping stone for the future utilisation of ADs by researchers.

Electronic supplementary material

The online version of this article (doi:10.1186/s13063-015-0958-9) contains supplementary material, which is available to authorized users.

Clinical trialist perspectives on the ethics of adaptive clinical trials: a mixed-methods analysis

BACKGROUND

In an adaptive clinical trial (ACT), key trial characteristics may be altered during the course of the trial according to predefined rules in response to information that accumulates within the trial itself. In addition to having distinguishing scientific features, adaptive trials also may involve ethical considerations that differ from more traditional randomized trials. Better understanding of clinical trial experts' views about the ethical aspects of adaptive designs could assist those planning ACTs. Our aim was to elucidate the opinions of clinical trial experts regarding their beliefs about ethical aspects of ACTs.

METHODS

We used a convergent, mixed-methods design employing a 22-item ACTs beliefs survey with visual analog scales and open-ended questions and mini-focus groups. We developed a coding scheme to conduct thematic searches of textual data, depicted responses to visual analog scales on box-plot diagrams, and integrated findings thematically. Fifty-three clinical trial experts from four constituent groups participated: academic biostatisticians (n = 5); consultant biostatisticians (n = 6); academic clinicians (n = 22); and other stakeholders including patient advocacy, National Institutes of Health, and U.S. Food and Drug Administration representatives (n = 20).

RESULTS

The respondents recognized potential ethical benefits of ACTs, including a higher probability of receiving an effective intervention for participants, optimizing resource utilization, and accelerating treatment discovery. Ethical challenges voiced include developing procedures so trial participants can make informed decisions about taking part in ACTs and plausible, though unlikely risks of research personnel altering enrollment patterns.

CONCLUSIONS

Clinical trial experts recognize ethical advantages but also pose potential ethical challenges of ACTs. The four constituencies differ in their weighing of ACT ethical considerations based on their professional vantage points. These data suggest further discussion about the ethics of ACTs is needed to facilitate ACT planning, design and conduct, and ultimately better allow planners to weigh ethical implications of competing trial designs.

OARSI Clinical Trials Recommendations: Design and conduct of implementation trials of interventions for osteoarthritis

Rigorous implementation research is important for testing strategies to improve the delivery of effective osteoarthritis (OA) interventions. The objective of this manuscript is to describe principles of implementation research, including conceptual frameworks, study designs and methodology, with specific recommendations for randomized clinical trials of OA treatment and management. This manuscript includes a comprehensive review of prior research and recommendations for implementation trials. The review of literature included identification of seminal articles on implementation research methods, as well as examples of previous exemplar studies using these methods. In addition to a comprehensive summary of this literature, this manuscript provides key recommendations for OA implementation trials. This review concluded that to date there have been relatively few implementation trials of OA interventions, but this is an emerging area of research. Future OA clinical trials should routinely consider incorporation of implementation aims to enhance translation of findings.

Choosing inclusion criteria that minimize the time and cost of clinical trials

AIM: To present statistical tools to model and optimize the cost of a randomized clinical trial as a function of the stringency of patient inclusion criteria.

METHODS: We consider a two treatment, dichotomous outcome trial that includes a proportion of patients who are strong responders to the tested intervention. Patients are screened for inclusion using an arbitrary number of test results that are combined into an aggregate suitability score. The screening score is regarded as a diagnostic test for the responsive phenotype, having a specific cutoff value for inclusion and a particular sensitivity and specificity. The cutoff is a measure of stringency of inclusion criteria. Total cost is modeled as a function of the cutoff value, number of patients screened, the number of patients included, the case occurrence rate, response probabilities for control and experimental treatments, and the trial duration required to produce a statistically significant result with a specified power. Regression methods are developed to estimate relevant model parameters from pilot data in an adaptive trial design.

RESULTS: The patient numbers and total cost are strongly related to the choice of the cutoff for inclusion. Clear cost minimums exist between 5.6 and 6.1 on a representative 10-point scale of exclusiveness. Potential cost savings for typical trial scenarios range in millions of dollars. As the response rate for controls approaches 50%, the proper choice of inclusion criteria can mean the difference between a successful trial and a failed trial.

CONCLUSION: Early formal estimation of optimal inclusion criteria allows planning of clinical trials to avoid high costs, excessive delays, and moral hazards of Type II errors.

Physicochemical and functional characterization of vaccine antigens and adjuvants

As novel vaccine antigens and adjuvants are being tested in humans, understanding of critical quality attributes essential for eliciting optimal vaccine response and vaccine antigen-adjuvant interactions is pivotal for vaccine safety and eliciting 'protective' immune responses. Therefore, the efforts to better characterize and evaluate vaccine antigen and antigen-adjuvant drug products need to begin very early during the discovery and development phase. In this review, we discuss the importance of characterization of physicochemical and functional properties in vaccine antigen, adjuvant and the final antigen-adjuvant drug product and emphasize the greater need for more extensive understanding of vaccine antigen-adjuvant interactions. We highlight the key parameters and quality attributes that are critical to measure during preclinical and clinical testing of the vaccine and discuss in some detail the technologies, and their limitations, used in analyzing the key physicochemical and functional attributes of vaccine antigen and antigen-adjuvant drug product.

Designing clinical trials for dystonia

With advances in the understanding of the pathophysiology of dystonia, novel therapeutics are being developed. Such therapies will require clinical investigation ranging from exploratory studies to examine safety, tolerability, dosage selection, and preliminary efficacy to confirmatory studies to evaluate efficacy definitively. As dystonia is a rare and complex disorder with clinical and etiological heterogeneity, clinical trials will require careful consideration of the trial design, including enrollment criteria, concomitant medication use, and outcome measures. Given the complexities of designing and implementing efficient clinical trials, it is important for clinicians and statisticians to collaborate closely throughout the clinical development process and that each has a basic understanding of both the clinical and statistical issues that must be addressed. To facilitate designing appropriate clinical trials in this field, we review important general clinical trial and regulatory principles, and discuss the critical components of trials with an emphasis on considerations specific to dystonia. Additionally, we discuss designs used in early exploratory, late exploratory, and confirmatory phases, including adaptive designs.

Overview, hurdles, and future work in adaptive designs: perspectives from a National Institutes of Health-funded workshop

BACKGROUND

The clinical trials community has a never-ending search for dependable and reliable ways to improve clinical research. This exploration has led to considerable interest in adaptive clinical trial designs, which provide the flexibility to adjust trial characteristics on the basis of data reviewed at interim stages. Statisticians and clinical investigators have proposed or implemented a wide variety of adaptations in clinical trials, but specific approaches have met with differing levels of support. Within industry, investigators are actively exploring the benefits and pitfalls associated with adaptive designs (ADs). For example, a Drug Information Association (DIA) working group on ADs has engaged regulatory agencies in discussions. Many researchers working on publicly funded clinical trials, however, are not yet fully engaged in this discussion. We organized the Scientific Advances in Adaptive Clinical Trial Designs Workshop to begin a conversation about using ADs in publicly funded research. Held in November of 2009, the 1½-day workshop brought together representatives from the National Institutes of Health (NIH), the Food and Drug Administration (FDA), the European Medicines Agency (EMA), the pharmaceutical industry, nonprofit foundations, the patient advocacy community, and academia. The workshop offered a forum for participants to address issues of ADs that arise at the planning, designing, and execution stages of clinical trials, and to hear the perspectives of influential members of the clinical trials community. The participants also set forth recommendations for guiding action to promote the appropriate use of ADs. These recommendations have since been presented, discussed, and vetted in a number of venues including the University of Pennsylvania Conference on Statistical Issues in Clinical Trials and the Society for Clinical Trials annual meeting.

PURPOSE

To provide a brief overview of ADs, describe the rationale behind conducting the workshop, and summarize the main recommendations that were produced as a result of this workshop.

CONCLUSIONS

There is a growing interest in the use of adaptive clinical trial designs. However, a number of logistical barriers need to be addressed in order to obtain the potential advantages of an AD. Currently, the pharmaceutical industry is well ahead of academic trialists with respect to addressing these barriers. Academic trialists will need to address important issues such as education, infrastructure, modifications to existing funding models, and the impact on Data and Safety Monitoring Boards (DSMB) in order to achieve the possible benefits of adaptive clinical trial designs.

Adaptive trial designs: a review of barriers and opportunities

Adaptive designs allow planned modifications based on data accumulating within a study. The promise of greater flexibility and efficiency stimulates increasing interest in adaptive designs from clinical, academic, and regulatory parties. When adaptive designs are used properly, efficiencies can include a smaller sample size, a more efficient treatment development process, and an increased chance of correctly answering the clinical question of interest. However, improper adaptations can lead to biased studies. A broad definition of adaptive designs allows for countless variations, which creates confusion as to the statistical validity and practical feasibility of many designs. Determining properties of a particular adaptive design requires careful consideration of the scientific context and statistical assumptions. We first review several adaptive designs that garner the most current interest. We focus on the design principles and research issues that lead to particular designs being appealing or unappealing in particular applications. We separately discuss exploratory and confirmatory stage designs in order to account for the differences in regulatory concerns. We include adaptive seamless designs, which combine stages in a unified approach. We also highlight a number of applied areas, such as comparative effectiveness research, that would benefit from the use of adaptive designs. Finally, we describe a number of current barriers and provide initial suggestions for overcoming them in order to promote wider use of appropriate adaptive designs. Given the breadth of the coverage all mathematical and most implementation details are omitted for the sake of brevity. However, the interested reader will find that we provide current references to focused reviews and original theoretical sources which lead to details of the current state of the art in theory and practice.

Sample size re-estimation in an on-going NIH-sponsored clinical trial: the secondary prevention of small subcortical strokes experience

BACKGROUND AND PURPOSE

When planning clinical trials, decisions regarding sample size are often based on educated guesses of parameters, which may in fact prove to be over- or under-estimates. For example, after initiation of the SPS3 study, published data indicated that the recurrent stroke rates might be lower than initially planned for the study. Failure to account for this could result in an under-powered study. Thus, we performed a sample size re-estimation, and describe the experience herein.

METHODS

We evaluated different scenarios based on a re-estimated overall event rate, including increasing the sample size and increasing the follow-up time, to determine their impact on both type I error and the power to detect the initially planned treatment difference.

RESULTS

We found that by increasing the sample size from 2500 to 3000 and by following the patients for one year after the end of recruitment, we would maintain our planned type I error rate, and increase the power to detect the prespecified clinically meaningful difference to between 67% and 87%, depending on the rate of recruitment.

CONCLUSIONS

We successfully implemented this unplanned design modification in the SPS3 study, in order to allow for sufficient power to detect the planned treatment differences.

CLINICAL TRIALS REGISTRATION INFORMATION

Clinical Trials Registration - http://clinicaltrials.gov/show/NCT00059306. Unique identifier: NCT00059306.

Current drug development challenges in chronic kidney disease (CKD)--identification of individualized determinants of renal progression and premature cardiovascular disease (CVD)

Chronic kidney disease (CKD) and end-stage renal disease (ESRD) are currently considered as major health burdens. Notably, CKD can be regarded as an interesting clinical model of accelerated cardiovascular disease (CVD) and ageing, which offers exciting new perspectives and challenges for pharmaceutical drug development. However, during the last decades, therapeutic advances to slow down the progression of CKD and reduce CVD risk have largely failed due to several possible reasons including (i) the lack of profound understanding of the pathophysiology of chronic renal damage and its associated CVD; (ii) an inadequate characterization of molecular mechanisms of currently approved therapies such as renin-angiotensin-aldosterone-system (RAAS) blockade; (iii) the unclear biochemical property needs required for novel therapeutic approaches; (iv) the missing quantity and quality of clinical trials in the nephrology field; and, most importantly, (v) the absence of prognostic renal biomarkers that reflect the severity of the structural organ damage and predict ESRD as well as CVD mortality. There is clearly an insufficient understanding of why a significant proportion of CKD patients progress to ESRD and/or die from CVD whereas others rather remain stable. In this article, we urge renal researchers to develop novel experimental and clinical tools for rational and translational drug discovery. Identification of individualized determinants of CKD progression and/or premature CVD will enable personalized medicine and lead to novel innovative nephro- and/or cardioprotective pharmacological treatment in these high-risk patients.

An urn based covariate adjusted response adaptive allocation design

An urn based covariate adjusted response adaptive allocation design with dichotomous responses is proposed incorporating the ordered nature of the covariate. The allocation procedure is assessed both numerically and theoretically. The performance of the allocation design is also investigated in a relevant hypothetical clinical trial.

Bayesian adaptive clinical trials: a dream for statisticians only?

Adaptive or 'flexible' designs have emerged, mostly within frequentist frameworks, as an effective way to speed up the therapeutic evaluation process. Because of their flexibility, Bayesian methods have also been proposed for Phase I through Phase III adaptive trials; however, it has been reported that they are poorly used in practice. We aim to describe the international scientific production of Bayesian clinical trials by investigating the actual development and use of Bayesian 'adaptive' methods in the setting of clinical trials. A bibliometric study was conducted using the PubMed and Science Citation Index-Expanded databases. Most of the references found were biostatistical papers from various teams around the world. Most of the authors were from the US, and a large proportion was from the MD Anderson Cancer Center (University of Texas, Houston, TX). The spread and use of these articles depended heavily on their topic, with 3.1% of the biostatistical articles accumulating at least 25 citations within 5 years of their publication compared with 15% of the reviews and 32% of the clinical articles. We also examined the reasons for the limited use of Bayesian adaptive design methods in clinical trials and the areas of current and future research to address these challenges. Efforts to promote Bayesian approaches among statisticians and clinicians appear necessary.

Public health nursing case management for women receiving temporary assistance for needy families: a randomized controlled trial using community-based participatory research

OBJECTIVES

We evaluated the effectiveness of a community-based participatory research-grounded intervention among women receiving Temporary Assistance for Needy Families (TANF) with chronic health conditions in increasing (1) health care visits, (2) Medicaid knowledge and skills, and (3) health and functional status.

METHODS

We used a randomized controlled trial design to assign 432 women to a public health nurse case management plus Medicaid intervention or a wait-control group. We assessed Medicaid outcomes pre- and posttraining; other outcomes were assessed at 3, 6, and 9 months.

RESULTS

Medicaid knowledge and skills improved (P < .001 for both). Intervention group participants were more likely to have a new mental health visit (odds ratio [OR] = 1.92; P = .007), and this likelihood increased in higher-risk subgroups (OR = 2.03 and 2.83; P = .04 and .006, respectively). Depression and functional status improved in the intervention group over time (P = .016 for both). No differences were found in routine or preventive care, or general health.

CONCLUSIONS

Health outcomes among women receiving TANF can be improved with public health interventions. Additional strategies are needed to further reduce health disparities in this population.

Adaptive trial designs

We review adaptive designs for clinical trials, giving special attention to the control of the Type I error in late-phase confirmatory trials, when the trial planner wishes to adjust the final sample size of the study in response to an unblinded analysis of interim estimates of treatment effects. We point out that there is considerable inefficiency in using the adaptive designs that employ conditional power calculations to reestimate the sample size and that maintain the Type I error by using certain weighted test statistics. Although these adaptive designs have little advantage over familiar group-sequential designs, our review also describes recent developments in adaptive designs that are both flexible and efficient. We also discuss the use of Bayesian designs, when the context of use demands control over operating characteristics (Type I and II errors) and correction of the bias of estimated treatment effects.

HIV-1 vaccines and adaptive trial designs

Developing a vaccine against the human immunodeficiency virus (HIV) poses an exceptional challenge. There are no documented cases of immune-mediated clearance of HIV from an infected individual, and no known correlates of immune protection. Although nonhuman primate models of lentivirus infection have provided valuable data about HIV pathogenesis, such models do not predict HIV vaccine efficacy in humans. The combined lack of a predictive animal model and undefined biomarkers of immune protection against HIV necessitate that vaccines to this pathogen be tested directly in clinical trials. Adaptive clinical trial designs can accelerate vaccine development by rapidly screening out poor vaccines while extending the evaluation of efficacious ones, improving the characterization of promising vaccine candidates and the identification of correlates of immune protection.

Formal feasibility studies in palliative care: why they are important and how to conduct them

The concept of clinical trial feasibility is of great interest to the community of palliative care researchers, clinicians, and granting agencies. Significant allocation of resources is required in the form of funding, time, intellect, and motivation to carry out clinical research, and understandably, clinical investigators, institutions, and granting agencies are disappointed when funded trials are unsuccessfully conducted. We argue that for many trials conducted in palliative care, the feasibility of conducting the proposed trial should be formally explored before implementation. There is substantial information available within the literature on the topic of study feasibility but no singular guide on how one can pragmatically apply this advice in the palliative care setting. We suggest that a Formal Feasibility Study for palliative care trials should be commonly conducted before development of a larger pivotal trial, to prospectively identify barriers to research, develop strategies to address these barriers, and predict whether the larger study is feasible. If a Formal Feasibility Study is not required, elements of feasibility can be specifically tested before launching clinical trials. The purpose of this article is to offer a draft framework for the design and conduct of a Formal Feasibility Study that, if implemented, could concretely support successful completion of high-quality research in a timely fashion. Additionally, we hope to foster dialogue within the palliative care research community regarding the relevance of establishing feasibility before initiation of definitive trials in the palliative care population.

Adaptive design clinical trials: Methodology, challenges and prospect

New drug development is a time-consuming and expensive process. Recently, there has been stagnation in the development of novel compounds. Moreover, the attrition rate in clinical research is also on the rise. Fearing more stagnation, the Food and Drug Administration released the critical path initiative in 2004 and critical path opportunity list in 2006 thus highlighting the need of advancing innovative trial designs. One of the innovations suggested was the adaptive designed clinical trials, a method promoting introduction of pre-specified modifications in the design or statistical procedures of an on-going trial depending on the data generated from the concerned trial thus making a trial more flexible. The adaptive design trials are proposed to boost clinical research by cutting on the cost and time factor. Although the concept of adaptive designed clinical trials is round-the-corner for the last 40 years, there is still lack of uniformity and understanding on this issue. This review highlights important adaptive designed methodologies besides covering the regulatory positions on this issue.

Implications of comparative effectiveness research for radiation oncology

PURPOSE

The essence of comparative effectiveness research (CER) is to understand what health interventions work, for which patients, and under what conditions. The objective of this article is to introduce the relative strengths and weaknesses of several forms of evidence to illustrate the potential for CER evidence generation within radiation oncology.

METHODS

We introduce the underlying concepts of effectiveness and efficacy. We describe the design of traditional explanatory randomized trials (RCTs). We introduce the rationale, strengths, and weaknesses of several alternative study designs for comparative effectiveness, including pragmatic clinical trials, adaptive trials, and observational (nonrandomized) studies.

RESULTS

Explanatory RCTs are designed to assess the efficacy of an intervention while achieving a high degree of internal validity. Pragmatic clinical trials (PCTs) are prospective studies performed in typical, real-world clinical practice settings. The emphasis of PCTs is to maintain a degree of internal validity while also maximizing external validity. Adaptive trials can be modified at interim stages using existing or evolving evidence in the course of a trial, which may allow trials to maintain clinical relevance by studying current treatments. Observational data are becoming increasingly important, given substantial funding for clinical registries and greater availability of electronic medical records and claims databases, but need to address well-known limitations such as selection bias.

CONCLUSION

With the rapid proliferation of new and evolving radiotherapy technologies, it is incumbent upon our field to invest in building the evidence base for radiotherapy CER and to actively participate in current initiatives for generating comparative evidence.

Internal pilots for observational studies

Study planning often involves selecting an appropriate sample size. Power calculations require specifying an effect size and estimating "nuisance" parameters, e.g. the overall incidence of the outcome. For observational studies, an additional source of randomness must be estimated: the rate of the exposure. A poor estimate of any of these parameters will produce an erroneous sample size. Internal pilot (IP) designs reduce the risk of this error - leading to better resource utilization - by using revised estimates of the nuisance parameters at an interim stage to adjust the final sample size. In the clinical trials setting, where allocation to treatment groups is pre-determined, IP designs have been shown to achieve the targeted power without introducing substantial inflation of the type I error rate. It has not been demonstrated whether the same general conclusions hold in observational studies, where exposure-group membership cannot be controlled by the investigator. We extend the IP to observational settings. We demonstrate through simulations that implementing an IP, in which prevalence of the exposure can be re-estimated at an interim stage, helps ensure optimal power for observational research with little inflation of the type I error associated with the final data analysis.

Food and drug administration's critical path initiative and innovations in drug development paradigm: Challenges, progress, and controversies

During the last decade, despite increased investment in drug research and development related activity, stagnation in new drug discovery has been documented. Despite a 70% increase in investment in research and development-related activities, a 40% fall in launch of new chemical entities was seen during 1994-2004. A steep rise in the attrition rate of drug development has complicated the matter. Rising cost and increased attrition rates proved major barriers to investment in higher risk drugs or in therapies for uncommon diseases or diseases that predominantly afflict the poor. This prompted Food and Drug Administration (FDA) to highlight this problem in a 2004 white paper classified as "Critical Path Initiative" (CPI) and to initiate steps to target stagnation and rise in attrition rates. Many new drug development projects have started worldwide taking cue from CPI; adopting microdosing, adaptive designs and taking advantage of newly developed biomarkers under the CPI. This review discusses the various strategies adopted under CPI to decrease attrition rate and stagnation of new drug development, and the challenges and controversies associated with CPI.

The design of phase II clinical trials testing cancer therapeutics: consensus recommendations from the clinical trial design task force of the national cancer institute investigational drug steering committee

The optimal design of phase II studies continues to be the subject of vigorous debate, especially studies of newer molecularly targeted agents. The observations that many new therapeutics "fail" in definitive phase III studies, coupled with the numbers of new agents to be tested as well as the increasing costs and complexity of clinical trials, further emphasize the critical importance of robust and efficient phase II design. The Clinical Trial Design Task Force (CTD-TF) of the National Cancer Institute (NCI) Investigational Drug Steering Committee (IDSC) has published a series of discussion papers on phase II trial design in Clinical Cancer Research. The IDSC has developed formal recommendations about aspects of phase II trial design that are the subject of frequent debate, such as endpoints (response versus progression-free survival), randomization (single-arm designs versus randomization), inclusion of biomarkers, biomarker-based patient enrichment strategies, and statistical design (e.g., two-stage designs versus multiple-group adaptive designs). Although these recommendations in general encourage the use of progression-free survival as the primary endpoint, randomization, inclusion of biomarkers, and incorporation of newer designs, we acknowledge that objective response as an endpoint and single-arm designs remain relevant in certain situations. The design of any clinical trial should always be carefully evaluated and justified based on characteristic specific to the situation.

Pivotal studies of orphan drugs approved for neurological diseases

OBJECTIVE

To identify design elements of clinical trials leading to US Food and Drug Administration approval of drugs for neurological diseases with and without orphan indications.

METHODS

We used publicly available information to identify approvals for drugs for neurological diseases with an orphan indication (n = 19) and compared them with recent approvals for drugs for neurological diseases without an orphan indication (n = 20). We identified "pivotal trials" from drug labels and drug approval packages, and assessed them on four elements of clinical trial design: control, blinding, randomization, and size.

RESULTS

All drugs for neurological diseases (100%) approved without an orphan indication included at least two randomized, double-blind, placebo-controlled trials. In comparison, 32% of drugs with an orphan indication had at least two such trials (p < 0.001) and 74% had at least one (p = 0.02). Thirty-three pivotal trials were conducted for the 19 drugs approved with an orphan indication. Of the 33 trials, 11 (33%) did not use a placebo control, 9 (27%) were not double blind, and 4 (12%) were not randomized. Drugs approved without an orphan indication had more pivotal trials per drug (3.8 vs 1.7 trials; p < 0.001) and a larger mean trial size (506 vs 164 trial participants; p < 0.001).

INTERPRETATION

The US Food and Drug Administration has approved orphan drugs for neurological diseases without randomized, doubled-blind, placebo-controlled pivotal trials. As orphan drug development grows, demand will likely increase for alternative designs for conducting adequate and well-controlled studies to demonstrate drug efficacy.

Phase II trial of CoQ10 for ALS finds insufficient evidence to justify phase III

OBJECTIVE

Amyotrophic lateral sclerosis (ALS) is a devastating, and currently incurable, neuromuscular disease in which oxidative stress and mitochondrial impairment are contributing to neuronal loss. Coenzyme Q10 (CoQ10), an antioxidant and mitochondrial cofactor, has shown promise in ALS transgenic mice, and in clinical trials for neurodegenerative diseases other than ALS. Our aims were to choose between two high doses of CoQ10 for ALS, and to determine if it merits testing in a Phase III clinical trial.

METHODS

We designed and implemented a multicenter trial with an adaptive, two-stage, bias-adjusted, randomized, placebo-controlled, double-blind, Phase II design (n = 185). The primary outcome in both stages was a decline in the ALS Functional Rating Scale-revised (ALSFRSr) score over 9 months. Stage 1 (dose selection, 35 participants per group) compared CoQ10 doses of 1,800 and 2,700 mg/day. Stage 2 (futility test, 75 patients per group) compared the dose selected in Stage 1 against placebo.

RESULTS

Stage 1 selected the 2,700 mg dose. In Stage 2, the pre-specified primary null hypothesis that this dose is superior to placebo was not rejected. It was rejected, however, in an accompanying prespecified sensitivity test, and further supplementary analyses. Prespecified secondary analyses showed no significant differences between CoQ10 at 2,700 mg/day and placebo. There were no safety concerns.

INTERPRETATION

CoQ10 at 2,700 mg daily for 9 months shows insufficient promise to warrant Phase III testing. Given this outcome, the adaptive Phase II design incorporating a dose selection and a futility test avoided the need for a much larger conventional Phase III trial.