Directions for new developments on statistical design and analysis of small population group trials

Two-stage randomized clinical trials with a right-censored endpoint: Comparison of frequentist and Bayesian adaptive designs

Adaptive randomized clinical trials are of major interest when dealing with a time-to-event outcome in a prolonged observation window. No consensus exists either to define stopping boundaries or to combinep $$ p $$ values or test statistics in the terminal analysis in the case of a frequentist design and sample size adaptation. In a one-sided setting, we compared three frequentist approaches using stopping boundaries relying onα $$ \alpha $$ -spending functions and a Bayesian monitoring setting with boundaries based on the posterior distribution of the log-hazard ratio. All designs comprised a single interim analysis with an efficacy stopping rule and the possibility of sample size adaptation at this interim step. Three frequentist approaches were defined based on the terminal analysis: combination of stagewise statistics (Wassmer) or ofp $$ p $$ values (Desseaux), or on patientwise splitting (Jörgens), and we compared the results with those of the Bayesian monitoring approach (Freedman). These different approaches were evaluated in a simulation study and then illustrated on a real dataset from a randomized clinical trial conducted in elderly patients with chronic lymphocytic leukemia. All approaches controlled for the type I error rate, except for the Bayesian monitoring approach, and yielded satisfactory power. It appears that the frequentist approaches are the best in underpowered trials. The power of all the approaches was affected by the violation of the proportional hazards (PH) assumption. For adaptive designs with a survival endpoint and a one-sided alternative hypothesis, the Wassmer and Jörgens approaches after sample size adaptation should be preferred, unless violation of PH is suspected.

Efficacy and safety of vigabatrin in patients with tuberous sclerosis complex and infantile epileptic spasm syndrome: a systematic review

INTRODUCTION

Tuberous sclerosis complex (TSC) is a common genetic cause of epilepsy. Infantile epileptic spasm syndrome (IESS) is often the presenting neurologic feature, progressively evolving into refractory epilepsy. Vigabatrin (VGB) is often used in clinical practice as a first-line therapy in TSC with IESS. This systematic review aims to collect and analyze the efficacy data about VGB in TSC cases with IESS, in order to evaluate the strength of evidence in the literature.

METHODS

A systematic search of trials, observational studies, and case series involving patients with TSC and IESS treated with VGB was performed using MEDLINE, CENTRAL, and the US NIH Clinical Trials Registry. Single case studies, animal and non-English language studies were excluded. Seventeen studies were selected, of which 3 were RCTs and 14 were observational studies.

RESULTS

An overall response rate of 67% (231/343 responders) resulted from the analysis, with a spasm-free rate restricted to RCTs of 88% (29/33 subjects).

CONCLUSIONS

Although all the studies analyzed reported beneficial effects of VGB in TSC patients with IESS, with higher response rates in comparison to non-TSC subjects with IESS, a low level of evidence and high heterogeneity do not guarantee sufficient strength for therapeutic recommendations.

Patient Centricity: Design and Conduct of Clinical Trials in Orphan Diseases: Third of Three Sets of Expanded Proceedings from the 2020 ISCTM Autumn Conference on Pediatric Drug Development

This article expands on a session, titled "Patient Centricity: Design and Conduct of Clinical Trials in Orphan Diseases," that was presented as part of a two-day meeting on Pediatric Drug Development at the International Society for Central Nervous System (CNS) Clinical Trials and Methodology (ISCTM) Autumn Conference in Boston, Massachusetts, in October 2020. Speakers from various areas of pediatric drug development addressed a variety of implications of including children in drug development programs, including implications for rare/orphan diseases. The speakers have written summaries of their talks. The session's lead Chair was Dr. Joan Busner, who wrote introductory and closing comments. Dr. Simon Day, regulatory consultant, outlined some of the past mistakes that have plagued trials that did not consult with patient groups in the early design phase. Dr. Atul Mahableshwarkar provided an industry perspective of a recent trial that benefited from the inclusion of patient input. Drs. Lucas Kempf and Maria Sheean provided regulatory input from the perspectives of the United States (US) Food and Drug Administration (FDA) and European Medicines Agency (EMA), respectively. Dr. Judith Dunn outlined a novel approach for assessing and rank ordering patient and clinician clinical meaningfulness and the disconnect that may occur. Dr. Busner provided closing comments, tied together the presented issues, and provided a synopsis of the lively discussion that followed the session. In addition to the speakers above, the discussion included two representatives from patient advocacy groups, as well as an additional speaker who described the challenges of conducting a pediatric trial in the US and European Union (EU), given the often competing regulatory requirements. This article should serve as an expert-informed reference to those interested and involved in CNS drug development programs that are aimed at children and rare diseases and seek to ensure a patient-centric approach.

Inclusion of Effect Size Measures and Clinical Relevance in Research Papers

BACKGROUND

There are multiple issues that arise when researchers focus on and only report "statistical significance" of study findings. An important element that is often not included in reports is a discussion of clinical relevance.

OBJECTIVES

The authors address issues related to significance, the use of effect sizes, confidence or credible intervals, and the inclusion of clinical relevance in reports of research findings.

METHODS

Measures of magnitude, precision, and relevance such as effect sizes, confidence intervals (CIs), and clinically relevant effects are described in detail. In addition, recommendations for reporting and evaluating effect sizes and CIs are included. Example scenarios are presented to illustrate the interplay of statistical significance and clinical relevance.

RESULTS

There are several issues that may arise when significance is the focus of clinical research reporting. One issue is the lack of attention to nonsignificant findings in published works although findings show clinical relevance. Another issue is that significance is interpreted as clinical relevance. As well, clinically relevant results from small-sample studies are often not considered for publication, and thus, findings might not be available for meta-analysis.

DISCUSSION

Findings in research reports should address effect sizes and clinical relevance and significance. Failure to publish clinically relevant effects and CIs may preclude the inclusion of clinically relevant studies in systematic reviews and meta-analyses, thereby limiting the advancement of evidence-based practice. Several accessible resources for researchers to generate, report, and evaluate measures of magnitude, precision, and relevance are included in this article.

Treating neutropenia and neutrophil dysfunction in glycogen storage disease type Ib with an SGLT2 inhibitor

Neutropenia and neutrophil dysfunction cause serious infections and inflammatory bowel disease in glycogen storage disease type Ib (GSD-Ib). Our discovery that accumulating 1,5-anhydroglucitol-6-phosphate (1,5AG6P) caused neutropenia in a glucose-6-phosphatase 3 (G6PC3)-deficient mouse model and in 2 rare diseases (GSD-Ib and G6PC3 deficiency) led us to repurpose the widely used antidiabetic drug empagliflozin, an inhibitor of the renal glucose cotransporter sodium glucose cotransporter 2 (SGLT2). Off-label use of empagliflozin in 4 GSD-Ib patients with incomplete response to granulocyte colony-stimulating factor (GCSF) treatment decreased serum 1,5AG and neutrophil 1,5AG6P levels within 1 month. Clinically, symptoms of frequent infections, mucosal lesions, and inflammatory bowel disease resolved, and no symptomatic hypoglycemia was observed. GCSF could be discontinued in 2 patients and tapered by 57% and 81%, respectively, in the other 2. The fluctuating neutrophil numbers in all patients were increased and stabilized. We further demonstrated improved neutrophil function: normal oxidative burst (in 3 of 3 patients tested), corrected protein glycosylation (2 of 2), and normal neutrophil chemotaxis (1 of 1), and bactericidal activity (1 of 1) under treatment. In summary, the glucose-lowering SGLT2 inhibitor empagliflozin, used for type 2 diabetes, was successfully repurposed for treating neutropenia and neutrophil dysfunction in the rare inherited metabolic disorder GSD-Ib without causing symptomatic hypoglycemia. We ascribe this to an improvement in neutrophil function resulting from the reduction of the intracellular concentration of 1,5AG6P.

Bayesian Approaches for Confirmatory Trials in Rare Diseases: Opportunities and Challenges

The aim of this narrative review is to introduce the reader to Bayesian methods that, in our opinion, appear to be the most important in the context of rare diseases. A disease is defined as rare depending on the prevalence of the affected patients in the considered population, for example, about 1 in 1500 people in U.S.; about 1 in 2500 people in Japan; and fewer than 1 in 2000 people in Europe. There are between 6000 and 8000 rare diseases and the main issue in drug development is linked to the challenge of achieving robust evidence from clinical trials in small populations. A better use of all available information can help the development process and Bayesian statistics can provide a solid framework at the design stage, during the conduct of the trial, and at the analysis stage. The focus of this manuscript is to provide a review of Bayesian methods for sample size computation or reassessment during phase II or phase III trial, for response adaptive randomization and of for meta-analysis in rare disease. Challenges regarding prior distribution choice, computational burden and dissemination are also discussed.

Translational perspectives to treat Epidermolysis bullosa-Where do we stand?

Epidermolysis bullosa (EB) is the prototypical example of genetic skin fragility disorders. Genotypic heterogeneity, modifier genes, epigenetic, biochemical and environmental factors alter and determine pathogenic traits and, ultimately, the wide and striking phenotypic variability in EB. Besides the primary structural-functional defect, chronic tissue damage with induction and dysregulation of inflammatory pathways is a common pathogenic mechanism in EB. In localized variants, the inflammatory aberrations may mainly affect the micromilieu of lesional skin, while a systemic inflammatory response was shown to contribute to the systemic morbidity in severe EB subtypes with extensive cutaneous involvement. Our continued understanding of the pathophysiology of EB, as well as advances in molecular technologies, has paved the way for translational therapeutic approaches. The spectrum comprises of corrective and symptom-relieving therapies that include innovative therapeutic options garnered from the bench, repurposed drugs approved for other diseases, as well as strategies for gene-, protein- and cell-based therapies. Immunological traits further define new targets of therapy, aimed at improving skin barrier restoration, microbial surveillance and infection control, wound healing and anti-neoplastic effects. Clinical availability and feasibility of these approaches for all EB patients and subtypes are currently limited, reflecting issues of efficacy, specificity, tolerability and safety. A multistep targeting approach and highly individualized, risk-stratified combinatory treatment plans will thus be essential for sustained efficacy and improved overall quality of life in EB.

Profiling trial burden and patients' attitudes to improve clinical research in epidermolysis bullosa

Background

Epidermolysis bullosa (EB) comprises inherited mechanobullous dermatoses with considerable morbidity and mortality. While current treatments are symptomatic, a growing number of innovative therapeutic compounds are evaluated in clinical trials. Clinical research in rare diseases like EB, however, faces many challenges, including sample size requirements and recruitment failures. The objective of this study was to determine attitudes of EB patients towards clinical research and trial participation as well as the assessment of contextual motivating and discouraging factors in an effort to support patient-centered RD trial designing.

Methods

A 53-items questionnaire was handed over to EB patients (of all types and ages) in contact with the EB House Austria, a designated national center of expertise for EB care. Main categories included level of interest in and personal knowledge about clinical studies, pros/cons for participation and extent of individual expenses considered acceptable for participation in a clinical study. Descriptive subgroup analysis was calculated with SPSS 20.0 and Microsoft Excel.

Results

Thirty-six individuals (mean age 25.7 years), diagnosed for recessive dystrophic EB (36.1%), EB simplex (33.4%), junctional EB (8.3%), dominant dystrophic EB (2.8%) and acral peeling syndrome (2.8%) participated. Motivation for participation in and the desire to increase personal knowledge about clinical trials were (outmost) high in 57.2 and 66.7%, respectively. Altruism was the major motivating factor, followed by hope that alleviation of the own symptoms can be achieved. The greatest hurdle was travel distance, followed by concerns about possible adverse reactions. Patients diagnosed for severe subgroups (RDEB, JEB) were more impaired by the extent of scheduled invasive investigations and possible adverse reactions of the study medication. Patients with generally milder EB forms and older patients were accepting more frequent outpatient study visits, blood takes, skin biopsies and inpatient admissions in association with trial participation.

Conclusions

This study provides additional indications to better determine and address attitudes towards clinical research among EB patients as well as guidance to improve clinical trial protocols for patient centricity.

The application of multi-criteria decision analysis to inform in resource allocation

Background: There is a perception held by payers that orphan products are expensive. As a result, the current health technology assessment systems might be too restrictive for orphan drugs, therefore potentially denying patients access to life-saving medicines. While price is important, it should be considered in relation to a broader range of disease-related product attributes that are not necessarily considered by many health technology assessment agencies. To overcome these challenges, multi-criteria decision analysis has been proposed as an alternative to evaluate technologies. Methods: A targeted literature review was conducted to identify the most frequently cited attributes in multi-criteria decision analysis (MCDA) in rare diseases. From the leading attributes identified, we developed a multi-criteria decision analysis framework with which to aggregate the orphan drug values. We subsequently reviewed and plotted the relationship between single attributes and the average annual treatment costs for 8 drugs used in the treatment of rare endocrine diseases. The annual treatment costs were based on UK list prices for the average daily dose per patient. Results: The five most frequently mentioned attributes in the literature were as follows: Disease severity, Unmet need (or availability of therapeutic alternatives), Comparative effectiveness or efficacy, Quality of evidence and Safety & tolerability. Results from the multi-criteria decision analysis framework indicate a wide range of average annual per-patients costs for drugs intended for the same diseases, and likewise for diseases with a similar level of Disease severity. Conclusions: Multi-criteria decision analysis may offer a viable alternative to support discussion in reimbursement decisions for orphan drugs. The analyses can be used to inform investigations on the application of MCDAs in rare diseases.

Integrating molecular nuclear imaging in clinical research to improve anticancer therapy

Effective patient selection before or early during treatment is important to increasing the therapeutic benefits of anticancer treatments. This selection process is often predicated on biomarkers, predominantly biospecimen biomarkers derived from blood or tumour tissue; however, such biomarkers provide limited information about the true extent of disease or about the characteristics of different, potentially heterogeneous tumours present in an individual patient. Molecular imaging can also produce quantitative outputs; such imaging biomarkers can help to fill these knowledge gaps by providing complementary information on tumour characteristics, including heterogeneity and the microenvironment, as well as on pharmacokinetic parameters, drug-target engagement and responses to treatment. This integrative approach could therefore streamline biomarker and drug development, although a range of issues need to be overcome in order to enable a broader use of molecular imaging in clinical trials. In this Perspective article, we outline the multistage process of developing novel molecular imaging biomarkers. We discuss the challenges that have restricted the use of molecular imaging in clinical oncology research to date and outline future opportunities in this area.

Inherited Retinal Disorders: Using Evidence as a Driver for Implementation

Incremental advances in the field of retinal genetics have transformed our understanding of inherited retinal disorders and have led to the development of powerful diagnostic tests and promising gene-based therapies. Despite this, successful integration of these developments into routine healthcare is frequently ineffective. Providing robust evidence of benefit can accelerate the implementation of clinical genetic interventions. For example, the adoption of a genetic test is much more likely when the test's clinical utility (i.e. its ability to influence management and health outcomes) has been clearly demonstrated. However, accruing such evidence for rare conditions like inherited retinal disorders is challenging. Conducting sufficiently powered studies requires both efficient study designs and large-scale, international collaboration. Reaching all populations and as many affected individuals as possible is key. Equally important are efforts to precisely and consistently capture phenotypic information, including natural history data. This article summarizes some of the current obstacles to implemen-tation and discusses approaches to overcome these barriers.

Recommendations for the design of therapeutic trials for neonatal seizures

Although seizures have a higher incidence in neonates than any other age group and are associated with significant mortality and neurodevelopmental disability, treatment is largely guided by physician preference and tradition, due to a lack of data from well-designed clinical trials. There is increasing interest in conducting trials of novel drugs to treat neonatal seizures, but the unique characteristics of this disorder and patient population require special consideration with regard to trial design. The Critical Path Institute formed a global working group of experts and key stakeholders from academia, the pharmaceutical industry, regulatory agencies, neonatal nurse associations, and patient advocacy groups to develop consensus recommendations for design of clinical trials to treat neonatal seizures. The broad expertise and perspectives of this group were invaluable in developing recommendations addressing: (1) use of neonate-specific adaptive trial designs, (2) inclusion/exclusion criteria, (3) stratification and randomization, (4) statistical analysis, (5) safety monitoring, and (6) definitions of important outcomes. The guidelines are based on available literature and expert consensus, pharmacokinetic analyses, ethical considerations, and parental concerns. These recommendations will ultimately facilitate development of a Master Protocol and design of efficient and successful drug trials to improve the treatment and outcome for this highly vulnerable population.

Regulatory strategies for rare diseases under current global regulatory statutes: a discussion with stakeholders

Rare or orphan diseases often are inherited and overwhelmingly affect children. Many of these diseases have no treatments, are incurable, and have a devastating impact on patients and their families. Regulatory standards for drug approval for rare diseases must ensure that patients receive safe and efficacious treatments. However, regulatory bodies have shown flexibility in applying these standards to drug development in rare diseases, given the unique challenges that hinder efficient and effective traditional clinical trials, including low patient numbers, limited understanding of disease pathology and progression, variability in disease presentation, and a lack of established endpoints.

To take steps toward improving rare disease clinical development strategies under current global regulatory statutes, Amicus Therapeutics, Inc. and BioNJ convened a 1-day meeting that included representatives from the Food and Drug Administration (FDA), biopharmaceutical industry, and not-for-profit agencies. The meeting focused on orphan diseases in pediatric and adult patients and was intended to identify potential strategies to overcome regulatory hurdles through open collaboration.

During this meeting, several strategies were identified to minimize the limitations associated with low patient numbers in rare diseases, including the use of natural history to generate historical control data in comparisons, simulations, and identifying inclusion/exclusion criteria and appropriate endpoints. Novel approaches to clinical trial design were discussed to minimize patient exposure to placebo and to reduce the numbers of patients and clinical trials needed for providing substantial evidence. Novel statistical analysis approaches were also discussed to address the inherent challenges of small patient numbers. Areas of urgent unmet need were identified, including the need to develop registries that protect patient identities, to establish close collaboration and communication between the sponsor and regulatory bodies to address methodological and statistical challenges, to collaborate in pre-competitive opportunities within multiple sponsors and in conjunction with academia and disease-specific patient advocacy groups for optimal data sharing, and to develop harmonized guidelines for data extrapolation from source to target pediatric populations. Ultimately, these innovations will help in solving many regulatory challenges in rare disease drug development and encourage the availability of new treatments for patients with rare diseases.

The patient's view on rare disease trial design - a qualitative study

BACKGROUND

Clinical trials in rare diseases are more challenging than trials in frequent diseases. Small numbers of eligible trial participants, often complicated by heterogeneity among rare disease patients, hamper the design and conduct of a 'classical' Randomized Controlled Trial. Therefore, novel designs are developed by statisticians. However, it is important to be aware of possible design aspects that may jeopardize the feasibility of trial conduct. If the burden of participation is considered out of proportion by patients or parents, recruitment may fail or participants may drop out before trial completion. In order to maximize the chance of success of trials in small populations, it is important to know which aspects of trial design are considered important by patients.

RESULTS

We have interviewed all ten members of the Patient Think Tank (PTT) of the ASTERIX project, a European research consortium on methodology for clinical trials in small populations. The PTT members are rare disease patient representatives who have completed extensive training in clinical trial methodology. We have analyzed the interviews qualitatively according to Grounded Theory using a thematic analysis, and we structured the topics in four chronologically ordered themes: 1. Involvement in trial design; 2. Opinions on trial design; 3. Trial participation; 4. Phase after the trial. Our main findings are that the PTT-members recommend that patients are involved in trial design from an early stage on, and have influence on the outcomes and measurement instruments that are chosen in the trial, the length of the study, the choice of participants, and the information that is sent to potential participants. Also, according to the PTT-members, patient groups should consider setting up disease registries, placebo groups should be minimized, and more education on clinical trials is advised.

CONCLUSIONS

Rare disease patient representatives who have been educated about clinical trial methodology think it is important to involve patient representatives in research at an early stage. They can be of advice in trial design in such a way that the ratio of potential benefit and burden of trial participation as well as the chosen outcome measures and in- and exclusion criteria are optimized.

Evidence supporting regulatory-decision making on orphan medicinal products authorisation in Europe: methodological uncertainties

BACKGROUND

To assess uncertainty in regulatory decision-making for orphan medicinal products (OMP), a summary of the current basis for approval is required; a systematic grouping of medical conditions may be useful in summarizing information and issuing recommendations for practice.

METHODS

A grouping of medical conditions with similar characteristics regarding the potential applicability of methods and designs was created using a consensus approach. The 125 dossiers for authorised OMP published between 1999 and 2014 on the EMA webpage were grouped accordingly and data was extracted from European Public Assessment Reports (EPARs) to assess the extent and robustness of the pivotal evidence supporting regulatory decisions.

RESULTS

88% (110/125) of OMP authorizations were based on clinical trials, with 35% (38/110) including replicated pivotal trials. The mean (SD) number of pivotal trials per indication was 1.4 (0.7), and the EPARs included a median of three additional non-pivotal supportive studies. 10% of OMPs (13/125) were authorised despite only negative pivotal trials. One-third of trials (53/159) did not include a control arm, one-third (50/159) did not use randomisation, half the trials (75/159) were open-label and 75% (119/159) used intermediate or surrogate variables as the main outcome. Chronic progressive conditions led by multiple system/organs, conditions with single acute episodes and progressive conditions led by one organ/system were the groups where the evidence deviated most from conventional standards. Conditions with recurrent acute episodes had the most robust datasets. The overall size of the exposed population at the time of authorisation of OMP - mean(SD) 190.5 (202.5) - was lower than that required for the qualification of clinically-relevant adverse reactions.

CONCLUSIONS

The regulatory evidence supporting OMP authorization showed substantial uncertainties, including weak protection against errors, substantial use of designs unsuited for conclusions on causality, use of intermediate variables, lack of a priorism and insufficient safety data to quantify risks of relevant magnitude. Grouping medical conditions based on clinical features and their methodological requirements may facilitate specific methodological and regulatory recommendations for the study of OMP to strengthen the evidence base.

Recommendations for the design of small population clinical trials

Background

Orphan drug development faces numerous challenges, including low disease prevalence, patient population heterogeneity, and strong presence of paediatric patient populations. Consequently, clinical trials for orphan drugs are often smaller than those of non-orphan drugs, and they require the development of efficient trial designs relevant to small populations to gain the most information from the available data. The International Rare Diseases Research Consortium (IRDiRC) is aimed at promoting international collaboration and advance rare diseases research worldwide, and has as one of its goals to contribute to 1000 new therapies for rare diseases. IRDiRC set up a Small Population Clinical Trials (SPCT) Task Force in order to address the shortcomings of our understanding in carrying out clinical trials in rare diseases.

Results

The IRDiRC SPCT Task Force met in March 2016 to discuss challenges faced in the design of small studies for rare diseases and present their recommendations, structured around six topics: different study methods/designs and their relation to different characteristics of medical conditions, adequate safety data, multi-arm trial designs, decision analytic approaches and rational approaches to adjusting levels of evidence, extrapolation, and patients’ engagement in study design.

Conclusions

Recommendations have been issued based on discussions of the Small Population Clinical Trials Task Force that aim to contribute towards successful therapy development and clinical use. While randomised clinical trials are still considered the gold standard, it is recommended to systematically take into consideration alternative trial design options when studying treatments for a rare disease. Combining different sources of safety data is important to give a fuller picture of a therapy’s safety profile. Multi-arm trials should be considered an opportunity for rare diseases therapy development, and funders are encouraged to support such trial design via international networks. Patient engagement is critical in trial design and therapy development, a process which sponsors are encouraged to incorporate when conducting trials and clinical studies. Input from multiple regulatory agencies is recommended early and throughout clinical development. Regulators are often supportive of new clinical trial designs, provided they are well thought through and justified, and they also welcome discussions and questions on this topic. Parallel advice for multiregional development programs should also be considered.

Exploring how non-inferiority and equivalence are assessed in paediatrics: a systematic review

OBJECTIVE

To review characteristics, methodology and reporting of non-inferiority and equivalence trials in the specific context of paediatrics.

DESIGN

PubMed and Cochrane databases were searched (up to September 2016) for non-inferiority/equivalence randomised controlled trials conducted in children published in high-impact-factor journals (>5.0 for general/specialist medical journals; >2.2 for paediatric journals).

RESULTS

We found that the statistical hypothesis was inconsistent with the objective in 12 (10%) of the 125 reports included. Non-inferiority (n=98) and equivalence trials (n=27) were mostly used to evaluate interventions with easier administration (45%, n=54/120) and/or better safety profile (34%, n=41/120). All the data needed for targeted sample size recalculation were available for 39 reports (31%). The margin-representing the largest difference between arms that would be clinically acceptable-was reported in 119 (95%), and 44/119 (37%) reported the method used for margin determination. The median sample size was 268 (IQR 125-531). Margins were wider in smaller trials (<125 randomised patients) than in larger trials (p=0.04/p<0.01 for binary/continuous outcomes, respectively). We did not agree with the authors' conclusions in 11% (11/103) of the reports that provided sufficient information.

CONCLUSIONS

There is still a need to improve the quality of methodology, reporting and interpretation of non-inferiority/equivalence trials in paediatrics. In particular, the margins were often not justified and the conclusion was often not supported by the design and/or the results. As researchers have to cope with small sample size and with lack of evidence, methods for non-inferiority/equivalence trials need to be used and/or developed in this vulnerable population.

Recent advances in methodology for clinical trials in small populations: the InSPiRe project

Where there are a limited number of patients, such as in a rare disease, clinical trials in these small populations present several challenges, including statistical issues. This led to an EU FP7 call for proposals in 2013. One of the three projects funded was the Innovative Methodology for Small Populations Research (InSPiRe) project. This paper summarizes the main results of the project, which was completed in 2017.The InSPiRe project has led to development of novel statistical methodology for clinical trials in small populations in four areas. We have explored new decision-making methods for small population clinical trials using a Bayesian decision-theoretic framework to compare costs with potential benefits, developed approaches for targeted treatment trials, enabling simultaneous identification of subgroups and confirmation of treatment effect for these patients, worked on early phase clinical trial design and on extrapolation from adult to pediatric studies, developing methods to enable use of pharmacokinetics and pharmacodynamics data, and also developed improved robust meta-analysis methods for a small number of trials to support the planning, analysis and interpretation of a trial as well as enabling extrapolation between patient groups. In addition to scientific publications, we have contributed to regulatory guidance and produced free software in order to facilitate implementation of the novel methods.

Rare disease registries: potential applications towards impact on development of new drug treatments

BACKGROUND

Low prevalence, lack of knowledge about the disease course, and phenotype heterogeneity hamper the development of drugs for rare diseases. Rare disease registries (RDRs) can be helpful by playing a role in understanding the course of the disease, and providing information necessary for clinical trial design, if designed and maintained properly. We describe the potential applications of a RDR and what type of information should be incorporated to support the design of clinical trials in the process of drug development, based on a broad inventory of registry experience. We evaluated two existing RDRs in more detail to check the completeness of these RDRs for trial design.

RESULTS

Before and during the application for regulatory approval a RDR can improve the efficiency and quality in clinical trial design by informing the sample size calculation and expected disease course. In exceptional circumstances information from RDRs has been used as historical controls for a one-armed clinical trial, and high quality RDRs may be used for registry-based randomized controlled trials. In the post marketing phase of (conditional) drug approval a disease-specific RDR is likely to provide more relevant information than a product-specific registry.

CONCLUSIONS

A RDR can be very helpful to improve the efficiency and quality of clinical trial design in several ways. To enable the applicability and optimal use of a RDR longitudinal data collection is indispensable, and specific data collection, prepared for repeated measurement, is needed. The developed checklist can help to define the appropriate variables to include. Attention should be paid to the inclusion of patient-relevant outcome measures in the RDR from the start. More research and experience is needed on the possibilities and limitations of combining RDR information with clinical trial data to maximize the availability of relevant evidence for regulatory decisions in rare diseases.

A proposal for a new PhD level curriculum on quantitative methods for drug development

This paper provides an overview of "Improving Design, Evaluation and Analysis of early drug development Studies" (IDEAS), a European Commission-funded network bringing together leading academic institutions and small- to large-sized pharmaceutical companies to train a cohort of graduate-level medical statisticians. The network is composed of a diverse mix of public and private sector partners spread across Europe, which will host 14 early-stage researchers for 36 months. IDEAS training activities are composed of a well-rounded mixture of specialist methodological components and generic transferable skills. Particular attention is paid to fostering collaborations between researchers and supervisors, which span academia and the private sector. Within this paper, we review existing medical statistics programmes (MSc and PhD) and highlight the training they provide on skills relevant to drug development. Motivated by this review and our experiences with the IDEAS project, we propose a concept for a joint, harmonised European PhD programme to train statisticians in quantitative methods for drug development.

Physical Therapy and Exercise Interventions in Huntington's Disease: A Mixed Methods Systematic Review

Background:

A number of studies evaluating physical therapy and exercise interventions in Huntington’s disease have been conducted over the past 15 years. However, an assessment of the quality and strength of the evidence in support of these interventions is lacking.

Objective:

The purpose of this systematic review was to investigate the effectiveness of physical therapy and exercise interventions in people with Huntington’s disease, and to examine the perceptions of patients, families and caregivers of these interventions.

Methods:

This mixed-methods systematic review utilized the Joanna Briggs Institute (JBI) approach and extraction tools to evaluate the literature from January 2003 until May 2016. The review considered interventions that included exercise and physical therapy interventions, and included both quantitative and qualitative outcome measures.

Results:

Twenty (20) studies met the inclusion criteria, including eighteen (18) that had quantitative outcome measures and two (2) that utilized qualitative methods. JBI Levels of evidence for the 18 quantitative studies were as follows: Eight studies were at evidence Level 1, seven were at Level 2, two were at Level 3, and one was at Level 4.

Conclusions:

Our review suggests that there is preliminary support for the benefits of exercise and physical activity in Huntington’s disease in terms of motor function, gait speed, and balance, as well as a range of physical and social benefits identified through patient-reported outcomes. Variability in mode of intervention as well as outcome measures limits the interpretability of these studies, and high-quality studies that incorporate adaptive trial designs for this rare disease are needed.

Improving the analysis of composite endpoints in rare disease trials

BACKGROUND

Composite endpoints are recommended in rare diseases to increase power and/or to sufficiently capture complexity. Often, they are in the form of responder indices which contain a mixture of continuous and binary components. Analyses of these outcomes typically treat them as binary, thus only using the dichotomisations of continuous components. The augmented binary method offers a more efficient alternative and is therefore especially useful for rare diseases. Previous work has indicated the method may have poorer statistical properties when the sample size is small. Here we investigate small sample properties and implement small sample corrections.

METHODS

We re-sample from a previous trial with sample sizes varying from 30 to 80. We apply the standard binary and augmented binary methods and determine the power, type I error rate, coverage and average confidence interval width for each of the estimators. We implement Firth's adjustment for the binary component models and a small sample variance correction for the generalized estimating equations, applying the small sample adjusted methods to each sub-sample as before for comparison.

RESULTS

For the log-odds treatment effect the power of the augmented binary method is 20-55% compared to 12-20% for the standard binary method. Both methods have approximately nominal type I error rates. The difference in response probabilities exhibit similar power but both unadjusted methods demonstrate type I error rates of 6-8%. The small sample corrected methods have approximately nominal type I error rates. On both scales, the reduction in average confidence interval width when using the adjusted augmented binary method is 17-18%. This is equivalent to requiring a 32% smaller sample size to achieve the same statistical power.

CONCLUSIONS

The augmented binary method with small sample corrections provides a substantial improvement for rare disease trials using composite endpoints. We recommend the use of the method for the primary analysis in relevant rare disease trials. We emphasise that the method should be used alongside other efforts in improving the quality of evidence generated from rare disease trials rather than replace them.

Lessons learned from IDeAl - 33 recommendations from the IDeAl-net about design and analysis of small population clinical trials

BACKGROUND

IDeAl (Integrated designs and analysis of small population clinical trials) is an EU funded project developing new statistical design and analysis methodologies for clinical trials in small population groups. Here we provide an overview of IDeAl findings and give recommendations to applied researchers.

METHOD

The description of the findings is broken down by the nine scientific IDeAl work packages and summarizes results from the project's more than 60 publications to date in peer reviewed journals. In addition, we applied text mining to evaluate the publications and the IDeAl work packages' output in relation to the design and analysis terms derived from in the IRDiRC task force report on small population clinical trials.

RESULTS

The results are summarized, describing the developments from an applied viewpoint. The main result presented here are 33 practical recommendations drawn from the work, giving researchers a comprehensive guidance to the improved methodology. In particular, the findings will help design and analyse efficient clinical trials in rare diseases with limited number of patients available. We developed a network representation relating the hot topics developed by the IRDiRC task force on small population clinical trials to IDeAl's work as well as relating important methodologies by IDeAl's definition necessary to consider in design and analysis of small-population clinical trials. These network representation establish a new perspective on design and analysis of small-population clinical trials.

CONCLUSION

IDeAl has provided a huge number of options to refine the statistical methodology for small-population clinical trials from various perspectives. A total of 33 recommendations developed and related to the work packages help the researcher to design small population clinical trial. The route to improvements is displayed in IDeAl-network representing important statistical methodological skills necessary to design and analysis of small-population clinical trials. The methods are ready for use.

In silico clinical trials for pediatric orphan diseases

To date poor treatment options are available for patients with congenital pseudarthrosis of the tibia (CPT), a pediatric orphan disease. In this study we have performed an in silico clinical trial on 200 virtual subjects, generated from a previously established model of murine bone regeneration, to tackle the challenges associated with the small, pediatric patient population. Each virtual subject was simulated to receive no treatment and bone morphogenetic protein (BMP) treatment. We have shown that the degree of severity of CPT is significantly reduced with BMP treatment, although the effect is highly subject-specific. Using machine learning techniques we were also able to stratify the virtual subject population in adverse responders, non-responders, responders and asymptomatic. In summary, this study shows the potential of in silico medicine technologies as well as their implications for other orphan diseases.

A systematic literature review of evidence-based clinical practice for rare diseases: what are the perceived and real barriers for improving the evidence and how can they be overcome?

BACKGROUND

Evidence-based clinical practice is challenging in all fields, but poses special barriers in the field of rare diseases. The present paper summarises the main barriers faced by clinical research in rare diseases, and highlights opportunities for improvement.

METHODS

Systematic literature searches without meta-analyses and internal European Clinical Research Infrastructure Network (ECRIN) communications during face-to-face meetings and telephone conferences from 2013 to 2017 within the context of the ECRIN Integrating Activity (ECRIN-IA) project.

RESULTS

Barriers specific to rare diseases comprise the difficulty to recruit participants because of rarity, scattering of patients, limited knowledge on natural history of diseases, difficulties to achieve accurate diagnosis and identify patients in health information systems, and difficulties choosing clinically relevant outcomes.

CONCLUSIONS

Evidence-based clinical practice for rare diseases should start by collecting clinical data in databases and registries; defining measurable patient-centred outcomes; and selecting appropriate study designs adapted to small study populations. Rare diseases constitute one of the most paradigmatic fields in which multi-stakeholder engagement, especially from patients, is needed for success. Clinical research infrastructures and expertise networks offer opportunities for establishing evidence-based clinical practice within rare diseases.

Demonstrating significant benefit of orphan medicines: analysis of 15 years of experience in Europe

In the European Union demonstration of 'significant benefit' is mandatory if satisfactory methods exist for a disease targeted by a new orphan medicinal product. Significant benefit is required at the time of orphan designation, when it can be supported by preclinical studies, and at the time of marketing authorization, when clinical data are needed. For the first time, our work has identified, defined and organized the scientific grounds on which significant benefit is granted in the European Union, based on a review of the orphan medicinal products authorized in the years 2000-2015, and on the working experience of the Committee of Orphan Medicinal Products. The resulting conceptual framework is a tool for medicine developers to reflect on potential areas of advantage of their candidate products, and for a broad range of stakeholders to stimulate the discussion on the added value of orphan medicines across the whole development lifecycle.

Does the low prevalence affect the sample size of interventional clinical trials of rare diseases? An analysis of data from the aggregate analysis of clinicaltrials.gov

BACKGROUND

Clinical trials are typically designed using the classical frequentist framework to constrain type I and II error rates. Sample sizes required in such designs typically range from hundreds to thousands of patients which can be challenging for rare diseases. It has been shown that rare disease trials have smaller sample sizes than non-rare disease trials. Indeed some orphan drugs were approved by the European Medicines Agency based on studies with as few as 12 patients. However, some studies supporting marketing authorisation included several hundred patients. In this work, we explore the relationship between disease prevalence and other factors and the size of interventional phase 2 and 3 rare disease trials conducted in the US and/or EU. We downloaded all clinical trials from Aggregate Analysis of ClinialTrials.gov (AACT) and identified rare disease trials by cross-referencing MeSH terms in AACT with the list from Orphadata. We examined the effects of prevalence and phase of study in a multiple linear regression model adjusting for other statistically significant trial characteristics.

RESULTS

Of 186941 ClinicalTrials.gov trials only 1567 (0.8%) studied a single rare condition with prevalence information from Orphadata. There were 19 (1.2%) trials studying disease with prevalence <1/1,000,000, 126 (8.0%) trials with 1-9/1,000,000, 791 (50.5%) trials with 1-9/100,000 and 631 (40.3%) trials with 1-5/10,000. Of the 1567 trials, 1160 (74%) were phase 2 trials. The fitted mean sample size for the rarest disease (prevalence <1/1,000,000) in phase 2 trials was the lowest (mean, 15.7; 95% CI, 8.7-28.1) but were similar across all the other prevalence classes; mean, 26.2 (16.1-42.6), 33.8 (22.1-51.7) and 35.6 (23.3-54.3) for prevalence 1-9/1,000,000, 1-9/100,000 and 1-5/10,000, respectively. Fitted mean size of phase 3 trials of rarer diseases, <1/1,000,000 (19.2, 6.9-53.2) and 1-9/1,000,000 (33.1, 18.6-58.9), were similar to those in phase 2 but were statistically significant lower than the slightly less rare diseases, 1-9/100,000 (75.3, 48.2-117.6) and 1-5/10,000 (77.7, 49.6-121.8), trials.

CONCLUSIONS

We found that prevalence was associated with the size of phase 3 trials with trials of rarer diseases noticeably smaller than the less rare diseases trials where phase 3 rarer disease (prevalence <1/100,000) trials were more similar in size to those for phase 2 but were larger than those for phase 2 in the less rare disease (prevalence ≥1/100,000) trials.

A framework: make it useful to guide and improve practice of clinical trial design in smaller populations

The increased attention to design and analysis of randomised clinical trials in small populations has triggered thinking regarding the most appropriate design methods for a particular clinical research question. Decision schemes and algorithms have been proposed, with varying starting points and foci. Parmar et al. (BMC Medicine 14:183, 2016) proposed a framework designed to assist the clinical trial team in design choices during protocol preparation. Herein, further stimulus is given regarding the extent to which a framework may help change practice for the better, the careful considerations for changing the usual error levels applied and the room for innovation in clinical trial design.Please see related article: http://bmcmedicine.biomedcentral.com/articles/10.1186/s12916-016-0722-3 .