Bayesian methods for the design and interpretation of clinical trials in very rare diseases

Sharing information across patient subgroups to draw conclusions from sparse treatment networks

Network meta-analysis (NMA) usually provides estimates of the relative effects with the highest possible precision. However, sparse networks with few available studies and limited direct evidence can arise, threatening the robustness and reliability of NMA estimates. In these cases, the limited amount of available information can hamper the formal evaluation of the underlying NMA assumptions of transitivity and consistency. In addition, NMA estimates from sparse networks are expected to be imprecise and possibly biased as they rely on large-sample approximations that are invalid in the absence of sufficient data. We propose a Bayesian framework that allows sharing of information between two networks that pertain to different population subgroups. Specifically, we use the results from a subgroup with a lot of direct evidence (a dense network) to construct informative priors for the relative effects in the target subgroup (a sparse network). This is a two-stage approach where at the first stage, we extrapolate the results of the dense network to those expected from the sparse network. This takes place by using a modified hierarchical NMA model where we add a location parameter that shifts the distribution of the relative effects to make them applicable to the target population. At the second stage, these extrapolated results are used as prior information for the sparse network. We illustrate our approach through a motivating example of psychiatric patients. Our approach results in more precise and robust estimates of the relative effects and can adequately inform clinical practice in presence of sparse networks.

Estimating the effect of nintedanib on forced vital capacity in children and adolescents with fibrosing interstitial lung disease using a Bayesian dynamic borrowing approach

BACKGROUND

The rarity of childhood interstitial lung disease (chILD) makes it challenging to conduct powered trials. In the InPedILD trial, among 39 children and adolescents with fibrosing ILD, there was a numerical benefit of nintedanib versus placebo on change in forced vital capacity (FVC) over 24 weeks (difference in mean change in FVC % predicted of 1.21 [95% confidence interval: -3.40, 5.81]). Nintedanib has shown a consistent effect on FVC across populations of adults with different diagnoses of fibrosing ILD.

METHODS

In a Bayesian dynamic borrowing analysis, prespecified before data unblinding, we incorporated data on the effect of nintedanib in adults and the data from the InPedILD trial to estimate the effect of nintedanib on FVC in children and adolescents with fibrosing ILD. The data from adults were represented as a meta-analytic predictive (MAP) prior distribution with mean 1.69 (95% credible interval: 0.49, 3.08). The adult data were weighted according to expert judgment on their relevance to the efficacy of nintedanib in chILD, obtained in a formal elicitation exercise.

RESULTS

Combined data from the MAP prior and InPedILD trial analyzed within the Bayesian framework resulted in a median difference between nintedanib and placebo in change in FVC % predicted at Week 24 of 1.63 (95% credible interval: -0.69, 3.40). The posterior probability for superiority of nintedanib versus placebo was 95.5%, reaching the predefined success criterion of at least 90%.

CONCLUSION

These findings, together with the safety data from the InPedILD trial, support the use of nintedanib in children and adolescents with fibrosing ILDs.

Bayesian sample size determination using commensurate priors to leverage preexperimental data

This paper develops Bayesian sample size formulae for experiments comparing two groups, where relevant preexperimental information from multiple sources can be incorporated in a robust prior to support both the design and analysis. We use commensurate predictive priors for borrowing of information and further place Gamma mixture priors on the precisions to account for preliminary belief about the pairwise (in)commensurability between parameters that underpin the historical and new experiments. Averaged over the probability space of the new experimental data, appropriate sample sizes are found according to criteria that control certain aspects of the posterior distribution, such as the coverage probability or length of a defined density region. Our Bayesian methodology can be applied to circumstances that compare two normal means, proportions, or event times. When nuisance parameters (such as variance) in the new experiment are unknown, a prior distribution can further be specified based on preexperimental data. Exact solutions are available based on most of the criteria considered for Bayesian sample size determination, while a search procedure is described in cases for which there are no closed-form expressions. We illustrate the application of our sample size formulae in the design of clinical trials, where pretrial information is available to be leveraged. Hypothetical data examples, motivated by a rare-disease trial with an elicited expert prior opinion, and a comprehensive performance evaluation of the proposed methodology are presented.

Practical approaches to Bayesian sample size determination in non-inferiority trials with binary outcomes

Bayesian analysis of a non-inferiority trial is advantageous in allowing direct probability statements to be made about the relative treatment difference rather than relying on an arbitrary and often poorly justified non-inferiority margin. When the primary analysis will be Bayesian, a Bayesian approach to sample size determination will often be appropriate for consistency with the analysis. We demonstrate three Bayesian approaches to choosing sample size for non-inferiority trials with binary outcomes and review their advantages and disadvantages. First, we present a predictive power approach for determining sample size using the probability that the trial will produce a convincing result in the final analysis. Next, we determine sample size by considering the expected posterior probability of non-inferiority in the trial. Finally, we demonstrate a precision-based approach. We apply these methods to a non-inferiority trial in antiretroviral therapy for treatment of HIV-infected children. A predictive power approach would be most accessible in practical settings, because it is analogous to the standard frequentist approach. Sample sizes are larger than with frequentist calculations unless an informative analysis prior is specified, because appropriate allowance is made for uncertainty in the assumed design parameters, ignored in frequentist calculations. An expected posterior probability approach will lead to a smaller sample size and is appropriate when the focus is on estimating posterior probability rather than on testing. A precision-based approach would be useful when sample size is restricted by limits on recruitment or costs, but it would be difficult to decide on sample size using this approach alone.

Modern Statistical Methods for the Surgeon Scientist: The Clash of Frequentist versus Bayesian Paradigms

The practice of evidence-based medicine is the result of a multitude of research and trials aimed toward improving health-care outcomes. An understanding of the associated data remains paramount toward optimizing patient outcomes. Medical statistics commonly revolve around frequentist concepts that are convoluted and nonintuitive for nonstatisticians. Within this article, we will discuss frequentist statistics, their limitations, as well as introduce Bayesian statistics as an alternative approach for data interpretation. By doing so, we intend to highlight the importance of correct statistical interpretations through clinically relevant examples while providing a deeper understanding of the underlying philosophies of frequentist and Bayesian statistics.

Informative g-Priors for Mixed Models

Zellner’s objective g-prior has been widely used in linear regression models due to its simple interpretation and computational tractability in evaluating marginal likelihoods. However, the g-prior further allows portioning the prior variability explained by the linear predictor versus that of pure noise. In this paper, we propose a novel yet remarkably simple g-prior specification when a subject matter expert has information on the marginal distribution of the response yi. The approach is extended for use in mixed models with some surprising but intuitive results. Simulation studies are conducted to compare the model fitting under the proposed g-prior with that under other existing priors.

Vaccine clinical trials with dynamic borrowing of historical controls: Two retrospective studies

Traditional vaccine efficacy trials usually use fixed designs with fairly large sample sizes. Recruiting a large number of subjects requires longer time and higher costs. Furthermore, vaccine developers are more than ever facing the need to accelerate vaccine development to fulfill the public's medical needs. A possible approach to accelerate development is to use the method of dynamic borrowing of historical controls in clinical trials. In this paper, we evaluate the feasibility and the performance of this approach in vaccine development by retrospectively analyzing two real vaccine studies: a relatively small immunological trial (typical early phase study) and a large vaccine efficacy trial (typical Phase 3 study) assessing prophylactic human papillomavirus vaccine. Results are promising, particularly for early development immunological studies, where the adaptive design is feasible, and control of type I error is less relevant.

A Bayesian Sample Size Estimation Procedure Based on a B-Splines Semiparametric Elicitation Method

Sample size estimation is a fundamental element of a clinical trial, and a binomial experiment is the most common situation faced in clinical trial design. A Bayesian method to determine sample size is an alternative solution to a frequentist design, especially for studies conducted on small sample sizes. The Bayesian approach uses the available knowledge, which is translated into a prior distribution, instead of a point estimate, to perform the final inference. This procedure takes the uncertainty in data prediction entirely into account. When objective data, historical information, and literature data are not available, it may be indispensable to use expert opinion to derive the prior distribution by performing an elicitation process. Expert elicitation is the process of translating expert opinion into a prior probability distribution. We investigated the estimation of a binomial sample size providing a generalized version of the average length, coverage criteria, and worst outcome criterion. The original method was proposed by Joseph and is defined in a parametric framework based on a Beta-Binomial model. We propose a more flexible approach for binary data sample size estimation in this theoretical setting by considering parametric approaches (Beta priors) and semiparametric priors based on B-splines.

Informed Bayesian survival analysis

BACKGROUND

We provide an overview of Bayesian estimation, hypothesis testing, and model-averaging and illustrate how they benefit parametric survival analysis. We contrast the Bayesian framework to the currently dominant frequentist approach and highlight advantages, such as seamless incorporation of historical data, continuous monitoring of evidence, and incorporating uncertainty about the true data generating process.

METHODS

We illustrate the application of the outlined Bayesian approaches on an example data set, retrospective re-analyzing a colon cancer trial. We assess the performance of Bayesian parametric survival analysis and maximum likelihood survival models with AIC/BIC model selection in fixed-n and sequential designs with a simulation study.

RESULTS

In the retrospective re-analysis of the example data set, the Bayesian framework provided evidence for the absence of a positive treatment effect of adding Cetuximab to FOLFOX6 regimen on disease-free survival in patients with resected stage III colon cancer. Furthermore, the Bayesian sequential analysis would have terminated the trial 10.3 months earlier than the standard frequentist analysis. In a simulation study with sequential designs, the Bayesian framework on average reached a decision in almost half the time required by the frequentist counterparts, while maintaining the same power, and an appropriate false-positive rate. Under model misspecification, the Bayesian framework resulted in higher false-negative rate compared to the frequentist counterparts, which resulted in a higher proportion of undecided trials. In fixed-n designs, the Bayesian framework showed slightly higher power, slightly elevated error rates, and lower bias and RMSE when estimating treatment effects in small samples. We found no noticeable differences for survival predictions. We have made the analytic approach readily available to other researchers in the RoBSA R package.

CONCLUSIONS

The outlined Bayesian framework provides several benefits when applied to parametric survival analyses. It uses data more efficiently, is capable of considerably shortening the length of clinical trials, and provides a richer set of inferences.

Novel clinical trial design and analytic methods to tackle challenges in therapeutic development in rare diseases

While only a fraction of the worldwide population may have a particular rare disorder, millions of people worldwide are affected across the over 6,000 rare disorders and do not have a safe and effective approved therapy to help them live or manage complications from the disorder. Challenges to clinical development of new therapies in rare disorders include difficulty in powering and recruiting into a study in small and often heterogenous population, scarcity of natural history data informing critical design elements such as endpoint selection and study duration, and ethical and recruitment challenges in randomizing patients to a placebo arm. In this review, we describe some existing and novel strategies to tackle these challenges, by efficient utilization of available resources. We discuss the role of natural history studies and endpoint selection as they remain critical features that apply across designs and disorders. We also review some novel clinical trial designs including incorporating external control and/or longitudinal measures, master protocol designs, and adaptive designs. Additionally, we review some analytic strategies that are often associated with these designs, such as the use of causal inference methods, and Bayesian methods. We hope this review will raise awareness of these novel approaches and encourage their use in studies of rare diseases.

Efficacy of Intravenous Elosulfase Alfa for Mucopolysaccharidosis Type IVA: A Systematic Review and Meta-Analysis

Mucopolysaccharidosis type IVA (MPS IVA or Morquio A), a lysosomal storage disease with an autosomal recessive inherited pattern, is induced by GALNS gene mutations causing deficiency in N-acetylgalactosamine-6-sulfatase activity (GALNS; EC 3.1.6.4). Currently, intravenous (IV) enzyme replacement therapy (ERT) with elosulfase alfa is employed for treating MPS IVA patients. A systematic literature review was conducted to evaluate the efficacy and safety of IV elosulfase alfa for MPS IVA by searching the National Center for Biotechnology Information, U.S. National Library of Medicine National Institutes of Health (PubMed), Excerpta Medica dataBASE, and Cochrane Library databases, limited to clinical trials. Four cohort studies and two randomized controlled trials, with a total of 550 participants (327 on ERT treatment versus 223 on placebo treatment), satisfied the inclusion criteria. Pooled analysis of proportions and confidence intervals were also utilized to systematically review clinical cohort studies and trials. Per the pooled proportions analysis, the difference in means of urinary keratan sulfate (uKS), 6-min walk test, 3-min stair climb test, self-care MPS-Health Assessment Questionnaire, caregiver assistance and mobility, forced vital capacity, the first second of forced expiration, and maximal voluntary ventilation between the ERT and placebo treatment groups were -0.260, -0.102, -0.182, -0.360, -0.408, -0.587, -0.293, -0.311, and -0.213, respectively. Based on the currently available data, our meta-analysis showed that there is uKS, physical performance, quality of life, and respiratory function improvements with ERT in MPS IVA patients. It is optimal to start ERT after diagnosis.

Optimal sample size determination for single-arm trials in pediatric and rare populations with Bayesian borrowing

In many therapeutic areas with unmet medical needs, such as pediatric oncology and rare diseases, one of the deterrent factors for clinical trial interpretability is the limited sample size with less-than-ideal operating characteristics. Single arm is usually the only viable design due to feasibility and ethical concerns. For the trial results to be more interpretable and conclusive, the evaluation of operating characteristics, such as type I error rate and power, and the appropriate utilization of prior information for study design, shall be prespecified and fully investigated during the trial planning phase. So far, very few existing literature addressed optimal sample size determination issues for the planning of pediatric and rare population trials, with majority of research focusing on analysis perspective with focus on Bayesian borrowing. In practice, when a single-arm trial is designed for rare population, it is not uncommon that the only information available is from an earlier trial and/or a few clinical publications based on observational studies, often constituting mixed or uncertain conclusions. In light of this, an optimal Bayesian sample size determination method for single-arm trial with binary or continuous endpoint is proposed, where conflicting prior beliefs can be readily incorporated. Prior effective sample size can be calculated to assess the robustness as well as the prior information borrowed. Moreover, due to the lack of closed-form posterior distributions in general, an alternative approach for calculating Bayesian power is described. Simulation studies are provided to demonstrate the utility of the proposed methods. In addition, a case study in pediatric patients with leukemia is included to illustrate the proposed method with the existing approaches.

Application of Bayesian methods to accelerate rare disease drug development: scopes and hurdles

Background

Design and analysis of clinical trials for rare and ultra-rare disease pose unique challenges to the practitioners. Meeting conventional power requirements is infeasible for diseases where sample sizes are inherently very small. Moreover, rare disease populations are generally heterogeneous and widely dispersed, which complicates study enrollment and design. Leveraging all available information in rare and ultra-rare disease trials can improve both drug development and informed decision-making processes.

Main text

Bayesian statistics provides a formal framework for combining all relevant information at all stages of the clinical trial, including trial design, execution, and analysis. This manuscript provides an overview of different Bayesian methods applicable to clinical trials in rare disease. We present real or hypothetical case studies that address the key needs of rare disease drug development highlighting several specific Bayesian examples of clinical trials. Advantages and hurdles of these approaches are discussed in detail. In addition, we emphasize the practical and regulatory aspects in the context of real-life applications.

Conclusion

The use of innovative trial designs such as master protocols and complex adaptive designs in conjunction with a Bayesian approach may help to reduce sample size, select the correct treatment and population, and accurately and reliably assess the treatment effect in the rare disease setting.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13023-022-02342-5.

Bayesian sample size determination using commensurate priors to leverage pre-experimental data

This paper develops Bayesian sample size formulae for experiments comparing two groups, where relevant pre-experimental information from multiple sources can be incorporated in a robust prior to support both the design and analysis. We use commensurate predictive priors for borrowing of information, and further place Gamma mixture priors on the precisions to account for preliminary belief about the pairwise (in)commensurability between parameters that underpin the historical and new experiments. Averaged over the probability space of the new experimental data, appropriate sample sizes are found according to criteria that control certain aspects of the posterior distribution, such as the coverage probability or length of a defined density region. Our Bayesian methodology can be applied to circumstances that compare two normal means, proportions or event times. When nuisance parameters (such as variance) in the new experiment are unknown, a prior distribution can further be specified based on pre-experimental data. Exact solutions are available based on most of the criteria considered for Bayesian sample size determination, while a search procedure is described in cases for which there are no closed-form expressions. We illustrate the application of our sample size formulae in the design of clinical trials, where pre-trial information is available to be leveraged. Hypothetical data examples, motivated by a rare-disease trial with elicited expert prior opinion, and a comprehensive performance evaluation of the proposed methodology are presented.

A stochastically curtailed single‐arm phase II trial design for binary outcomes

Phase II clinical trials are a critical aspect of the drug development process. With drug development costs ever increasing, novel designs that can improve the efficiency of phase II trials are extremely valuable.Phase II clinical trials for cancer treatments often measure a binary outcome. The final trial decision is generally to continue or cease development. When this decision is based solely on the result of a hypothesis test, the result may be known with certainty before the planned end of the trial. Unfortunately, there is often no opportunity for early stopping when this occurs.Some existing designs do permit early stopping in this case, accordingly reducing the required sample size and potentially speeding up drug development. However, more improvements can be achieved by stopping early when the final trial decision is very likely, rather than certain, known as stochastic curtailment. While some authors have proposed approaches of this form, these approaches have various limitations.In this work we address these limitations by proposing new design approaches for single-arm phase II binary outcome trials that use stochastic curtailment. We use exact distributions, avoid simulation, consider a wider range of possible designs and permit early stopping for promising treatments. As a result, we are able to obtain trial designs that have considerably reduced sample sizes on average.

Improving the assessment of the probability of success in late stage drug development

There are several steps to confirming the safety and efficacy of a new medicine. A sequence of trials, each with its own objectives, is usually required. Quantitative risk metrics can be useful for informing decisions about whether a medicine should transition from one stage of development to the next. To obtain an estimate of the probability of regulatory approval, pharmaceutical companies may start with industry-wide success rates and then apply to these subjective adjustments to reflect program-specific information. However, this approach lacks transparency and fails to make full use of data from previous clinical trials. We describe a quantitative Bayesian approach for calculating the probability of success (PoS) at the end of phase II which incorporates internal clinical data from one or more phase IIb studies, industry-wide success rates, and expert opinion or external data if needed. Using an example, we illustrate how PoS can be calculated accounting for differences between the phase II data and future phase III trials, and discuss how the methods can be extended to accommodate accelerated drug development pathways.

Bivariate Bayesian hypothesis testing with missing data in components

Multiple endpoints and historical data borrowing may be simultaneously incorporated for enhancing efficiency and speeding up the new drug development process in the pharmaceutical industry. O'Brien's test is a widely used weighted combination test for multiplicity adjustment on multiple endpoints to control the overall type error rate in a weak sense. In this research, a modification on the O'Brien's test more specifically on the weights is considered for a trial with two primary endpoints to potentially increase power. The method can handle missing data in the current study and in the prior derivation for dynamic historical data borrowing. Simulations are conducted to compare the performances of different methods. A data example is used to illustrate the applications of the methods.

Randomized and non-randomized designs for causal inference with longitudinal data in rare disorders

In the United States, approximately 7000 rare diseases affect 30 million patients, and only 10% of these diseases have existing therapies. Sound study design and causal inference methods are essential to demonstrate the therapeutic efficacy, safety, and effectiveness of new therapies. In the rare diseases setting, several factors challenge the use of typical parallel control designs: the small patient population size, genotypic and phenotypic diversity, and the complexity and incomplete understanding of the disorder’s progression. Repeated measures, when spaced appropriately relative to disease progression and exploited in design and analysis, can increase study power and reduce variability in treatment effect estimation. This paper reviews these longitudinal designs and draws the parallel between some new and existing randomized studies in rare diseases and their less well-known controlled observational study designs. We show that self-controlled randomized crossover and N-of-1 designs have similar considerations as the observational case series and case-crossover designs. Also, randomized sequential designs have similar considerations to longitudinal cohort studies using sequential matching or weighting to control confounding. We discuss design and analysis considerations for valid causal inference and illustrate them with examples of analyses in multiple rare disorders, including urea cycle disorder and cystic fibrosis.

Use of external evidence for design and Bayesian analysis of clinical trials: a qualitative study of trialists' views

BACKGROUND

Evidence from previous studies is often used relatively informally in the design of clinical trials: for example, a systematic review to indicate whether a gap in the current evidence base justifies a new trial. External evidence can be used more formally in both trial design and analysis, by explicitly incorporating a synthesis of it in a Bayesian framework. However, it is unclear how common this is in practice or the extent to which it is considered controversial. In this qualitative study, we explored attitudes towards, and experiences of, trialists in incorporating synthesised external evidence through the Bayesian design or analysis of a trial.

METHODS

Semi-structured interviews were conducted with 16 trialists: 13 statisticians and three clinicians. Participants were recruited across several universities and trials units in the United Kingdom using snowball and purposeful sampling. Data were analysed using thematic analysis and techniques of constant comparison.

RESULTS

Trialists used existing evidence in many ways in trial design, for example, to justify a gap in the evidence base and inform parameters in sample size calculations. However, no one in our sample reported using such evidence in a Bayesian framework. Participants tended to equate Bayesian analysis with the incorporation of prior information on the intervention effect and were less aware of the potential to incorporate data on other parameters. When introduced to the concepts, many trialists felt they could be making more use of existing data to inform the design and analysis of a trial in particular scenarios. For example, some felt existing data could be used more formally to inform background adverse event rates, rather than relying on clinical opinion as to whether there are potential safety concerns. However, several barriers to implementing these methods in practice were identified, including concerns about the relevance of external data, acceptability of Bayesian methods, lack of confidence in Bayesian methods and software, and practical issues, such as difficulties accessing relevant data.

CONCLUSIONS

Despite trialists recognising that more formal use of external evidence could be advantageous over current approaches in some areas and useful as sensitivity analyses, there are still barriers to such use in practice.

Prior elicitation of the efficacy and tolerability of Methotrexate and Mycophenolate Mofetil in Juvenile Localised Scleroderma

Background

Evidence is lacking for safe and effective treatments for juvenile localised scleroderma (JLS). Methotrexate (MTX) is commonly used first line and mycophenolate mofetil (MMF) second line, despite a limited evidence base. A head to head trial of these two medications would provide data on relative efficacy and tolerability. However, a frequentist approach is difficult to deliver in JLS, because of the numbers needed to sufficiently power a trial. A Bayesian approach could be considered.

Methods

An international consensus meeting was convened including an elicitation exercise where opinion was sought on the relative efficacy and tolerability of MTX compared to MMF to produce prior distributions for a future Bayesian trial. Secondary aims were to achieve consensus agreement on critical aspects of a future trial.

Results

An international group of 12 clinical experts participated. Opinion suggested superior efficacy and tolerability of MMF compared to MTX; where most likely value of efficacy of MMF was 0.70 (95% confidence interval (CI) 0.34-0.90) and of MTX was 0.68 (95% CI 0.41-0.8). The most likely value of tolerability of MMF was 0.77 (95% CI 0.3-0.94) and of MTX was 0.62 (95% CI 0.32-0.84). The wider CI for MMF highlights that experts were less sure about relative efficacy and tolerability of MMF compared to MTX. Despite using a Bayesian approach, power calculations still produced a total sample size of 240 participants, reflecting the uncertainty amongst experts about the performance of MMF.

Conclusions

Key factors have been defined regarding the design of a future Bayesian approach clinical trial including elicitation of prior opinion of the efficacy and tolerability of MTX and MMF in JLS. Combining further efficacy data on MTX and MMF with prior opinion could potentially reduce the pre-trial uncertainty so that, when combined with smaller trial sample sizes a compelling evidence base is available.

Mycophenolate Mofetil Versus Cyclophosphamide for Remission Induction in Childhood Polyarteritis Nodosa: An Open-Label, Randomized, Bayesian Noninferiority Trial

OBJECTIVE

Cyclophosphamide (CYC) is used in clinical practice off-label for the induction of remission in childhood polyarteritis nodosa (PAN). Mycophenolate mofetil (MMF) might offer a less toxic alternative. This study was undertaken to explore the relative effectiveness of CYC and MMF treatment in a randomized controlled trial (RCT).

METHODS

This was an international, open-label, Bayesian RCT to investigate the relative effectiveness of CYC and MMF for remission induction in childhood PAN. Eleven patients with newly diagnosed childhood PAN were randomized (1:1) to receive MMF or intravenous CYC; all patients received the same glucocorticoid regimen. The primary end point was remission within 6 months while compliant with glucocorticoid taper. Bayesian distributions for remission rates were established a priori for MMF and CYC by experienced clinicians and updated to posterior distributions on trial completion.

RESULTS

Baseline disease activity and features were similar between the 2 treatment groups. The primary end point was met in 4 of 6 patients (67%) in the MMF group and 4 of 5 patients (80%) in the CYC group. Time to remission was shorter in the MMF group compared to the CYC group (median 7.1 weeks versus 17.6 weeks). No relapses occurred in either group within 18 months. Two serious infections were found to be likely linked to MMF treatment. Physical and psychosocial quality-of-life scores were superior in the MMF group compared to the CYC group at 6 months and 18 months. Combining the prior expert opinion with results from the present study provided posterior estimates of remission of 71% for MMF (90% credibility interval [90% CrI] 51, 83) and 75% for CYC (90% CrI 57, 86).

CONCLUSION

The present results, taken together with prior opinion, indicate that rates of remission induction in childhood PAN are similar with MMF treatment and CYC treatment, and MMF treatment might be associated with better health-related quality of life than CYC treatment.

Effectiveness of initial methotrexate-based treatment approaches in early rheumatoid arthritis: an elicitation of rheumatologists' beliefs

OBJECTIVES

To quantify rheumatologists' beliefs about the effectiveness of triple therapy (MTX + HCQ + SSZ) and other commonly used initial treatments for RA.

METHODS

In a Bayesian belief elicitation exercise, 40 rheumatologists distributed 20 chips, each representing 5% of their total weight of belief on the probability that a typical patient with moderate-severe early RA would have an ACR50 response within 6 months with MTX (oral and s.c.), MTX + HCQ (dual therapy) and triple therapy. Parametric distributions were fit, and used to calculate pairwise median relative risks (RR), with 95% credible intervals, and estimate sample sizes for new trials to shift these beliefs.

RESULTS

In the pooled analysis, triple therapy was perceived to be superior to MTX (RR 1.97; 1.35, 2.89) and dual therapy (RR 1.32; 1.03, 1.73). A pessimistic subgroup (n = 10) perceived all treatments to be similar, whereas an optimistic subgroup (n = 10) believed triple therapy to be most effective of all (RR 4.03; 2.22, 10.12). Similar variability was seen for the comparison between oral and s.c. MTX. Assuming triple therapy is truly more effective than MTX, a trial of 100 patients would be required to convince the pessimists; if triple therapy truly has no-modest effect (RR <1.5), a non-inferiority trial of 475 patients would be required to convince the optimists.

CONCLUSION

Rheumatologists' beliefs regarding the effectiveness of triple therapy vary, which may partially explain the variability in its use. Owing to the strength of beliefs, some may be reluctant to shift, even with new evidence.

Issues in Designing and Interpreting Small Clinical Trials

The randomised controlled trial (RCT) is a powerful approach for testing the effectiveness of various clinical interventions. Cardiology often benefits from large RCTs, which may be used to inform practice decisions ranging from primary prevention to advanced cardiac disease and/or acute cardiac care. RCTs in cardiology often need to be quite large to test for meaningful effects on clinical outcomes, because effect sizes are typically modest and clinical outcomes may take several years to occur after treatment initiation. However, a variety of small clinical trials are also carried out in the biomedical research enterprise; these are often difficult to design and interpret, because the objectives and needs of small clinical trials are quite variable. Some are pilot trials that may be used to refine processes or as part of the planning in advance of a larger trial designed to test therapeutic efficacy. Some are first-in-human or proof-of-concept studies that, also, will eventually be followed by one or more larger trials to test therapeutic efficacy. Some are intended to be stand-alone trials that are small for other reasons. In this paper, we explore some key issues related to design and interpretation of small clinical trials in cardiology. We broadly classify small trials into 4 types: 1) pilot trials, 2) early-stage or proof-of-concept trials, 3) rare diseases or difficult-to-recruit populations, and 4) underpowered trials. For each, we describe the appropriate objectives, analysis, and interpretation.

Bounds for the weight of external data in shrinkage estimation

Shrinkage estimation in a meta-analysis framework may be used to facilitate dynamical borrowing of information. This framework might be used to analyze a new study in the light of previous data, which might differ in their design (e.g., a randomized controlled trial and a clinical registry). We show how the common study weights arise in effect and shrinkage estimation, and how these may be generalized to the case of Bayesian meta-analysis. Next we develop simple ways to compute bounds on the weights, so that the contribution of the external evidence may be assessed a priori. These considerations are illustrated and discussed using numerical examples, including applications in the treatment of Creutzfeldt-Jakob disease and in fetal monitoring to prevent the occurrence of metabolic acidosis. The target study's contribution to the resulting estimate is shown to be bounded below. Therefore, concerns of evidence being easily overwhelmed by external data are largely unwarranted.

On weakly informative prior distributions for the heterogeneity parameter in Bayesian random-effects meta-analysis

The normal-normal hierarchical model (NNHM) constitutes a simple and widely used framework for meta-analysis. In the common case of only few studies contributing to the meta-analysis, standard approaches to inference tend to perform poorly, and Bayesian meta-analysis has been suggested as a potential solution. The Bayesian approach, however, requires the sensible specification of prior distributions. While noninformative priors are commonly used for the overall mean effect, the use of weakly informative priors has been suggested for the heterogeneity parameter, in particular in the setting of (very) few studies. To date, however, a consensus on how to generally specify a weakly informative heterogeneity prior is lacking. Here we investigate the problem more closely and provide some guidance on prior specification.

Prior Elicitation for Use in Clinical Trial Design and Analysis: A Literature Review

Bayesian inference is increasingly popular in clinical trial design and analysis. The subjective knowledge derived from an expert elicitation procedure may be useful to define a prior probability distribution when no or limited data is available. This work aims to investigate the state-of-the-art Bayesian prior elicitation methods with a focus on clinical trial research. A literature search on the Current Index to Statistics (CIS), PubMed, and Web of Science (WOS) databases, considering “prior elicitation” as a search string, was run on 1 November 2020. Summary statistics and trend of publications over time were reported. Finally, a Latent Dirichlet Allocation (LDA) model was developed to recognise latent topics in the pertinent papers retrieved. A total of 460 documents pertinent to the Bayesian prior elicitation were identified. Of these, 213 (45.4%) were published in the “Probability and Statistics” area. A total of 42 articles pertain to clinical trial and the majority of them (81%) reports parametric techniques as elicitation method. The last decade has seen an increased interest in prior elicitation and the gap between theory and application getting narrower and narrower. Given the promising flexibility of non-parametric approaches to the experts’ elicitation, more efforts are needed to ensure their diffusion also in applied settings.

Management of severe hyperinflammation in the COVID-19 era: the role of the rheumatologist

Objectives

The objectives of this study were (i) to describe the clinical presentation, treatment and outcome of paediatric inflammatory multisystem syndrome temporally related to Sars-CoV-2 (PIMS-TS) in children; (ii) to propose a framework to guide multidisciplinary team (MDT) management; and (iii) to highlight the role of the paediatric rheumatologist in this context.

Methods

This study involved a retrospective case notes review of patients referred to a single specialist paediatric centre with suspected PIMS-TS, with a focus on clinical presentation, laboratory parameters, treatment, and outcome in the context of an MDT framework.

Results

Nineteen children of median age 9.1 years fulfilled the definition of PIMS-TS and were managed within an MDT framework: 5/19 were female; 14/19 were of Black, Asian or minority ethnicity; 9/19 also fulfilled diagnostic criteria for complete or incomplete Kawasaki disease (KD). Severe systemic inflammation, shock, and abdominal pain were ubiquitous. Treatment was stratified within an MDT framework and included CSs in all; i.v. immunoglobulin in all; anakinra in 4/19; infliximab in 1/19; and antiviral (aciclovir) in 4/19.

Conclusions

We observed significant diagnostic equipoise using a current definition of PIMS-TS, overlapping with KD. Outside of clinical trials, an MDT approach is vital. The role of the paediatric rheumatologist is to consider differential diagnoses of hyperinflammation in the young, to advise on empiric immunomodulatory therapy, to set realistic therapeutic targets, to gauge therapeutic success, to oversee timely step-down of immunomodulation, and to contribute to the longer-term MDT follow-up of any late inflammatory sequelae.

Bayesian Approaches for Confirmatory Trials in Rare Diseases: Opportunities and Challenges

The aim of this narrative review is to introduce the reader to Bayesian methods that, in our opinion, appear to be the most important in the context of rare diseases. A disease is defined as rare depending on the prevalence of the affected patients in the considered population, for example, about 1 in 1500 people in U.S.; about 1 in 2500 people in Japan; and fewer than 1 in 2000 people in Europe. There are between 6000 and 8000 rare diseases and the main issue in drug development is linked to the challenge of achieving robust evidence from clinical trials in small populations. A better use of all available information can help the development process and Bayesian statistics can provide a solid framework at the design stage, during the conduct of the trial, and at the analysis stage. The focus of this manuscript is to provide a review of Bayesian methods for sample size computation or reassessment during phase II or phase III trial, for response adaptive randomization and of for meta-analysis in rare disease. Challenges regarding prior distribution choice, computational burden and dissemination are also discussed.

Developments in evidence creation for treatments of inborn errors of metabolism

Inborn errors of metabolism (IEM) represent the first group of genetic disorders, amenable to causal therapies. In addition to traditional medical diet and cofactor treatments, new treatment strategies such as enzyme replacement and small molecule therapies, solid organ transplantation, and cell-and gene-based therapies have become available. Inherent to the rare nature of the single conditions, generating high-quality evidence for these treatments in clinical trials and under real-world conditions has been challenging. Guidelines developed with standardized methodologies have contributed to improve the practice of care and long-term clinical outcomes. Adaptive trial designs allow for changes in sample size, group allocation and trial duration as the trial proceeds. n-of-1 studies may be used in small sample sized when participants are clinically heterogeneous. Multicenter observational and registry-based clinical trials are promoted via international research networks. Core outcome and standard data element sets will enhance comparative analysis of clinical trials and observational studies. Patient-centered outcome-research as well as patient-led research initiatives will further accelerate the development of therapies for IEM.

Expert Elicitation for Latent Growth Curve Models: The Case of Posttraumatic Stress Symptoms Development in Children With Burn Injuries

Experts provide an alternative source of information to classical data collection methods such as surveys. They can provide additional insight into problems, supplement existing data, or provide insights when classical data collection is troublesome. In this paper, we explore the (dis)similarities between expert judgments and data collected by traditional data collection methods regarding the development of posttraumatic stress symptoms (PTSSs) in children with burn injuries. By means of an elicitation procedure, the experts' domain expertise is formalized and represented in the form of probability distributions. The method is used to obtain beliefs from 14 experts, including nurses and psychologists. Those beliefs are contrasted with questionnaire data collected on the same issue. The individual and aggregated expert judgments are contrasted with the questionnaire data by means of Kullback-Leibler divergences. The aggregated judgments of the group that mainly includes psychologists resemble the questionnaire data more than almost all of the individual expert judgments.

Surgery for women with endometrioma prior to in vitro fertilisation: proposal for a feasible multicentre randomised clinical trial in the UK

STUDY QUESTION

Is it feasible to undertake a randomised controlled trial to establish whether surgical removal of endometrioma or not, improves live birth rates from IVF?

SUMMARY ANSWER

A randomised controlled trial (RCT) comparing surgery versus no surgery to endometrioma prior to IVF is only feasible in UK if an adaptive rather than traditional study design is used; this would minimise resource wastage and complete the trial in an acceptable time frame.

WHAT IS KNOWN ALREADY

There is wide variation in the management of endometriomas prior to IVF, with decisions about treatment being influenced by personal preferences.

STUDY DESIGN, SIZE, AND DURATION

This was a mixed-methods study consisting of an online survey of clinicians, a focus group and individual interviews with potential trial participants.

PARTICIPANTS/MATERIALS, SETTING, METHODS

Endometriosis and fertility experts across the UK were invited to participate in an online anonymised questionnaire. Potential future trial participants were recruited from a tertiary care fertility centre and invited to participate in either individual interviews or focus groups.

MAIN RESULTS AND THE ROLE OF CHANCE

Clinicians and potential trial participants confirmed the need for an RCT to inform the management of an endometrioma prior to IVF. There were 126 clinicians who completed the survey, and the majority (70%) were willing to recruit to a trial. Half of those who responded indicated that they see at least 10 eligible women each year. The main barriers to recruitment were waiting lists for surgery and access to public funding for IVF. One focus group (n = 7) and five interviews were conducted with potential trial participants (n = 3) and their partners (n = 2). The findings from these discussions highlighted that recruitment and retention in a potential RCT could be improved by coordination between IVF and surgical services such that an operation does not delay IVF or affect access to public funding. Live birth was considered the most important outcome with an improvement of at least 10% considered the minimum acceptable by both patients and clinicians.

LIMITATIONS, REASONS FOR CAUTION

This feasibility study captured views of clinicians across the UK, but as patients were from a single Scottish centre, their views may not be representative of other areas with limited public funding for IVF.

WIDER IMPLICATIONS OF THE FINDINGS

There is a need for an appropriately powered RCT to establish whether or not surgical treatment of endometrioma prior to IVF improves live birth rates. There are logistical issues to be considered due to limited number of participants, funding of IVF and waiting times. These could be overcome in a RCT by using an adaptive design which would include a prospectively planned opportunity for modification of specified aspects of the study design based on interim analysis of the data, coordination of IVF treatments and endometriosis surgeries and international collaboration. Similar principles could be used for other questions in fertility where a traditional approach for randomised trials is not feasible.

STUDY FUNDING/COMPETING INTEREST(S)

Funding was received from the NHS Grampian R&D pump priming fund (RG14437-12). S.B. is Editor-in-Chief of HROPEN, and A.W.H. is Deputy Editor of HROPEN. Neither was involved in the review of this manuscript. L.S. reports grants from CSO and NIHR to do endometriosis research, outside the submitted work. K.C. reports grants from NIHR/HTA and CSO during the conduct of the study. J.H.e., A.W.H., J.D., S.B.r., K.B., G.B., J.H.u. and K.G. report no conflict of interest.

Methodologic considerations in randomized clinical trials in reproductive medicine

Randomized controlled trials (RCTs) are the cornerstone of evidence-based medicine. In this series in Fertility and Sterility, several aspects of RCTs are discussed, with contributions on multicenter RCTs, different international settings, and integrity of RCTs. The present contribution deals with methodologic issues. We discuss different types of RCTs based on null hypothesis (superiority vs. noninferiority vs. equivalence) as well as frequentist versus Bayesian interpretation. We also discuss the use of RCTs in the era of personalized medicine and RCTs to address diagnostic and prognostic questions. Finally, we address the use of big data compared with the use of RCTs.

Optimizing interim analysis timing for Bayesian adaptive commensurate designs

In developing products for rare diseases, statistical challenges arise due to the limited number of patients available for participation in drug trials and other clinical research. Bayesian adaptive clinical trial designs offer the possibility of increased statistical efficiency, reduced development cost and ethical hazard prevention via their incorporation of evidence from external sources (historical data, expert opinions, and real-world evidence), and flexibility in the specification of interim looks. In this paper, we propose a novel Bayesian adaptive commensurate design that borrows adaptively from historical information and also uses a particular payoff function to optimize the timing of the study's interim analysis. The trial payoff is a function of how many samples can be saved via early stopping and the probability of making correct early decisions for either futility or efficacy. We calibrate our Bayesian algorithm to have acceptable long-run frequentist properties (Type I error and power) via simulation at the design stage. We illustrate our approach using a pediatric trial design setting testing the effect of a new drug for a rare genetic disease. The optimIA R package available at https://github.com/wxwx1993/Bayesian_IA_Timing provides an easy-to-use implementation of our approach.

Latent process model of the 6-minute walk test in Duchenne muscular dystrophy : A Bayesian approach to quantifying rare disease progression

Duchenne muscular dystrophy (DMD) is a rare X-linked genetic pediatric disease characterized by a lack of functional dystrophin production in the body, resulting in muscle deterioration. Lower body muscle weakness progresses to non-ambulation typically by early teenage years, followed by upper body muscle deterioration and ultimately death by the late twenties. The objective of this study was to enhance the quantitative understanding of DMD disease progression through nonlinear mixed effects modeling of the population mean and variability of the 6-min walk test (6MWT) clinical endpoint. An indirect response model with a latent process was fit to digitized literature data using full Bayesian estimation. The modeling data set consisted of 22 healthy controls and 218 DMD patients from one interventional and four observational trials. The model reasonably described the central tendency and population variability of the 6MWT in healthy subjects and DMD patients. An exploratory categorical covariate analysis indicated that there was no apparent effect of corticosteroid administration on DMD disease progression. The population predicted 6MWT began to rise at 1.32 years of age, plateauing at 654 meters (m) at 17.2 years of age for the healthy population. The DMD trajectory reached a maximum of 411 m at 8.90 years before declining and falling below 1 m at age 18.0. The model has potential to be used as a Bayesian estimation and posterior simulation tool to make informed model-based drug development decisions that incorporate prior knowledge with new data.

What Do You Think? Using Expert Opinion to Improve Predictions of Response Propensity Under a Bayesian Framework

Responsive survey designs introduce protocol changes to survey operations based on accumulating paradata. Case-level predictions, including response propensity, can be used to tailor data collection features in pursuit of cost or quality goals. Unfortunately, predictions based only on partial data from the current round of data collection can be biased, leading to ineffective tailoring. Bayesian approaches can provide protection against this bias. Prior beliefs, which are generated from data external to the current survey implementation, contribute information that may be lacking from the partial current data. Those priors are then updated with the accumulating paradata. The elicitation of the prior beliefs, then, is an important characteristic of these approaches. While historical data for the same or a similar survey may be the most natural source for generating priors, eliciting prior beliefs from experienced survey managers may be a reasonable choice for new surveys, or when historical data are not available. Here, we fielded a questionnaire to survey managers, asking about expected attempt-level response rates for different subgroups of cases, and developed prior distributions for attempt-level response propensity model coefficients based on the mean and standard error of their responses. Then, using respondent data from a real survey, we compared the predictions of response propensity when the expert knowledge is incorporated into a prior to those based on a standard method that considers accumulating paradata only, as well as a method that incorporates historical survey data.

Practical help for specifying the target difference in sample size calculations for RCTs: the DELTA2 five-stage study, including a workshop

BACKGROUND

The randomised controlled trial is widely considered to be the gold standard study for comparing the effectiveness of health interventions. Central to its design is a calculation of the number of participants needed (the sample size) for the trial. The sample size is typically calculated by specifying the magnitude of the difference in the primary outcome between the intervention effects for the population of interest. This difference is called the 'target difference' and should be appropriate for the principal estimand of interest and determined by the primary aim of the study. The target difference between treatments should be considered realistic and/or important by one or more key stakeholder groups.

OBJECTIVE

The objective of the report is to provide practical help on the choice of target difference used in the sample size calculation for a randomised controlled trial for researchers and funder representatives.

METHODS

The Difference ELicitation in TriAls2 (DELTA2) recommendations and advice were developed through a five-stage process, which included two literature reviews of existing funder guidance and recent methodological literature; a Delphi process to engage with a wider group of stakeholders; a 2-day workshop; and finalising the core document.

RESULTS

Advice is provided for definitive trials (Phase III/IV studies). Methods for choosing the target difference are reviewed. To aid those new to the topic, and to encourage better practice, 10 recommendations are made regarding choosing the target difference and undertaking a sample size calculation. Recommended reporting items for trial proposal, protocols and results papers under the conventional approach are also provided. Case studies reflecting different trial designs and covering different conditions are provided. Alternative trial designs and methods for choosing the sample size are also briefly considered.

CONCLUSIONS

Choosing an appropriate sample size is crucial if a study is to inform clinical practice. The number of patients recruited into the trial needs to be sufficient to answer the objectives; however, the number should not be higher than necessary to avoid unnecessary burden on patients and wasting precious resources. The choice of the target difference is a key part of this process under the conventional approach to sample size calculations. This document provides advice and recommendations to improve practice and reporting regarding this aspect of trial design. Future work could extend the work to address other less common approaches to the sample size calculations, particularly in terms of appropriate reporting items.

FUNDING

Funded by the Medical Research Council (MRC) UK and the National Institute for Health Research as part of the MRC-National Institute for Health Research Methodology Research programme.

Provocative questions in osteosarcoma basic and translational biology: A report from the Children's Oncology Group

Patients who are diagnosed with osteosarcoma (OS) today receive the same therapy that patients have received over the last 4 decades. Extensive efforts to identify more effective or less toxic regimens have proved disappointing. As we enter a postgenomic era in which we now recognize OS not as a cancer of mutations but as one defined by p53 loss, chromosomal complexity, copy number alteration, and profound heterogeneity, emerging threads of discovery leave many hopeful that an improving understanding of biology will drive discoveries that improve clinical care. Under the organization of the Bone Tumor Biology Committee of the Children's Oncology Group, a team of clinicians and scientists sought to define the state of the science and to identify questions that, if answered, have the greatest potential to drive fundamental clinical advances. Having discussed these questions in a series of meetings, each led by invited experts, we distilled these conversations into a series of seven Provocative Questions. These include questions about the molecular events that trigger oncogenesis, the genomic and epigenomic drivers of disease, the biology of lung metastasis, research models that best predict clinical outcomes, and processes for translating findings into clinical trials. Here, we briefly present each Provocative Question, review the current scientific evidence, note the immediate opportunities, and speculate on the impact that answered questions might have on the field. We do so with an intent to provide a framework around which investigators can build programs and collaborations to tackle the hardest problems and to establish research priorities for those developing policies and providing funding.

Defining consensus opinion to develop randomised controlled trials in rare diseases using Bayesian design: An example of a proposed trial of adalimumab versus pamidronate for children with CNO/CRMO

Introduction

Chronic nonbacterial osteomyelitis (CNO) is a rare autoinflammatory bone disorder primarily affecting children and adolescents. It can lead to chronic pain, bony deformities and fractures. The pathophysiology of CNO is incompletely understood. Scientific evidence suggests dysregulated expression of pro- and anti-inflammatory cytokines to be centrally involved. Currently, treatment is largely based on retrospective observational studies and expert opinion. Treatment usually includes nonsteroidal anti-inflammatory drugs and/or glucocorticoids, followed by a range of drugs in unresponsive cases. While randomised clinical trials are lacking, retrospective and prospective non-controlled studies suggest effectiveness of TNF inhibitors and bisphosphonates. The objective of the Bayesian consensus meeting was to quantify prior expert opinion.

Methods

Twelve international CNO experts were randomly chosen to be invited to a Bayesian prior elicitation meeting.

Results

Results showed that a typical new patient treated with pamidronate would have an 84% chance of improvement in their pain score relative to baseline at 26 weeks and an 83% chance on adalimumab. Experts thought there was a 50% chance that a new typical patient would record a pain score of 28mm (pamidronate) to 30mm (adalimumab) or better at 26 weeks. There was a modest trend in prior opinion to indicate an advantage of pamidronate vs adalimumab, with a 68% prior chance that pamidronate is superior to adalimumab by some margin. However, it is clear that there is considerable uncertainty about the precise relative merits of the two treatments.

Conclusions

The rarity of CNO leads to challenges in conducting randomised controlled trials with sufficient power to provide a definitive outcome. We address this using a Bayesian design, and here describe the process and outcome of the elicitation exercise to establish expert prior opinion. This opinion will be tested in the planned prospective CNO study. The process for establishing expert consensus opinion in CNO will be helpful for developing studies in other rare paediatric diseases.

Mendelian randomization with Egger pleiotropy correction and weakly informative Bayesian priors

Background

The MR-Egger (MRE) estimator has been proposed to correct for directional pleiotropic effects of genetic instruments in an instrumental variable (IV) analysis. The power of this method is considerably lower than that of conventional estimators, limiting its applicability. Here we propose a novel Bayesian implementation of the MR-Egger estimator (BMRE) and explore the utility of applying weakly informative priors on the intercept term (the pleiotropy estimate) to increase power of the IV (slope) estimate.

Methods

This was a simulation study to compare the performance of different IV estimators. Scenarios differed in the presence of a causal effect, the presence of pleiotropy, the proportion of pleiotropic instruments and degree of 'Instrument Strength Independent of Direct Effect' (InSIDE) assumption violation. Based on empirical plasma urate data, we present an approach to elucidate a prior distribution for the amount of pleiotropy.

Results

A weakly informative prior on the intercept term increased power of the slope estimate while maintaining type 1 error rates close to the nominal value of 0.05. Under the InSIDE assumption, performance was unaffected by the presence or absence of pleiotropy. Violation of the InSIDE assumption biased all estimators, affecting the BMRE more than the MRE method.

Conclusions

Depending on the prior distribution, the BMRE estimator has more power at the cost of an increased susceptibility to InSIDE assumption violations. As such the BMRE method is a compromise between the MRE and conventional IV estimators, and may be an especially useful approach to account for observed pleiotropy.

Personalized Pancreatic Cancer Management: A Systematic Review of How Machine Learning Is Supporting Decision-making

This review critically analyzes how machine learning is being used to support clinical decision-making in the management of potentially resectable pancreatic cancer. Following PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-analyses) guidelines, electronic searches of MEDLINE, Embase, PubMed, and Cochrane Database were undertaken. Studies were assessed using the checklist for critical appraisal and data extraction for systematic reviews of prediction modeling studies (CHARMS) checklist. In total 89,959 citations were retrieved. Six studies met the inclusion criteria. Three studies were Markov decision-analysis models comparing neoadjuvant therapy versus upfront surgery. Three studies predicted survival time using Bayesian modeling (n = 1) and artificial neural network (n = 1), and one study explored machine learning algorithms including Bayesian network, decision trees, k-nearest neighbor, and artificial neural networks. The main methodological issues identified were limited data sources, which limits generalizability and potentiates bias; lack of external validation; and the need for transparency in methods of internal validation, consecutive sampling, and selection of candidate predictors. The future direction of research relies on expanding our view of the multidisciplinary team to include professionals from computing and data science with algorithms developed in conjunction with clinicians and viewed as aids, not replacement, to traditional clinical decision-making.

Commentary on the EMA Reflection Paper on the use of extrapolation in the development of medicines for paediatrics

Adopted guidelines reflect a harmonised European approach to a specific scientific issue and should reflect the most recent scientific knowledge. However, whilst EU regulations are mandatory for all member states and EU directives must be followed by national laws in line with the directive, EMA guidelines do not have legal force and alternative approaches may be taken, but these obviously require more justification. This new series of the BJCP, developed in collaboration with the EMA, aims to address this issue by providing an annotated version of some relevant EMA guidelines and regulatory documents by experts. Hopefully, this will help in promoting their diffusion and in opening a forum for discussion with our readers.

Dynamically borrowing strength from another study through shrinkage estimation

Meta-analytic methods may be used to combine evidence from different sources of information. Quite commonly, the normal–normal hierarchical model (NNHM) including a random-effect to account for between-study heterogeneity is utilized for such analyses. The same modeling framework may also be used to not only derive a combined estimate, but also to borrow strength for a particular study from another by deriving a shrinkage estimate. For instance, a small-scale randomized controlled trial could be supported by a non-randomized study, e.g. a clinical registry. This would be particularly attractive in the context of rare diseases. We demonstrate that a meta-analysis still makes sense in this extreme case, effectively based on a synthesis of only two studies, as illustrated using a recent trial and a clinical registry in Creutzfeld-Jakob disease. Derivation of a shrinkage estimate within a Bayesian random-effects meta-analysis may substantially improve a given estimate even based on only a single additional estimate while accounting for potential effect heterogeneity between the studies. Alternatively, inference may equivalently be motivated via a model specification that does not require a common overall mean parameter but considers the treatment effect in one study, and the difference in effects between the studies. The proposed approach is quite generally applicable to combine different types of evidence originating, e.g. from meta-analyses or individual studies. An application of this more general setup is provided in immunosuppression following liver transplantation in children.

Applicability and added value of novel methods to improve drug development in rare diseases

BACKGROUND

The ASTERIX project developed a number of novel methods suited to study small populations. The objective of this exercise was to evaluate the applicability and added value of novel methods to improve drug development in small populations, using real world drug development programmes as reported in European Public Assessment Reports.

METHODS

The applicability and added value of thirteen novel methods developed within ASTERIX were evaluated using data from 26 European Public Assessment Reports (EPARs) for orphan medicinal products, representative of rare medical conditions as predefined through six clusters. The novel methods included were 'innovative trial designs' (six methods), 'level of evidence' (one method), 'study endpoints and statistical analysis' (four methods), and 'meta-analysis' (two methods) and they were selected from the methods developed within ASTERIX based on their novelty; methods that discussed already available and applied strategies were not included for the purpose of this validation exercise. Pre-requisites for application in a study were systematized for each method, and for each main study in the selected EPARs it was assessed if all pre-requisites were met. This direct applicability using the actual study design was firstly assessed. Secondary, applicability and added value were explored allowing changes to study objectives and design, but without deviating from the context of the drug development plan. We evaluated whether differences in applicability and added value could be observed between the six predefined condition clusters.

RESULTS AND DISCUSSION

Direct applicability of novel methods appeared to be limited to specific selected cases. The applicability and added value of novel methods increased substantially when changes to the study setting within the context of drug development were allowed. In this setting, novel methods for extrapolation, sample size re-assessment, multi-armed trials, optimal sequential design for small sample sizes, Bayesian sample size re-estimation, dynamic borrowing through power priors and fall-back tests for co-primary endpoints showed most promise - applicable in more than 40% of evaluated EPARs in all clusters. Most of the novel methods were applicable to conditions in the cluster of chronic and progressive conditions, involving multiple systems/organs. Relatively fewer methods were applicable to acute conditions with single episodes. For the chronic clusters, Goal Attainment Scaling was found to be particularly applicable as opposed to other (non-chronic) clusters.

CONCLUSION

Novel methods as developed in ASTERIX can improve drug development programs. Achieving optimal added value of these novel methods often requires consideration of the entire drug development program, rather than reconsideration of methods for a specific trial. The novel methods tested were mostly applicable in chronic conditions, and acute conditions with recurrent episodes.

Model averaging for robust extrapolation in evidence synthesis

Extrapolation from a source to a target, eg, from adults to children, is a promising approach to utilize external information when data are sparse. In the context of meta-analyses, one is commonly faced with a small number of studies, whereas potentially relevant additional information may also be available. Here, we describe a simple extrapolation strategy using heavy-tailed mixture priors for effect estimation in meta-analysis, which effectively results in a model-averaging technique. The described method is robust in the sense that a potential prior-data conflict, ie, a discrepancy between source and target data, is explicitly anticipated. The aim of this paper is to develop a solution for this particular application to showcase the ease of implementation by providing R code, and to demonstrate the robustness of the general approach in simulations.

Using a meta-narrative literature review and focus groups with key stakeholders to identify perceived challenges and solutions for generating robust evidence on the effectiveness of treatments for rare diseases

INTRODUCTION

For many rare diseases, strong analytic study designs for evaluating the efficacy and effectiveness of interventions are challenging to implement because of small, geographically dispersed patient populations and underlying clinical heterogeneity. The objective of this study was to integrate perspectives from published literature and key rare disease stakeholders to better understand the perceived challenges and proposed methodological approaches to research on clinical interventions for rare diseases.

METHODS

We used a meta-narrative literature review and focus group interviews with key rare disease stakeholders to better understand the perceived challenges in generating and synthesizing treatment effectiveness evidence, and to describe various research methods for mitigating these identified challenges. Data from both components of this study were synthesized narratively according to research paradigms that emerged from our data.

RESULTS

Results from our meta-narrative literature review and focus group interviews revealed three fundamental challenges in generating robust treatment effectiveness evidence for rare diseases: i) limitations in recruiting a sufficient sample size to achieve planned statistical power; ii) inability to account for clinical heterogeneity and assess treatment effects across a clinical spectrum; and iii) reliance on short-term, surrogate outcomes whose clinical relevance is often unclear. We mapped these challenges and associated solutions to three interrelated research paradigms: i) explanatory evidence generation; ii) comparative effectiveness/pragmatic evidence generation; and iii) patient-oriented evidence generation. Within each research paradigm, numerous criticisms and potential solutions have been described with respect to overcoming these challenges from a research study design perspective.

CONCLUSIONS

Over time, discussions about clinical research for interventions for rare diseases have moved beyond methodological approaches to overcome challenges related to explanatory evidence generation, with increased recognition of the importance of pragmatic and patient-oriented evidence. Future directions for our work include developing a framework to expand current evidence synthesis practices to take into consideration many of the concepts discussed in this paper.

How to use prior knowledge and still give new data a chance?

A common challenge for the development of drugs in rare diseases and special populations, eg, paediatrics, is the small numbers of patients that can be recruited into clinical trials. Extrapolation can be used to support development and licensing in paediatrics through the structured integration of available data in adults and prospectively generated data in paediatrics to derive conclusions that support licensing decisions in the target paediatric population. In this context, Bayesian analyses have been proposed to obtain formal proof of efficacy of a new drug or therapeutic principle by using additional information (data, opinion, or expectation), expressed through a prior distribution. However, little is said about the impact of the prior assumptions on the evaluation of outcome and prespecified strategies for decision‐making as required in the regulatory context.

On the basis of examples, we explore the use of data‐based Bayesian meta‐analytic–predictive methods and compare these approaches with common frequentist and Bayesian meta‐analysis models. Noninformative efficacy prior distributions usually do not change the conclusions irrespective of the chosen analysis method. However, if heterogeneity is considered, conclusions are highly dependent on the heterogeneity prior. When using informative efficacy priors based on previous study data in combination with heterogeneity priors, these may completely determine conclusions irrespective of the data generated in the target population. Thus, it is important to understand the impact of the prior assumptions and ensure that prospective trial data in the target population have an appropriate chance, to change prior belief to avoid trivial and potentially erroneous conclusions.

Incorporating historical information in biosimilar trials: Challenges and a hybrid Bayesian-frequentist approach

For the approval of biosimilars, it is, in most cases, necessary to conduct large Phase III clinical trials in patients to convince the regulatory authorities that the product is comparable in terms of efficacy and safety to the originator product. As the originator product has already been studied in several trials beforehand, it seems natural to include this historical information into the showing of equivalent efficacy. Since all studies for the regulatory approval of biosimilars are confirmatory studies, it is required that the statistical approach has reasonable frequentist properties, most importantly, that the Type I error rate is controlled-at least in all scenarios that are realistic in practice. However, it is well known that the incorporation of historical information can lead to an inflation of the Type I error rate in the case of a conflict between the distribution of the historical data and the distribution of the trial data. We illustrate this issue and confirm, using the Bayesian robustified meta-analytic-predictive (MAP) approach as an example, that simultaneously controlling the Type I error rate over the complete parameter space and gaining power in comparison to a standard frequentist approach that only considers the data in the new study, is not possible. We propose a hybrid Bayesian-frequentist approach for binary endpoints that controls the Type I error rate in the neighborhood of the center of the prior distribution, while improving the power. We study the properties of this approach in an extensive simulation study and provide a real-world example.

Decision-theoretic designs for a series of trials with correlated treatment effects using the Sarmanov multivariate beta-binomial distribution

The motivation for the work in this article is the setting in which a number of treatments are available for evaluation in phase II clinical trials and where it may be infeasible to try them concurrently because the intended population is small. This paper introduces an extension of previous work on decision-theoretic designs for a series of phase II trials. The program encompasses a series of sequential phase II trials with interim decision making and a single two-arm phase III trial. The design is based on a hybrid approach where the final analysis of the phase III data is based on a classical frequentist hypothesis test, whereas the trials are designed using a Bayesian decision-theoretic approach in which the unknown treatment effect is assumed to follow a known prior distribution. In addition, as treatments are intended for the same population it is not unrealistic to consider treatment effects to be correlated. Thus, the prior distribution will reflect this. Data from a randomized trial of severe arthritis of the hip are used to test the application of the design. We show that the design on average requires fewer patients in phase II than when the correlation is ignored. Correspondingly, the time required to recommend an efficacious treatment for phase III is quicker.