Meta-analysis of few small studies in orphan diseases

Methods for evidence synthesis in the case of very few studies

In systematic reviews, meta-analyses are routinely applied to summarize the results of the relevant studies for a specific research question. If one can assume that in all studies the same true effect is estimated, the application of a meta-analysis with common effect (commonly referred to as fixed-effect meta-analysis) is adequate. If between-study heterogeneity is expected to be present, the method of choice is a meta-analysis with random effects. The widely used DerSimonian and Laird method for meta-analyses with random effects has been criticized due to its unfavorable statistical properties, especially in the case of very few studies. A working group of the Cochrane Collaboration recommended the use of the Knapp-Hartung method for meta-analyses with random effects. However, as heterogeneity cannot be reliably estimated if only very few studies are available, the Knapp-Hartung method, while correctly accounting for the corresponding uncertainty, has very low power. Our aim is to summarize possible methods to perform meaningful evidence syntheses in the situation with only very few (ie, 2-4) studies. Some general recommendations are provided on which method should be used when. Our recommendations are based on the existing literature on methods for meta-analysis with very few studies and consensus of the authors. The recommendations are illustrated by 2 examples coming from dossier assessments of the Institute for Quality and Efficiency in Health Care.

Increasing antipsychotic dose for non response in schizophrenia

BACKGROUND

Many people with schizophrenia do not reach a satisfactory clinical response with a standard dose of an initially prescribed antipsychotic drug. In such cases, clinicians face the dilemma of increasing the antipsychotic dose in order to enhance antipsychotic efficacy.

OBJECTIVES

To examine the efficacy of increasing antipsychotic dose compared to keeping the same dose in the treatment of people with schizophrenia who have not responded (as defined in the individual studies) to an initial antipsychotic drug trial. We also examine the adverse effects associated with such a procedure.

SEARCH METHODS

We searched the Cochrane Schizophrenia Group Trials Register (10 June 2014, 6 October 2015, and 30 March 2017). We examined references of all included studies for further trials.

SELECTION CRITERIA

All relevant randomised controlled trials (RCTs), reporting useable data, comparing increasing the antipsychotic dose rather than maintaining the original dose for people with schizophrenia who do not respond to their initial antipsychotic treatment.

DATA COLLECTION AND ANALYSIS

At least two review authors independently extracted data . We analysed dichotomous data using relative risks (RR) and the 95% confidence intervals (CI). We analysed continuous data using mean differences (MD) and their 95% CI. We assessed risk of bias for included studies and used GRADE to create a 'Summary of findings' table.

MAIN RESULTS

Ten relevant RCTs with 675 participants are included in this review. All trials were double blind except one single blind. All studies had a run-in phase to confirm they did not respond to their initial antipsychotic treatment. The trials were published between 1980 and 2016. In most studies the methods of randomisation, allocation and blinding were poorly reported. In addition sample sizes were often small, limiting the overall quality of the evidence. Overall, no clear difference was found between groups in terms of the number of participants who showed clinically relevant response (RR 1.09, 95% CI 0.86 to 1.40, 9 RCTs, N = 533, low-quality evidence), or left the study early due to adverse effects (RR 1.63, 95% CI 0.52 to 5.07, very low quality evidence), or due to any reason (RR 1.30, 95% CI 0.89 to 1.90, 5 RCTs, N = 353, low-quality evidence). Similarly, no clear difference was found in general mental state as measured by PANSS total score change (MD -1.44, 95% CI -6.85 to 3.97, 3 RCTs, N = 258, very low quality evidence). At least one adverse effect was equivocal between groups (RR 0.91, 95% CI 0.55 to 1.50, 2 RCTs, N = 191, very low quality evidence). Data were not reported for time in hospital or quality-of-life outcomes. Finally, subgroup and sensitivity analyses did not show any effect on the primary outcome but these analyses were clearly underpowered.

AUTHORS' CONCLUSIONS

Current data do not show any clear differences between increasing or maintaining the antipsychotic dose for people with schizophrenia who do not respond to their initial antipsychotic treatment. Adverse effect reporting was limited and poor. There is an urgent need for further trials in order to determine the optional treatment strategy in such cases.

Economic Modeling Considerations for Rare Diseases

OBJECTIVES

To identify challenges that affect the feasibility and rigor of economic models in rare diseases and strategies that manufacturers have employed in health technology assessment submissions to demonstrate the value of new orphan products that have limited study data.

METHODS

Targeted reviews of PubMed, the National Institute for Health and Care Excellence's (NICE's) Highly Specialised Technologies (HST), and the Scottish Medicines Consortium's (SMC's) ultra-orphan submissions were performed.

RESULTS

A total of 19 PubMed studies, 3 published NICE HSTs, and 11 ultra-orphan SMC submissions were eligible for inclusion. In rare diseases, a number of different factors may affect the model's ability to comply with good practice recommendations. Many products for the treatment of rare diseases have an incomplete efficacy and safety profile at product launch. In addition, there is often limited available natural history and epidemiology data. Information on the direct and indirect cost burden of an orphan disease also may be limited, making it difficult to estimate the potential economic benefit of treatment. These challenges can prevent accurate estimation of a new product's benefits in relation to costs. Approaches that can address such challenges include using patient and/or clinician feedback to inform model assumptions; data from disease analogues; epidemiological techniques, such as matching-adjusted indirect comparison; and long-term data collection.

CONCLUSIONS

Modeling in rare diseases is often challenging; however, a number of approaches are available to support the development of model structures and the collation of input parameters and to manage uncertainty.

How to use prior knowledge and still give new data a chance?

A common challenge for the development of drugs in rare diseases and special populations, eg, paediatrics, is the small numbers of patients that can be recruited into clinical trials. Extrapolation can be used to support development and licensing in paediatrics through the structured integration of available data in adults and prospectively generated data in paediatrics to derive conclusions that support licensing decisions in the target paediatric population. In this context, Bayesian analyses have been proposed to obtain formal proof of efficacy of a new drug or therapeutic principle by using additional information (data, opinion, or expectation), expressed through a prior distribution. However, little is said about the impact of the prior assumptions on the evaluation of outcome and prespecified strategies for decision‐making as required in the regulatory context.

On the basis of examples, we explore the use of data‐based Bayesian meta‐analytic–predictive methods and compare these approaches with common frequentist and Bayesian meta‐analysis models. Noninformative efficacy prior distributions usually do not change the conclusions irrespective of the chosen analysis method. However, if heterogeneity is considered, conclusions are highly dependent on the heterogeneity prior. When using informative efficacy priors based on previous study data in combination with heterogeneity priors, these may completely determine conclusions irrespective of the data generated in the target population. Thus, it is important to understand the impact of the prior assumptions and ensure that prospective trial data in the target population have an appropriate chance, to change prior belief to avoid trivial and potentially erroneous conclusions.

Interval estimation of the overall treatment effect in a meta-analysis of a few small studies with zero events

When a meta-analysis consists of a few small trials that report zero events, accounting for heterogeneity in the (interval) estimation of the overall effect is challenging. Typically, we predefine meta-analytical methods to be employed. In practice, data poses restrictions that lead to deviations from the pre-planned analysis, such as the presence of zero events in at least one study arm. We aim to explore heterogeneity estimators behaviour in estimating the overall effect across different levels of sparsity of events. We performed a simulation study that consists of two evaluations. We considered an overall comparison of estimators unconditional on the number of observed zero cells and an additional one by conditioning on the number of observed zero cells. Estimators that performed modestly robust when (interval) estimating the overall treatment effect across a range of heterogeneity assumptions were the Sidik-Jonkman, Hartung-Makambi and improved Paul-Mandel. The relative performance of estimators did not materially differ between making a predefined or data-driven choice. Our investigations confirmed that heterogeneity in such settings cannot be estimated reliably. Estimators whose performance depends strongly on the presence of heterogeneity should be avoided. The choice of estimator does not need to depend on whether or not zero cells are observed.

A systematic literature review of evidence-based clinical practice for rare diseases: what are the perceived and real barriers for improving the evidence and how can they be overcome?

BACKGROUND

Evidence-based clinical practice is challenging in all fields, but poses special barriers in the field of rare diseases. The present paper summarises the main barriers faced by clinical research in rare diseases, and highlights opportunities for improvement.

METHODS

Systematic literature searches without meta-analyses and internal European Clinical Research Infrastructure Network (ECRIN) communications during face-to-face meetings and telephone conferences from 2013 to 2017 within the context of the ECRIN Integrating Activity (ECRIN-IA) project.

RESULTS

Barriers specific to rare diseases comprise the difficulty to recruit participants because of rarity, scattering of patients, limited knowledge on natural history of diseases, difficulties to achieve accurate diagnosis and identify patients in health information systems, and difficulties choosing clinically relevant outcomes.

CONCLUSIONS

Evidence-based clinical practice for rare diseases should start by collecting clinical data in databases and registries; defining measurable patient-centred outcomes; and selecting appropriate study designs adapted to small study populations. Rare diseases constitute one of the most paradigmatic fields in which multi-stakeholder engagement, especially from patients, is needed for success. Clinical research infrastructures and expertise networks offer opportunities for establishing evidence-based clinical practice within rare diseases.

Evidence synthesis from aggregate recurrent event data for clinical trial design and analysis

Information from historical trials is important for the design, interim monitoring, analysis, and interpretation of clinical trials. Meta-analytic models can be used to synthesize the evidence from historical data, which are often only available in aggregate form. We consider evidence synthesis methods for trials with recurrent event endpoints, which are common in many therapeutic areas. Such endpoints are typically analyzed by negative binomial regression. However, the individual patient data necessary to fit such a model are usually unavailable for historical trials reported in the medical literature. We describe approaches for back-calculating model parameter estimates and their standard errors from available summary statistics with various techniques, including approximate Bayesian computation. We propose to use a quadratic approximation to the log-likelihood for each historical trial based on 2 independent terms for the log mean rate and the log of the dispersion parameter. A Bayesian hierarchical meta-analysis model then provides the posterior predictive distribution for these parameters. Simulations show this approach with back-calculated parameter estimates results in very similar inference as using parameter estimates from individual patient data as an input. We illustrate how to design and analyze a new randomized placebo-controlled exacerbation trial in severe eosinophilic asthma using data from 11 historical trials.

Predictors of mortality and ICD shock therapy in primary prophylactic ICD patients-A systematic review and meta-analysis

Background

There is evidence that the benefit of a primary prophylactic ICD therapy is not equal in all patients.

Purpose

To evaluate risk factors of appropriate shocks and all- cause mortality in patients with a primary prophylactic ICD regarding contemporary studies.

Data source

PubMed, LIVIVO, Cochrane CENTRAL between 2010 and 2016.

Study selection

Studies were eligible if at least one of the endpoints of interest were reported.

Data extraction

All abstracts were independently reviewed by at least two authors. The full text of all selected studies was then analysed in detail.

Data synthesis

Our search strategy retrieved 608 abstracts. After exclusion of unsuitable studies, 36 papers with a total patient number of 47282 were included in our analysis. All-cause mortality was significantly associated with increasing age (HR 1.41, CI 1.29–1.53), left ventricular function (LVEF; HR 1.21, CI 1.14–1.29), ischemic cardiomyopathy (ICM; HR 1.37, CI 1.14–1.66) and co-morbidities such as impaired renal function (HR 2.30, CI 1.97–2.69). Although, younger age (HR 0.96, CI 0.85–1.09), impaired LVEF (HR 1.26, CI 0.89–1.78) and ischemic cardiomyopathy (HR 2.22, CI 0.83–5.93) were associated with a higher risk of appropriate shocks, none of these factors reached statistical significance.

Limitations

Individual patient data were not available for most studies.

Conclusion

In this meta-analysis of contemporary clinical studies, all-cause mortality is predicted by a variety of clinical characteristics including LVEF. On the other hand, the risk of appropriate shocks might be associated with impaired LVEF and ischemic cardiomyopathy. Further prospective studies are required to verify risk factors for appropriate shocks other than LVEF to help select appropriate patients for primary prophylactic ICD-therapy.

Meta-analysis of two studies in the presence of heterogeneity with applications in rare diseases

Random-effects meta-analyses are used to combine evidence of treatment effects from multiple studies. Since treatment effects may vary across trials due to differences in study characteristics, heterogeneity in treatment effects between studies must be accounted for to achieve valid inference. The standard model for random-effects meta-analysis assumes approximately normal effect estimates and a normal random-effects model. However, standard methods based on this model ignore the uncertainty in estimating the between-trial heterogeneity. In the special setting of only two studies and in the presence of heterogeneity, we investigate here alternatives such as the Hartung-Knapp-Sidik-Jonkman method (HKSJ), the modified Knapp-Hartung method (mKH, a variation of the HKSJ method) and Bayesian random-effects meta-analyses with priors covering plausible heterogeneity values; R code to reproduce the examples is presented in an appendix. The properties of these methods are assessed by applying them to five examples from various rare diseases and by a simulation study. Whereas the standard method based on normal quantiles has poor coverage, the HKSJ and mKH generally lead to very long, and therefore inconclusive, confidence intervals. The Bayesian intervals on the whole show satisfying properties and offer a reasonable compromise between these two extremes.

Using phase II data for the analysis of phase III studies: An application in rare diseases

BACKGROUND

Clinical research and drug development in orphan diseases are challenging, since large-scale randomized studies are difficult to conduct. Formally synthesizing the evidence is therefore of great value, yet this is rarely done in the drug-approval process. Phase III designs that make better use of phase II data can facilitate drug development in orphan diseases.

METHODS

A Bayesian meta-analytic approach is used to inform the phase III study with phase II data. It is particularly attractive, since uncertainty of between-trial heterogeneity can be dealt with probabilistically, which is critical if the number of studies is small. Furthermore, it allows quantifying and discounting the phase II data through the predictive distribution relevant for phase III. A phase III design is proposed which uses the phase II data and considers approval based on a phase III interim analysis. The design is illustrated with a non-inferiority case study from a Food and Drug Administration approval in herpetic keratitis (an orphan disease). Design operating characteristics are compared to those of a traditional design, which ignores the phase II data.

RESULTS

An analysis of the phase II data reveals good but insufficient evidence for non-inferiority, highlighting the need for a phase III study. For the phase III study supported by phase II data, the interim analysis is based on half of the patients. For this design, the meta-analytic interim results are conclusive and would justify approval. In contrast, based on the phase III data only, interim results are inconclusive and require further evidence.

CONCLUSION

To accelerate drug development for orphan diseases, innovative study designs and appropriate methodology are needed. Taking advantage of randomized phase II data when analyzing phase III studies looks promising because the evidence from phase II supports informed decision-making. The implementation of the Bayesian design is straightforward with public software such as R.

Restrictive versus liberal blood transfusion for gastrointestinal bleeding: a systematic review and meta-analysis of randomised controlled trials

BACKGROUND

Acute upper gastrointestinal bleeding is a leading indication for red blood cell (RBC) transfusion worldwide, although optimal thresholds for transfusion are debated.

METHODS

We searched MEDLINE, Embase, CENTRAL, CINAHL, and the Transfusion Evidence Library from inception to Oct 20, 2016, for randomised controlled trials comparing restrictive and liberal RBC transfusion strategies for acute upper gastrointestinal bleeding. Main outcomes were mortality, rebleeding, ischaemic events, and mean RBC transfusion. We computed pooled estimates for each outcome by random effects meta-analysis, and individual participant data for a cluster randomised trial were re-analysed to facilitate meta-analysis. We compared treatment effects between patient subgroups, including patients with liver cirrhosis, patients with non-variceal upper gastrointestinal bleeding, and patients with ischaemic heart disease at baseline.

FINDINGS

We included four published and one unpublished randomised controlled trial, totalling 1965 participants. The number of RBC units transfused was lower in the restrictive transfusion group than in the liberal transfusion group (mean difference -1·73 units, 95% CI -2·36 to -1·11, p<0·0001). Restrictive transfusion was associated with lower risk of all-cause mortality (relative risk [RR] 0·65, 95% CI 0·44-0·97, p=0·03) and rebleeding overall (0·58, 0·40-0·84, p=0·004). We detected no difference in risk of ischaemic events. There were no statistically significant differences in the subgroups.

INTERPRETATION

These results support more widespread implementation of restrictive transfusion policies for adults with acute upper gastrointestinal bleeding.

FUNDING

None.

Meta-analysis of few small studies in orphan diseases

Meta-analyses in orphan diseases and small populations generally face particular problems, including small numbers of studies, small study sizes and heterogeneity of results. However, the heterogeneity is difficult to estimate if only very few studies are included. Motivated by a systematic review in immunosuppression following liver transplantation in children, we investigate the properties of a range of commonly used frequentist and Bayesian procedures in simulation studies. Furthermore, the consequences for interval estimation of the common treatment effect in random-effects meta-analysis are assessed. The Bayesian credibility intervals using weakly informative priors for the between-trial heterogeneity exhibited coverage probabilities in excess of the nominal level for a range of scenarios considered. However, they tended to be shorter than those obtained by the Knapp-Hartung method, which were also conservative. In contrast, methods based on normal quantiles exhibited coverages well below the nominal levels in many scenarios. With very few studies, the performance of the Bayesian credibility intervals is of course sensitive to the specification of the prior for the between-trial heterogeneity. In conclusion, the use of weakly informative priors as exemplified by half-normal priors (with a scale of 0.5 or 1.0) for log odds ratios is recommended for applications in rare diseases. © 2016 The Authors. Research Synthesis Methods published by John Wiley & Sons Ltd.