Use of implicit and explicit bayesian methods in health technology assessment

HTA Barriers for Conditional Approval Drugs

BACKGROUND

Conditional approval pathways facilitate accelerated marketing authorisation based on immature clinical evidence for drugs that address an unmet medical need in a life-threatening or chronically debilitating condition. Lowering evidence requirements for marketing authorisation results in higher clinical uncertainty, which may present challenges for the health technology assessment (HTA) of these products.

OBJECTIVES

The objective of this study is to assess whether conditionally approved drugs face higher probabilities of HTA rejection or delays in HTA approval relative to drugs with standard marketing authorisation.

METHODS

This paper adopts a mixed-methods approach to provide a meta-analysis of HTA outcomes across 80 drug-indication pairs in France, England, Scotland and Canada. Differences in the characteristics (i.e. disease rarity and clinical trial design) of conditionally approved drugs and drugs with standard marketing authorisation and drivers of HTA outcomes are assessed through logistics regressions. Delays in HTA approval are assessed through a survival analysis.

RESULTS

Relative to standard approval drugs, conditionally approved drugs are less likely to include phase III trial designs, less likely to include clinical endpoints and less likely to include an active comparator. Uncertainties in clinical and economic evidence are raised more frequently by HTA agencies for conditionally approved drugs, which have a marginally lower probability of receiving HTA approval relative to drugs with standard approval. Conditionally approved drugs face moderate delays (an average of 6 months) in receiving HTA approval relative to standard approval drugs.

CONCLUSIONS

Overall, conditionally approved drugs likely face increased barriers at the HTA level.

A Bayesian framework for health economic evaluation in studies with missing data

Health economics studies with missing data are increasingly using approaches such as multiple imputation that assume that the data are "missing at random." This assumption is often questionable, as-even given the observed data-the probability that data are missing may reflect the true, unobserved outcomes, such as the patients' true health status. In these cases, methodological guidelines recommend sensitivity analyses to recognise data may be "missing not at random" (MNAR), and call for the development of practical, accessible approaches for exploring the robustness of conclusions to MNAR assumptions. Little attention has been paid to the problem that data may be MNAR in health economics in general and in cost-effectiveness analyses (CEA) in particular. In this paper, we propose a Bayesian framework for CEA where outcome or cost data are missing. Our framework includes a practical, accessible approach to sensitivity analysis that allows the analyst to draw on expert opinion. We illustrate the framework in a CEA comparing an endovascular strategy with open repair for patients with ruptured abdominal aortic aneurysm, and provide software tools to implement this approach.

Keeping Children Safe: a multicentre programme of research to increase the evidence base for preventing unintentional injuries in the home in the under-fives

Background

Unintentional injuries among 0- to 4-year-olds are a major public health problem incurring substantial NHS, individual and societal costs. However, evidence on the effectiveness and cost-effectiveness of preventative interventions is lacking.

Aim

To increase the evidence base for thermal injury, falls and poisoning prevention for the under-fives.

Methods

Six work streams comprising five multicentre case–control studies assessing risk and protective factors, a study measuring quality of life and injury costs, national surveys of children’s centres, interviews with children’s centre staff and parents, a systematic review of barriers to, and facilitators of, prevention and systematic overviews, meta-analyses and decision analyses of home safety interventions. Evidence from these studies informed the design of an injury prevention briefing (IPB) for children’s centres for preventing fire-related injuries and implementation support (training and facilitation). This was evaluated by a three-arm cluster randomised controlled trial comparing IPB and support (IPB+), IPB only (no support) and usual care. The primary outcome was parent-reported possession of a fire escape plan. Evidence from all work streams subsequently informed the design of an IPB for preventing thermal injuries, falls and poisoning.

Results

Modifiable risk factors for falls, poisoning and scalds were found. Most injured children and their families incurred small to moderate health-care and non-health-care costs, with a few incurring more substantial costs. Meta-analyses and decision analyses found that home safety interventions increased the use of smoke alarms and stair gates, promoted safe hot tap water temperatures, fire escape planning and storage of medicines and household products, and reduced baby walker use. Generally, more intensive interventions were the most effective, but these were not always the most cost-effective interventions. Children’s centre and parental barriers to, and facilitators of, injury prevention were identified. Children’s centres were interested in preventing injuries, and believed that they could prevent them, but few had an evidence-based strategic approach and they needed support to develop this. The IPB was implemented by children’s centres in both intervention arms, with greater implementation in the IPB+ arm. Compared with usual care, more IPB+ arm families received advice on key safety messages, and more families in each intervention arm attended fire safety sessions. The intervention did not increase the prevalence of fire escape plans [adjusted odds ratio (AOR) IPB only vs. usual care 0.93, 95% confidence interval (CI) 0.58 to 1.49; AOR IPB+ vs. usual care 1.41, 95% CI 0.91 to 2.20] but did increase the proportion of families reporting more fire escape behaviours (AOR IPB only vs. usual care 2.56, 95% CI 1.38 to 4.76; AOR IPB+ vs. usual care 1.78, 95% CI 1.01 to 3.15). IPB-only families were less likely to report match play by children (AOR 0.27, 95% CI 0.08 to 0.94) and reported more bedtime fire safety routines (AOR for a 1-unit increase in the number of routines 1.59, 95% CI 1.09 to 2.31) than usual-care families. The IPB-only intervention was less costly and marginally more effective than usual care. The IPB+ intervention was more costly and marginally more effective than usual care.

Limitations

Our case–control studies demonstrate associations between modifiable risk factors and injuries but not causality. Some injury cost estimates are imprecise because of small numbers. Systematic reviews and meta-analyses were limited by the quality of the included studies, the small numbers of studies reporting outcomes and significant heterogeneity, partly explained by differences in interventions. Network meta-analysis (NMA) categorised interventions more finely, but some variation remained. Decision analyses are likely to underestimate cost-effectiveness for a number of reasons. IPB implementation varied between children’s centres. Greater implementation may have resulted in changes in more fire safety behaviours.

Conclusions

Our studies provide new evidence about the effectiveness of, as well as economic evaluation of, home safety interventions. Evidence-based resources for preventing thermal injuries, falls and scalds were developed. Providing such resources to children’s centres increases their injury prevention activity and some parental safety behaviours.

Future work

Further randomised controlled trials, meta-analyses and NMAs are needed to evaluate the effectiveness and cost-effectiveness of home safety interventions. Further work is required to measure NHS, family and societal costs and utility decrements for childhood home injuries and to evaluate complex multicomponent interventions such as home safety schemes using a single analytical model.

Trial registration

Current Controlled Trials ISRCTN65067450 and ClinicalTrials.gov NCT01452191.

Funding

The National Institute for Health Research (NIHR) Programme Grants for Applied Research programme and will be published in full inProgramme Grants for Applied Research; Vol. 5, No. 14. See the NIHR Journals Library website for further project information.

Reporting of Bayesian analysis in epidemiologic research should become more transparent

OBJECTIVES

The objective of this systematic review is to investigate the use of Bayesian data analysis in epidemiology in the past decade and particularly to evaluate the quality of research papers reporting the results of these analyses.

STUDY DESIGN AND SETTING

Complete volumes of five major epidemiological journals in the period 2005-2015 were searched via PubMed. In addition, we performed an extensive within-manuscript search using a specialized Java application. Details of reporting on Bayesian statistics were examined in the original research papers with primary Bayesian data analyses.

RESULTS

The number of studies in which Bayesian techniques were used for primary data analysis remains constant over the years. Though many authors presented thorough descriptions of the analyses they performed and the results they obtained, several reports presented incomplete method sections and even some incomplete result sections. Especially, information on the process of prior elicitation, specification, and evaluation was often lacking.

CONCLUSION

Though available guidance papers concerned with reporting of Bayesian analyses emphasize the importance of transparent prior specification, the results obtained in this systematic review show that these guidance papers are often not used. Additional efforts should be made to increase the awareness of the existence and importance of these checklists to overcome the controversy with respect to the use of Bayesian techniques. The reporting quality in epidemiological literature could be improved by updating existing guidelines on the reporting of frequentist analyses to address issues that are important for Bayesian data analyses.

Network meta-analysis to evaluate the effectiveness of interventions to prevent falls in children under age 5 years

BACKGROUND

This study aimed to simultaneously evaluate the effectiveness of a range of interventions to increase the possession of safety equipment or behaviours to prevent falls in children under 5 years of age in the home.

METHODS

A recently published systematic review identified studies to be included in a network meta-analysis; an extension of pairwise meta-analysis that enables comparison of all evaluated interventions simultaneously, including comparisons not directly compared in individual studies.

RESULTS

29 primary studies were identified, of which 16 were included in at least 1 of 4 network meta-analyses. For increasing possession of a fitted stair gate, the most intensive intervention (including education, low cost/free home safety equipment, home safety inspection and fitting) was the most likely to be the most effective, with an OR versus usual care of 7.80 (95% CrI 3.08 to 21.3). For reducing possession or use of a baby walker: education only was most likely to be most effective, with an OR versus usual care of 0.48 (95% CrI 0.31 to 0.84). Little difference was found between interventions for possession of window locks (most intensive intervention versus usual care OR=1.56 (95% CrI 0.02 to 89.8)) and for not leaving a child alone on a high surface (education vs usual care OR=0.89 (95% CrI 0.10 to 9.67)). There was insufficient evidence for network meta-analysis for possession and use of bath mats.

CONCLUSIONS

These results will inform healthcare providers of the most effective components of interventions and can be used in cost-effectiveness analyses.

An exploration of synthesis methods in public health evaluations of interventions concludes that the use of modern statistical methods would be beneficial

OBJECTIVES

To review the methods currently used to synthesize evidence in public health evaluations and demonstrate the availability of more sophisticated approaches.

STUDY DESIGN AND SETTING

A systematic review of National Institute for Health and Care Excellence (NICE) public health appraisals published between 2006 and 2012 was performed to assess the methods used for the synthesis of effectiveness evidence. The ability of new developments in evidence synthesis methodology to address the challenges and opportunities present in a public health context is demonstrated.

RESULTS

Nine (23%) of the 39 NICE appraisals included in the review performed pairwise meta-analyses as part of the effectiveness review with one of these also including a network meta-analysis. Of the remainder, 29 (74.4%) presented narrative summaries of the evidence only, and 1 (2.6%) appraisal did not present any review of effectiveness and/or cost-effectiveness evidence. Heterogeneity of outcomes, methods, and interventions were the main reasons given for not pooling the data. Exploration of quantitative synthesis methods shows that pairwise meta-analyses can be extended to incorporate individual participant data (when it is available), extend the number of interventions being compared using a network meta-analysis, and adjust for both subject- and summary-level covariates. All these can contribute to ensuring the analysis answers directly the policy-relevant questions.

CONCLUSION

More sophisticated methods in evidence synthesis should be considered to make evaluations in public health more useful for decision makers.