Statistical inference for cost-effectiveness ratios

What data counts in policymaking and programming evaluation - Relevant data sources for triangulation according to main epistemologies and philosophies within social science

Policy analysis and program evaluation quality guides how impact is measured, revisions are made, and allocations of resources is deployed. As interdisciplinary research grows in contemporary policy science, the importance of defining truth can be contentious among different social scientists. This research traces the history of triangulation to its contemporary version within the social sciences. The study examines the philosophical evolution influencing the environment in which triangulation develops including: consilience and the comparative linguistic structure of the traditions of thought; and the historical development of methods and emergence of triangulation in research. It also offers contrasting interpretations of triangulation within the various epistemologies and philosophies of science that have arisen in recent movements within social science. As research strives to address novel 21st century issues including big data, pandemics, misinformation, and globalization, there is a need for rigorous social science and policy-based research to be aware of different interpretations of empirical data and valid research methods across disciplines, examine new and old phenomena using multi-method approaches, and validate data informing policy through methods of triangulation.

Comparing the cost-effectiveness of the Otago Exercise Programme among older women and men: A secondary analysis of a randomized controlled trial

Objective

Using stratified analyses, we examined the cost-effectiveness of the Otago Exercise Programme (OEP), from a health care system perspective, among older women and men who have previously fallen.

Methods

This study was a secondary stratified analysis (by women and men), of a 12-month prospective economic evaluation of a randomized clinical trial (OEP compared with usual care). Three hundred and forty four community-dwelling older adults (≥70; 172 OEP (110 women; 62 men), 172 usual care (119 women; 53 men)) who sustained a fall in the past 12 months and received a baseline assessment at the Vancouver Falls Prevention Clinic, Canada were included. A gender by OEP/usual care interaction was examined for the falls incidence rate ratio (IRR). Outcome measures stratified by gender included: falls IRR, incremental cost-per fall prevented (ICER), incremental cost per quality adjusted life year (QALY, ICUR) gained, and mean total health care resource utilization costs.

Results

Men were frailer than women at baseline. Men incurred higher mean total healthcare costs $6794 (SD: $11906)). There was no significant gender by OEP/usual care interaction on falls IRR. The efficacy of the OEP did not vary by gender. The adjusted IRR for the OEP group demonstrated a 39% (IRR: 0.61, CI: 0.40–0.93) significant reduction in falls among men but not women (32% reduction (IRR: 0.69, CI: 0.47–1.02)). The ICER showed the OEP was effective in preventing falls and less costly for men, while it was costlier for women by $42. The ICUR showed the OEP did not impact quality of life.

Conclusion

Future studies should explore gender factors (i.e., health seeking behaviours, gender related frailty) that may explain observed variation in the cost-effectiveness of the OEP as a secondary falls prevention strategy.

Trial registrations

ClinicalTrials.gov Protocol Registration System

Identifier: NCT01029171; URL: https://clinicaltrials.gov/ct2/show/NCT01029171

Identifier: NCT00323596; URL: https://clinicaltrials.gov/ct2/show/NCT00323596

Action Seniors! Cost-Effectiveness Analysis of a Secondary Falls Prevention Strategy Among Community-Dwelling Older Fallers

BACKGROUND

The Otago Exercise Program (OEP) has demonstrated cost-effectiveness for the primary prevention of falls in a general community setting. The cost-effectiveness of exercise as a secondary falls prevention (ie, preventing falls among those who have already fallen) strategy remains unknown. The primary objective was to estimate the cost-effectiveness (incremental cost-effectiveness/utility ratio) of the OEP from a healthcare system perspective.

DESIGN

A concurrent 12-month prospective economic evaluation conducted alongside the Action Seniors! randomized critical trial (OEP compared with usual care).

SETTING

Vancouver Falls Prevention Clinic (Vancouver, BC, Canada; http://www.fallsclinic.ca).

PARTICIPANTS

A total of 344 community-dwelling older adults, aged 70 years and older, who attended a geriatrician-led Falls Prevention Clinic in Vancouver, after sustaining a fall in the previous 12 months.

MEASUREMENTS

Main outcome measures included: incidence rate ratio for falls, healthcare costs, incremental cost per fall prevented, and incremental cost per quality-adjusted life year (QALY) gained.

RESULTS

The OEP costs $393 CAD per participant to implement. The incremental cost per fall prevented resulted in a savings of $2 CAD. The incremental cost per QALY gained (where QALYs were estimated using the Euro-Qol 5D three-level version [EQ-5D-3L]) indicated the OEP was less effective than usual care. The incremental cost per QALY gained (where QALYs were estimated using the Short Form 6D [SF-6D]) indicated the OEP was more effective and less costly than usual care. The incremental QALYs estimated using the EQ-5D-3L and the SF-6D were not clinically significant and close to zero, indicating no change in quality of life.

CONCLUSION

Compared with usual care, healthcare system costs are saved and falls are prevented when older fallers who attend a geriatrician-led falls clinic are allocated to, and provided, the physiotherapist-guided exercise-based falls prevention program (the OEP).

Economic evaluation of aerobic exercise training in older adults with vascular cognitive impairment: PROMoTE trial

BACKGROUND/OBJECTIVES

Evidence suggests that aerobic exercise may slow the progression of subcortical ischaemic vascular cognitive impairment (SIVCI) by modifying cardiovascular risk factors. Yet the economic consequences relating to aerobic training (AT) remain unknown. Therefore, our primary objective was to estimate the incremental cost per quality-adjusted life years (QALYs) gained of a thrice weekly AT intervention compared with usual care.

DESIGN

Cost-utility analysis alongside a randomised trial.

SETTING

Vancouver, British Columbia, Canada.

PARTICIPANTS

70 adults (mean age of 74 years, 51% women) who meet the diagnostic criteria for mild SIVCI.

INTERVENTION

A 6-month, thrice weekly, progressive aerobic exercise training programme compared with usual care (CON; comparator) with a follow-up assessment 6 months after formal cessation of aerobic exercise training.

MEASUREMENTS

Healthcare resource usage was estimated over the 6-month intervention and 6-month follow-up period. Health status (using the EQ-5D-3L) at baseline and trial completion and 6-month follow-up was used to calculate QALYs. The incremental cost-utility ratio (cost per QALY gained) was calculated.

RESULTS

QALYs were both modestly greater, indicating a health gain. Total healthcare costs (ie, 1791±1369 {2015 $CAD} at 6 months) were greater, indicating a greater cost for the thrice weekly AT group compared with CON. From the Canadian healthcare system perspective, the incremental cost-utility ratios for thrice weekly AT were cost-effective compared with CON, when using a willingness to pay threshold of $CAD 20 000 per QALY gained or higher.

CONCLUSIONS

AT represents an attractive and potentially cost-effective strategy for older adults with mild SIVCI.

TRIAL REGISTRATION NUMBER

NCT01027858.

Tolerance limits and tolerance intervals for ratios of normal random variables using a bootstrap calibration

This paper addresses the problem of deriving one-sided tolerance limits and two-sided tolerance intervals for a ratio of two random variables that follow a bivariate normal distribution, or a lognormal/normal distribution. The methodology that is developed uses nonparametric tolerance limits based on a parametric bootstrap sample, coupled with a bootstrap calibration in order to improve accuracy. The methodology is also adopted for computing confidence limits for the median of the ratio random variable. Numerical results are reported to demonstrate the accuracy of the proposed approach. The methodology is illustrated using examples where ratio random variables are of interest: an example on the radioactivity count in reverse transcriptase assays and an example from the area of cost-effectiveness analysis in health economics.

Cost-effectiveness analysis: a proposal of new reporting standards in statistical analysis

Cost-effectiveness analysis (CEA) is a method for evaluating the outcomes and costs of competing strategies designed to improve health, and has been applied to a variety of different scientific fields. Yet there are inherent complexities in cost estimation and CEA from statistical perspectives (e.g., skewness, bidimensionality, and censoring). The incremental cost-effectiveness ratio that represents the additional cost per unit of outcome gained by a new strategy has served as the most widely accepted methodology in the CEA. In this article, we call for expanded perspectives and reporting standards reflecting a more comprehensive analysis that can elucidate different aspects of available data. Specifically, we propose that mean- and median-based incremental cost-effectiveness ratios and average cost-effectiveness ratios be reported together, along with relevant summary and inferential statistics, as complementary measures for informed decision making.

An economic evaluation of resistance training and aerobic training versus balance and toning exercises in older adults with mild cognitive impairment

Background

Mild cognitive impairment (MCI) represents a critical window to intervene against dementia. Exercise training is a promising intervention strategy, but the efficiency (i.e., relationship of costs and consequences) of such types of training remains unknown. Thus, we estimated the incremental cost-effectiveness of resistance training or aerobic training compared with balance and tone exercises in terms of changes in executive cognitive function among senior women with probable MCI.

Methods

Economic evaluation conducted concurrently with a six-month three arm randomized controlled trial including eighty-six community dwelling women aged 70 to 80 years living in Vancouver, Canada. Participants received twice-weekly resistance training (n = 28), twice weekly aerobic training (n = 30) or twice-weekly balance and tone (control group) classes (n = 28) for 6 months. The primary outcome measure of the Exercise for Cognition and Everyday Living (EXCEL) study assessed executive cognitive function, a test of selective attention and conflict resolution (i.e., Stroop Test). We collected healthcare resource utilization costs over six months.

Results

Based on the bootstrapped estimates from our base case analysis, we found that both the aerobic training and resistance training interventions were less costly than twice weekly balance and tone classes. Compared with the balance and tone group, the resistance-training group had significantly improved performance on the Stroop Test (p = 0.04).

Conclusions

Resistance training and aerobic training result in health care cost saving and are more effective than balance and tone classes after only 6 months of intervention. Resistance training is a promising strategy to alter the trajectory of cognitive decline in seniors with MCI.

Trial Registration

ClinicalTrials.gov NCT00958867.

Average cost-effectiveness ratio with censored data

In cost-effectiveness analysis, interest could lie foremost in the incremental cost-effectiveness ratio (ICER), which is the ratio of the incremental cost to the incremental benefit of two competing interventions. The average cost-effectiveness ratio (ACER) is the ratio of the cost to benefit of an intervention without reference to a comparator. A vast literature is available for statistical inference of the ICERs, but limited methods have been developed for the ACERs, particularly in the presence of censoring. Censoring is a common feature in prospective studies, and valid analyses should properly adjust for censoring in cost as well as in effectiveness. In this article, we propose statistical methods for constructing a confidence interval for the ACER from censored data. Different methods-Fieller, Taylor, bootstrap-are proposed, and through simulation studies and data analysis, we address the performance characteristics of these methods.

Individualized cost-effectiveness analysis

John Ioannidis and Alan Garber discuss how to use incremental cost-effectiveness ratios (ICER) and related metrics so they can be useful for decision-making at the individual level, whether used by clinicians or individual patients.

Economic evaluation of dose-response resistance training in older women: a cost-effectiveness and cost-utility analysis

SUMMARY

We estimated the incremental cost-effectiveness of a once-weekly or twice-weekly resistance training intervention compared with balance and tone classes in terms of falls prevented and quality-adjusted life years (QALYs) gained. Both resistance training interventions were more likely to save health care resource money and offer better health outcomes for falls prevention than balance and tone classes.

INTRODUCTION

This study aims to estimate the incremental cost-effectiveness and cost-utility of a once-weekly or twice-weekly resistance training intervention compared with twice-weekly balance and tone classes in terms of falls prevented and QALYs gained.

METHODS

Economic evaluation was conducted concurrently with a three-arm randomized controlled trial including 155 community-dwelling women aged 65 to 75 years, Mini Mental State Examination ≥24, and visual acuity 20/40 or better. Participants received the once-weekly resistance training (n = 54), the twice-weekly resistance training (n = 51) or the twice-weekly balance and tone (the comparator) classes (n = 50) for 1 year. Measurements included the number of falls for each participant, healthcare resource utilization, and associated costs over 9 months; health status was assessed using the EQ-5D and SF-6D to calculate QALYs.

RESULTS

Based on the point estimates from our base case analysis, we found that both once- and twice-weekly resistance training groups were less costly (p < 0.05) and more effective than twice-weekly balance and tone classes. The incremental QALYs assessed using the SF-6D were 0.003 for both the once- and twice-weekly resistance training groups, compared with the twice-weekly balance and tone classes. The incremental QALYs assessed using the EQ-5D were 0.084 for the once-weekly and 0.179 for the twice-weekly resistance training groups, respectively, compared with the twice-weekly balance and tone classes.

CONCLUSIONS

An individually tailored resistance training intervention delivered once or twice weekly provided better value for money for falls prevention than balance and tone classes.

Regression models for analyzing costs and their determinants in health care: an introductory review

OBJECTIVE

This article aims to describe the various approaches in multivariable modelling of healthcare costs data and to synthesize the respective criticisms as proposed in the literature.

METHODS

We present regression methods suitable for the analysis of healthcare costs and then apply them to an experimental setting in cardiovascular treatment (COSTAMI study) and an observational setting in diabetes hospital care.

RESULTS

We show how methods can produce different results depending on the degree of matching between the underlying assumptions of each method and the specific characteristics of the healthcare problem.

CONCLUSIONS

The matching of healthcare cost models to the analytic objectives and characteristics of the data available to a study requires caution. The study results and interpretation can be heavily dependent on the choice of model with a real risk of spurious results and conclusions.

A clinician's guide to correct cost-effectiveness analysis: think incremental not average

OBJECTIVE

To explain how to correctly report the results from a cost-effectiveness analysis (CEA).

METHODS

Results were used from a hypothetical clinical trial to illustrate how different ways of reporting economic results affect both presentation of findings and formulation of conclusions. To provide context, we reviewed some high-profile exchanges in the scientific literature.

RESULTS

The critical issue with which decision makers must grapple involves the trade-offs introduced by a new treatment or intervention. Specifically, are decision makers willing to pay the additional cost for the additional outcomes? This question cannot be considered without estimates of the additional cost and additional outcomes. Correct cost-effectiveness measures, such as the incremental cost-effectiveness ratio or the incremental net benefit, address this issue.

CONCLUSIONS

As decision makers face the challenge of balancing increasing health care demand with cost containment, it will be crucial to identify cost-effective ways of providing care. Health care providers and other decision makers should not be misled by the results of improperly reported CEAs. Decisions around adoption of pharmaceuticals or implementation of new programs or interventions may be affected by which cost-effectiveness summary measure is reported. Thus consumers of CEA must have a basic understanding of why different methods give different results, and how the results should be interpreted.

Unrelated medical care in life years gained and the cost utility of primary prevention: in search of a 'perfect' cost-utility ratio

An important subject of debate in cost-utility analysis of health care programmes is whether to include costs of unrelated medical care in life years gained. The inclusion of such costs is likely to be of consequence in the case of primary prevention. This paper presents different strategies regarding the inclusion not only of the costs, but also of the health effects of unrelated medical care in economic evaluations. Four different cost-utility ratios are presented and related to the criterion of internal consistency. In addition, the possibility to relate the ratios to a well-posed decision problem is analysed. An example computes the different ratios for smoking cessation interventions in different age groups. Including health care costs of unrelated medical care in life years gained increases cost utility ratios, but excluding unrelated medical costs favours smoking cessation interventions targeted at older smokers over those at younger smokers. We conclude that for primary prevention only a cost utility ratio that includes both the costs and effects of unrelated medical care meets the criterion of internal consistency and is related to a meaningful decision problem. Therefore, this type of cost-utility ratio should be preferred even if the data requirements may be substantial.

Nonparametric statistical methods for cost-effectiveness analyses

Two measures often used in a cost-effectiveness analysis are the incremental cost-effectiveness ratio (ICER) and the net health benefit (NHB). Inferences on these two quantities are often hindered by highly skewed cost data. In this article, we derive the Edgeworth expansions for the studentized t-statistics for the two measures and show how they could be used to guide inferences. In particular, we use the expansions to study the theoretical performance of existing confidence intervals based on normal theory and to derive new confidence intervals for the ICER and the NHB. We conduct a simulation study to compare our new intervals with several existing methods. The methods evaluated include Taylor's interval, Fieller's interval, the bootstrap percentile interval, and the bootstrap bias-corrected acceleration interval. We found that our new intervals give good coverage accuracy and are narrower compared to the current recommended intervals.

How sensitive are cost-effectiveness analyses to choice of parametric distributions?

BACKGROUND

Cost-effectiveness analyses of clinical trial data are based on assumptions about the distributions of costs and effects. Cost data usually have very skewed distributions and can be difficult to model. The authors investigate whether choice of distribution can make a difference to the conclusions drawn.

METHODS

The authors compare 3 distributions for cost data-normal, gamma, and lognormal-using similar parametric models for the cost-effectiveness analyses. Inferences on the cost-effectiveness plane are derived, together with cost-effectiveness acceptability curves. These methods are applied to data from a trial of rapid magnetic resonance imaging (rMRI) investigation in patients with low back pain.

RESULTS

The gamma and lognormal distributions fitted the cost data much better than the normal distribution. However, in terms of inferences about cost-effectiveness, it was the normal and gamma distributions that gave similar results. Using the lognormal distribution led to the conclusion that rMRI was cost-effective for a range of willingness-to-pay values where assuming a gamma or normal distribution did not.

CONCLUSIONS

Conclusions from cost-effectiveness analyses are sensitive to choice of distribution and, in particular, to how the upper tail of the cost distribution beyond the observed data is modeled. How well a distribution fits the data is an insufficient guide to model choice. A sensitivity analysis is therefore necessary to address uncertainty about choice of distribution.

Cost-effectiveness of bupropion, nortriptyline, and psychological intervention in smoking cessation

Sustained-release bupropion and nortriptyline have been shown to be efficacious in treating cigarette smoking. Psychological intervention is also recognized as efficacious. The cost and cost-effectiveness of the 2 drug therapies have not been estimated. It was hypothesized that nortriptyline would be more cost-effective than bupropion. Hypotheses were not originally proposed concerning the cost-effectiveness of psychological versus drug treatment, but the 2 were compared using exploratory analyses. This was a 3 (bupropion versus nortriptyline versus placebo) by 2 (medical management alone versus medical management plus psychological intervention) randomized trial. Participants were 220 cigarette smokers. Outcome measures were cost and cost-effectiveness computed at week 52. Nortriptyline cost less than bupropion. Nortriptyline was more cost-effective than bupropion; the difference was not statistically significant. Psychological intervention cost less than the 2 drug treatments, and was more cost-effective, but not significantly so. Prospective investigations of the cost and cost-effectiveness of psychological and pharmacological intervention, using adequate sample sizes, are warranted.

Optimal allocation of resources over health care programmes: dealing with decreasing marginal utility and uncertainty

This paper addresses the problem of how to value health care programmes with different ratios of costs to effects, specifically when taking into account that these costs and effects are uncertain. First, the traditional framework of maximising health effects with a given health care budget is extended to a flexible budget using a value function over money and health effects. Second, uncertainty surrounding costs and effects is included in the model using expected utility. Other approaches to uncertainty that do not specify a utility function are discussed and it is argued that these also include implicit notions about risk attitude.

Using covariates to reduce uncertainty in the economic evaluation of clinical trial data

As part of their practice, policymakers have to make economic evaluations using clinical trial data. Recent interest has been expressed in determining how cost-effectiveness analysis can be undertaken in a regression framework. In this respect, published research basically provides a general method for prognostic factor adjustment in the presence of imbalance, emphasizing sub-group analysis. In this paper, we present an alternative method from a Bayesian approach. We propose the use of covariates in Bayesian health technology assessment in order to reduce uncertainty about the effect of treatments. We show its advantages by comparison with another published method that do not adjust for covariates using simulated data.

An examination of quinone toxicity using the yeast Saccharomyces cerevisiae model system

The toxicity of quinones is generally thought to occur by two mechanisms: the formation of covalent bonds with biological molecules by Michael addition chemistry and the catalytic reduction of oxygen to superoxide and other reactive oxygen species (ROS) (redox cycling). In an effort to distinguish between these general mechanisms of toxicity, we have examined the toxicity of five quinones to yeast cells as measured by their ability to reduce growth rate. Yeast cells can grow in the presence and absence of oxygen and this feature was used to evaluate the role of redox cycling in the toxicity of each quinone. Furthermore, yeast mutants deficient in superoxide dismutase (SOD) activity were used to assess the role of this antioxidant enzyme in protecting cells against quinone-induced reactive oxygen toxicity. The effects of different quinones under different conditions of exposure were compared using IC50 values (the concentration of quinone required to inhibit growth rate by 50%). For the most part, the results are consistent with the chemical properties of each quinone with the exception of 9,10-phenanthrenequinone (9,10-PQ). This quinone, which is not an electrophile, exhibited an unexpected toxicity under anaerobic conditions. Further examination revealed a potent induction of cell viability loss which poorly correlated with decreases in the GSH/2GSSG ratio but highly correlated (r2 > 0.7) with inhibition of the enzyme glyceraldehyde-3-phosphate dehydrogenase (GAPDH), suggesting disruption of glycolysis by this quinone. Together, these observations suggest an unexpected oxygen-independent mechanism in the toxicity of 9,10-phenanthrenequinone.

Economic endpoints in clinical trials

Healthcare decision makers are increasingly requesting information on the cost and cost-effectiveness of new medicines at the time of product launch. In order to provide this information, data on healthcare resource utilization and, in some cases, costs, may be collected in clinical trials. In this paper, we discuss some of the issues statisticians need to address when it is appropriate to include these economic endpoints in the trial. Several design issues are discussed, including the alternative types of and methods for collecting economic endpoint data, sample size and generalizability. Alternative approaches in the analysis of resource utilization, cost and cost-effectiveness are also presented. Finally, several of the analytic approaches are applied to actual data from a clinical trial.

Bayesian estimation of cost-effectiveness from censored data

We describe a Bayesian methodology for estimating the cost-effectiveness of a new treatment compared to a standard in a clinical trial, when censoring of survival, the effectiveness variable, induces censoring of total cost. The statistical model assumes that survival follows a Weibull distribution and that total health care cost follows a gamma distribution whose mean has a linear regression on survival time. We summarize the posterior distributions of key parameters by importance sampling. We illustrate the method with an analysis of data from a randomized clinical trial of a treatment for cardiovascular disease.

A Bayesian approach to aid in formulary decision making: incorporating institution-specific cost-effectiveness data with clinical trial results

Pharmacy and therapeutics committees commonly cite a lack of generalizability as a reason for not incorporating cost-effectiveness information into decision making. To address this concern, many committees undertake site-specific economic evaluations, which are often limited by small sample sizes and nonrandomized designs. We show how 2 complementary approaches were used to minimize these limitations in an economic evaluation of abciximab at 1 institution. Using a propensity score methodology, we selected patients who did not receive abciximab for the comparison cohort. Then, we adopted a Bayesian, hierarchical, random-effects model to integrate site-specific and clinical trial data. We applied the posterior distributions of effectiveness with local cost data in a traditional decision-analytic model. In 74% of the simulations, abciximab was cost-effective at 1 institution at the $50,000 per life year saved threshold, assuming a 50:50 split of patients undergoing coronary stenting and angioplasty. Among patients undergoing coronary stenting, the cost-effectiveness ratio of the addition of abciximab was at or below the $50,000 per life year saved threshold in 66.0% of the simulations.

Bayesian methods for design and analysis of cost-effectiveness trials in the evaluation of health care technologies

We review the development of Bayesian statistical methods for the design and analysis of randomized controlled trials in the assessment of the cost-effectiveness of health care technologies. We place particular emphasis on the benefits of the Bayesian approach; the implications of skew cost data; the need to model the data appropriately to generate efficient and robust inferences instead of relying on distribution-free methods; the importance of making full use of quantitative and structural prior information to produce realistic inferences; and issues in the determination of sample size. Several new examples are presented to illustrate the methods. We conclude with a discussion of the key areas for future research.

Statistical determination of cost-effectiveness frontier based on net health benefits

Statistical methods are given for producing a cost-effectiveness frontier for an arbitrary number of programs. In the deterministic case, the net health benefit (NHB) decision rule is optimal; the rule funds the program with the largest positive NHB at each lambda, the amount a decision-maker is willing to pay for an additional unit of effectiveness. For bivariate normally distributed cost and effectiveness variables and a specified lambda, a statistical procedure is presented, based on the method of constrained multiple comparisons with the best (CMCB), for determining the program with the largest NHB. A one-tailed t test is used to determine if the NHB is positive. To obtain a statistical frontier in the lambda-NHB plane, we develop a method to produce the region in which each program has the largest NHB, by pivoting a CMCB confidence interval. A one-sided version of Fieller's theorem is used to determine the region where the NHB of each program is positive. At each lambda, the pointwise error rate is bounded by a prespecified alpha. Upper bounds on the familywise error rate, the probability of an error at any value of lambda, are given. The methods are applied to a hypothetical clinical trial of antipsychotic agents.

The familywise error rate of a simultaneous confidence band for the incremental net health benefit

Interest in the use of net health benefit in cost-effectiveness analysis derives from its optimality property for decision-making. A description of the results of an economic evaluation of health care interventions is incomplete if it does not include point and interval estimates of this outcome measure. A simultaneous confidence band for the incremental net health benefit, INHB(lambda), for all lambda may be obtained by forming a confidence interval based on student's t statistic, and letting the willingness-to-pay value, lambda, run over all values. The familywise error rate (FWER) of the simultaneous confidence band is the probability that the confidence interval does not cover the true INHB(lambda) for some value of lambda. We show that the FWER equals P(T(2)>t(2)), where T(2) follows Hotelling's central distribution and that the simultaneous confidence band does not cover the true INHB(lambda) if and only if a T(2) based confidence ellipsoid does not cover the true mean c-e vector.

The probability of cost-effectiveness

BACKGROUND

The study of cost-effectiveness comparisons between competing medical interventions has led to a variety of proposals for quantifying cost-effectiveness. The differences between the various approaches can be subtle, and one purpose of this article is to clarify some important distinctions.

DISCUSSION

We discuss alternative measures in the framework of individual, patient-level, incremental net benefits. In particular we examine the probability of cost-effectiveness for an individual, proposed by Willan.

SUMMARY

We argue that this is a useful addition to the range of cost-effectiveness measures, but will be of secondary interest to most decision makers. We also demonstrate that Willan's proposed estimate of this probability is logically flawed.

Probabilistic cost-effectiveness analysis of HIV prevention. Comparing a Bayesian approach with traditional deterministic sensitivity analysis

In cost-effectiveness analysis, the incremental cost-effectiveness ratio is used to measure economic efficiency of a new intervention, relative to an existing one. However, costs and effects are seldom known with certainty. Uncertainty arises from two main sources: uncertainty regarding correct values of intervention-related parameters and uncertainty associated with sampling variation. Recently, attention has focused on Bayesian techniques for quantifying uncertainty. We computed the Bayesian-based 95% credible interval estimates of the incremental cost-effectiveness ratio of several related HIV prevention interventions and compared these results with univariate sensitivity analyses. The conclusions were comparable, even though the probabilistic technique provided additional information.

A framework for cost-effectiveness analysis from clinical trial data

We present a general Bayesian framework for cost-effectiveness analysis (CEA) from clinical trial data. This framework allows for very flexible modelling of both cost and efficacy related trial data. A common CEA technique is established for this wide class of models through linking mean efficacy and mean cost to the parameters of any given model. Examples are given in which efficacy may be measured as a continuous, binary, ordinal or time-to-event outcome, and in which costs are modelled as distributed normally, log-normally, as a mixture or non-parametrically. A case study is presented, illustrating the methodology and illuminating the role of prior information.

Statistical cost-effectiveness analysis of two treatments based on net health benefits

Statistical methods for cost-effectiveness analysis (CEA) for two treatments that mimic the deterministic optimal rules of CEA are presented. In these rules the objective is to determine the treatment with the maximal effectiveness whose unit cost is less than an amount, lambda, that a decision-maker is willing to pay (WTP). This is accomplished by identifying the treatment with the largest positive net health benefit (NHB), which is a function of lambda, while controlling the familywise error rate both when the WTP value is given and when it is unspecified. Fieller's theorem is used to determine a region of WTP values where the NHBs of the treatments are not distinguishable. For each lambda outside of the confidence region, the larger treatment is identified. A newly developed one-tailed analogue of Fieller's theorem is used to determine the WTP values where a treatment's NHB is positive. The situation in which both treatments are experimental is distinguished from the case where one of the treatments is usual care. The one-tailed confidence region is used in the latter case to obtain the lambda values where the NHBs are not different, and determining the region of positivity of the NHBs may be unnecessary. An example is presented in which the cost-effectiveness of two antipsychotic treatments is evaluated.

Bayesian cost-effectiveness analysis from clinical trial data

A key tool for assessing the relative cost-effectiveness of two treatments in health economics is the incremental C/E acceptability curve. We present Bayesian computations for this curve in the case where data on both costs and efficacy are available from a clinical trial. Analysis is given under various formulations of prior information. A case study is analysed in which reasonable prior information is shown to strengthen substantially the posterior inference, leading to a more conclusive assessment of cost-effectiveness. Calculations can be performed using readily available Bayesian software.

The death of cost-minimization analysis?

Four different types of evaluation methods, cost-benefit analysis (CBA), cost-utility analysis (CUA), cost-effectiveness analysis (CEA) and cost-minimization analysis (CMA), are usually distinguished. In this note, we pronounce the (near) death of CMA by showing the rare circumstances under which CMA is an appropriate method of analysis. We argue that it is inappropriate for separate and sequential hypothesis tests on differences in effects and costs to determine whether incremental cost-effectiveness (or cost-utility) should be estimated. We further argue that the analytic focus should be on the estimation of the joint density of cost and effect differences, the quantification of uncertainty surrounding the incremental cost-effectiveness ratio and the presentation of such data as cost-effectiveness acceptability curves. Two examples from recently published CEA are employed to illustrate the issues. The first shows a situation where analysts might be tempted (inappropriately) to employ CMA rather than CEA. The second illustrates one of the rare circumstances in which CMA may be justified as a legitimate form of analysis.

Parameter solicitation for planning cost effectiveness studies with dichotomous outcomes

When economic endpoints are included alongside clinical effectiveness measures in randomized clinical trials (RCT), they are summarized together by the incremental cost effectiveness ratio (ICER). Adding economic endpoints to an RCT complicates the planning of experiments because investigators must now solicit their beliefs about costs, but even more challenging, they must also specify their association with effectiveness. Solicitation of correlations between costs and effects can be unintuitive, and so potentially highly inaccurate. This is unfortunate because power is highly sensitive to the association between costs and effects. Mis-specification in this association may lead to substantially underpowered or overpowered studies. We show that when clinical effectiveness measures are dichotomous, specification of the correlation between costs and effects can be avoided by instead describing their association with a mixture model. This representation leads to simple and highly intuitive parameter specifications. It may also be used to generate realistic raw data that can be used to evaluate experiment power with simulation. We give particular attention to evaluating and interpreting power when Fieller's theorem method (FTM) is used to calculate confidence for, and test hypotheses about, the ICER. Data from a previously published clinical trial are used to demonstrate the use of this new method to calculate sample size for a cost effectiveness study.

Use of an angular transformation for ratio estimation in cost-effectiveness analysis

Economic evaluations of medical technologies involve a consideration of both costs and clinical benefits, and an increasing number of clinical studies include a specific objective of assessing cost-effectiveness. These studies measure the trade-off between costs and benefits using the cost-effectiveness ratio (CE ratio), which is defined as the net incremental cost per unit of benefit provided by the candidate therapy. In this paper we review the statistical methods which have been proposed for estimating 95 per cent confidence intervals for cost-effectiveness ratios. We show that the use of an angular transformation of the standardized ratio stabilizes the variance of the estimated CE ratio, and provides a clearer interpretation of study results. An estimate of the 95 per cent confidence interval for the CE ratio in the transformed scale is easily made using the jack-knife or bootstrap. The available methods are compared using data from a long term study of mortality in patients with congestive heart failure.

Fieller's method and net health benefits

Statistical and conceptual difficulties complicate the estimation of the incremental cost-effectiveness ratio (ICER). An alternative approach is to measure cost-effectiveness by the incremental net health benefit (INHB), defined as the difference in mean effectiveness of a new treatment compared with a standard, adjusted for cost difference by subtracting the health foregone if purchasing care at the rate of a marginally cost-effective therapy. Because net health benefit (NHB) is dependent on this threshold rate, one can construct confidence intervals for the INHB at various values of the rate. It turns out that the set of rates where new and standard are not significantly different is equal to the Fieller's method confidence set for the ICER. We review the derivation of the Fieller's method confidence set, present numerical examples, and discuss the implications of our result for the calculation and interpretation of NHB analyses.

Power and sample assessments for tests of hypotheses on cost-effectiveness ratios

We address the issue of statistical power and sample size for cost-effectiveness studies. Tests of hypotheses on the cost-effectiveness ratio (CER) are constructed from the net cost and incremental effectiveness measures. When the difference in effectiveness is known, we derive formulae for statistical power and sample size assessments for one- and two-sided tests of hypotheses of the CER. We also construct a test of the joint hypothesis of cost-effectiveness and effectiveness and derive an expression connecting power and sample size. Our methods account for the correlation between cost and effectiveness and lead to smaller sample size requirements than comparative methods that ignore the correlation. The implications of our formulae for cost-effectiveness studies are illustrated through numerical examples. When compared with trials designed to demonstrate effectiveness alone, our results indicate that a trial appropriately powered to demonstrate cost-effectiveness might require sample sizes many times greater.

Inference for the cost-effectiveness acceptability curve and cost-effectiveness ratio

The aim of this article is to consider Bayesian and frequentist inference methods for measures of incremental cost effectiveness in data obtained via a clinical trial. The most useful measure is the cost-effectiveness (C/E) acceptability curve. Recent publications on Bayesian estimation have assumed a normal posterior distribution, which ignores uncertainty in estimated variances, and suggest unnecessarily complicated methods of computation. We present a simple Bayesian computation for the C/E acceptability curve and a simple frequentist analogue. Our approach takes account of errors in estimated variances, resulting in calculations that are based on distributions rather than normal distributions. If inference is required about the C/E ratio, we argue that the standard frequentist procedures give unreliable or misleading inferences, and present instead a Bayesian interval.

Resampling and multiplicity in cost-effectiveness inference

We compare published methods for placing statistical confidence limits around incremental cost-effectiveness ratio statistics and show that only a nonparametric, bootstrap approach using polar angles gives completely reasonable results when neither treatment has significant advantages in cost or effectiveness. We also discuss alternative ways to report analytical results using plots, confidence or tolerance limits, and quadrant acceptability fractions. Finally, we use simulation to study the multiplicity bias that can be introduced into ICER confidence limits when only the most favorable results are reported over several possible choices of numerator cost measure and denominator effectiveness measure.

Stochastic cost-effectiveness analysis: a simultaneous marginal-effect approach

OBJECTIVE

The purpose of this study is to develop a cost-effectiveness methodology in the context of a simultaneous modeling framework that provides consistent point and interval estimates.

METHODS

A simultaneous model of cost and effectiveness functions was developed to measure the incremental cost-effectiveness ratio for competing medical interventions. A feasible nonlinear least-squares method was suggested to estimate the simultaneous model. Using a series of hypothetical data, a simulation analysis was performed to show the superior performance of the proposed model, relative to the average-effect model, a widely used approach to cost-effectiveness estimation.

RESULTS

The traditional average-effect approach has two shortcomings. First, it assumes two strong conditions: truly random distributions of all the significant nontreatment variables (both observed and unobserved) across study groups, and the independence of cost and effectiveness variables. Second, it does not give the confidence interval, an important measure to assess the stochastic nature and robustness of point estimates. In contrast, the simultaneous modeling approach provides marginal-effect estimates, imposing no restrictions on the random distributions of the individual characteristics across study groups. Furthermore, it takes into account the simultaneity of cost and effectiveness functions being estimated. The simulation analysis showed that the simultaneous modeling approach is significantly more unbiased and efficient in predicting the true cost-effectiveness ratio.

CONCLUSION

The simultaneous modeling approach is superior to the average-effect approach in the estimation of incremental cost-effectiveness ratios using data with significant nontreatment confounding factors. The advantages of the simultaneous modeling approach are particularly appealing for evaluative studies dealing with large-scale retrospective data at the patient level.

Power and sample size in cost-effectiveness analysis

For resource allocation under a constrained budget, optimal decision rules for mutually exclusive programs require that the treatment with the highest incremental cost-effectiveness ratio (ICER) below a willingness-to-pay (WTP) criterion be funded. This is equivalent to determining the treatment with the smallest net health cost. The designer of a cost-effectiveness study needs to select a sample size so that the power to reject the null hypothesis, the equality of the net health costs of two treatments, is high. A recently published formula derived under normal distribution theory overstates sample-size requirements. Using net health costs, the authors present simple methods for power analysis based on conventional normal and on nonparametric statistical theory.

On the use of survival analysis techniques to estimate medical care costs

Measurement of treatment costs is important in the evaluation of medical interventions. Accurate cost estimation is problematic, when cost records are incomplete. Methods from the survival analysis literature have been proposed for estimating costs using available data. In this article, we clarify assumptions necessary for validity of these techniques. We demonstrate how assumptions needed for valid survival analysis may be violated when these methods are applied to cost estimation. Our observations are confirmed through simulations and empirical data analysis. We conclude that survival analysis approaches are not generally appropriate for the analysis of medical costs and review several valid alternatives.

Bayesian estimation of cost-effectiveness ratios from clinical trials

Estimation of the incremental cost-effectiveness ratio (ICER) is difficult for several reasons: treatments that decrease both cost and effectiveness and treatments that increase both cost and effectiveness can yield identical values of the ICER; the ICER is a discontinuous function of the mean difference in effectiveness; and the standard estimate of the ICER is a ratio. To address these difficulties, we have developed a Bayesian methodology that involves computing posterior probabilities for the four quadrants and separate interval estimates of ICER for the quadrants of interest. We compute these quantities by simulating draws from the posterior distribution of the cost and effectiveness parameters and tabulating the appropriate posterior probabilities and quantiles. We demonstrate the method by re-analysing three previously published clinical trials.

Performing cost-effectiveness analysis by integrating randomized trial data with a comprehensive decision model: application to treatment of acute ischemic stroke

A recent national panel on cost-effectiveness in health and medicine has recommended that cost-effectiveness analysis (CEA) of randomized controlled trials (RCTs) should reflect the effect of treatments on long-term outcomes. Because the follow-up period of RCTs tends to be relatively short, long-term implications of treatments must be assessed using other sources. We used a comprehensive simulation model of the natural history of stroke to estimate long-term outcomes after a hypothetical RCT of an acute stroke treatment. The RCT generates estimates of short-term quality-adjusted survival and cost and also the pattern of disability at the conclusion of follow-up. The simulation model incorporates the effect of disability on long-term outcomes, thus supporting a comprehensive CEA. Treatments that produce relatively modest improvements in the pattern of outcomes after ischemic stroke are likely to be cost-effective. This conclusion was robust to modifying the assumptions underlying the analysis. More effective treatments in the acute phase immediately following stroke would generate significant public health benefits, even if these treatments have a high price and result in relatively small reductions in disability. Simulation-based modeling can provide the critical link between a treatment's short-term effects and its long-term implications and can thus support comprehensive CEA.