Sample size and power issues in estimating incremental cost-effectiveness ratios from clinical trials data

An overview of methodological considerations regarding adaptive stopping, arm dropping, and randomization in clinical trials

BACKGROUND AND OBJECTIVES

Adaptive features may increase flexibility and efficiency of clinical trials, and improve participants' chances of being allocated to better interventions. Our objective is to provide thorough guidance on key methodological considerations for adaptive clinical trials.

METHODS

We provide an overview of key methodological considerations for clinical trials employing adaptive stopping, adaptive arm dropping, and response-adaptive randomization. We cover pros and cons of different decisions and provide guidance on using simulation to compare different adaptive trial designs. We focus on Bayesian multi-arm adaptive trials, although the same general considerations apply to frequentist adaptive trials.

RESULTS

We provide guidance on 1) interventions and possible common control, 2) outcome selection, follow-up duration and model choice, 3) timing of adaptive analyses, 4) decision rules for adaptive stopping and arm dropping, 5) randomization strategies, 6) performance metrics, their prioritization, and arm selection strategies, and 7) simulations, assessment of performance under different scenarios, and reporting. Finally, we provide an example using a newly developed R simulation engine that may be used to evaluate and compare different adaptive trial designs.

CONCLUSION

This overview may help trialists design better and more transparent adaptive clinical trials and to adequately compare them before initiation.

Cost-effectiveness of Web-Based and Home-Based Postnatal Psychoeducational Interventions for First-time Mothers: Economic Evaluation Alongside Randomized Controlled Trial

Background

The cost-effectiveness of interventions has attracted increasing interest among researchers. Although web-based and home-based psychoeducational interventions have been developed to improve first-time mothers’ postnatal health outcomes, very limited studies have reported their cost-effectiveness.

Objective

The aim of this study was to evaluate the cost-effectiveness of web-based and home-based postnatal psychoeducational interventions for first-time mothers during the early postpartum period.

Methods

A randomized controlled 3-group pretest and posttest design was adopted, and cost-effectiveness analysis from the health care’s perspective was conducted. A total of 204 primiparas were recruited from a public tertiary hospital in Singapore from October 2016 to August 2017 who were randomly allocated to the web-based intervention (n=68), home-based intervention (n=68), or control (n=68) groups. Outcomes of maternal parental self-efficacy, social support, postnatal depression, anxiety, and health care resource utilization were measured using valid and reliable instruments at baseline and at 1 month, 3 months, and 6 months after childbirth. The generalized linear regression models on effectiveness and cost were used to assess the incremental cost-effectiveness ratios of the web-based and home-based intervention programs compared to routine care. Projections of cumulative cost over 5 years incurred by the 3 programs at various coverage levels (ie, 10%, 50%, and 100%) were also estimated.

Results

The web-based intervention program dominated the other 2 programs (home-based program and routine care) with the least cost (adjusted costs of SGD 376.50, SGD 457.60, and SGD 417.90 for web-based, home-based, and control group, respectively; SGD 1=USD 0.75) and the best improvements in self-efficacy, social support, and psychological well-being. When considering the implementation of study programs over the next 5 years by multiplying the average cost per first-time mother by the estimated average number of first-time mothers in Singapore during the 5-year projection period, the web-based program was the least costly program at all 3 coverage levels. Based on the 100% coverage, the reduced total cost reached nearly SGD 7.1 million and SGD 11.3 million when compared to control and home-based programs at the end of the fifth year, respectively.

Conclusions

The web-based approach was promisingly cost-effective to deliver the postnatal psychoeducational intervention to first-time mothers and could be adopted by hospitals as postnatal care support.

Trial Registration

ISRCTN registry ISRCTN45202278; https://www.isrctn.com/ISRCTN45202278

Preventing postnatal maternal mental health problems using a psychoeducational intervention: the cost-effectiveness of What Were We Thinking

OBJECTIVES

Postnatal maternal mental health problems, including depression and anxiety, entail a significant burden globally, and finding cost-effective preventive solutions is a public policy priority. This paper presents a cost-effectiveness analysis of the intervention, What Were We Thinking (WWWT), for the prevention of postnatal maternal mental health problems.

DESIGN

The economic evaluation, including cost-effectiveness and cost-utility analyses, was conducted alongside a cluster-randomised trial.

SETTING

48 Maternal and Child Health Centres in Victoria, Australia.

PARTICIPANTS

Participants were English-speaking first-time mothers attending participating Maternal and Child Health Centres. Full data were collected for 175 participants in the control arm and 184 in the intervention arm.

INTERVENTION

WWWT is a psychoeducational intervention targeted at the partner relationship, management of infant behaviour and parental fatigue.

OUTCOME MEASURES

The evaluation considered public sector plus participant out-of-pocket costs, while outcomes were expressed in the 30-day prevalence of depression, anxiety and adjustment disorders, and quality-adjusted life years (QALYs). Incremental costs and outcomes were estimated using regression analyses to account for relevant sociodemographic, prognostic and clinical characteristics.

RESULTS

The intervention was estimated to cost $A118.16 per participant. The analysis showed no statistically significant difference between the intervention and control groups in costs or outcomes. The incremental cost-effectiveness ratios were $A36 451 per QALY gained and $A152 per percentage-point reduction in 30-day prevalence of depression, anxiety and adjustment disorders. The estimate lies under the unofficial cost-effectiveness threshold of $A55 000 per QALY; however, there was considerable uncertainty surrounding the results, with a 55% probability that WWWT would be considered cost-effective at that threshold.

CONCLUSIONS

The results suggest that, although WWWT shows promise as a preventive intervention for postnatal maternal mental health problems, further research is required to reduce the uncertainty over its cost-effectiveness as there were no statistically significant differences in costs or outcomes.

TRIAL REGISTRATION NUMBER

ACTRN12613000506796; results.

Cost-effectiveness of collaborative care for the treatment of depressive disorders in primary care: a systematic review

BACKGROUND

For the treatment of depressive disorders, the framework of collaborative care has been recommended, which showed improved outcomes in the primary care sector. Yet, an earlier literature review did not find sufficient evidence to draw robust conclusions on the cost-effectiveness of collaborative care.

PURPOSE

To systematically review studies on the cost-effectiveness of collaborative care, compared with usual care for the treatment of patients with depressive disorders in primary care.

METHODS

A systematic literature search in major databases was conducted. Risk of bias was assessed using the Cochrane Collaboration's tool. Methodological quality of the articles was assessed using the Consensus on Health Economic Criteria (CHEC) list. To ensure comparability across studies, cost data were inflated to the year 2012 using country-specific gross domestic product inflation rates, and were adjusted to international dollars using purchasing power parities (PPP).

RESULTS

In total, 19 cost-effectiveness analyses were reviewed. The included studies had sample sizes between n = 65 to n = 1,801, and time horizons between six to 24 months. Between 42% and 89% of the CHEC quality criteria were fulfilled, and in only one study no risk of bias was identified. A societal perspective was used by five studies. Incremental costs per depression-free day ranged from dominance to US$PPP 64.89, and incremental costs per QALY from dominance to US$PPP 874,562.

CONCLUSION

Despite our review improved the comparability of study results, cost-effectiveness of collaborative care compared with usual care for the treatment of patients with depressive disorders in primary care is ambiguous depending on willingness to pay. A still considerable uncertainty, due to inconsistent methodological quality and results among included studies, suggests further cost-effectiveness analyses using QALYs as effect measures and a time horizon of at least 1 year.

OARSI Clinical Trials Recommendations: Design and conduct of clinical trials for primary prevention of osteoarthritis by joint injury prevention in sport and recreation

The risk of post-traumatic osteoarthritis (PTOA) substantially increases following joint injury. Research efforts should focus on investigating the efficacy of preventative strategies in high quality randomized controlled trials (RCT). The objective of these OARSI RCT recommendations is to inform the design, conduct and analytical approaches to RCTs evaluating the preventative effect of joint injury prevention strategies. Recommendations regarding the design, conduct, and reporting of RCTs evaluating injury prevention interventions were established based on the consensus of nine researchers internationally with expertise in epidemiology, injury prevention and/or osteoarthritis (OA). Input and resultant consensus was established through teleconference, face to face and email correspondence over a 1 year period. Recommendations for injury prevention RCTs include context specific considerations regarding the research question, research design, study participants, randomization, baseline characteristics, intervention, outcome measurement, analysis, implementation, cost evaluation, reporting and future considerations including the impact on development of PTOA. Methodological recommendations for injury prevention RCTs are critical to informing evidence-based practice and policy decisions in health care, public health and the community. Recommendations regarding the interpretation and conduct of injury prevention RCTs will inform the highest level of evidence in the field. These recommendations will facilitate between study comparisons to inform best practice in injury prevention that will have the greatest public health impact.

The Cost-Effectiveness of Measures to Prevent Recurrent Ankle Sprains: Results of a 3-Arm Randomized Controlled Trial

BACKGROUND

Ankle sprains are the most common sports-related injury, associated with a high rate of recurrence and societal costs. Recent studies have emphasized the effectiveness of both neuromuscular training and bracing for the secondary prevention of ankle sprains.

PURPOSE

To evaluate the cost-effectiveness of the separate and combined use of bracing and neuromuscular training for the prevention of the recurrence of ankle sprains.

STUDY DESIGN

Economic and decision analysis; Level of evidence, 2.

METHODS

A total of 340 athletes (157 male and 183 female; aged 12-70 years) who had sustained a lateral ankle sprain up to 2 months before inclusion were randomized to a neuromuscular training group (n = 107), brace group (n = 113), and combined intervention group (n = 120). Randomization was stratified by medical treatment of the inclusion sprain. Participants in the neuromuscular training group underwent an 8-week home-based exercise program. Participants in the brace group received a semirigid ankle brace to be worn during all sports activities for a period of 12 months. Participants allocated to the combined group underwent both interventions, with the ankle brace to be worn during all sports activities for a period of 8 weeks. The recurrence of ankle sprains and associated costs were registered during the 1-year follow-up.

RESULTS

There were no differences between groups at baseline with regard to age, sex, sports participation, previous injury, or knowledge of preventive measures. The incremental cost-effectiveness ratio (ICER) of the brace group in comparison with the combined group was -€2828.30 (approximately--US$3865.00), based on a difference in the mean cost of -€76.16 (approximately--US$104.00) and a difference in the mean effects of 2.68%. The ICER of the neuromuscular training group in comparison with the combined group was €310.08 (approximately US$424.00), based on a difference in the mean cost of -€28.37 (approximately--US$39.00) and a difference in the mean effects of 9.15%.

CONCLUSION

Bracing was found to be the dominant secondary preventive intervention over both neuromuscular training and the combination of both measures.

A C++ program to calculate sample sizes for cost-effectiveness trials in a Bayesian framework

Cost-Effectiveness Analysis (CEA) has become an increasingly important component of clinical trials. However, formal sample size calculations for such studies are not common. One of the reasons for this might be due to the absence of readily available computer software to perform complex calculations, particularly in a Bayesian setting. In this paper, a C++ program (using NAG library functions/subroutines) is presented to estimate the sample sizes for cost-effectiveness clinical trials in a Bayesian framework. The program can equally be used to calculate sample sizes for efficacy trials. The Bayesian approach to sample size calculation is based on that of O'Hagan and Stevens (A. O'Hagan, J.W. Stevens, Bayesian assessment of sample size for clinical trials of cost-effectiveness, Medical Decision Making 21 (2001) 219-230). With this program, the user can calculate sample sizes for various thresholds of willingness to pay and under various assumptions of the correlations between cost and effects. Under some prior, the program produces frequentist sample size as well. The program runs under windows environment and running time is very short.

Sample size determination for cost-effectiveness trials

Methods for determining sample size requirements for cost-effectiveness studies are reviewed and illustrated. Traditional methods based on tests of hypothesis and power arguments are given for the incremental cost-effectiveness ratio and incremental net benefit (INB). In addition, a full Bayesian approach using decision theory to determine optimal sample size is given for INB. The full Bayesian approach, based on the value of information, is proposed in reaction to concerns that traditional methods rely on arbitrarily chosen error probabilities and an ill-defined notion of the smallest clinically important difference. Furthermore, the results of cost-effectiveness studies are used for decision making (e.g. should a new intervention be adopted or the old one retained), and employing decision theory, which permits optimal use of current information and the optimal design of new studies, provides a more consistent approach.

Sample size and power for cost-effectiveness analysis (part 1)

Basic sample size and power formulae for cost-effectiveness analysis have been established in the literature. These formulae are reviewed and the similarities and differences between sample size and power for cost-effectiveness analysis and for the analysis of other continuous variables such as changes in blood pressure or weight are described. The types of sample size and power tables that are commonly calculated for cost-effectiveness analysis are also described and the impact of varying the assumed parameter values on the resulting sample size and power estimates is discussed. Finally, the way in which the data for these calculations may be derived are discussed.

The cost effectiveness of rapid-acting insulin aspart compared with human insulin in type 2 diabetes patients: an analysis from the Japanese third-party payer perspective

OBJECTIVES

The Nippon Ultra-Rapid Insulin and Diabetic Complication Evaluation Study (NICE Study) (NCT00575172) was a 5-year, open-label, randomised controlled trial which compared cardiovascular outcomes in Japanese type 2 diabetes patients intensively treated with regular human insulin or insulin aspart (NovoRapid; Novo Nordisk A/S, Bagsvaerd, Denmark), a rapid-acting insulin analogue. The aim of the present analysis was to evaluate the cost effectiveness of insulin aspart versus regular human insulin from the perspective of a Japanese third-party healthcare payer.

RESEARCH DESIGN AND METHODS

A discrete event-simulation model was developed in Microsoft Excel to assess the within-trial cost effectiveness and make longer-term clinical projections in patients treated with regular human insulin or insulin aspart. In addition to severe hypoglycaemia, the model captured myocardial and cerebral infarction events and percutaneous coronary intervention and coronary artery bypass graft procedures. Within-trial mortality, incidence of severe hypoglycaemia and cardiovascular event probabilities were derived from the annual rates observed during the trial period, while post-trial outcomes were calculated using the event rates from the trial, adjusted for increasing patient age. Event costs were accounted from the healthcare payer perspective and expressed in 2008 Japanese yen (JPY), while health-related quality of life (HRQoL) was captured using event and state utilities. Future costs and clinical benefits were discounted at 3% annually. Life expectancy, quality-adjusted life expectancy, cardiovascular event rates and costs were evaluated over 5- and 10-year time horizons and sensitivity analyses were performed to assess variability in model outcomes.

RESULTS

Over 5 years of treatment, insulin aspart dominated human insulin both in incremental life expectancy and in incremental quality-adjusted life-years (QALYS). Insulin aspart was associated with a small improvement in discounted life expectancy of 0.005 years (4.688 vs. 4.684 years) and an increase of 0.023 quality-adjusted life-years (QALYs) (3.800 vs. 3.776 QALYs) when compared with regular human insulin. Insulin aspart also incurred lower costs (JPY 481,586 vs. 594,717, difference -113,131) which resulted from the decreased incidence of cardiovascular events with insulin aspart (0.013 events per patient year vs. 0.030 on regular human insulin). Breakdown of costs indicated that pharmacy costs were higher with insulin aspart (JPY 346,608 vs. 278,468), but these costs were more than offset by the reduced costs associated with cardiovascular complications and hypoglycaemia over 5 years of treatment (JPY 134,978 vs. 316,249). Sensitivity analysis showed that insulin aspart was still cost-effective in the case where only 18% of the within-trial cardiovascular and mortality benefit over regular human insulin was captured in the model (assuming a willingness-to-pay threshold of JPY 5,000,000).

LIMITATIONS

The NICE study cohort was relatively small (n = 325), meaning that caution should be exercised when calculating and interpreting the incremental cost-effectiveness ratio. Also, despite the differences in cardiovascular risk profile between the Japanese and UK populations, UKPDS-derived risk equations were used to project MI outcomes and PCI and CABG procedures and UKPDS HRQoL scores were applied to all health states. While these risk formulas and HRQoL utilities may not be directly applicable to the Japanese population, no equivalent Japanese-specific data are currently available.

CONCLUSIONS

In a Japanese type 2 diabetes population, prescribing rapid-acting insulin aspart significantly reduced cardiovascular complications over 5- and 10-year time horizons, resulting in increased quality of life and decreased costs when compared with human insulin.

Incremental cost-effectiveness ratio and incremental net-health benefit: two sides of the same coin

In recent years, an alternative framework for cost-effectiveness analyses has been growing in popularity. Instead of the incremental cost-effectiveness ratio for which statistical inference is often difficult, the incremental net-health benefit (INHB), a linear transformation of incremental costs and effectiveness, has been utilized. The linear structure of this statistic allows easy computation and interpretation of confidence intervals, hypothesis tests and acceptability curves. It is often difficult, however, to switch decision-making procedures without first verifying the appropriateness of the new methods. In this paper, we demonstrate the decision-making similarities between the INHB and the incremental cost-effectiveness ratio and describe how the INHB can be used to clarify inference of the incremental cost-effectiveness ratio. We also describe the two statistics in terms of the DeltaE-DeltaC plane, thus allowing both a mathematical and graphical comparison of these similarities. We conclude with a general discussion of cost-effectiveness analyses and advocate Bayesian, rather than frequentist inference as the more intuitive and powerful decision-making framework.

The ISPOR Good Practices for Quality Improvement of Cost-Effectiveness Research Task Force Report

OBJECTIVES

The International Society for Pharmacoeconomics and Outcomes Research (ISPOR) Health Science Policy Council recommended and the ISPOR Board of Directors approved the formation of a Task Force to critically examine the major issues related to Quality Improvement in Cost-effectiveness Research (QICER). The Council's primary recommendation for this Task Force was that it should report on the quality of cost-effectiveness research and make recommendations to facilitate the improvement of pharmacoeconomics and health outcomes research and its use in stimulating better health care and policy. Task force members were knowledgeable and experienced in medicine, pharmacy, biostatistics, health policy and health-care decision-making, biomedical knowledge transfer, health economics, and pharmacoeconomics. They were drawn from industry, academia, consulting organizations, and advisors to governments and came from Japan, the Netherlands, Canada and the United States.

METHODS

Face-to-face meetings of the Task Force were held at ISPOR North American and European meetings and teleconferences occurred every few months. Literature reviews and surveys were conducted and the first preliminary findings presented at an open forum at the May 2008 ISPOR meeting in Toronto. The final draft report was circulated to the expert reviewer group and then to the entire membership for comment. The draft report was posted on the ISPOR Web site in April 2009. All formal comments received were posted to the association Web site and presented for discussion at the Task Force forum during the ISPOR 14th Annual International Meeting in May 2009. Comments and feedback from the forums, reviewers and membership were considered in the final report. Once Task Force consensus was reached, the article was submitted to Value in Health.

CONCLUSIONS

The QICER Task Force recommends that ISPOR implement the following: * With respect to CER guidelines, that ISPOR promote harmonization of guidelines, allowing for differences in application, regional needs and politics; evaluate available instruments or promote development of a new one that will allow standardized quantification of the impact of CER guidelines on the quality of CER studies; report periodically on those countries or regions that have developed guidelines; periodically evaluate the quality of published studies (those journals with CER guidances) or those submitted to decision-making bodies (as public transparency increases). * With respect to methodologies, that ISPOR promote publication of methodological guidelines in more applied journals in more easily understandable format to transfer knowledge to researchers who need to apply more rigorous methods; promote full availability of models in electronic format to combat space restrictions in hardcopy publications; promote consistency of methodological review for all CER studies; promote adoption of explicit best practices guidelines among regulatory and reimbursement authorities; periodically update all ISPOR Task Force reports; periodically review use of ISPOR Task Force guidelines; periodically report on statistical and methodological challenges in HE; evaluate periodically whether ISPOR's methodological guidelines lead to improved quality; and support training and knowledge transfer of rigorous CER methodologies to researchers and health care decision-makers. * With respect to publications, that ISPOR develop standard CER guidances to which journals will be able to refer their authors and their reviewers; lobby to establish these guidances within the International Committee for Medical Journal Editors (ICMJE) Requirements to which most journals refer in their Author Instructions; provide support in terms of additional reviewer expertise to those journals lacking appropriate reviewers; periodically report on journals publishing CER research; periodically report on the quality of CER publications; and support training and knowledge transfer of the use of these guidelines to researchers and reviewers. * With respect to evidence-based health-care decision-making, that ISPOR recognize at its annual meetings those countries/agencies/private companies/researchers using CER well, and those practitioners and researchers supporting good patient use of CER in decision-making; and promote public presentation of case studies of applied use of CER concepts or guidelines.

A behavioral Bayes method to determine the sample size of a clinical trial considering efficacy and safety

It is necessary for the calculation of sample size to achieve the best balance between the cost of a clinical trial and the possible benefits from a new treatment. Gittins and Pezeshk developed an innovative (behavioral Bayes) approach, which assumes that the number of users is an increasing function of the difference in performance between the new treatment and the standard treatment. The better a new treatment, the more the number of patients who want to switch to it. The optimal sample size is calculated in this framework. This BeBay approach takes account of three decision-makers, a pharmaceutical company, the health authority and medical advisers. Kikuchi, Pezeshk and Gittins generalized this approach by introducing a logistic benefit function, and by extending to the more usual unpaired case, and with unknown variance. The expected net benefit in this model is based on the efficacy of the new drug but does not take account of the incidence of adverse reactions. The present paper extends the model to include the costs of treating adverse reactions and focuses on societal cost-effectiveness as the criterion for determining sample size. The main application is likely to be to phase III clinical trials, for which the primary outcome is to compare the costs and benefits of a new drug with a standard drug in relation to national health-care.

Feasibility of using administrative claims data for cost-effectiveness analysis of a clinical trial

OBJECTIVE

This study was performed retrospectively to determine if Medicare claims data could be used to evaluate the cost effectiveness, from a payer perspective, of different radiation treatment schedules evaluated in a national clinical trial.

METHODS

Medicare costs from all providers and all places of service were obtained from the Centers for Medicare & Medicaid Services for patients treated in the period 1992-1996 on Radiation Therapy Oncology Group 90-03, and combined with data on outcomes from the trial.

RESULTS

Of the 1,113 patients entered, Medicare cost data and clinical outcomes were available for 187 patients. Significant differences in tolerance of treatment and outcome were noted between patients with Medicare data included in the study and patients without Medicare data, and non-Medicare patients excluded from it. Ninety-five percent confidence ellipses on the incremental cost-effectiveness scatterplots crossed both axes, indicating non-significant differences in cost effectiveness between radiation treatment schedules.

CONCLUSIONS

Claims data permit estimation of cost effectiveness, but Medicare data provide inadequate representation of results applicable to patients from the general population.

An opportunity cost approach to sample size calculation in cost-effectiveness analysis

The inclusion of economic evaluations as part of clinical trials has led to concerns about the adequacy of trial sample size to support such analysis. The analytical tool of cost-effectiveness analysis is the incremental cost-effectiveness ratio (ICER), which is compared with a threshold value (lambda) as a method to determine the efficiency of a health-care intervention. Accordingly, many of the methods suggested to calculating the sample size requirements for the economic component of clinical trials are based on the properties of the ICER. However, use of the ICER and a threshold value as a basis for determining efficiency has been shown to be inconsistent with the economic concept of opportunity cost. As a result, the validity of the ICER-based approaches to sample size calculations can be challenged. Alternative methods for determining improvements in efficiency have been presented in the literature that does not depend upon ICER values. In this paper, we develop an opportunity cost approach to calculating sample size for economic evaluations alongside clinical trials, and illustrate the approach using a numerical example. We compare the sample size requirement of the opportunity cost method with the ICER threshold method. In general, either method may yield the larger required sample size. However, the opportunity cost approach, although simple to use, has additional data requirements. We believe that the additional data requirements represent a small price to pay for being able to perform an analysis consistent with both concept of opportunity cost and the problem faced by decision makers.

Cost-Effectiveness Inferences from Bootstrap Quadrant Confidence Levels: Three Degrees of Dominance

When with at least 95% confidence a new treatment is shown to be not only less costly (LC), but also more effective (ME), than a current treatment, that new treatment can be said to "strictly dominate" the current treatment statistically. But what can be said when head-to-head treatment comparisons turn out to be less clear-cut than this? Here, we propose two additional sets of specific LC and/or ME confidence thresholds to define the concepts of "some dominance" and "much dominance." Confidence levels associated with entire quadrants of the incremental cost-effectiveness (ICE) plane are easily computed using the same bootstrapping techniques used to estimate an "acceptability curve." Our two proposed additional "degrees" of dominance, although less stringent than strict dominance, are nevertheless more stringent than commonly accepted approaches using ICE ratio or net benefit calculations. To illustrate analysis concepts, we use data from a randomized, double-blind, placebo- and active comparator-controlled clinical registration trial for treatment of major depressive disorder (MDD). As is typical, our case study is rather small and short term, providing outcome information for a total of only 264 patients during their initial 8 weeks of acute-phase MDD treatment. Thus, we focus attention on sensitivity analyses, showing that the bootstrap distribution of cost-effectiveness uncertainty is robust across two alternative ways of measuring overall effectiveness and three alternative ways of imputing missing values. Evaluation of the balance between cost and benefit is particularly difficult when a new pharmacological treatment is first introduced, yet information of this sort is highly desired by decision makers. We show that, even with only a relatively modest amount of clinical trial information, sensitivity analyses can still confirm that cost-effectiveness comparisons are being made in a consistent fashion. In contrast, extensive follow-up comparisons using data from actual clinical practice will almost always ultimately be needed to better inform health policy makers.

Estimation, power and sample size calculations for stochastic cost and effectiveness analysis

Various methods have been proposed to address uncertainty in economic evaluations of healthcare programmes. One approach suggested in the literature is to estimate separate confidence intervals for the incremental costs and effects of a new health programme in comparison with an existing programme. These intervals are then combined to generate a rectangular confidence region in the cost-effectiveness plane that implicitly defines a corresponding confidence interval for the incremental cost-effectiveness ratio (ICER). The same approach has been used to calculate sample sizes and study power. This application of the rectangle method is consistent with the adoption of ICERs and a threshold as a decision rule, this being the most commonly used approach in empirical applications of cost-effectiveness analysis, as well as the one recommended by agencies that assess medical technology around the world. In this paper, we first outline the rectangle method, and then propose a modification that recognises that separate inferences are being drawn on the cost and effectiveness domains, and that corrects for multiple statistical comparisons. The confidence rectangle is otherwise too small, the corresponding confidence interval for the ICER is too narrow and sample sizes are under-estimated. Our modification corrects these problems. A further difficulty is that the placement of the confidence rectangle around the null value is somewhat arbitrary, and does not correspond to a unique value of ICERs. As a result, different values of sample size and power for the estimation of ICERs can be obtained, depending on the null values of the cost and effectiveness. We conclude that it is important to clearly identify the analytic goal in terms of estimating differential costs, differential effects or a combination of the two using the ICER index. These ideas are illustrated using numerical examples.

Costs and effects in lumbar spinal fusion. A follow-up study in 136 consecutive patients with chronic low back pain

Although cost-effectiveness is becoming the foremost evaluative criterion within health service management of spine surgery, scientific knowledge about cost-patterns and cost-effectiveness is limited. The aims of this study were (1) to establish an activity-based method for costing at the patient-level, (2) to investigate the correlation between costs and effects, (3) to investigate the influence of selected patient characteristics on cost-effectiveness and, (4) to investigate the incremental cost-effectiveness ratio of (a) posterior instrumentation and (b) intervertebral anterior support in lumbar spinal fusion. We hypothesized a positive correlation between costs and effects, that determinants of effects would also determine cost-effectiveness, and that posterolateral instrumentation and anterior intervertebral support are cost-effective adjuncts in posterolateral lumbar fusion. A cohort of 136 consecutive patients with chronic low back pain, who were surgically treated from January 2001 through January 2003, was followed until 2 years postoperatively. Operations took place at University Hospital of Aarhus and all patients had either (1) non-instrumented posterolateral lumbar spinal fusion, (2) instrumented posterolateral lumbar spinal fusion, or (3) instrumented posterolateral lumbar spinal fusion + anterior intervertebral support. Analysis of costs was performed at the patient-level, from an administrator's perspective, by means of Activity-Based-Costing. Clinical effects were measured by means of the Dallas Pain Questionnaire and the Low Back Pain Rating Scale at baseline and 2 years postoperatively. Regression models were used to reveal determinants for costs and effects. Costs and effects were analyzed as a net-benefit measure to reveal determinants for cost-effectiveness, and finally, adjusted analysis (for non-random allocation of patients) was performed in order to reveal the incremental cost-effectiveness ratios of (a) posterior instrumentation and (b) anterior support. The costs of non-instrumented posterolateral spinal fusion were estimated at DKK 88,285(95% CI 81,369;95,546), instrumented posterolateral spinal fusion at DKK 94,396(95% CI 89,865;99,574) and instrumented posterolateral lumbar spinal fusion + anterior intervertebral support at DKK 120,759(95% CI 111,981;133,738). The net-benefit of the regimens was significantly affected by smoking and functional disability in psychosocial life areas. Multi-level fusion and surgical technique significantly affected the net-benefit as well. Surprisingly, no correlation was found between treatment costs and treatment effects. Incremental analysis suggested that the probability of posterior instrumentation being cost-effective was limited, whereas the probability of anterior intervertebral support being cost-effective escalates as willingness-to-pay per effect unit increases. This study reveals useful and hitherto unknown information both about cost-patterns at the patient-level and determinants of cost-effectiveness. The overall conclusion of the present investigation is a recommendation to focus further on determinants of cost-effectiveness. For example, patient characteristics that are modifiable at a relatively low expense may have greater influence on cost-effectiveness than the surgical technique itself--at least from an administrator's perspective.

In search of pharmacoeconomic evaluations for fibromyalgia treatments: a review

Fibromyalgia is characterised by chronic widespread pain of unknown aetiology and affects approximately 2% of the population. It can cause significant patient disability, sizeable economic costs, complex management decisions and controversy for healthcare providers. In lieu of uniformly approved treatments for fibromyalgia, patients may try multiple pharmacological and non-pharmacological therapies with questionable efficacy. The literature lacks pharmacoeconomic studies that balance the cost and benefit of interventions. In the absence of this work, cost outcomes are reviewed in this paper. Due to inconclusive results, further study is needed on fibromyalgia treatment cost-effectiveness. These analyses could provide useful information for policy and evidence-based practice guidelines toward optimal disease management. Medical professionals should be a driving force in understanding the clinical and economic challenges of fibromyalgia.

Incremental cost-effectiveness ratios (ICERs): The silence of the lambda

Despite the central role of the threshold incremental cost-effectiveness ratio (ICER), or lambda (lambda), in the methods and application of cost-effective analysis (CEA), little attention has been given to the determining the value of lambda. In this paper we consider 'what explains the silence of the lambda'? The concept of the threshold ICER is critically appraised. We show that there is 'silence of the lambda' with respect to justification of the value of ICER thresholds, their use in decision-making and their relationship to the opportunity cost of marginal resources. Moreover, the 'sound of silence' extends to both 'automatic cut-off' and more sophisticated approaches to the use of lambda in determining recommendations about health care programs. We argue that the threshold value provides no useful information for determining the efficiency of using available resources to support new health care programs. On the contrary, the threshold approach has lead to decisions that resulted in increased expenditures on health care programs and concerns about the sustainability of public funding for health care programs without any evidence of increases in total health gains. To improve efficiency in resource allocation, decision-makers need information about the opportunity costs of programs.

Health economic evaluation in lumbar spinal fusion: a systematic literature review anno 2005

The goal of this systematic literature review was to assess the evidence for cost-effectiveness of various surgical techniques in lumbar spinal fusion in conformity with the guidelines provided by the Cochrane Back Review Group. As new technology continuously emerges and divergent directions in clinical practice are present, economic evaluation is needed in order to facilitate the decision-makers' budget allocations. NHS Economic Evaluation Database, MEDLINE, EMBASE and Cochrane Library were searched. Two independent reviewers (one clinical content expert and one economic content expert) applied the eligibility criteria. A list of criteria for methodological quality assessment was established by merging the criteria recommended by leading health economists with the criteria recommended by the Cochrane Back Review Group. The two reviewers independently scored the selected literature and the disagreement was resolved by means of consensus following discussion. Key data were extracted and the level of evidence concluded. Seven studies were eligible; these studies reflected the diversified choices of economic methodology, study populations (diagnosis), outcome measures and comparators. At the conclusion of quality assessment, the methodological quality of three studies was judged credible. Two studies investigated posteolateral fusion (PLF) +/- instrumentation in different populations: one investigated non-specific low back pain and one investigated degenerative stenosis + spondylolisthesis. Both studies reflected that cost-effectiveness of instrumentation in PLF is not convincing. The third study concerned the question of circumferential vs anterior lumbar interbody fusion and found a non-significant difference between the techniques. In conclusion, the literature is limited and, in view of the fact that the clinical effects are statistically synonymous, it does not support the use of high-cost techniques. There is a great potential for improvement of methodological quality in economic evaluations of lumbar spinal fusion and further research is imperative.

Cost-effectiveness analysis for multinational clinical trials

Clinical trials of cost-effectiveness are often conducted in more than one country. The two most common ways of dealing with the multinational nature of the data are either to calculate a pooled estimate or to stratify results by country. Since the between-country heterogeneity in costs is potentially substantial, pooled estimates may be difficult to interpret for any one country. Policy decisions are often made at a national level, and so country-specific results are important. However, country-specific analyses will be based on fewer patients and will often fail to provide adequate precision for statistical analyses. Shrinkage estimation is a compromise between these two methods and has been used successfully in other fields. These estimates are country-specific yet less variable than those derived through a subgroup approach. Univariate and multivariate shrinkage estimators for costs and effects are proposed, then compared with one another and to the traditional methods in a simulation study. The methods are illustrated using data from a multinational trial evaluating the cost-effectiveness of three thrombolytic drug regimens in patients with acute myocardial infarction.

Unfinished symphony: a tribute to the life and career of Bernie O'Brien (1959-2004)

The death of Bernie O'Brien in February 2004 brought a premature end to one of the most productive and influential careers in the area of health technology assessment and economic evaluation. A long-term member of the Society for Medical Decision Making, Bernie will be remembered for his research contributions in areas including outcome valuation, decision modeling, statistical methods in economic evaluation, and applied cost-effectiveness studies. He was also an excellent communicator and teacher and, above all, a fun guy to work with. In this article, the authors provide a review of Bernie's academic contributions.

Optimisation versus certainty: understanding the issue of heterogeneity in economic evaluation

This paper considers how the information provided by economic evaluation for decision-makers can fail to optimise use of health resources and how assessment of the relative cost-effectiveness of health care interventions can be misleading unless heterogeneity within populations is taken into account. The cost-effectiveness of an intervention is not a point estimate but an average chosen from within a distribution of different results. The normal interpretation of the distribution around that point is often mistakenly assumed to be the 'white noise' of measurement error. In reality this variance is a combination of measurement error and true heterogeneity of results. There remains an overemphasis on pursuing certainty which stems from the fact that the methods involved were originally devised to measure dichotomous outcomes not continuous ones such as cost-effectiveness ratios. It is argued in this paper that more consideration be given to the heterogeneous nature of costs and effects across populations in analysis and policy making.

Economic evaluations and randomized trials in spinal disorders: principles and methods

STUDY DESIGN

Descriptive methodologic recommendations.

OBJECTIVE

To help researchers designing, conducting, and reporting economic evaluations in the field of back and neck pain.

SUMMARY OF BACKGROUND DATA

Economic evaluations of both existing and new therapeutic interventions are becoming increasingly important. There is a need to improve the methods of economic evaluations in the field of spinal disorders.

MATERIALS AND METHODS

To improve the methods of economic evaluations in the field of spinal disorders, this article describes the various steps in an economic evaluation, using as example a study on the cost-effectiveness of manual therapy, physiotherapy, and usual care provided by the general practitioner for patients with neck pain.

RESULTS

An economic evaluation is a study in which two or more interventions are systematically compared with regard to both costs and effects. There are four types of economic evaluations, based on analysis of: (1) cost-effectiveness, (2) cost-utility, (3) cost-minimization, and (4) cost-benefit. The cost-utility analysis is a special case of cost-effectiveness analysis. The first step in all these economic evaluations is to identify the perspective of the study. The choice of the perspective will have consequences for the identification of costs and effects. Secondly, the alternatives that will be compared should be identified. Thirdly, the relevant costs and effects should be identified. Economic evaluations are usually performed from a societal perspective and include consequently direct health care costs, direct nonhealth care costs, and indirect costs. Fourthly, effect data are collected by means of questionnaires or interviews, and relevant cost data with regard to effect measures and health care utilization, work absenteeism, travel expenses, use of over-the-counter medication, and help from family and friends, are collected by means of cost diaries, questionnaires, or (telephone) interviews. Fifthly, real costs are calculated, or the costs are estimated on the basis of real costs, guideline prices, or tariffs. Finally, in the statistical analysis the mean direct, indirect, and total costs of the alternatives are compared, using bootstrapping techniques. Incremental cost-effectiveness ratios are graphically presented on a cost-effectiveness plane and acceptability curves are calculated.

CONCLUSION

Economic evaluations require specific methods. These recommendations may be helpful in improving the quality of economic evaluations of new and existing therapeutic interventions in the field of spinal disorders.

Incremental net benefit in randomized clinical trials with quality-adjusted survival

Owing to induced dependent censoring, estimating mean costs and quality-adjusted survival in a cost-effectiveness comparison of two groups using standard life-table methods leads to biased results. In this paper we propose methods for estimating the difference in mean costs and the difference in mean effectiveness, together with their respective variances and covariance in the presence of dependent censoring. We consider the situation in which the measure of effectiveness is either the probability of surviving a duration of interest or mean quality-adjusted survival time over a duration of interest. The methods are illustrated in an example using an incremental net benefit analysis.

Advantages of using the net-benefit approach for analysing uncertainty in economic evaluation studies

No consensus has yet been reached on how to analyse uncertainty in economic evaluation studies where individual patient data are available for costs and health effects. This paper summarises the available results regarding the analysis of uncertainty on the cost-effectiveness plane and argues for using the net-benefit approach when analysing uncertainty in cost-effectiveness studies. The net-benefit approach avoids the interpretation and statistical problems related to the incremental cost effectiveness ratio and implies several advantages. First, traditional statistical methods can be used for confidence-interval estimation and hypothesis testing. Second, calculation of the optimal sample size and the power of the study are facilitated allowing the correlation between costs and effects to vary within and between patient groups. Third, the use of a Bayesian approach to cost-effectiveness analysis is facilitated. Fourth, a formal relation between cost-effectiveness acceptability curves and statistical inference is provided. Finally, the net-benefit approach gives the Fieller's limits of the confidence interval for the incremental cost-effectiveness ratio in the cost-effectiveness plane. Based on these advantages the net-benefit approach should strongly be considered when analysing uncertainty in cost-effectiveness analyses.

Bayesian methods for design and analysis of cost-effectiveness trials in the evaluation of health care technologies

We review the development of Bayesian statistical methods for the design and analysis of randomized controlled trials in the assessment of the cost-effectiveness of health care technologies. We place particular emphasis on the benefits of the Bayesian approach; the implications of skew cost data; the need to model the data appropriately to generate efficient and robust inferences instead of relying on distribution-free methods; the importance of making full use of quantitative and structural prior information to produce realistic inferences; and issues in the determination of sample size. Several new examples are presented to illustrate the methods. We conclude with a discussion of the key areas for future research.

Using inverse-weighting in cost-effectiveness analysis with censored data

Due to induced dependent censoring, estimating mean costs and quality-adjusted survival in a cost-effectiveness analysis using standard life-table methods leads to biased results. In this paper we propose methods for estimating the difference in mean costs and the difference in effectiveness, together with their respective variances and covariance in the presence of dependent censoring. We consider the situation in which the measure of effectiveness is either the probability of patients surviving a duration of interest, mean survival time over a duration of interest or mean quality-adjusted survival time over a duration of interest. The method of inverse-weighting is used for censored cost and quality of life data. The methods are illustrated in an example using an incremental net benefit analysis.

Analysis, sample size, and power for estimating incremental net health benefit from clinical trial data

Stinnett and Mullahy recently introduced the concept of net health benefit as an alternative to cost-effectiveness ratios for the statistical analysis of patient-level data on the costs and health effects of competing interventions. Net health benefit addresses a number of problems associated with cost-effectiveness ratios by assuming a value for the willingness-to-pay for a unit of effectiveness. We extend the concept of net health benefit to demonstrate that standard statistical procedures can be used for the analysis, power, and sample size determinations of cost-effectiveness data. We also show that by varying the value of the willingness-to-pay, the point estimate and confidence interval for the incremental cost-effectiveness ratio can be determined. An example is provided.

Parameter solicitation for planning cost effectiveness studies with dichotomous outcomes

When economic endpoints are included alongside clinical effectiveness measures in randomized clinical trials (RCT), they are summarized together by the incremental cost effectiveness ratio (ICER). Adding economic endpoints to an RCT complicates the planning of experiments because investigators must now solicit their beliefs about costs, but even more challenging, they must also specify their association with effectiveness. Solicitation of correlations between costs and effects can be unintuitive, and so potentially highly inaccurate. This is unfortunate because power is highly sensitive to the association between costs and effects. Mis-specification in this association may lead to substantially underpowered or overpowered studies. We show that when clinical effectiveness measures are dichotomous, specification of the correlation between costs and effects can be avoided by instead describing their association with a mixture model. This representation leads to simple and highly intuitive parameter specifications. It may also be used to generate realistic raw data that can be used to evaluate experiment power with simulation. We give particular attention to evaluating and interpreting power when Fieller's theorem method (FTM) is used to calculate confidence for, and test hypotheses about, the ICER. Data from a previously published clinical trial are used to demonstrate the use of this new method to calculate sample size for a cost effectiveness study.

Power and sample assessments for tests of hypotheses on cost-effectiveness ratios

We address the issue of statistical power and sample size for cost-effectiveness studies. Tests of hypotheses on the cost-effectiveness ratio (CER) are constructed from the net cost and incremental effectiveness measures. When the difference in effectiveness is known, we derive formulae for statistical power and sample size assessments for one- and two-sided tests of hypotheses of the CER. We also construct a test of the joint hypothesis of cost-effectiveness and effectiveness and derive an expression connecting power and sample size. Our methods account for the correlation between cost and effectiveness and lead to smaller sample size requirements than comparative methods that ignore the correlation. The implications of our formulae for cost-effectiveness studies are illustrated through numerical examples. When compared with trials designed to demonstrate effectiveness alone, our results indicate that a trial appropriately powered to demonstrate cost-effectiveness might require sample sizes many times greater.