Sample size determination for cost-effectiveness trials

Power and sample size calculation for incremental net benefit in cost effectiveness analyses with applications to trials conducted by the Canadian Cancer Trials Group

BACKGROUND

Historically, a priori power and sample size calculations have not been routinely performed cost-effectiveness analyses (CEA), partly because the absence of published cost and effectiveness correlation and variance data, which are essential for power and sample size calculations. Importantly, the empirical correlation between cost and effectiveness has not been examined with respect to the estimation of value-for-money in clinical literature. Therefore, it is not well established if cost-effectiveness studies embedded within randomized-controlled-trials (RCTs) are under- or over-powered to detect changes in value-for-money. However, recently guidelines (such as those from ISPOR) and funding agencies have suggested sample size and power calculations should be considered in CEAs embedded in clinical trials.

METHODS

We examined all RCTs conducted by the Canadian Cancer Trials Group with an embedded cost-effectiveness analysis. Variance and correlation of effectiveness and costs were derived from original-trial data. The incremental net benefit method was used to calculate the power of the cost-effectiveness analysis, with exploration of alternative correlation and willingness-to-pay values.

RESULTS

We identified four trials for inclusion. We observed that a hypothetical scenario of correlation coefficient of zero between cost and effectiveness led to a conservative estimate of sample size. The cost-effectiveness analysis was under-powered to detect changes in value-for-money in two trials, at willingness-to-pay of $100,000. Based on our observations, we present six considerations for future economic evaluations, and an online program to help analysts include a priori sample size and power calculations in future clinical trials.

CONCLUSION

The correlation between cost and effectiveness had a potentially meaningful impact on the power and variance of value-for-money estimates in the examined cost-effectiveness analyses. Therefore, the six considerations and online program, may facilitate a priori power calculations in embedded cost-effectiveness analyses in future clinical trials.

Cost-Effectiveness of Robotic vs. Laparoscopic Surgery for Different Surgical Procedures: Protocol for a Prospective, Multicentric Study (ROBOCOSTES)

Background

The studies which address the impact of costs of robotic vs. laparoscopic approach on quality of life (cost-effectiveness studies) are scares in general surgery.

Methods

The Spanish national study on cost-effectiveness differences among robotic and laparoscopic surgery (ROBOCOSTES) is designed as a prospective, multicentre, national, observational study. The aim is to determine in which procedures robotic surgery is more cost-effective than laparoscopic surgery. Several surgical operations and patient populations will be evaluated (distal pancreatectomy, gastrectomy, sleeve gastrectomy, inguinal hernioplasty, rectal resection for cancer, Heller cardiomiotomy and Nissen procedure).

Discussion

The results of this study will demonstrate which treatment (laparoscopic or robotic) and in which population is more cost-effective. This study will also assess the impact of previous surgical experience on main outcomes.

Economic evaluations for intensive care unit randomised clinical trials in Australia and New Zealand: Practical recommendations for researchers

OBJECTIVES

Economic evaluations of intensive care unit (ICU) interventions have specific considerations, including how to cost ICU stays and accurately measure quality of life in survivors. The aim of this article was to develop best practice recommendations for economic evaluations alongside future ICU randomised controlled trials (RCTs).

REVIEW METHODS

We collated our experience based on expert consensus across several recent economic evaluations to provide best-practice, practical recommendations for researchers conducting economic evaluations alongside RCTs in the ICU. Recommendations were structured according to the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) Consolidated Health Economic Evaluation Reporting Standards (CHEERS) Task Force Report.

RESULTS

We discuss recommendations across the components of economic evaluations, including: types of economic evaluation, the population and sample size, study perspective, comparators, time horizon, choice of health outcomes, measurement of effectiveness, measurement and valuation of quality of life, estimating resources and costs, analytical methods, and the increment cost-effectiveness ratio. We also provide future directions for research with regard to developing more robust economic evaluations for the ICU.

CONCLUSION

Economic evaluations should be built alongside ICU RCTs and should be designed a priori using appropriate follow-up and data collection to capture patient-relevant outcomes. Further work is needed to improve the quality of data available for linkage in Australia as well as developing costing methods for the ICU and appropriate quality of life measurements.

Practical help for specifying the target difference in sample size calculations for RCTs: the DELTA2 five-stage study, including a workshop

BACKGROUND

The randomised controlled trial is widely considered to be the gold standard study for comparing the effectiveness of health interventions. Central to its design is a calculation of the number of participants needed (the sample size) for the trial. The sample size is typically calculated by specifying the magnitude of the difference in the primary outcome between the intervention effects for the population of interest. This difference is called the 'target difference' and should be appropriate for the principal estimand of interest and determined by the primary aim of the study. The target difference between treatments should be considered realistic and/or important by one or more key stakeholder groups.

OBJECTIVE

The objective of the report is to provide practical help on the choice of target difference used in the sample size calculation for a randomised controlled trial for researchers and funder representatives.

METHODS

The Difference ELicitation in TriAls2 (DELTA2) recommendations and advice were developed through a five-stage process, which included two literature reviews of existing funder guidance and recent methodological literature; a Delphi process to engage with a wider group of stakeholders; a 2-day workshop; and finalising the core document.

RESULTS

Advice is provided for definitive trials (Phase III/IV studies). Methods for choosing the target difference are reviewed. To aid those new to the topic, and to encourage better practice, 10 recommendations are made regarding choosing the target difference and undertaking a sample size calculation. Recommended reporting items for trial proposal, protocols and results papers under the conventional approach are also provided. Case studies reflecting different trial designs and covering different conditions are provided. Alternative trial designs and methods for choosing the sample size are also briefly considered.

CONCLUSIONS

Choosing an appropriate sample size is crucial if a study is to inform clinical practice. The number of patients recruited into the trial needs to be sufficient to answer the objectives; however, the number should not be higher than necessary to avoid unnecessary burden on patients and wasting precious resources. The choice of the target difference is a key part of this process under the conventional approach to sample size calculations. This document provides advice and recommendations to improve practice and reporting regarding this aspect of trial design. Future work could extend the work to address other less common approaches to the sample size calculations, particularly in terms of appropriate reporting items.

FUNDING

Funded by the Medical Research Council (MRC) UK and the National Institute for Health Research as part of the MRC-National Institute for Health Research Methodology Research programme.

Choosing the target difference and undertaking and reporting the sample size calculation for a randomised controlled trial - the development of the DELTA2 guidance

BACKGROUND

A key step in the design of a randomised controlled trial is the estimation of the number of participants needed. The most common approach is to specify a target difference in the primary outcome between the randomised groups and then estimate the corresponding sample size. The sample size is chosen to provide reassurance that the trial will have high statistical power to detect the target difference at the planned statistical significance level. Alternative approaches are also available, though most still require specification of a target difference. The sample size has many implications for the conduct of the study, as well as incurring scientific and ethical aspects. Despite the critical role of the target difference for the primary outcome in the design of a randomised controlled trial (RCT), the manner in which it is determined has received little attention. This article reports the development of the DELTA² guidance on the specification and reporting of the target difference for the primary outcome in a sample size calculation for a RCT.

METHODS

The DELTA² (Difference ELicitation in TriAls) project has five components comprising systematic literature reviews of recent methodological developments (stage 1) and existing funder guidance (stage 2), a Delphi study (stage 3), a 2-day consensus meeting bringing together researchers, funders and patient representatives (stage 4), and the preparation and dissemination of a guidance document (stage 5).

RESULTS

The project started in April 2016. The literature search identified 28 articles of methodological developments relevant to a method for specifying a target difference. A Delphi study involving 69 participants, along with a 2-day consensus meeting were conducted. In addition, further engagement sessions were held at two international conferences. The main guidance text was finalised on April 18, 2018, after revision informed by feedback gathered from stages 2 and 3 and from funder representatives.

DISCUSSION

The DELTA² Delphi study identified a number of areas (such as practical recommendations and examples, greater coverage of different trial designs and statistical approaches) of particular interest amongst stakeholders which new guidance was desired to meet. New relevant references were identified by the review. Such findings influenced the scope, drafting and revision of the guidance. While not all suggestions could be accommodated, it is hoped that the process has led to a more useful and practical document.

Bayesian sample size determination for cost-effectiveness studies with censored data

Cost-effectiveness models are commonly utilized to determine the combined clinical and economic impact of one treatment compared to another. However, most methods for sample size determination of cost-effectiveness studies assume fully observed costs and effectiveness outcomes, which presents challenges for survival-based studies in which censoring exists. We propose a Bayesian method for the design and analysis of cost-effectiveness data in which costs and effectiveness may be censored, and the sample size is approximated for both power and assurance. We explore two parametric models and demonstrate the flexibility of the approach to accommodate a variety of modifications to study assumptions.

The Center for Epidemiologic Studies Depression Scale is an adequate screening instrument for depression and anxiety disorder in adults with congential heart disease

BACKGROUND

The Center for Epidemiological Studies Depression Scale (CES-D) is an instrument that is commonly used to screen for depression in patients with chronic disease, but the characteristics of the CES-D in adults with congenital heart disease (CHD) have not yet been studied. The aim of this study was to investigate the criterion validities and the predictive powers of the CES-D for depression and anxiety disorders in adults with CHD.

METHODS

Two hundred patients were screened with the CES-D and secondarily interviewed with a diagnostic instrument, i.e., the Mini International Neuropsychiatric Instrument. The sensitivity and specificity values of the CES-D were calculated by cross-tabulation at different cutoff scores. Receiver operating characteristic (ROC) curves were used to assess the optimal cutoff point for each disorder and to assess the predictive power of the instrument.

RESULTS

The CES-D exhibited satisfactory criterion validities for depression and for all combinations of depression and/or anxiety. With a desired sensitivity of at least 80%, the optimal cutoff scores were 18. The predictive power of the CES-D in the patients was best for major depression and dysthymia (area under the ROC curve: 0.92) followed by the score for any combination of depression and/or anxiety (0.88).

CONCLUSION

The use of CES-D to simultaneously screen for both depression and anxiety disorders may be useful in adults with CHD.

TRIAL REGISTRATION

CESDEP 212. Registered 2 March 2014 (retrospectively registered).

Value of Information Analysis Informing Adoption and Research Decisions in a Portfolio of Health Care Interventions

Background: Value of information (VOI) analysis quantifies the value of additional research in reducing decision uncertainty. It addresses adoption and research decisions simultaneously by comparing the expected benefits and costs of research studies. Nevertheless, the application of this approach in practice remains limited. Objectives: To apply VOI analysis in health care interventions to guide adoption decisions, optimize trial design, and prioritize research. Methods: The analysis was from the perspective of Queensland Health, Australia. It included four interventions: clinically indicated catheter replacement, tissue adhesive for securing catheters, negative pressure wound therapy (NPWT) in caesarean sections, and nutritional support for preventing pressure ulcers. For each intervention, cost-effectiveness analysis was performed, decision uncertainty characterized, and VOI calculated using Monte Carlo simulations. The benefits and costs of additional research were considered together with the costs and consequences of acting now versus waiting for more information. All values are reported in 2014 Australian dollars (AU$). Results: All interventions were cost-effective, but with various levels of decision uncertainty. The current evidence is sufficient to support the adoption of clinically indicated catheter replacement. For the tissue adhesive, an additional study before adoption is worthwhile with a four-arm trial of 220 patients per arm. Additional research on NPWT before adoption is worthwhile with a two-arm trial of 200 patients per arm. Nutritional support should be adopted with a two-arm trial of 1200 patients per arm. Based on the expected net monetary benefits, the studies were ranked as follows: 1) NPWT (AU$1.2 million), 2) tissue adhesive (AU$0.3 milliion), and 3) nutritional support (AU$0.1 million). Conclusions: VOI analysis is a useful and practical approach to inform adoption and research decisions. Efforts should be focused on facilitating its integration into decision making frameworks.

The effect of fines on nonattendance in public hospital outpatient clinics: study protocol for a randomized controlled trial

Background

Nonattendance at scheduled appointments in public hospitals presents a challenge for efficient resource use and may ultimately affect health outcomes due to longer waiting times. Seven percent of all scheduled outpatient appointments in the United Kingdom are estimated to be nonattended. Various reminder systems have been shown to moderately reduce nonattendance, although the effect of issuing fines for nonattendance has not yet been tested in a randomized context. However, such use of financial incentives could impact access to care differently across the different socioeconomic groups. The aim of this study is to assess the effect of fines on hospital outpatient nonattendance.

Methods/design

A 1:1 randomized controlled trial of scheduled outpatient appointments was used, with follow-ups until the date of appointment. The setting is an orthopedic clinic at a regional hospital in Denmark. Appointments for users who are scheduled for diagnostics, treatment, surgery, or follow-ups were included from May 2015 to November 2015. Appointments assigned to the intervention arm include an attachment of the appointment letter explaining that a fine will be issued in the case of nonattendance without prior notice. Appointments assigned to the control arm follow usual practice (same system but no letter attachment). The primary outcome is the proportion of nonattendance. Secondary outcomes are proportions of cancellations, sociodemographics, and health-problem characteristics. Furthermore, the intervention costs and production value of nonattended appointments will be measured. An analysis of effect and cost-effectiveness will be conducted based on a 5 % significance level.

Discussion

The study is initiated and funded by the Danish Regions, which have the responsibility for the Danish public healthcare sector. The results are expected to inform future decisions about the introduction of fines for nonattendance at public hospitals.

Trial registration

Current Controlled Trials, ISRCTN61925912. Registered on 6 July 2015.

Optimal and maximin sample sizes for multicentre cost-effectiveness trials

This paper deals with the optimal sample sizes for a multicentre trial in which the cost-effectiveness of two treatments in terms of net monetary benefit is studied. A bivariate random-effects model, with the treatment-by-centre interaction effect being random and the main effect of centres fixed or random, is assumed to describe both costs and effects. The optimal sample sizes concern the number of centres and the number of individuals per centre in each of the treatment conditions. These numbers maximize the efficiency or power for given research costs or minimize the research costs at a desired level of efficiency or power. Information on model parameters and sampling costs are required to calculate these optimal sample sizes. In case of limited information on relevant model parameters, sample size formulas are derived for so-called maximin sample sizes which guarantee a power level at the lowest study costs. Four different maximin sample sizes are derived based on the signs of the lower bounds of two model parameters, with one case being worst compared to others. We numerically evaluate the efficiency of the worst case instead of using others. Finally, an expression is derived for calculating optimal and maximin sample sizes that yield sufficient power to test the cost-effectiveness of two treatments.

A non-linear beta-binomial regression model for mapping EORTC QLQ- C30 to the EQ-5D-3L in lung cancer patients: a comparison with existing approaches

BACKGROUND

The performance of the Beta Binomial (BB) model is compared with several existing models for mapping the EORTC QLQ-C30 (QLQ-C30) on to the EQ-5D-3L using data from lung cancer trials.

METHODS

Data from 2 separate non small cell lung cancer clinical trials (TOPICAL and SOCCAR) are used to develop and validate the BB model. Comparisons with Linear, TOBIT, Quantile, Quadratic and CLAD models are carried out. The mean prediction error, R(2), proportion predicted outside the valid range, clinical interpretation of coefficients, model fit and estimation of Quality Adjusted Life Years (QALY) are reported and compared. Monte-Carlo simulation is also used.

RESULTS

The Beta-Binomial regression model performed 'best' among all models. For TOPICAL and SOCCAR trials, respectively, residual mean square error (RMSE) was 0.09 and 0.11; R(2) was 0.75 and 0.71; observed vs. predicted means were 0.612 vs. 0.608 and 0.750 vs. 0.749. Mean difference in QALY's (observed vs. predicted) were 0.051 vs. 0.053 and 0.164 vs. 0.162 for TOPICAL and SOCCAR respectively. Models tested on independent data show simulated 95% confidence from the BB model containing the observed mean more often (77% and 59% for TOPICAL and SOCCAR respectively) compared to the other models. All algorithms over-predict at poorer health states but the BB model was relatively better, particularly for the SOCCAR data.

CONCLUSION

The BB model may offer superior predictive properties amongst mapping algorithms considered and may be more useful when predicting EQ-5D-3L at poorer health states. We recommend the algorithm derived from the TOPICAL data due to better predictive properties and less uncertainty.

Methodological developments in randomized controlled trial-based economic evaluations

Economic evaluation is a key contributor to decision making in health care, and it is important that it is carried out as effectively and reliably as possible. Studies carried out alongside randomised controlled trials are required to contribute real-world evidence to the decision-making process. However, the requirement that resource use be measured as well as effectiveness data within a trial results in additional complexity for trialists, and there are a number of methodological areas in which improvement is needed. This article reviews the literature in methodological work carried out to inform economic evaluation studies conducted alongside randomised controlled trials. Recent advances in areas including overall trial design, measuring resource use, measuring outcomes and reporting economic evaluations are discussed.

Value of information analysis optimizing future trial design from a pilot study on catheter securement devices

Background:

Value of information analysis has been proposed as an alternative to the standard hypothesis testing approach, which is based on type I and type II errors, in determining sample sizes for randomized clinical trials. However, in addition to sample size calculation, value of information analysis can optimize other aspects of research design such as possible comparator arms and alternative follow-up times, by considering trial designs that maximize the expected net benefit of research, which is the difference between the expected cost of the trial and the expected value of additional information.

Purpose:

To apply value of information methods to the results of a pilot study on catheter securement devices to determine the optimal design of a future larger clinical trial.

Methods:

An economic evaluation was performed using data from a multi-arm randomized controlled pilot study comparing the efficacy of four types of catheter securement devices: standard polyurethane, tissue adhesive, bordered polyurethane and sutureless securement device. Probabilistic Monte Carlo simulation was used to characterize uncertainty surrounding the study results and to calculate the expected value of additional information. To guide the optimal future trial design, the expected costs and benefits of the alternative trial designs were estimated and compared.

Results:

Analysis of the value of further information indicated that a randomized controlled trial on catheter securement devices is potentially worthwhile. Among the possible designs for the future trial, a four-arm study with 220 patients/arm would provide the highest expected net benefit corresponding to 130% return-on-investment. The initially considered design of 388 patients/arm, based on hypothesis testing calculations, would provide lower net benefit with return-on-investment of 79%.

Limitations:

Cost-effectiveness and value of information analyses were based on the data from a single pilot trial which might affect the accuracy of our uncertainty estimation. Another limitation was that different follow-up durations for the larger trial were not evaluated.

Conclusion:

The value of information approach allows efficient trial design by maximizing the expected net benefit of additional research. This approach should be considered early in the design of randomized clinical trials.

Value of information analysis in healthcare: a review of principles and applications

BACKGROUND

Economic evaluations are increasingly utilized to inform decisions in healthcare; however, decisions remain uncertain when they are not based on adequate evidence. Value of information (VOI) analysis has been proposed as a systematic approach to measure decision uncertainty and assess whether there is sufficient evidence to support new technologies.

SCOPE

The objective of this paper is to review the principles and applications of VOI analysis in healthcare. Relevant databases were systematically searched to identify VOI articles. The findings from the selected articles were summarized and narratively presented.

FINDINGS

Various VOI methods have been developed and applied to inform decision-making, optimally designing research studies and setting research priorities. However, the application of this approach in healthcare remains limited due to technical and policy challenges.

CONCLUSION

There is a need to create more awareness about VOI analysis, simplify its current methods, and align them with the needs of decision-making organizations.

Competing in value-based health care: keys to winning the foot race

The US health care system is transitioning to a value-based model of health care in which providers will be rewarded for delivering services that achieve excellent clinical outcomes with efficient cost utilization. The concept of "value" in health care (defined as health outcomes achieved per dollar spent) is rapidly spreading as physicians and health systems brace for the paradigm shift from "fee-for-volume" to "fee-for-value" reimbursement. What constitutes good value versus poor value in health care remains nebulous at this time. Various specialties across medicine and within orthopaedics are seeking to better demonstrate value delivered to patients, payers, and policy makers. The objective of this article is to develop a framework for defining and measuring value in foot and ankle surgery. In this new era of health care, we believe that a working knowledge of value and its determinants will be imperative for foot and ankle surgeons to unify research and quality improvement efforts so as to demonstrate the value of services rendered within the subspecialty.

LEVEL OF EVIDENCE

Level V, expert opinion.

A C++ program to calculate sample sizes for cost-effectiveness trials in a Bayesian framework

Cost-Effectiveness Analysis (CEA) has become an increasingly important component of clinical trials. However, formal sample size calculations for such studies are not common. One of the reasons for this might be due to the absence of readily available computer software to perform complex calculations, particularly in a Bayesian setting. In this paper, a C++ program (using NAG library functions/subroutines) is presented to estimate the sample sizes for cost-effectiveness clinical trials in a Bayesian framework. The program can equally be used to calculate sample sizes for efficacy trials. The Bayesian approach to sample size calculation is based on that of O'Hagan and Stevens (A. O'Hagan, J.W. Stevens, Bayesian assessment of sample size for clinical trials of cost-effectiveness, Medical Decision Making 21 (2001) 219-230). With this program, the user can calculate sample sizes for various thresholds of willingness to pay and under various assumptions of the correlations between cost and effects. Under some prior, the program produces frequentist sample size as well. The program runs under windows environment and running time is very short.