Cost-effectiveness analysis for multinational clinical trials

Multivariate Generalized Linear Mixed-Effects Models for the Analysis of Clinical Trial-Based Cost-Effectiveness Data

Economic evaluations conducted alongside randomized controlled trials are a popular vehicle for generating high-quality evidence on the incremental cost-effectiveness of competing health care interventions. Typically, in these studies, resource use (and by extension, economic costs) and clinical (or preference-based health) outcomes data are collected prospectively for trial participants to estimate the joint distribution of incremental costs and incremental benefits associated with the intervention. In this article, we extend the generalized linear mixed-model framework to enable simultaneous modeling of multiple outcomes of mixed data types, such as those typically encountered in trial-based economic evaluations, taking into account correlation of outcomes due to repeated measurements on the same individual and other clustering effects. We provide new wrapper functions to estimate the models in Stata and R by maximum and restricted maximum quasi-likelihood and compare the performance of the new routines with alternative implementations across a range of statistical programming packages. Empirical applications using observed and simulated data from clinical trials suggest that the new methods produce broadly similar results as compared with Stata’s merlin and gsem commands and a Bayesian implementation in WinBUGS. We highlight that, although these empirical applications primarily focus on trial-based economic evaluations, the new methods presented can be generalized to other health economic investigations characterized by multivariate hierarchical data structures.

Taking the Analysis of Trial-Based Economic Evaluations to the Next Level: The Importance of Accounting for Clustering

OBJECTIVES

The aim of this study was to assess the performance and impact of multilevel modelling (MLM) compared with ordinary least squares (OLS) regression in trial-based economic evaluations with clustered data.

METHODS

Three thousand datasets with balanced and unbalanced clusters were simulated with correlation coefficients between costs and effects of - 0.5, 0, and 0.5, and intraclass correlation coefficients (ICCs) varying between 0.05 and 0.30. Each scenario was analyzed using both MLM and OLS. Statistical uncertainty around MLM and OLS estimates was estimated using bootstrapping. Performance measures were estimated and compared between approaches, including bias, root mean squared error (RMSE) and coverage probability. Cost and effect differences, and their corresponding confidence intervals and standard errors, incremental cost-effectiveness ratios, incremental net-monetary benefits and cost-effectiveness acceptability curves were compared.

RESULTS

Cost-effectiveness outcomes were similar between OLS and MLM. MLM produced larger statistical uncertainty and coverage probabilities closer to nominal levels than OLS. The higher the ICC, the larger the effect on statistical uncertainty between MLM and OLS. Significant cost-effectiveness outcomes as estimated by OLS became non-significant when estimated by MLM. At all ICCs, MLM resulted in lower probabilities of cost effectiveness than OLS, and this difference became larger with increasing ICCs. Performance measures and cost-effectiveness outcomes were similar across scenarios with varying correlation coefficients between costs and effects.

CONCLUSIONS

Although OLS produced similar cost-effectiveness outcomes, it substantially underestimated the amount of variation in the data compared with MLM. To prevent suboptimal conclusions and a possible waste of scarce resources, it is important to use MLM in trial-based economic evaluations when data are clustered.

The Impact of Treatments for OSA on Monetized Health Economic Outcomes: A Systematic Review

OBJECTIVE

To review systematically the published literature regarding the impact of treatment for OSA on monetized health economic outcomes.

METHODS

Customized structured searches were performed in PubMed, Embase (Embase.com), and the Cochrane Central Register of Controlled Trials (Wiley) databases. Reference lists of eligible studies were also analyzed. Titles and abstracts were examined, and articles were identified for full-text review. Studies that met inclusion criteria were evaluated in detail, and study characteristics were extracted using a standardized template. Quantitative characteristics of the studies were summarized, and a qualitative synthesis was performed.

RESULTS

Literature searches identified 2,017 nonredundant abstracts, and 196 full-text articles were selected for review. Seventeen studies met inclusion criteria and were included in the final synthesis. Seven studies included formal cost-effectiveness or cost-utility analyses. Ten studies employed cohort designs, and four studies employed randomized controlled trial or quasi-experimental designs. Positive airway pressure was the most common treatment modality, but oral appliances and surgical approaches were also included. The most common health economic outcomes were health-care use (HCU) and quality-adjusted life years (QALYs). Follow-ups ranged from 6 weeks to 5 years. Overall, 15 of 18 comparisons found that treatment of OSA resulted in a positive economic impact. Treatment adherence and OSA severity were positively associated with cost-effectiveness.

CONCLUSIONS

Although study methodologies varied widely, evidence consistently suggested that treatment of OSA was associated with favorable economic outcomes, including QALYs, within accepted ranges of cost-effectiveness, reduced HCU, and reduced monetized costs.

[Economic analysis of multinational clinical trials in oncology]

In oncology, as in other fields of medicine, international multicentre clinical trials came into being so as to include a sufficient number of subjects to investigate a clinical situation. The existence of tight budgetary constraints and the desire to make the best use of the resources available have resulted in the development of economic evaluations associated with these trials, which, thanks to their level of evidence and their size, provide particularly relevant material. Nonetheless, economic evaluations alongside international clinical trials raise specific questions of methodology with regard to both the design and the analysis of the results. Indeed, the costs of goods and services consumed, the types and quantities of resources, and medical practices vary from one country to another and within an individual country. Economic data from the different countries involved must be available so as to study and to take into account this variability, and appropriate techniques for cost estimations and analysis must be implemented to aggregate the results from several countries. From a review of the literature, the aim of this work was to provide an overview of the specific methodological features of economic evaluations alongside international clinical trials: analysis of efficacy data from several countries, collection of resources and real costs, methods to establish the monetary value of resources, methods to aggregate results accounting for the trial effect.

Conducting EQ-5D Valuation Studies in Resource-Constrained Countries: The Potential Use of Shrinkage Estimators to Reduce Sample Size

BACKGROUND

Resource-constrained countries have difficulty conducting large EQ-5D valuation studies, which limits their ability to conduct cost-utility analyses using a value set specific to their own population. When estimates of similar but related parameters are available, shrinkage estimators reduce uncertainty and yield estimators with smaller mean square error (MSE). We hypothesized that health utilities based on shrinkage estimators can reduce MSE and mean absolute error (MAE) when compared to country-specific health utilities.

METHODS

We conducted a simulation study (1,000 iterations) based on the observed means and standard deviations (or standard errors) of the EQ-5D-3L valuation studies from 14 counties. In each iteration, the simulated data were fitted with the model based on the country-specific functional form of the scoring algorithm to create country-specific health utilities ("naïve" estimators). Shrinkage estimators were calculated based on the empirical Bayes estimation methods. The performance of shrinkage estimators was compared with those of the naïve estimators over a range of different sample sizes based on MSE, MAE, mean bias, standard errors and the width of confidence intervals.

RESULTS

The MSE of the shrinkage estimators was smaller than the MSE of the naïve estimators on average, as theoretically predicted. Importantly, the MAE of the shrinkage estimators was also smaller than the MAE of the naïve estimators on average. In addition, the reduction in MSE with the use of shrinkage estimators did not substantially increase bias. The degree of reduction in uncertainty by shrinkage estimators is most apparent in valuation studies with small sample size.

CONCLUSION

Health utilities derived from shrinkage estimation allow valuation studies with small sample size to "borrow strength" from other valuation studies to reduce uncertainty.

Mind the Gap! A Multilevel Analysis of Factors Related to Variation in Published Cost-Effectiveness Estimates within and between Countries

Background. Published cost-effectiveness estimates can vary considerably, both within and between countries. Despite extensive discussion, little is known empirically about factors relating to these variations. Objectives. To use multilevel statistical modeling to integrate cost-effectiveness estimates from published economic evaluations to investigate potential causes of variation. Methods. Cost-effectiveness studies of statins for cardiovascular disease prevention were identified by systematic review. Estimates of incremental costs and effects were extracted from reported base case, sensitivity, and subgroup analyses, with estimates grouped in studies and in countries. Three bivariate models were developed: a cross-classified model to accommodate data from multinational studies, a hierarchical model with multinational data allocated to a single category at country level, and a hierarchical model excluding multinational data. Covariates at different levels were drawn from a long list of factors suggested in the literature. Results. We found 67 studies reporting 2094 cost-effectiveness estimates relating to 23 countries (6 studies reporting for more than 1 country). Data and study-level covariates included patient characteristics, intervention and comparator cost, and some study methods (e.g., discount rates and time horizon). After adjusting for these factors, the proportion of variation attributable to countries was negligible in the cross-classified model but moderate in the hierarchical models (14%−19% of total variance). Country-level variables that improved the fit of the hierarchical models included measures of income and health care finance, health care resources, and population risks. Conclusions. Our analysis suggested that variability in published cost-effectiveness estimates is related more to differences in study methods than to differences in national context. Multinational studies were associated with much lower country-level variation than single-country studies. These findings are for a single clinical question and may be atypical.

Cost-effectiveness analysis alongside clinical trials II-An ISPOR Good Research Practices Task Force report

Clinical trials evaluating medicines, medical devices, and procedures now commonly assess the economic value of these interventions. The growing number of prospective clinical/economic trials reflects both widespread interest in economic information for new technologies and the regulatory and reimbursement requirements of many countries that now consider evidence of economic value along with clinical efficacy. As decision makers increasingly demand evidence of economic value for health care interventions, conducting high-quality economic analyses alongside clinical studies is desirable because they broaden the scope of information available on a particular intervention, and can efficiently provide timely information with high internal and, when designed and analyzed properly, reasonable external validity. In 2005, ISPOR published the Good Research Practices for Cost-Effectiveness Analysis Alongside Clinical Trials: The ISPOR RCT-CEA Task Force report. ISPOR initiated an update of the report in 2014 to include the methodological developments over the last 9 years. This report provides updated recommendations reflecting advances in several areas related to trial design, selecting data elements, database design and management, analysis, and reporting of results. Task force members note that trials should be designed to evaluate effectiveness (rather than efficacy) when possible, should include clinical outcome measures, and should obtain health resource use and health state utilities directly from study subjects. Collection of economic data should be fully integrated into the study. An incremental analysis should be conducted with an intention-to-treat approach, complemented by relevant subgroup analyses. Uncertainty should be characterized. Articles should adhere to established standards for reporting results of cost-effectiveness analyses. Economic studies alongside trials are complementary to other evaluations (e.g., modeling studies) as information for decision makers who consider evidence of economic value along with clinical efficacy when making resource allocation decisions.

Optimal and maximin sample sizes for multicentre cost-effectiveness trials

This paper deals with the optimal sample sizes for a multicentre trial in which the cost-effectiveness of two treatments in terms of net monetary benefit is studied. A bivariate random-effects model, with the treatment-by-centre interaction effect being random and the main effect of centres fixed or random, is assumed to describe both costs and effects. The optimal sample sizes concern the number of centres and the number of individuals per centre in each of the treatment conditions. These numbers maximize the efficiency or power for given research costs or minimize the research costs at a desired level of efficiency or power. Information on model parameters and sampling costs are required to calculate these optimal sample sizes. In case of limited information on relevant model parameters, sample size formulas are derived for so-called maximin sample sizes which guarantee a power level at the lowest study costs. Four different maximin sample sizes are derived based on the signs of the lower bounds of two model parameters, with one case being worst compared to others. We numerically evaluate the efficiency of the worst case instead of using others. Finally, an expression is derived for calculating optimal and maximin sample sizes that yield sufficient power to test the cost-effectiveness of two treatments.

Transferability of health cost evaluation across locations in oncology: cluster and principal component analysis as an explorative tool

Background

The transferability of economic evaluation in health care is of increasing interest in today’s globalized environment. Here, we propose a methodology for assessing the variability of data elements in cost evaluations in oncology. This method was tested in the context of the European Network of Excellence “Connective Tissues Cancers Network”.

Methods

Using a database that was previously aimed at exploring sarcoma management practices in Rhône-Alpes (France) and Veneto (Italy), we developed a model to assess the transferability of health cost evaluation across different locations. A nested data structure with 60 final factors of variability (e.g., unit cost of chest radiograph) within 16 variability areas (e.g., unit cost of imaging) within 12 objects (e.g., diagnoses) was produced in Italy and France, separately. Distances between objects were measured by Euclidean distance, Mahalanobis distance, and city-block metric. A hierarchical structure using cluster analysis (CA) was constructed. The objects were also represented by their projections and area of variability through correlation studies using principal component analysis (PCA). Finally, a hierarchical clustering based on principal components was performed.

Results

CA suggested four clusters of objects: chemotherapy in France; follow-up with relapse in Italy; diagnosis, surgery, radiotherapy, chemotherapy, and follow-up without relapse in Italy; and diagnosis, surgery, and follow-up with or without relapse in France. The variability between clusters was high, suggesting a lower transferability of results. Also, PCA showed a high variability (i.e. lower transferability) for diagnosis between both countries with regard to the quantities and unit costs of biopsies.

Conclusion

CA and PCA were found to be useful for assessing the variability of cost evaluations across countries. In future studies, regression methods could be applied after these methods to elucidate the determinants of the differences found in these analyses.

Electronic supplementary material

The online version of this article (doi:10.1186/s12913-014-0537-x) contains supplementary material, which is available to authorized users.

The road not taken: transferability issues in multinational trials

BACKGROUND

National regulatory agencies often have to use cost-effectiveness (CE) data from multinational randomized controlled trials (RCTs) for national decision making on reimbursement of new drugs. We need to make the best use of these patient-level data to obtain estimates of country-specific CE. Several methods, ranging from simple to statistically complex, have existed for years. We investigated which of these methods are used to estimate CE ratios in economic evaluations performed alongside recent, multinational RCTs that enrolled at least 500 patients.

METHODS

In this systematic literature review, studies were classified based on whether resource use, unit costs, health outcomes and utility value sets were obtained from all countries, a subset of countries or one country. We recorded if the study presented trial-wide and country-specific CE results and reported the statistical analyses that were used to estimate them.

RESULTS

We included 21 studies, of which the majority used measurements of health care utilization and health outcomes from all countries to estimate CE. Thirteen studies used a one-country valuation of health care utilization; six used a multi-country valuation. Despite the availability of country-specific utility value sets, none of the studies that presented quality-adjusted life-years (QALYs) used multi-country valuation. Valuation of health care utilization and health outcomes was not always consistent within a study: three studies combined a multi-country valuation of health care utilization, with a one-country valuation of health outcomes. Most studies calculated trial-wide CE estimates, while 11 studies calculated country- or region-specific estimates. Thirteen studies used relatively simple methods, which do not take the possible interaction between the country and treatment effect on health care utilization and health outcomes into account. Eight studies used more advanced statistical methods. Three of them used a fixed-effects modeling approach. Five studies explicitly took the hierarchical structure of the data into account, which leads to more appropriate estimates of population average results and associated standard errors. In this way, they help improve transferability of the published results.

CONCLUSION

Based on this systematic review, we concluded that the uptake of more advanced statistical methods has been relatively slow, while simpler naïve methods are still routinely employed.

Accounting for between-study variation in incremental net benefit in value of information methodology

Previous applications of value of information methods for determining optimal sample size in randomized clinical trials have assumed no between-study variation in mean incremental net benefit. By adopting a hierarchical model, we provide a solution for determining optimal sample size with this assumption relaxed. The solution is illustrated with two examples from the literature. Expected net gain increases with increasing between-study variation, reflecting the increased uncertainty in incremental net benefit and reduced extent to which data are borrowed from previous evidence. Hence, a trial can become optimal where current evidence is sufficient assuming no between-study variation. However, despite the expected net gain increasing, the optimal sample size in the illustrated examples is relatively insensitive to the amount of between-study variation. Further percentage losses in expected net gain were small even when choosing sample sizes that reflected widely different between-study variation.

Empirical shrinkage estimator for consistency assessment of treatment effects in multi-regional clinical trials

Multi-regional clinical trials have been widely used for efficient global new drug developments. Both a fixed-effect model and a random-effect model can be used for trial design and data analysis of a multi-regional clinical trial. In this paper, we first compare these two models in terms of the required sample size, type I error rate control, and the interpretability of trial results. We then apply the empirical shrinkage estimation approach based on the random-effect model to two criteria of consistency assessment of treatment effects across regions. As demonstrated in our computations, compared with the sample estimator, the shrinkage estimator of the treatment effect of an individual region borrowing information from the other regions is much closer to the estimator of the overall treatment effect, has smaller variability, and therefore provides much higher probability for demonstrating consistency. We use a multinational trial example with time to event endpoint to illustrate the application of the method.

Developing appropriate methods for cost-effectiveness analysis of cluster randomized trials

AIM

Cost-effectiveness analyses (CEAs) may use data from cluster randomized trials (CRTs), where the unit of randomization is the cluster, not the individual. However, most studies use analytical methods that ignore clustering. This article compares alternative statistical methods for accommodating clustering in CEAs of CRTs.

METHODS

Our simulation study compared the performance of statistical methods for CEAs of CRTs with 2 treatment arms. The study considered a method that ignored clustering--seemingly unrelated regression (SUR) without a robust standard error (SE)--and 4 methods that recognized clustering--SUR and generalized estimating equations (GEEs), both with robust SE, a "2-stage" nonparametric bootstrap (TSB) with shrinkage correction, and a multilevel model (MLM). The base case assumed CRTs with moderate numbers of balanced clusters (20 per arm) and normally distributed costs. Other scenarios included CRTs with few clusters, imbalanced cluster sizes, and skewed costs. Performance was reported as bias, root mean squared error (rMSE), and confidence interval (CI) coverage for estimating incremental net benefits (INBs). We also compared the methods in a case study.

RESULTS

Each method reported low levels of bias. Without the robust SE, SUR gave poor CI coverage (base case: 0.89 v. nominal level: 0.95). The MLM and TSB performed well in each scenario (CI coverage, 0.92-0.95). With few clusters, the GEE and SUR (with robust SE) had coverage below 0.90. In the case study, the mean INBs were similar across all methods, but ignoring clustering underestimated statistical uncertainty and the value of further research.

CONCLUSIONS

MLMs and the TSB are appropriate analytical methods for CEAs of CRTs with the characteristics described. SUR and GEE are not recommended for studies with few clusters.

Extending DerSimonian and Laird's methodology to perform multivariate random effects meta-analyses

Multivariate meta-analysis is increasingly used in medical statistics. In the univariate setting, the non-iterative method proposed by DerSimonian and Laird is a simple and now standard way of performing random effects meta-analyses. We propose a natural and easily implemented multivariate extension of this procedure which is accessible to applied researchers and provides a much less computationally intensive alternative to existing methods. In a simulation study, the proposed procedure performs similarly in almost all ways to the more established iterative restricted maximum likelihood approach. The method is applied to some real data sets and an extension to multivariate meta-regression is described.

Improving costing methods in multicentre economic evaluation: the use of multiple imputation for unit costs

Economic evaluations must use appropriate costing methods. However, in multicentre cost-effectiveness analyses (CEA) a fundamental issue of how best to measure and analyse unit costs has been neglected. Multicentre CEA commonly take the mean unit cost from a national database, such as NHS reference costs. This approach does not recognise that unit costs vary across centres and are unavailable in some centres. This paper proposes the use of multiple imputation (MI) to predict those centre-specific unit costs that are not available, while recognising the statistical uncertainty surrounding this imputation.We illustrate MI with a CEA of a multicentre randomised trial (1014 patients, 60 centres), implemented using multilevel modelling. We use MI to derive centre-specific unit costs, based on centre characteristics including average casemix, and compare this to using mean NHS reference costs. In this case study, using MI unit costs rather than mean reference costs led to less heterogeneity across centres, more precise estimates of incremental cost, but similar estimates of incremental cost-effectiveness.We conclude that using MI to predict unit costs can preserve correlations, maximise the use of available data, and, when combined with multilevel modelling is an appropriate method for recognising the statistical uncertainty in multicentre CEA.

The ISPOR Good Practices for Quality Improvement of Cost-Effectiveness Research Task Force Report

OBJECTIVES

The International Society for Pharmacoeconomics and Outcomes Research (ISPOR) Health Science Policy Council recommended and the ISPOR Board of Directors approved the formation of a Task Force to critically examine the major issues related to Quality Improvement in Cost-effectiveness Research (QICER). The Council's primary recommendation for this Task Force was that it should report on the quality of cost-effectiveness research and make recommendations to facilitate the improvement of pharmacoeconomics and health outcomes research and its use in stimulating better health care and policy. Task force members were knowledgeable and experienced in medicine, pharmacy, biostatistics, health policy and health-care decision-making, biomedical knowledge transfer, health economics, and pharmacoeconomics. They were drawn from industry, academia, consulting organizations, and advisors to governments and came from Japan, the Netherlands, Canada and the United States.

METHODS

Face-to-face meetings of the Task Force were held at ISPOR North American and European meetings and teleconferences occurred every few months. Literature reviews and surveys were conducted and the first preliminary findings presented at an open forum at the May 2008 ISPOR meeting in Toronto. The final draft report was circulated to the expert reviewer group and then to the entire membership for comment. The draft report was posted on the ISPOR Web site in April 2009. All formal comments received were posted to the association Web site and presented for discussion at the Task Force forum during the ISPOR 14th Annual International Meeting in May 2009. Comments and feedback from the forums, reviewers and membership were considered in the final report. Once Task Force consensus was reached, the article was submitted to Value in Health.

CONCLUSIONS

The QICER Task Force recommends that ISPOR implement the following: * With respect to CER guidelines, that ISPOR promote harmonization of guidelines, allowing for differences in application, regional needs and politics; evaluate available instruments or promote development of a new one that will allow standardized quantification of the impact of CER guidelines on the quality of CER studies; report periodically on those countries or regions that have developed guidelines; periodically evaluate the quality of published studies (those journals with CER guidances) or those submitted to decision-making bodies (as public transparency increases). * With respect to methodologies, that ISPOR promote publication of methodological guidelines in more applied journals in more easily understandable format to transfer knowledge to researchers who need to apply more rigorous methods; promote full availability of models in electronic format to combat space restrictions in hardcopy publications; promote consistency of methodological review for all CER studies; promote adoption of explicit best practices guidelines among regulatory and reimbursement authorities; periodically update all ISPOR Task Force reports; periodically review use of ISPOR Task Force guidelines; periodically report on statistical and methodological challenges in HE; evaluate periodically whether ISPOR's methodological guidelines lead to improved quality; and support training and knowledge transfer of rigorous CER methodologies to researchers and health care decision-makers. * With respect to publications, that ISPOR develop standard CER guidances to which journals will be able to refer their authors and their reviewers; lobby to establish these guidances within the International Committee for Medical Journal Editors (ICMJE) Requirements to which most journals refer in their Author Instructions; provide support in terms of additional reviewer expertise to those journals lacking appropriate reviewers; periodically report on journals publishing CER research; periodically report on the quality of CER publications; and support training and knowledge transfer of the use of these guidelines to researchers and reviewers. * With respect to evidence-based health-care decision-making, that ISPOR recognize at its annual meetings those countries/agencies/private companies/researchers using CER well, and those practitioners and researchers supporting good patient use of CER in decision-making; and promote public presentation of case studies of applied use of CER concepts or guidelines.

Bayesian Hierarchical Models for Cost-Effectiveness Analyses that Use Data from Cluster Randomized Trials

Cost-effectiveness analyses (CEA) may be undertaken alongside cluster randomized trials (CRTs) where randomization is at the level of the cluster (for example, the hospital or primary care provider) rather than the individual. Costs (and outcomes) within clusters may be correlated so that the assumption made by standard bivariate regression models, that observations are independent, is incorrect. This study develops a flexible modeling framework to acknowledge the clustering in CEA that use CRTs. The authors extend previous Bayesian bivariate models for CEA of multicenter trials to recognize the specific form of clustering in CRTs. They develop new Bayesian hierarchical models (BHMs) that allow mean costs and outcomes, and also variances, to differ across clusters. They illustrate how each model can be applied using data from a large (1732 cases, 70 primary care providers) CRT evaluating alternative interventions for reducing postnatal depression. The analyses compare cost-effectiveness estimates from BHMs with standard bivariate regression models that ignore the data hierarchy. The BHMs show high levels of cost heterogeneity across clusters (intracluster correlation coefficient, 0.17). Compared with standard regression models, the BHMs yield substantially increased uncertainty surrounding the cost-effectiveness estimates, and altered point estimates. The authors conclude that ignoring clustering can lead to incorrect inferences. The BHMs that they present offer a flexible modeling framework that can be applied more generally to CEA that use CRTs.

Transferability of economic evaluations across jurisdictions: ISPOR Good Research Practices Task Force report

ABSTRACT A growing number of jurisdictions now request economic data in support of their decision-making procedures for the pricing and/or reimbursement of health technologies. Because more jurisdictions request economic data, the burden on study sponsors and researchers increases. There are many reasons why the cost-effectiveness of health technologies might vary from place to place. Therefore, this report of an ISPOR Good Practices Task Force reviews what national guidelines for economic evaluation say about transferability, discusses which elements of data could potentially vary from place to place, and recommends good research practices for dealing with aspects of transferability, including strategies based on the analysis of individual patient data and based on decision-analytic modeling.

Cost-effectiveness analysis of a multinational RCT with a binary measure of effectiveness and an interacting covariate

In a recent multinational randomized clinical trial, 1356 patients from 14 countries were randomized between two arms. The primary measure of effectiveness was 30-day survival. Health care utilization was collected on all patients and was combined with a single country's price weights to provide patient-level cost data. The purpose of this paper is to report the results of the cost-effectiveness analysis for the country that provided the cost weights, so as to provide a case study for illustrating recently proposed methodologies that account for skewed cost data, the between-country variation in treatment effects, possible interactions between treatment and baseline covariates, and the difficulty of estimated adjusted risk differences. A hierarchal model is used to account for the two sources of variation (between country and between patients, within a country). The model, which uses gamma distributions for cost data and recent methods for estimating adjusted risk differences, provides overall and country-specific estimates of treatment effects. Model estimation is facilitated by Markov chain Monte Carlo methods using the WinBUGS software. In addition, the theory of expected value of information is used to determine if the data provided by the trial are sufficient for decision making.

A Bayesian approach for incorporating economic factors in sample size design for clinical trials of individual drugs and portfolios of drugs

Classical approaches to clinical trial design ignore economic factors that determine economic viability of a new drug. We address the choice of sample size in Phase III trials as a decision theory problem using a hybrid approach that takes a Bayesian view from the perspective of a drug company and a classical Neyman-Pearson view from the perspective of regulatory authorities. We incorporate relevant economic factors in the analysis to determine the optimal sample size to maximize the expected profit for the company. We extend the analysis to account for risk by using a 'satisficing' objective function that maximizes the chance of meeting a management-specified target level of profit. We extend the models for single drugs to a portfolio of clinical trials and optimize the sample sizes to maximize the expected profit subject to budget constraints. Further, we address the portfolio risk and optimize the sample sizes to maximize the probability of achieving a given target of expected profit.

Multilevel models for estimating incremental net benefits in multinational studies

Multilevel models (MLMs) have been recommended for estimating incremental net benefits (INBs) in multicentre cost-effectiveness analysis (CEA). However, these models have assumed that the INBs are exchangeable and that there is a common variance across all centres. This paper examines the plausibility of these assumptions by comparing various MLMs for estimating the mean INB in a multinational CEA. The results showed that the MLMs that assumed the INBs were exchangeable and had a common variance led to incorrect inferences. The MLMs that included covariates to allow for systematic differences across the centres, and estimated different variances in each centre, made more plausible assumptions, fitted the data better and led to more appropriate inferences. We conclude that the validity of assumptions underlying MLMs used in CEA need to be critically evaluated before reliable conclusions can be drawn.

Cost-effectiveness analysis using data from multinational trials: the use of bivariate hierarchical modeling

Health care cost-effectiveness analysis (CEA) often uses individual patient data (IPD) from multinational randomized controlled trials. Although designed to account for between-patient sampling variability in the clinical and economic data, standard analytical approaches to CEA ignore the presence of between-location variability in the study results. This is a restrictive limitation given that countries often differ in factors that could affect the results of CEAs, such as the availability of health care resources, their unit costs, clinical practice, and patient case mix. The authors advocate the use of Bayesian bivariate hierarchical modeling to analyze multinational cost-effectiveness data. This analytical framework explicitly recognizes that patient-level costs and outcomes are nested within countries. Using real-life data, the authors illustrate how the proposed methods can be applied to obtain (a) more appropriate estimates of overall cost-effectiveness and associated measure of sampling uncertainty compared to standard CEA and (b) country-specific cost-effectiveness estimates that can be used to assess the between-location variability of the study results while controlling for differences in country-specific and patient-specific characteristics. It is demonstrated that results from standard CEA using IPD from multinational trials display a large degree of variability across the 17 countries included in the analysis, producing potentially misleading results. In contrast, "shrinkage estimates'' obtained from the modeling approach proposed here facilitate the appropriate quantification of country-specific cost-effectiveness estimates while weighting the results based on the level of information available within each country. The authors suggest that the methods presented here represent a general framework for the analysis of economic data collected from different locations.

Addressing the issues that arise in analysing multicentre cost data, with application to a multinational study

Differences in the mean, spread and skewness of cost data collected from different countries present problems for analysis and interpretation. Here we develop generalised linear multilevel models to estimate the effects of patient and national characteristics on costs. Using gamma distributions and multiplicative effects for patient characteristics fitted the data better than models which assumed normal distributions or estimated additive effects. A multilevel gamma model is employed to allow for heterogeneity in the effects of patient case-mix across centres. Analysis of multinational cost data must recognise differences in mean, spread and skewness across centres, as well as the data's hierarchical structure.

Assessing generalisability in model-based economic evaluation studies: a structured review in osteoporosis

To support decision making, many countries have now introduced some formal assessment process to evaluate whether health technologies represent good 'value for money'. These often take the form of decision models that can be used to explore elements of importance to generalisability of study results across clinical settings and jurisdictions. The objective of this review was to assess whether articles reporting decision-analytic models in the area of osteoporosis provided enough information to enable decision makers in different countries/jurisdictions to fully appreciate the variability of results according to location and be able to apply the evaluation to their own setting. Of the 18 articles included in the review, only three explicitly stated the decision-making audience. It was not possible to infer a decision-making audience in eight studies. The target population was well reported, as were resource and cost data, and clinical data used for estimates of relative risk reduction. However, baseline risk was rarely adapted to the relevant jurisdiction, and when no decision maker was explicit it was difficult to assess whether the reported cost and resource use data were in fact relevant. A few studies used sensitivity analysis to explore elements of generalisability, such as compliance rates and baseline fracture risk rates, although such analyses were generally restricted to evaluating parameter uncertainty. This review found that variability in cost effectiveness across locations is addressed to a varying extent in modelling studies in the field of osteoporosis, limiting their use for decision makers across different locations. Transparency of reporting is expected to increase as methodology develops and decision makers publish 'reference case' type guidance.

Economic evaluation of rivastigmine in patients with Parkinson's disease dementia

BACKGROUND

The positive results of a randomised clinical trial of rivastigmine in patients with dementia associated with Parkinson's disease have been published recently. Patient-level healthcare utilisation data were also collected, and this report is the economic evaluation based on these data.

OBJECTIVE

To determine the cost effectiveness of rivastigmine 3-12 mg/day in patients in whom mild to moderate dementia developed at least 2 years after they received a clinical diagnosis of Parkinson's disease.

METHODS

A cost-effectiveness analysis was performed by applying Canadian and UK cost weights (year 2004 values) to healthcare utilisation data collected prospectively during a randomised, double-blind, multinational, 24-week trial of rivastigmine 3-12 mg/day (n = 362) versus placebo (n = 179). Patients were > or =50 years of age, had a Mini-Mental State Examination (MMSE) score of between 20 and 24 and had contact with a responsible caregiver at least 3 days a week.Quality-adjusted survival time, transformed from MMSE scores, was the measure of effectiveness. Caregiver costs included paid and unpaid time, and direct costs included concomitant medications, outpatient care, hospitalisations, long-term care and study medications. Analysis was conducted from a societal perspective with a time horizon of 24 weeks.

RESULTS

Consistent with the improvement in clinical outcomes, there was an observed increase in quality-adjusted survival time in the rivastigmine arm of 2.81 quality-adjusted life-days (two-sided p-value 0.13 [90% CI -0.243, 5.86]). Using Canadian price weights, there was an observed increase in cost in the rivastigmine arm of Can 55.76 dollars(two-sided p-value 0.98 [90% CI -3431, 3543]), with a resulting incremental cost-effectiveness ratio of Can 7429 dollars per QALY. Using UK price weights, there was an observed decrease in cost in the rivastigmine arm of pound 26.18 (two-sided p-value 0.99 [90% CI -2407, 2355]).

CONCLUSION

Although no between-treatment differences in cost were seen, the small sample size, highly variable cost distributions and short time horizon prevent us from making strong conclusions with regard to the effect of rivastigmine on total costs and, by inference, on cost effectiveness.

'Lost in translation': accounting for between-country differences in the analysis of multinational cost-effectiveness data

Cost-effectiveness analysis has gained status over the last 15 years as an important tool for assisting resource allocation decisions in a budget-limited environment such as healthcare. Randomised (multicentre) multinational controlled trials are often the main vehicle for collecting primary patient-level information on resource use, cost and clinical effectiveness associated with alternative treatment strategies. However, trial-wide cost effectiveness results may not be directly applicable to any one of the countries that participate in a multinational trial, requiring some form of additional modelling to customise the results to the country of interest. This article proposes an algorithm to assist with the choice of the appropriate analytical strategy when facing the task of adapting the study results from one country to another. The algorithm considers different scenarios characterised by: (a) whether the country of interest participated in the trial; and (b) whether individual patient-level data (IPD) from the trial are available. The analytical options available range from the use of regression-based techniques to the application of decision-analytic models. Decision models are typically used when the evidence base is available exclusively in summary format whereas regression-based methods are used mainly when the country of interest actively recruited patients into the trial and there is access to IPD (or at least country-specific summary data). Whichever method is used to reflect between-country variability in cost-effectiveness data, it is important to be transparent regarding the assumptions made in the analysis and (where possible) assess their impact on the study results.

Analysis sans frontières: can we ever make economic evaluations generalisable across jurisdictions?

Over the last decade or so, a number of healthcare systems have used economic evaluations as a formal input into decisions about the coverage or reimbursement of new healthcare interventions. This change in the policy landscape has placed some important demands on the design and characteristics of economic evaluation and these are increasingly evident in studies being presented to decision makers. One challenge has been to make studies specific to the context in which the decision is being taken. This is because of the inevitable geographical variation in many of the parameters within an analysis. There has been a series of important contributions to the published literature in recent years on how to quantify geographical heterogeneity within economic analyses based on randomised controlled trials. However, there are good reasons for economic evaluation for decision making to be undertaken using methods of evidence synthesis and decision analytical modelling, but issues of geographical variation still need to be handled appropriately. The key requirements of economic evaluations for decision making within healthcare systems can be defined as follows: (i) a design that meets the objectives and constraints of the healthcare system; (ii) coherent and complete specification of the decision problem; (iii) inclusion of all relevant evidence; and (iv) recognition and appropriate handling of uncertainty. In satisfying these requirements, it is important to be aware of variation between jurisdictions, and this imposes some important analytical requirements on economic studies. While many agencies have produced guidelines on preferred methods for healthcare economic evaluation, these exhibit considerable variation. Some of this variation can be justified by genuine differences between systems in clinical practice, objectives and constraints, while some of the variation relates to differences of opinion about appropriate analysis given methodological uncertainty. However, some of the variation in guidance is difficult to justify and is inconsistent with the aims and objectives of the systems the analyses are seeking to inform. Decision makers and analysts need to work together to streamline and where possible harmonise guidelines on methods for economic evaluations, whilst recognising legitimate variation in the needs of different healthcare systems. Otherwise, there is the risk that scarce resources will be wasted in producing country-specific analyses in situations where these are not justified. Expected value of information analyses are also emerging as a tool that could be considered by decision makers to guide their policy on the acceptance or non-acceptance of data from other jurisdictions.

Conducting economic evaluations alongside multinational clinical trials: toward a research consensus

Demand for economic evaluations in multinational clinical trials is increasing, but there is little consensus about how such studies should be conducted and reported. At a workshop in Durham, North Carolina, we sought to identify areas of agreement about how the primary findings of economic evaluations in multinational clinical trials should be generated and presented. In this paper, we propose a framework for classifying multinational economic evaluations according to (a) the sources of an analyst's estimates of resource use and clinical effectiveness and (b) the analyst's method of estimating costs. We review existing studies in the cardiology literature in the context of the proposed framework. We then describe important methodological and practical considerations in conducting multinational economic evaluations and summarize the advantages and disadvantages of each approach. Finally, we describe opportunities for future research. Delineation of the various approaches to multinational economic evaluation may assist researchers, peer reviewers, journal editors, and decision makers in evaluating the strengths and limitations of particular studies.