Economic evaluation of smoking-cessation therapies: a critical and systematic review of simulation models

Should Nicotine Replacement Therapy be Provided Free of Charge? A Cost-Utility Analysis in Sweden

INTRODUCTION

Treatment with nicotine replacement therapy (NRT) during an attempt to quit smoking increases the likelihood of success by about 55%. However, out-of-pocket payment for NRT can hinder its use.

AIMS AND METHODS

This study aims therefore to assess the cost-effectiveness of subsidizing NRT in Sweden. A homogeneous cohort-based Markov model was used to assess the lifetime costs and effects of subsidized NRT from a payer and societal perspective. Data to populate the model were retrieved from the literature, and selected parameters were varied in deterministic and probabilistic sensitivity analyses to assess robustness of model outputs. Costs are presented in USD, year 2021.

RESULTS

A 12-week treatment with NRT was estimated to cost USD 632 (474-790) per person. From a societal perspective, subsidized NRT was a cost-saving alternative in 98.5% of the simulations. NRT is cost-saving across all ages, but the health and economic gains are somewhat larger among younger smokers from a societal perspective. When a payer perspective was used, the incremental cost-effectiveness ratio was estimated at 14 480 (11 721-18 515) USD per QALY which was cost-effective at a willingness to pay of 50 000 USD per QALY in 100 % of the simulations. Results were robust with realistic changes in the inputs during scenario and sensitivity analyses.

CONCLUSIONS

Subsidizing NRT is potentially a cost-saving smoking cessation strategy from a societal perspective and cost-effective from a payer perspective.

IMPLICATIONS

This study found that subsidizing NRT is potentially a cost-saving smoking cessation policy alternative compared to current practice from a societal perspective. From a healthcare payer perspective, subsidizing NRT is estimated to cost USD 14 480 to gain an extra QALY. NRT is cost-saving across all ages, but the health and economic gains are somewhat larger among younger smokers from a societal perspective. Moreover, subsidizing NRT removes the financial barriers that are mostly faced by socioeconomically disadvantaged smokers which might reduce health inequalities. Thus, future economic evaluations should further investigate the health inequality impacts with methods that are more suitable for this.

Economic evaluations of tobacco control interventions in low- and middle-income countries: a systematic review

BACKGROUND AND AIMS

Tobacco consumption and its associated adverse outcomes remain major public health issues, particularly in low- and middle-income countries. This systematic review aimed to identify and critically assess full economic evaluations for tobacco control interventions in low- and middle-income countries.

METHODS

Electronic databases, including EMBASE, MEDLINE and PsycINFO and the grey literature, were searched using terms such as 'tobacco', 'economic evaluation' and 'smoking' from 1994 to 2020. Study quality was assessed using the Consensus Health Economic Criteria and the Philips checklist. Studies were included which were full economic evaluations of tobacco control interventions in low- and middle-income settings. Reviews, commentaries, conference proceedings and abstracts were excluded. Study selection and quality assessment were conducted by two reviewers independently. A narrative synthesis was conducted to synthesize the findings of the studies.

RESULTS

This review identified 20 studies for inclusion. The studies evaluated a wide range of interventions, including tax increase, nicotine replacement therapy (nicotine patch/gum) and financial incentives. Overall, 12 interventions were reported to be cost-effective, especially tax increases for tobacco consumption and cessation counselling. There were considerable limitations regarding data sources (e.g. using cost data from other countries or assumptions due to the lack of local data) and the model structure; sensitivity analyses were inadequately described in many studies; and there were issues around the transferability of results to other settings. Additionally, the affordability of the interventions was only discussed in two studies.

CONCLUSIONS

There are few high-quality studies of the cost-effectiveness of tobacco use control interventions in low- and middle-income countries. The methodological limitations of the existing literatures could affect the generalizability of the findings.

Protocol for a systematic review of economic evaluations of preoperative smoking cessation interventions for preventing surgical complications

INTRODUCTION

The short-term economic benefit of embedding best practice tobacco dependence treatment (TDT) into healthcare services prior to surgery across different populations and jurisdictions is largely unknown. The aim of this systematic review is to summarise the cost-effectiveness of preoperative smoking cessation interventions for preventing surgical complications compared with usual care. The results will provide hospital managers, clinicians, healthcare professionals and policymakers with a critical summary of the economic evidence on providing TDT routinely before surgery, aiding the development and dissemination of unified, best practice guidelines, that is, implementation of article 14 of the WHO Framework Convention on Tobacco Control.

METHODS AND ANALYSIS

A comprehensive search of peer-reviewed literature will be conducted from database inception until 23 June 2021 (Cochrane, Econlit, Embase, Health Technology Assessment, Medline Complete, Scopus). Published, English-language articles describing economic evaluations of preoperative smoking cessation interventions for preventing surgical complications will be included. One researcher will complete the searches and two researchers will independently screen results for eligible studies. Any disagreement will be resolved by the third researcher. A narrative summary of included studies will be provided. Study characteristics, economic evaluation methods and cost-effectiveness results will be extracted by one reviewer and descriptive analyses will be undertaken. A second reviewer will review data extracted for accuracy from 10% of the included studies. Reporting and methodological quality of the included studies will be evaluated independently by two reviewers using the Consolidated Health Economic Evaluation Reporting Standards statement and the Quality of Health Economic Studies Instrument checklist, respectively.

ETHICS AND DISSEMINATION

This research does not require ethics approval because the study is a planned systematic review of published literature. Findings will be presented at health economic, public health and tobacco control conferences, published in a peer-reviewed journal and disseminated via social media.

TRIAL REGISTRATION NUMBER

CRD42021257740.

Long-Term Cost-Effectiveness of Smoking Cessation Interventions in People With Mental Disorders: A Dynamic Decision Analytical Model

OBJECTIVES

People with mental disorders are more likely to smoke than the general population. The objective of this study is to develop a decision analytical model that estimates long-term cost-effectiveness of smoking cessation interventions in this population.

METHODS

A series of Markov models were constructed to estimate average lifetime smoking-attributable inpatient cost and expected quality-adjusted life-years. The model parameters were estimated using a variety of data sources. The model incorporated uncertainty through probabilistic sensitivity analysis using Monte Carlo simulations. It also generated tables presenting incremental cost-effectiveness ratios of the proposed interventions with varying incremental costs and incremental quit rates. We used data from 2 published trials to demonstrate the model's ability to make projections beyond the observational time frame.

RESULTS

The average smoker's smoking-attributable inpatient cost was 3 times higher and health utility was 5% lower than ex-smokers. The intervention in the trial with a statistically insignificant difference in quit rate (19% vs 25%; P=.2) showed a 45% to 49% chance of being cost-effective compared with the control at willingness-to-pay thresholds of £20 000 to £30 000/quality-adjusted life-years. The second trial had a significant outcome (quit rate 35.9% vs 15.6%; P<.001), and the corresponding probability of the intervention being cost-effective was 65%.

CONCLUSIONS

This model provides a consistent platform for clinical trials to estimate the potential lifetime cost-effectiveness of smoking cessation interventions for people with mental disorders and could help commissioners direct resources to the most cost-effective programs. However, direct comparisons of results between trials must be interpreted with caution owing to their different designs and settings.

Assessing the impact of smoking on the health and productivity of the working-age Indonesian population using modelling

OBJECTIVES

To estimate the impact of smoking in the working-age Indonesian population in terms of costs, years of life, quality-adjusted life years (QALYs) and productivity-adjusted life years (PALYs) lost.

METHODS

Life table modelling of Indonesian smokers aged 15-54 years, followed up until 55 years (retirement age). Contemporary data on demographics, all-cause mortality, population attributable fractions and prevalence of smoking were derived from the Institute for Health Metrics and Evaluation. The quality of life and reduction in productivity due to smoking were derived from published sources. The analysis was repeated but with the assumption that the cohorts were non-smokers. The differences in results represented the losses incurred due to smoking. Gross domestic product (GDP) per equivalent full-time worker (US$11 765) was used for estimation of the cost of each PALY, and an annual discount rate of 3.0% was applied to all costs and outcomes.

RESULTS

The prevalences of smoking among Indonesian working-age men and women were 67.2% and 2.16%, respectively. This study estimated that smoking caused 846 123 excess deaths, 2.9 million years of life lost (0.40%), 41.6 million QALYs lost (5.9%) and 15.6 million PALYs lost (2.3%). The total cost of productivity loss due to smoking amounted to US$183.7 billion among the working-age population followed up until retirement. Healthcare cost was predicted to be US$1.8 trillion. Over a 1-year time horizon, US$10.2 billion was lost in GDP and 117 billion was lost in healthcare costs.

CONCLUSION

Smoking imposes significant health and economic burden in Indonesia. The findings stress the importance of developing effective tobacco control strategies at the macro and micro levels, which would benefit the country both in terms of health and wealth.

Economic evaluations of public health interventions for physical activity and healthy diet: A systematic review

Physical inactivity and unhealthy dietary habits are associated with an increased disease and economic burden. The aim of this systematic review was to identify economic evaluations of public health interventions targeting physical activity and healthy diet, and assess the quality and transferability of the findings to the Swedish context. A search of published economic evaluations was conducted through electronic databases including PubMed, Web of Science, PsycINFO, National Health Service Economic Evaluation Databases (NHS EED) and the Health Technology Assessment Database (HTA). An additional search was done using references of relevant systematic reviews and websites of relevant organizations were checked to find grey literature. Quality and transferability of the economic evaluations were appraised using a quality assessment tool developed by the Swedish Agency for Health Technology Assessment. Thirty-two economic evaluations, rated as moderate or high quality, of 178 interventions were included; thirteen studies targeting physical activity, thirteen targeting healthy diet and six targeting both. The interventions varied in terms of their content, setting, mode of delivery and target populations. A majority of the economic evaluations reported that the interventions were likely to be cost-effective; however, considerable variations in the methodological and reporting qualities were observed. Only half of the economic evaluations were rated to have a high probability of transferring to the Swedish context. Public health interventions targeting physical activity and dietary habits have a high potential to be cost-effective. However, decision makers should consider the variation in quality and transferability of the available evidence.

Simulating future public health benefits of tobacco control interventions: a systematic review of models

BACKGROUND

To prioritise tobacco control interventions, simulating their health impacts is valuable. We undertook a systematic review of tobacco intervention simulation models to assess model structure and input variations that may render model outputs non-comparable.

METHODS

We applied a Medline search with keywords intersecting modelling and tobacco. Papers were limited to those modelling health outputs (eg, mortality, health-adjusted life years), and at least two of cancer, cardiovascular and respiratory diseases. Data were extracted for each simulation model with ≥3 arising papers, including: model type, untimed or with time steps and trends in business-as-usual (BAU) tobacco prevalence and epidemiology.

RESULTS

Of 1911 papers, 186 met the inclusion criteria, including 13 eligible simulation models. The SimSmoke model had the largest number of publications (n=46), followed by Benefits of Smoking Cessation on Outcomes (n=12) and Tobacco Policy Model (n=10). Two of 13 models only estimated deaths averted, 1 had no time steps, 5 had no future trends in BAU tobacco prevalence, 9 had no future trends in BAU disease epidemiology and 7 had no time lags from quitting tobacco to reversal of health harm.

CONCLUSIONS

Considerable heterogeneity exists in simulation models, making outputs substantively non-comparable between models. Ranking of interventions by one model may be valid. However, this may not be true if, for example, interventions that differentially affect age groups (eg, a tobacco-free generation policy vs increased cessation among adults) do not account for plausible future trends. Greater standardisation of model structures and outputs will allow comparison across models and countries, and for comparisons of the impact of tobacco control interventions with other preventive interventions.

Cost-effectiveness of a high-intensity versus a low-intensity smoking cessation intervention in a dental setting: long-term follow-up

OBJECTIVES

The aim of this study was to conduct a cost-effectiveness analysis (CEA) of a high-intensity and a low-intensity smoking cessation treatment programme (HIT and LIT) using long-term follow-up effectiveness data and to validate the cost-effectiveness results based on short-term follow-up.

DESIGN AND OUTCOME MEASURES

Intervention effectiveness was estimated in a randomised controlled trial as numbers of abstinent participants after 1 and 5-8 years of follow-up. The economic evaluation was performed from a societal perspective using a Markov model by estimating future disease-related costs (in Euro (€) 2018) and health effects (in quality-adjusted life-years (QALYs)). Programmes were explicitly compared in an incremental analysis, and the results were presented as an incremental cost-effectiveness ratio.

SETTING

The study was conducted in dental clinics in Sweden.

PARTICIPANTS

294 smokers aged 19-71 years were included in the study.

INTERVENTIONS

Behaviour therapy, coaching and pharmacological advice (HIT) was compared with one counselling session introducing a conventional self-help programme (LIT).

RESULTS

The more costly HIT led to higher number of 6-month continuous abstinent participants after 1 year and higher number of sustained abstinent participants after 5-8 years, which translates into larger societal costs avoided and health gains than LIT. The incremental cost/QALY of HIT compared with LIT amounted to €918 and €3786 using short-term and long-term effectiveness, respectively, which is considered very cost-effective in Sweden.

CONCLUSION

CEA favours the more costly HIT if decision makers are willing to spend at least €4000/QALY for tobacco cessation treatment.

Smoking Cessation: A Comparison of Two Model Structures

BACKGROUND

Most economic evaluations of smoking cessation interventions have used cohort state-transition models. Discrete event simulations (DESs) have been proposed as a superior approach.

OBJECTIVE

We developed a state-transition model and a DES using the discretely integrated condition event (DICE) framework and compared the cost-effectiveness results. We performed scenario analysis using the DES to explore the impact of alternative assumptions.

METHODS

The models estimated the costs and quality-adjusted life years (QALYs) for the intervention and comparator from the perspective of the UK National Health Service and Personal Social Services over a lifetime horizon. The models considered five comorbidities: chronic obstructive pulmonary disease, myocardial infarction, coronary heart disease, stroke and lung cancer. The state-transition model used prevalence data, and the DES used incidence. The costs and utility inputs were the same between two models and consistent with those used in previous analyses for the National Institute for Health and Care Excellence.

RESULTS

In the state-transition model, the intervention produced an additional 0.16 QALYs at a cost of £540, leading to an incremental cost-effectiveness ratio (ICER) of £3438. The comparable DES scenario produced an ICER of £5577. The ICER for the DES increased to £18,354 when long-term relapse was included.

CONCLUSIONS

The model structures themselves did not influence smoking cessation cost-effectiveness results, but long-term assumptions did. When there is variation in long-term predictions between interventions, economic models need a structure that can reflect this.

Economic evaluation of a brief counselling for smoking cessation in dentistry: a case study comparing two health economic models

OBJECTIVES

This study aimed to compare the cost-effectiveness estimates of a brief counselling of smoking cessation in dentistry by using two different health economic models.

DESIGN AND OUTCOME MEASURES

Intervention effectiveness was estimated in a cluster randomised controlled trial. The number of quitters was estimated based on 7-day abstinence and on smoking reduction at follow-up. Health economic evaluation was performed using two models: (1) a population-based model employing potential impact fractions and (2) a Markov model estimating the cost-effectiveness of the intervention for the actual participants. The evaluation was performed from healthcare and societal perspectives, and health gains were expressed in quality-adjusted life-years (QALYs).

SETTING

Dental clinics in Sweden.

PARTICIPANTS

205 Swedish smokers aged 20-75 years.

INTERVENTIONS

A brief, structured behavioural intervention was compared with 'usual care'.

RESULTS

The cost per quitter was US$552 in the intervention and US$522 in the 'usual care' condition. The net saving estimated with the population-based model was US$17.3 million for intervention and US$49.9 million for 'usual care', with health gains of 1428 QALYs and 2369 QALYs, respectively, for the whole Swedish population during 10 years. The intervention was thus dominated by 'usual care'. The reverse was true when using the Markov model, showing net societal savings of US$71 000 for the intervention and US$57000 for 'usual care', with gains of 5.42 QALYs and 4.74 QALYs, respectively, for lifelong quitters.

CONCLUSION

The comparison of intervention and 'usual care' derived from small-scale studies may be highly sensitive to the choice of the model used to calculate cost-effectiveness.

TRIAL REGISTRATION

The cluster randomised trial is registered in the ISRCTN register of controlled trials with identification number ISRCTN50627997.

Model-based economic evaluations in smoking cessation and their transferability to new contexts: a systematic review

AIMS

To identify different types of models used in economic evaluations of smoking cessation, analyse the quality of the included models examining their attributes and ascertain their transferability to a new context.

METHODS

A systematic review of the literature on the economic evaluation of smoking cessation interventions published between 1996 and April 2015, identified via Medline, EMBASE, National Health Service (NHS) Economic Evaluation Database (NHS EED), Health Technology Assessment (HTA). The checklist-based quality of the included studies and transferability scores was based on the European Network of Health Economic Evaluation Databases (EURONHEED) criteria. Studies that were not in smoking cessation, not original research, not a model-based economic evaluation, that did not consider adult population and not from a high-income country were excluded.

FINDINGS

Among the 64 economic evaluations included in the review, the state-transition Markov model was the most frequently used method (n = 30/64), with quality adjusted life years (QALY) being the most frequently used outcome measure in a life-time horizon. A small number of the included studies (13 of 64) were eligible for EURONHEED transferability checklist. The overall transferability scores ranged from 0.50 to 0.97, with an average score of 0.75. The average score per section was 0.69 (range = 0.35-0.92). The relative transferability of the studies could not be established due to a limitation present in the EURONHEED method.

CONCLUSION

All existing economic evaluations in smoking cessation lack in one or more key study attributes necessary to be fully transferable to a new context.

Validation and calibration of structural models that combine information from multiple sources

INTRODUCTION

Mathematical models that attempt to capture structural relationships between their components and combine information from multiple sources are increasingly used in medicine. Areas covered: We provide an overview of methods for model validation and calibration and survey studies comparing alternative approaches. Expert commentary: Model validation entails a confrontation of models with data, background knowledge, and other models, and can inform judgments about model credibility. Calibration involves selecting parameter values to improve the agreement of model outputs with data. When the goal of modeling is quantitative inference on the effects of interventions or forecasting, calibration can be viewed as estimation. This view clarifies issues related to parameter identifiability and facilitates formal model validation and the examination of consistency among different sources of information. In contrast, when the goal of modeling is the generation of qualitative insights about the modeled phenomenon, calibration is a rather informal process for selecting inputs that result in model behavior that roughly reproduces select aspects of the modeled phenomenon and cannot be equated to an estimation procedure. Current empirical research on validation and calibration methods consists primarily of methodological appraisals or case-studies of alternative techniques and cannot address the numerous complex and multifaceted methodological decisions that modelers must make. Further research is needed on different approaches for developing and validating complex models that combine evidence from multiple sources.

Gender difference and effect of pharmacotherapy: findings from a smoking cessation service

BACKGROUND

Smoking cessation services are available in England to provide assistance to those wishing to quit smoking. Data from one such service were analysed in order to investigate differences in quit rate between males and females prescribed with different treatments.

METHODS

A logistic regression model was fitted to the data using the binary response of self-reported quit (failed attempt = 0, successful attempt = 1), validated by Carbon Monoxide (CO) monitoring, 4 weeks after commencing programme. Main effects fitted were: client gender; age; region; the type of advisory sessions; and pharmacotherapy, Nicotine Replacement Therapy (NRT) or Varenicline. A second model was fitted including all main effects plus two-way interactions except region. These models were repeated using 12-week self-reported quit as the outcome.

RESULTS

At 4 weeks, all main effects were statistically significant, with males more likely (odds ratio and 95 % CI, females v males = 0.88 [0.79-0.97]), older smokers more likely (adjusted odds ratios [OR] and 95 % confidence interval [CI] respectively for groups 20-29, 30-49, 50-69 and 70+ vs 12-19 age group: 1.79 [1.39-2.31], 2.12 [1.68-2.68], 2.30 [1.80-2.92] and 2.47 [1.81-3.37] and for overall difference between groups, χ2(4) = 53.5, p < 0.001) and clients being treated with Varenicline more likely to have successfully quit than those on NRT (adjusted OR and 95 % CI for Varenicline vs NRT = 1.41 [1.21-1.64]). Statistically significant interactions were observed between (i) gender and type of counselling, and (ii) age and type of counselling. Similar results were seen in relation to main effects at 12 weeks except that type of counselling was non-significant. The only significant interaction at this stage was between gender and pharmacotherapy (adjusted OR and 95 % CI for females using Varenicline versus all other groups = 1.43 [1.06-1.94]).

CONCLUSION

Gender and treatment options were identified as predictors of abstinence at both 4 and 12 weeks after quitting smoking. Furthermore, interactions were observed between gender and (i) type of counselling received (ii) pharmacotherapy. In particular, the quit rate in women at 12 weeks was significantly improved in conjunction with Varenicline use. These findings have implications for service delivery.

Unremarked or Unperformed? Systematic Review on Reporting of Validation Efforts of Health Economic Decision Models in Seasonal Influenza and Early Breast Cancer

BACKGROUND

Transparent reporting of validation efforts of health economic models give stakeholders better insight into the credibility of model outcomes. In this study we reviewed recently published studies on seasonal influenza and early breast cancer in order to gain insight into the reporting of model validation efforts in the overall health economic literature.

METHODS

A literature search was performed in Pubmed and Embase to retrieve health economic modelling studies published between 2008 and 2014. Reporting on model validation was evaluated by checking for the word validation, and by using AdViSHE (Assessment of the Validation Status of Health Economic decision models), a tool containing a structured list of relevant items for validation. Additionally, we contacted corresponding authors to ask whether more validation efforts were performed other than those reported in the manuscripts.

RESULTS

A total of 53 studies on seasonal influenza and 41 studies on early breast cancer were included in our review. The word validation was used in 16 studies (30 %) on seasonal influenza and 23 studies (56 %) on early breast cancer; however, in a minority of studies, this referred to a model validation technique. Fifty-seven percent of seasonal influenza studies and 71 % of early breast cancer studies reported one or more validation techniques. Cross-validation of study outcomes was found most often. A limited number of studies reported on model validation efforts, although good examples were identified. Author comments indicated that more validation techniques were performed than those reported in the manuscripts.

CONCLUSIONS

Although validation is deemed important by many researchers, this is not reflected in the reporting habits of health economic modelling studies. Systematic reporting of validation efforts would be desirable to further enhance decision makers' confidence in health economic models and their outcomes.

Cost-effectiveness of a voucher-based intervention for smoking cessation

BACKGROUND

Contingency management (CM) has been shown to be effective in reducing smoking consumption, but has traditionally been criticized for its costs.

OBJECTIVES

This study assessed the cost-effectiveness of using a voucher-based CM protocol added to a cognitive behavioral treatment (CBT) for smoking cessation among treatment-seeking patients from the general population.

METHODS

A total of 92 patients were randomly assigned to CBT or CBT plus CM for abstinence. Incremental cost-effectiveness ratios (ICERs) were calculated by dividing the increase in costs by the increase in effects (continuous abstinence, longest duration of abstinence at 6-month follow-up and cotinine results during the treatment). An acceptability curve illustrated the statistical uncertainty surrounding the cost-effectiveness estimate. We also determined the optimum cost per participant for predicting the smoking status at 6-month follow-up.

RESULTS

The average cost per participant in the CBT condition was €138.73 (US$ 150.23) as opposed to €411.61 (US$ 445.73) in the CBT plus CM condition (p < 0.01). The incremental cost of using voucher-based CM to increase the number of participants that maintained abstinence at 6-month follow-up by one extra participant was €68.22 (US$ 73.88), and to lengthen the longest duration of abstinence by 1 week was €53.92 (US$ 58.39). The incremental cost to obtain an extra cotinine-negative result was €181.90 (US$ 196.98).

CONCLUSION

Compared with CBT alone, the voucher-based protocol required additional costs but achieved significantly better outcomes. These results will allow stakeholders to make policy decisions about CM implementation for smoking cessation in the broader community.

Smoking cessation treatment and outcomes patterns simulation: a new framework for evaluating the potential health and economic impact of smoking cessation interventions

BACKGROUND

Most existing models of smoking cessation treatments have considered a single quit attempt when modelling long-term outcomes.

OBJECTIVE

To develop a model to simulate smokers over their lifetimes accounting for multiple quit attempts and relapses which will allow for prediction of the long-term health and economic impact of smoking cessation strategies.

METHODS

A discrete event simulation (DES) that models individuals' life course of smoking behaviours, attempts to quit, and the cumulative impact on health and economic outcomes was developed. Each individual is assigned one of the available strategies used to support each quit attempt; the outcome of each attempt, time to relapses if abstinence is achieved, and time between quit attempts is tracked. Based on each individual's smoking or abstinence patterns, the risk of developing diseases associated with smoking (chronic obstructive pulmonary disease, lung cancer, myocardial infarction and stroke) is determined and the corresponding costs, changes to mortality, and quality of life assigned. Direct costs are assessed from the perspective of a comprehensive US healthcare payer ($US, 2012 values). Quit attempt strategies that can be evaluated in the current simulation include unassisted quit attempts, brief counselling, behavioural modification therapy, nicotine replacement therapy, bupropion, and varenicline, with the selection of strategies and time between quit attempts based on equations derived from survey data. Equations predicting the success of quit attempts as well as the short-term probability of relapse were derived from five varenicline clinical trials.

RESULTS

Concordance between the five trials and predictions from the simulation on abstinence at 12 months was high, indicating that the equations predicting success and relapse in the first year following a quit attempt were reliable. Predictions allowing for only a single quit attempt versus unrestricted attempts demonstrate important differences, with the single quit attempt simulation predicting 19 % more smoking-related diseases and 10 % higher costs associated with smoking-related diseases. Differences are most prominent in predictions of the time that individuals abstain from smoking: 13.2 years on average over a lifetime allowing for multiple quit attempts, versus only 1.2 years with single quit attempts. Differences in abstinence time estimates become substantial only 5 years into the simulation. In the multiple quit attempt simulations, younger individuals survived longer, yet had lower lifetime smoking-related disease and total costs, while the opposite was true for those with high levels of nicotine dependence.

CONCLUSION

By allowing for multiple quit attempts over the course of individuals' lives, the simulation can provide more reliable estimates on the health and economic impact of interventions designed to increase abstinence from smoking. Furthermore, the individual nature of the simulation allows for evaluation of outcomes in populations with different baseline profiles. DES provides a framework for comprehensive and appropriate predictions when applied to smoking cessation over smoker lifetimes.

Model performance evaluation (validation and calibration) in model-based studies of therapeutic interventions for cardiovascular diseases : a review and suggested reporting framework

Decision analytic models play an increasingly important role in the economic evaluation of health technologies. Given uncertainties around the assumptions used to develop such models, several guidelines have been published to identify and assess 'best practice' in the model development process, including general modelling approach (e.g., time horizon), model structure, input data and model performance evaluation. This paper focuses on model performance evaluation. In the absence of a sufficient level of detail around model performance evaluation, concerns regarding the accuracy of model outputs, and hence the credibility of such models, are frequently raised. Following presentation of its components, a review of the application and reporting of model performance evaluation is presented. Taking cardiovascular disease as an illustrative example, the review investigates the use of face validity, internal validity, external validity, and cross model validity. As a part of the performance evaluation process, model calibration is also discussed and its use in applied studies investigated. The review found that the application and reporting of model performance evaluation across 81 studies of treatment for cardiovascular disease was variable. Cross-model validation was reported in 55 % of the reviewed studies, though the level of detail provided varied considerably. We found that very few studies documented other types of validity, and only 6 % of the reviewed articles reported a calibration process. Considering the above findings, we propose a comprehensive model performance evaluation framework (checklist), informed by a review of best-practice guidelines. This framework provides a basis for more accurate and consistent documentation of model performance evaluation. This will improve the peer review process and the comparability of modelling studies. Recognising the fundamental role of decision analytic models in informing public funding decisions, the proposed framework should usefully inform guidelines for preparing submissions to reimbursement bodies.