Event simulation and external validation applied in published health economic models for obesity: a systematic review

Does the Structure Matter? An External Validation and Health Economic Results Comparison of Event Simulation Approaches in Severe Obesity

OBJECTIVES

As obesity-associated events impact long-term survival, health economic (HE) modelling is commonly applied, but modelling approaches are diverse. This research aimed to compare the events simulation and the HE outcomes produced by different obesity modelling approaches.

METHODS

An external validation, using the Swedish obesity subjects (SOS) study, of three main structural event modelling approaches was performed: (1) continuous body mass index (BMI) approach; (2) risk equation approach; and (3) categorical BMI-related approach. Outcomes evaluated were mortality, cardiovascular events, and type 2 diabetes (T2D) for both the surgery and the control arms. Concordance between modelling results and the SOS study were investigated by different state-of-the-art measurements, and categorized by the grade of deviation observed (grades 1-4 expressing mild, moderate, severe, and very severe deviations). Furthermore, the costs per quality-adjusted life-year (QALY) gained of surgery versus controls were compared.

RESULTS

Overall and by study arm, the risk equation approach presented the lowest average grade of deviation (overall grade 2.50; control arm 2.25; surgery arm 2.75), followed by the continuous BMI approach (overall 3.25; control 3.50; surgery 3.00) and by the categorial BMI approach (overall 3.63; control 3.50; surgery 3.75). Considering different confidence interval limits, the costs per QALY gained were fairly comparable between all structural approaches (ranging from £2,055 to £6,206 simulating a lifetime horizon).

CONCLUSION

None of the structural approaches provided perfect external event validation, although the risk equation approach showed the lowest overall deviations. The economic outcomes resulting from the three approaches were fairly comparable.

Replication of Published Health Economic Obesity Models: Assessment of Facilitators, Hurdles and Reproduction Success

OBJECTIVES

This research aims to (1) replicate published health economic models, (2) compare reproduced results with original results, (3) identify facilitators and hurdles to model replicability and determine reproduction success, and (4) suggest model replication reporting standards to enhance model reproducibility, in the context of health economic obesity models.

METHODS

Four health economic obesity models simulating an adult UK population were identified, selected for replication, and evaluated using the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) checklist. Reproduction results were compared to original results, focusing on cost-effectiveness outcomes, and the resulting reproduction success was assessed by published criteria. Replication facilitators and hurdles were identified and transferred into related reporting standards.

RESULTS

All four case studies were state-transition models simulating costs and quality-adjusted life-years (QALYs). Comparing original versus reproduction outcomes, the following deviation ranges were observed: costs - 3.9 to 16.1% (mean over all model simulations 3.78%), QALYs - 3.7 to 2.1% (mean - 0.11%), and average cost-utility ratios - 3.0 to 17.9% (mean 4.28%). Applying different published criteria, an overall reproduction success was observed for three of four models. Key replication facilitators were input data tables and model diagrams, while missing standard deviations and missing formulas for equations were considered as key hurdles.

CONCLUSIONS

This study confirms the feasibility of rebuilding health economic obesity models, but minor to major assumptions were needed to fill reporting gaps. Model replications can help to assess the quality of health economic model documentation and can be used to validate current model reporting practices. Simple changes to actual CHEERS reporting criteria may solve identified replication hurdles.

External Validation of the Core Obesity Model to Assess the Cost-Effectiveness of Weight Management Interventions

BACKGROUND

For economic models to be considered fit for purpose, it is vital that their outputs can be interpreted with confidence by clinicians, budget holders and other stakeholders. Consequently, thorough validation of models should be carried out to enhance confidence in their predictions. Here, we present results of external dependent and independent validations of the Core Obesity Model (COM), which was developed to assess the cost-effectiveness of weight management interventions.

OBJECTIVE

The aim was to assess the external validity of the COM (version 6.1), in line with best practice guidance from the International Society for Pharmacoeconomics and Outcomes Research and the Society for Medical Decision Making.

METHODS

For validation, suitable sources and outcomes were identified, and used to populate the COM with relevant inputs to allow prediction of study outcomes. Study characteristics were entered into the COM to replicate either the studies used to develop the model (dependent validation) or those not included in the model (independent validation). The concordance between predicted and observed outcomes was then assessed using established statistical methods and generation of mean error estimates.

RESULTS

For most outcomes, the predictions of the COM showed good linear correlation with observed outcomes, as evidenced by the high coefficients of determination (R² values). The independent validation revealed a degree of underestimation in predictions of cardiovascular (CV) disease and mortality, and type 2 diabetes.

CONCLUSION

The predictions generated by the risk equations used in the COM showed good concordance both with the studies used to develop the model and with studies not included in the model. In particular, the concordance observed in the external dependent validation suggests that the COM accurately predicts obesity-related event rates observed in the studies used to develop the model. However, the impact of existing CV risk, as well as mortality, is a key area for future refinement of the COM. Our results should increase confidence in the estimates derived from the COM and reduce uncertainty associated with analyses using this model.

Lifestyle information and access to a commercial weight management group to promote maternal postnatal weight management and positive lifestyle behaviour: the SWAN feasibility RCT

Background

Increasing numbers of UK women have overweight or obese body mass index scores when they become pregnant, or gain excessive weight in pregnancy, increasing their risk of adverse outcomes. Failure to manage postnatal weight is linked to smoking, non-healthy dietary choices, lack of regular exercise and poorer longer-term health. Women living in areas of higher social deprivation are more likely to experience weight management problems postnatally.

Objectives

The objectives were to assess the feasibility of conducting a definitive randomised controlled trial to determine the clinical effectiveness and cost-effectiveness of lifestyle information and access to a commercial weight management group focusing on self-monitoring, goal-setting and motivation to achieve dietary change commencing 8–16 weeks postnatally to achieve and maintain weight management and positive lifestyle behaviour.

Design

The design was a randomised two-arm feasibility trial with a nested mixed-methods process evaluation.

Setting

The setting was a single centre in an inner city setting in the south of England.

Participants

Participants were women with body mass index scores of > 25 kg/m2 at antenatal ‘booking’ and women with normal body mass index scores (18.0–24.9 kg/m2) at antenatal booking who developed excessive gestational weight gain as assessed at 36 weeks’ gestation.

Main outcome measures

Recruitment, retention, acceptability of trial processes and identification of relevant economic data were the feasibility objectives. The proposed primary outcome was difference between groups in weight at 12 months postnatally, expressed as percentage weight change and weight loss from antenatal booking. Other proposed outcomes included assessment of diet, physical activity, smoking, alcohol consumption, body image, maternal esteem, mental health, infant feeding and NHS costs.

Results

Most objectives were achieved. A total of 193 women were recruited, 98 allocated to the intervention arm and 95 to the control arm. High follow-up rates (> 80%) were achieved to 12 months. There was an 8.8% difference in weight loss at 12 months between women allocated to the intervention arm and women allocated to the control arm (13.0% vs. 4.2%, respectively; p = 0.062); 47% of women in the intervention arm attended at least one weight management session, with low risk of contamination between arms. The greatest benefit was among women who attended ≥ 10 sessions. Barriers to attending sessions included capability, opportunity and motivation issues. Data collection tools were appropriate to support economic evaluation in a definitive trial, and economic modelling is feasible to quantify resource impacts and outcomes not directly measurable within a trial.

Limitations

The trial recruited from only one site. It was not possible to recruit women with normal body mass index scores who developed excessive pregnancy weight gain.

Conclusions

It was feasible to recruit and retain women with overweight or obese body mass index scores at antenatal booking to a trial comparing postnatal weight management plus standard care with standard care only and collect relevant data to assess outcomes. Approaches to recruit women with normal body mass index scores who gain excessive gestational weight need to be considered. Commercial weight management groups could support women’s weight management as assessed at 12 months postnatally, with probable greater benefit from attending ≥ 10 sessions. Process evaluation findings highlighted the importance of providing more information about the intervention on trial allocation, extended duration of time to commence sessions following birth and extended number of sessions offered to enhance uptake and retention. Results support the conduct of a future randomised controlled trial.

Trial registration

Current Controlled Trials ISRCTN39186148.

Funding

This project was funded by the National Institute for Health Research (NIHR) Public Health Research programme and will be published in full in Public Health Research; Vol. 8, No. 9. See the NIHR Journals Library website for further project information.