Modelling decay in effectiveness for evaluation of behaviour change interventions: a tutorial for public health economists

BACKGROUND AND PURPOSE

Recent methodological reviews of evaluations of behaviour change interventions in public health have highlighted that the decay in effectiveness over time has been mostly overlooked, potentially leading to suboptimal decision-making. While, in principle, discrete-time Markov chains-the most commonly used modelling approach-can be adapted to account for decay in effectiveness, this framework inherently lends itself to strong model simplifications. The application of formal and more appropriate modelling approaches has been supported, but limited progress has been made to date. The purpose of this paper is to encourage this shift by offering a practical guide on how to model decay in effectiveness using a continuous-time Markov chain (CTMC)-based approach.

METHODS

A CTMC approach is demonstrated, with a contextualized tutorial being presented to facilitate learning and uptake. A worked example based on the stylized case study in physical activity promotion is illustrated with accompanying R code.

DISCUSSION

The proposed framework presents a relatively small incremental change from the current modelling practice. CTMC represents a technical solution which, in absence of relevant data, allows for formally testing the sensitivity of results to assumptions regarding the long-term sustainability of intervention effects and improving model transparency.

CONCLUSIONS

The use of CTMC should be considered in evaluations where decay in effectiveness is likely to be a key factor to consider. This would enable more robust model-based evaluations of population-level programmes to promote behaviour change and reduce the uncertainty surrounding the decision to invest in these public health interventions.

A Scoping Review Exploring Whether a Free “Offer” Devalues or Widens Sport and Physical Activity Participation Amongst Children and Young Adults Aged 0–25?

Background

Socio-economic status continues to mediate physical activity engagement, despite a range of interventions aimed at reducing inequalities and widening sport and physical activity participation. As a result there has been increasing interest amongst policy makers, national governing bodies (NGB), county sports partnerships (CPS) and the sport and physical activity sector more broadly, in understanding how best to reduce inequalities and widen participation. The “price point” of offers and whether a “free offer” enables or devalues participation, has been a key area of interest. This scoping review aimed to explore this topic further by investigating whether “a free “offer” devalues or widens sport and physical activity participation amongst children and young adults aged 0-25?”.

Methods

This scoping review searched three electronic bibliographic databases (MEDLINE, PsycINFO, SPORTDiscus) using a structured search strategy to identify articles published between 2017 and January 2022. Studies were included using the PICO criteria of; Population: children and young adults aged 0-25; Intervention: free “offer” relating to physical activity; Context: areas of deprivation in the UK; Outcome: engagement, involvement, participation in sport and physical activity.

Results and Discussion

Five studies were eligible after screening 1301 titles and reviewing 14 full-text studies. Features reported included intervention design, outcomes, potential challenges and wider implications / future recommendations. Specifically, a narrative synthesis of the key themes of participation deprivation and cost effectiveness were outlined in more detail. A subsidized cost or free offer can improve participation generally and in attracting those from lower socio-economic backgrounds. However, the impact of such initiatives decrease with increasing deprivation highlighting that groups with the highest levels of deprivation have wider complexities affecting their participation. Competing priorities and potentially unrealistic expectations at stakeholders level was also identified.

Conclusion

Despite the paucity of current research exploring the impact of a “free offer” in children and young adults, recommendations for future research, practice and policy included the need for longitudinal, more holistic and participant centric approaches. Further research is required to explore the impact of a “free offer” from an individual, societal and policy-level perspective, in widening and increasing participation in sport and physical activity.

Copula Models for Addressing Sample Selection in the Evaluation of Public Health Programmes: An Application to the Leeds Let's Get Active Study

Sample selectivity is a recurrent problem in public health programmes and poses serious challenges to their evaluation. Traditional approaches to handle sample selection tend to rely on restrictive assumptions. The aim of this paper is to illustrate a copula-based selection model to handle sample selection in the evaluation of public health programmes. Motivated by a public health programme to promote physical activity in Leeds (England), we describe the assumptions underlying the copula selection, and its relative advantages compared with commonly used approaches to handle sample selection, such as inverse probability weighting and Heckman's selection model. We illustrate the methods in the Leeds Let's Get Active programme and show the implications of method choice for estimating the effect on individual's physical activity. The programme was associated with increased physical activity overall, but the magnitude of its effect differed according to adjustment method. The copula selection model led to a similar effect to the Heckman's approach but with relatively narrower 95% confidence intervals. These results remained relatively similar when different model specifications and alternative distributional assumptions were considered. The copula selection model can address important limitations of traditional approaches to address sample selection, such as the Heckman model, and should be considered in the evaluation of public health programmes, where sample selection is likely to be present.

Taking a local government perspective for economic evaluation of a population-level programme to promote exercise

BACKGROUND

In order to tackle the issue of physical inactivity, local governments have implemented population-level programmes to promote exercise. While evidence is accumulating on the cost-effectiveness of these interventions, studies have typically adopted a health sector perspective for economic evaluation. This approach has been challenged as it does not allow for key concerns by local governments, which are primary stakeholders, to be addressed.

OBJECTIVES

To show how taking a local government perspective for economic evaluation can be implemented in practice and this may affect the economic conclusions.

METHODS

Based on data from a case study, the health equity impact of the intervention and its opportunity cost from a service provider viewpoint were assessed. The cost-effectiveness implications of a change in perspective were subsequently estimated by means of scenario analysis.

FINDINGS

The intervention was found to provide adult residents living in the most deprived city areas with greater health benefits compared with the rest of the population. However, a negative net equity impact was found in the short-term. The opportunity cost of the intervention was estimated to be substantially lower than its financial cost (£2.77 per person/year), with significant implications for decision-making.

CONCLUSIONS

Taking a local government perspective can affect the conclusions drawn from the economic evaluation of population-level programmes to promote exercise, and therefore influence decision making.