Protocol for the economic evaluation of a complex intervention to improve the mental health of maltreated infants and children in foster care in the UK (The BeST? services trial)

A new framework for developing and evaluating complex interventions: update of Medical Research Council guidance

The UK Medical Research Council's widely used guidance for developing and evaluating complex interventions has been replaced by a new framework, commissioned jointly by the Medical Research Council and the National Institute for Health Research, which takes account of recent developments in theory and methods and the need to maximise the efficiency, use, and impact of research.

Framework for the development and evaluation of complex interventions: gap analysis, workshop and consultation-informed update

BACKGROUND

The Medical Research Council published the second edition of its framework in 2006 on developing and evaluating complex interventions. Since then, there have been considerable developments in the field of complex intervention research. The objective of this project was to update the framework in the light of these developments. The framework aims to help research teams prioritise research questions and design, and conduct research with an appropriate choice of methods, rather than to provide detailed guidance on the use of specific methods.

METHODS

There were four stages to the update: (1) gap analysis to identify developments in the methods and practice since the previous framework was published; (2) an expert workshop of 36 participants to discuss the topics identified in the gap analysis; (3) an open consultation process to seek comments on a first draft of the new framework; and (4) findings from the previous stages were used to redraft the framework, and final expert review was obtained. The process was overseen by a Scientific Advisory Group representing the range of relevant National Institute for Health Research and Medical Research Council research investments.

RESULTS

Key changes to the previous framework include (1) an updated definition of complex interventions, highlighting the dynamic relationship between the intervention and its context; (2) an emphasis on the use of diverse research perspectives: efficacy, effectiveness, theory-based and systems perspectives; (3) a focus on the usefulness of evidence as the basis for determining research perspective and questions; (4) an increased focus on interventions developed outside research teams, for example changes in policy or health services delivery; and (5) the identification of six 'core elements' that should guide all phases of complex intervention research: consider context; develop, refine and test programme theory; engage stakeholders; identify key uncertainties; refine the intervention; and economic considerations. We divide the research process into four phases: development, feasibility, evaluation and implementation. For each phase we provide a concise summary of recent developments, key points to address and signposts to further reading. We also present case studies to illustrate the points being made throughout.

LIMITATIONS

The framework aims to help research teams prioritise research questions and design and conduct research with an appropriate choice of methods, rather than to provide detailed guidance on the use of specific methods. In many of the areas of innovation that we highlight, such as the use of systems approaches, there are still only a few practical examples. We refer to more specific and detailed guidance where available and note where promising approaches require further development.

CONCLUSIONS

This new framework incorporates developments in complex intervention research published since the previous edition was written in 2006. As well as taking account of established practice and recent refinements, we draw attention to new approaches and place greater emphasis on economic considerations in complex intervention research. We have introduced a new emphasis on the importance of context and the value of understanding interventions as 'events in systems' that produce effects through interactions with features of the contexts in which they are implemented. The framework adopts a pluralist approach, encouraging researchers and research funders to adopt diverse research perspectives and to select research questions and methods pragmatically, with the aim of providing evidence that is useful to decision-makers.

FUTURE WORK

We call for further work to develop relevant methods and provide examples in practice. The use of this framework should be monitored and the move should be made to a more fluid resource in the future, for example a web-based format that can be frequently updated to incorporate new material and links to emerging resources.

FUNDING

This project was jointly funded by the Medical Research Council (MRC) and the National Institute for Health Research (Department of Health and Social Care 73514).

A video-feedback parenting intervention to prevent enduring behaviour problems in at-risk children aged 12-36 months: the Healthy Start, Happy Start RCT

BACKGROUND

Behaviour problems emerge early in childhood and place children at risk for later psychopathology.

OBJECTIVES

To evaluate the clinical effectiveness and cost-effectiveness of a parenting intervention to prevent enduring behaviour problems in young children.

DESIGN

A pragmatic, assessor-blinded, multisite, two-arm, parallel-group randomised controlled trial.

SETTING

Health visiting services in six NHS trusts in England.

PARTICIPANTS

A total of 300 at-risk children aged 12-36 months and their parents/caregivers.

INTERVENTIONS

Families were allocated in a 1 : 1 ratio to six sessions of Video-feedback Intervention to promote Positive Parenting and Sensitive Discipline (VIPP-SD) plus usual care or usual care alone.

MAIN OUTCOME MEASURES

The primary outcome was the Preschool Parental Account of Children's Symptoms, which is a structured interview of behaviour symptoms. Secondary outcomes included caregiver-reported total problems on the Child Behaviour Checklist and the Strengths and Difficulties Questionnaire. The intervention effect was estimated using linear regression. Health and social care service use was recorded using the Child and Adolescent Service Use Schedule and cost-effectiveness was explored using the Preschool Parental Account of Children's Symptoms.

RESULTS

In total, 300 families were randomised: 151 to VIPP-SD plus usual care and 149 to usual care alone. Follow-up data were available for 286 (VIPP-SD, n = 140; usual care, n = 146) participants and 282 (VIPP-SD, n = 140; usual care, n = 142) participants at 5 and 24 months, respectively. At the post-treatment (primary outcome) follow-up, a group difference of 2.03 on Preschool Parental Account of Children's Symptoms (95% confidence interval 0.06 to 4.01; p = 0.04) indicated a positive treatment effect on behaviour problems (Cohen's d = 0.20, 95% confidence interval 0.01 to 0.40). The effect was strongest for children's conduct [1.61, 95% confidence interval 0.44 to 2.78; p = 0.007 (d = 0.30, 95% confidence interval 0.08 to 0.51)] versus attention deficit hyperactivity disorder symptoms [0.29, 95% confidence interval -1.06 to 1.65; p = 0.67 (d = 0.05, 95% confidence interval -0.17 to 0.27)]. The Child Behaviour Checklist [3.24, 95% confidence interval -0.06 to 6.54; p = 0.05 (d = 0.15, 95% confidence interval 0.00 to 0.31)] and the Strengths and Difficulties Questionnaire [0.93, 95% confidence interval -0.03 to 1.9; p = 0.06 (d = 0.18, 95% confidence interval -0.01 to 0.36)] demonstrated similar positive treatment effects to those found for the Preschool Parental Account of Children's Symptoms. At 24 months, the group difference on the Preschool Parental Account of Children's Symptoms was 1.73 [95% confidence interval -0.24 to 3.71; p = 0.08 (d = 0.17, 95% confidence interval -0.02 to 0.37)]; the effect remained strongest for conduct [1.07, 95% confidence interval -0.06 to 2.20; p = 0.06 (d = 0.20, 95% confidence interval -0.01 to 0.42)] versus attention deficit hyperactivity disorder symptoms [0.62, 95% confidence interval -0.60 to 1.84; p = 0.32 (d = 0.10, 95% confidence interval -0.10 to 0.30)], with little evidence of an effect on the Child Behaviour Checklist and the Strengths and Difficulties Questionnaire. The primary economic analysis showed better outcomes in the VIPP-SD group at 24 months, but also higher costs than the usual-care group (adjusted mean difference £1450, 95% confidence interval £619 to £2281). No treatment- or trial-related adverse events were reported. The probability of VIPP-SD being cost-effective compared with usual care at the 24-month follow-up increased as willingness to pay for improvements on the Preschool Parental Account of Children's Symptoms increased, with VIPP-SD having the higher probability of being cost-effective at willingness-to-pay values above £800 per 1-point improvement on the Preschool Parental Account of Children's Symptoms.

LIMITATIONS

The proportion of participants with graduate-level qualifications was higher than among the general public.

CONCLUSIONS

VIPP-SD is effective in reducing behaviour problems in young children when delivered by health visiting teams. Most of the effect of VIPP-SD appears to be retained over 24 months. However, we can be less certain about its value for money.

TRIAL REGISTRATION

Current Controlled Trials ISRCTN58327365.

FUNDING

This project was funded by the National Institute for Health Research (NIHR) Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 25, No. 29. See the NIHR Journals Library website for further project information.

Going beyond the mean in examining relationships of adolescent non-cognitive skills with health-related quality of life and biomarkers in later-life

Several studies have established associations between early-life non-cognitive skills and later-life health and health behaviours. However, no study addresses the more important policy concern about how this relationship varies along the health distribution. We use unconditional quantile regression to analyse the effects of adolescent non-cognitive skills across the distributions of the health-related quality of life at age 50 and biomarkers at age 45 years. We examine the effects of measures of conscientiousness, agreeableness and neuroticism recorded at age 16 for 3585 individuals from the National Child Development Study. Adolescent conscientiousness is positively associated with ability to cope with stress and negatively associated with risk of cardiovascular disease in middle-age. Adolescent agreeableness is associated with higher health-related quality of life and lower physiological 'wear and tear', but negatively associated with ability to cope with stress in middle-age. Adolescent neuroticism is associated with lower health-related quality of life, higher physiological 'wear and tear', and a higher risk of cardiovascular disease in middle-age. All of these associations are stronger at the lower end of the health distribution except for the cardiovascular risk biomarkers. These associations are robust to correcting for attrition using inverse probability weighting and consistent with causal bounds assuming proportional selection on observables and unobservables. They suggest policies that improve non-cognitive skills in adolescence could offer most long-term health benefit to those with the poorest health.

Economic evaluation of culprit lesion only PCI vs. immediate multivessel PCI in acute myocardial infarction complicated by cardiogenic shock: the CULPRIT-SHOCK trial

BACKGROUND

The CULPRIT-SHOCK trial compared two treatment strategies for patients with acute myocardial infarction and multivessel coronary artery disease complicated by cardiogenic shock: (a) culprit vessel only percutaneous coronary intervention (CO-PCI), with additional staged revascularisation if indicated, and (b) immediate multivessel PCI (MV-PCI).

METHODS

A German societal and national health service perspective was considered for three different analyses. The cost utility analysis (CUA) estimated costs and quality adjusted life years (QALYs) based on a pre-trial decision analytic model taking a lifelong time horizon. In addition, a within trial CUA estimated QALYs and costs for 1 year. Finally, the cost effectiveness analysis (CEA) used the composite primary outcome, mortality and renal failure at 30-day follow-up, and the within trial costs. Econometric and survival analysis on the trial data was used for the estimation of the model parameters. Subgroup analysis was performed following an economic protocol.

RESULTS

The lifelong CUA showed an incremental cost effectiveness ratio (ICER), CO-PCI vs. MV-PCI, of €7010 per QALY and a probability of CO-PCI being the most cost-effective strategy > 64% at a €30,000 threshold. The ICER for the within trial CUA was €14,600 and the incremental cost per case of death/renal failure avoided at 30-day follow-up was €9010. Cost-effectiveness improved with patient age and for those without diabetes.

CONCLUSIONS

The estimates of cost-effectiveness for CO-PCI vs. MV-PCI have been shown to change depending on the time horizon and type of economic evaluation performed. The results favoured a long-term horizon analysis for avoiding underestimation of QALY gains from the CO-PCI arm.

A framework for conducting economic evaluations alongside natural experiments

Internationally, policy makers are increasingly focussed on reducing the detrimental consequences and rising costs associated with unhealthy diets, inactivity, smoking, alcohol and other risk factors on the health of their populations. This has led to an increase in the demand for evidence-based, cost-effective Population Health Interventions (PHIs) to reverse this trend. Given that research designs such as randomised controlled trials (RCTs) are often not suited to the evaluation of PHIs, Natural Experiments (NEs) are now frequently being used as a design to evaluate such complex, preventive PHIs. However, current guidance for economic evaluation focusses on RCT designs and therefore does not address the specific challenges of NE designs. Using such guidance can lead to sub-optimal design, data collection and analysis for NEs, leading to bias in the estimated effectiveness and cost-effectiveness of the PHI. As a consequence, there is a growing recognition of the need to identify a robust methodological framework for the design and conducting of economic evaluations alongside such NEs. This paper outlines the challenges inherent to the design and conduct of economic evaluations of PHIs alongside NEs, providing a comprehensive framework and outlining a research agenda in this area.