Communities in charge of alcohol (CICA): a protocol for a stepped-wedge randomised control trial of an alcohol health champions programme

A new framework for developing and evaluating complex interventions: update of Medical Research Council guidance

The UK Medical Research Council's widely used guidance for developing and evaluating complex interventions has been replaced by a new framework, commissioned jointly by the Medical Research Council and the National Institute for Health Research, which takes account of recent developments in theory and methods and the need to maximise the efficiency, use, and impact of research.

Does a local Alcohol Health Champion programme have a measurable impact on health and crime outcomes? A natural experiment evaluation of Communities in Charge of Alcohol (CICA) based on triangulation of methods

BACKGROUND AND AIM

Drinking alcohol may cause harm to an individual's health and social relationships, while a drinking culture may harm societies as it may increase crime rates and make an area feel less safe. Local councils in Greater Manchester, UK, developed the Communities in Charge of Alcohol (CICA) intervention, in which volunteers were trained to give alcohol-related advice to the public and taught how to influence policies to restrict when, where and how alcohol is sold. As part of a larger study, the aim of the current project is to measure the impact of CICA on health and crime outcomes at the lower super output (LSOA) geographical aggregation.

DESIGN

Quantitative evaluation using four time series analytic methods (stepped-wedge design, and comparisons to local controls, national controls and synthetic controls) with findings triangulated across these methods. A cost-benefit analysis was carried out alongside the effectiveness analysis.

SETTING AND PARTICIPANTS

The general public in Greater Manchester, UK, between 2010 and 2020.

MEASUREMENTS

The primary outcome of interest was alcohol-related hospital admissions. Secondary outcomes were accident and emergency (A&E) attendances, ambulance callouts, recorded crimes and anti-social behaviour incidents.

FINDINGS

Triangulation of the results did not indicate any consistent effect on area-level alcohol-related hospital admissions, A&E attendances, ambulance callouts, reported crimes or anti-social behaviour associated with the implementation of CICA. The primary stepped-wedge analysis indicated an increase in alcohol-related hospital admissions following the implementation of CICA of 13.4% (95% confidence interval -3.3%, +30.1%), which was consistent with analyses based on other methods with point estimates ranging from +3.4% to 16.4%.

CONCLUSION

There is no evidence of a measurable impact of the Communities in Charge of Alcohol (CICA) programme on area-level health and crime outcomes in Greater Manchester, UK, within 3 years of the programme start. The increase in alcohol-related hospital admissions was likely the result of other temporal trends rather than the CICA programme. Possible explanations include insufficient follow-up time, too few volunteers trained, volunteers being unwilling to get involved in licensing decisions or that the intervention has no direct impact on the selected outcomes.

Making it transparent: A worked example of articulating programme theory for a digital health application using Intervention Mapping

Objective

Digital health interventions for behaviour change are usually complex interventions, and intervention developers should 'articulate programme theory', that is, they should offer detailed descriptions of individual intervention components and their proposed mechanisms of action. However, such detailed descriptions often remain lacking. The objective of this work was to provide a conceptual case study with an applied example of 'articulating programme theory' for a newly developed digital health intervention.

Methods

Intervention Mapping methodology was applied to arrive at a detailed description of programme theory for a newly developed digital health intervention that aims to support cardiac rehabilitation patients in establishing heart-healthy physical activity habits. Based on a Predisposing, Reinforcing, and Enabling Constructs in Educational Diagnosis and Evaluation (PRECEDE) logic model of the problem, a logic model of change was developed. The proposed mechanisms of action were visualised in an acyclic behaviour change diagram.

Results

Programme theory for this digital health intervention includes 4 sub-behaviours of the main target behaviour (i.e. habitual heart-healthy physical activity), 8 personal determinants and 12 change objectives (i.e. changes needed at the determinant level to achieve the sub-behaviours). These are linked to 12 distinct features of the digital health intervention and 12 underlying behaviour change methods.

Conclusions

This case study offers a worked example of articulating programme theory for a digital health intervention using Intervention Mapping. Intervention developers and researchers may draw on this example to replicate the method, or to reflect on most suitable approaches for their own behaviour change interventions.

'A priori' external contextual factors and relationships with process indicators: a mixed methods study of the pre-implementation phase of 'Communities in Charge of Alcohol'

BACKGROUND

It is widely recognised that complex public health interventions roll out in distinct phases, within which external contextual factors influence implementation. Less is known about relationships with external contextual factors identified a priori in the pre-implementation phase. We investigated which external contextual factors, prior to the implementation of a community-centred approach to reducing alcohol harm called 'Communities in Charge of Alcohol' (CICA), were related to one of the process indicators: numbers of Alcohol Health Champions (AHCs) trained.

METHODS

A mixed methods design was used in the pre-implementation phase of CICA. We studied ten geographic communities experiencing both high levels of deprivation and alcohol-related harm in the North West of England. Qualitative secondary data were extracted from pre-implementation meeting notes, recorded two to three months before roll-out. Items were coded into 12 content categories using content analysis. To create a baseline 'infrastructure score', the number of external contextual factors documented was counted per area to a maximum score of 12. Descriptive data were collected from training registers detailing training numbers in the first 12 months. The relationship between the baseline infrastructure score, external contextual factors, and the number of AHCs trained was assessed using non-parametric univariable statistics.

RESULTS

There was a positive correlation between baseline infrastructure score and total numbers of AHCs trained (Rs = 0.77, p = 0.01). Four external contextual factors were associated with significantly higher numbers of lay people recruited and trained: having a health care provider to coordinate the intervention (p = 0.02); a pool of other volunteers to recruit from (p = 0.02); a contract in place with a commissioned service (p = 0.02), and; formal volunteer arrangements (p = 0.03).

CONCLUSIONS

Data suggest that there were four key components that significantly influenced establishing an Alcohol Health Champion programme in areas experiencing both high levels of deprivation and alcohol-related harm. There is added value of capturing external contextual factors a priori and then testing relationships with process indicators to inform the effective roll-out of complex interventions. Future research could explore a wider range of process indicators and outcomes, incorporating methods to rate individual factors to derive a mean score.

TRIAL REGISTRATION

ISRCTN81942890, date of registration 12/09/2017.

A mixed methods analysis evaluating an alcohol health champion community intervention: How do newly trained champions perceive and understand their training and role?

Globally, alcohol harm is recognised as one of the greatest population risks and reducing alcohol harm is a key priority for the UK Government. The Communities in Charge of Alcohol (CICA) programme took an asset-based approach in training community members across nine areas to become alcohol health champions (AHCs); trained in how to have informal conversations about alcohol and get involved with alcohol licensing. This paper reports on the experiences of AHCs taking part in the training through the analysis of: questionnaires completed pre- and post-training (n = 93) and semi-structured interviews with a purposive sample of five AHCs who had started their role. Questionnaires explored: characteristics of AHCs, perceived importance of community action around alcohol and health, and confidence in undertaking their role. Following training AHCs felt more confident to talk about alcohol harms, give brief advice and get involved in licensing decisions. Interviews explored: AHCs' experiences of the training, barriers and facilitators to the adoption of their role, and how they made sense of their role. Four overarching themes were identified through thematic analysis taking a framework approach: (a) perceptions of AHC training; (b) applying knowledge and skills in the AHC role; (c) barriers and facilitators to undertaking the AHC role; and (d) sustaining the AHC role. Findings highlight the challenges in establishing AHC roles can be overcome by combining the motivation of volunteers with environmental assets in a community setting: the most important personal asset being the confidence to have conversations with people about a sensitive topic, such as alcohol.

Framework for the development and evaluation of complex interventions: gap analysis, workshop and consultation-informed update

BACKGROUND

The Medical Research Council published the second edition of its framework in 2006 on developing and evaluating complex interventions. Since then, there have been considerable developments in the field of complex intervention research. The objective of this project was to update the framework in the light of these developments. The framework aims to help research teams prioritise research questions and design, and conduct research with an appropriate choice of methods, rather than to provide detailed guidance on the use of specific methods.

METHODS

There were four stages to the update: (1) gap analysis to identify developments in the methods and practice since the previous framework was published; (2) an expert workshop of 36 participants to discuss the topics identified in the gap analysis; (3) an open consultation process to seek comments on a first draft of the new framework; and (4) findings from the previous stages were used to redraft the framework, and final expert review was obtained. The process was overseen by a Scientific Advisory Group representing the range of relevant National Institute for Health Research and Medical Research Council research investments.

RESULTS

Key changes to the previous framework include (1) an updated definition of complex interventions, highlighting the dynamic relationship between the intervention and its context; (2) an emphasis on the use of diverse research perspectives: efficacy, effectiveness, theory-based and systems perspectives; (3) a focus on the usefulness of evidence as the basis for determining research perspective and questions; (4) an increased focus on interventions developed outside research teams, for example changes in policy or health services delivery; and (5) the identification of six 'core elements' that should guide all phases of complex intervention research: consider context; develop, refine and test programme theory; engage stakeholders; identify key uncertainties; refine the intervention; and economic considerations. We divide the research process into four phases: development, feasibility, evaluation and implementation. For each phase we provide a concise summary of recent developments, key points to address and signposts to further reading. We also present case studies to illustrate the points being made throughout.

LIMITATIONS

The framework aims to help research teams prioritise research questions and design and conduct research with an appropriate choice of methods, rather than to provide detailed guidance on the use of specific methods. In many of the areas of innovation that we highlight, such as the use of systems approaches, there are still only a few practical examples. We refer to more specific and detailed guidance where available and note where promising approaches require further development.

CONCLUSIONS

This new framework incorporates developments in complex intervention research published since the previous edition was written in 2006. As well as taking account of established practice and recent refinements, we draw attention to new approaches and place greater emphasis on economic considerations in complex intervention research. We have introduced a new emphasis on the importance of context and the value of understanding interventions as 'events in systems' that produce effects through interactions with features of the contexts in which they are implemented. The framework adopts a pluralist approach, encouraging researchers and research funders to adopt diverse research perspectives and to select research questions and methods pragmatically, with the aim of providing evidence that is useful to decision-makers.

FUTURE WORK

We call for further work to develop relevant methods and provide examples in practice. The use of this framework should be monitored and the move should be made to a more fluid resource in the future, for example a web-based format that can be frequently updated to incorporate new material and links to emerging resources.

FUNDING

This project was jointly funded by the Medical Research Council (MRC) and the National Institute for Health Research (Department of Health and Social Care 73514).

How can communities influence alcohol licensing at a local level? Licensing officers' perspectives of the barriers and facilitators to sustaining engagement in a volunteer-led alcohol harm reduction approach

Despite the World Health Organization's assertion that communities need to become involved in reducing alcohol harm, evidence of community engagement in alcohol licensing decision-making in England remains limited. The evaluation of the Communities in Charge of Alcohol (CICA) programme offers policymakers, Licensing authorities and public health practitioners, evidence regarding a specific volunteer-led, place-based approach, designed to enable community engagement in licensing with the aim of reducing localised alcohol harm. This study explored factors affecting the sustainable involvement of volunteers in alcohol licensing decision-making from six licensing officers' perspectives, through semi-structured interviews. Routinely collected crime, disorder, and hospital admissions data were reviewed for further context as proxy indicators for alcohol-related harm. Licensing officers perceived sustainable engagement to be impacted by: (i) the extent of alignment with statutory requirements and local political support; (ii) the ability of licensing officers to operationalise CICA and support local assets, and; (iii) the opportunity for, and ability of, volunteers to raise licensing issues. The perspectives of licensing officers indicate complexities inherent in seeking to empower residents to engage in licensing decision-making at a community level. These relate to statutory and political factors, funding, social norms regarding engagement in licensing decision-making, and the need for networks between critical actors including responsible authorities and communities. The evidence indicates that after increasing community capacity to influence alcohol availability decision-making at a local level, communities continue to struggle to influence statutory processes to affect alcohol availability where they live and work. More understanding of how to enable effective community engagement is required.

An asset-based community development approach to reducing alcohol harm: Exploring barriers and facilitators to community mobilisation at initial implementation stage

Globally there is a need to engage communities in actions to reduce alcohol harm. This paper reports on the initial implementation phase of an asset-based community development (ABCD) approach to reducing alcohol harm in ten pre-identified areas across Greater Manchester (UK). This qualitative study highlights the experiences of stakeholders responsible for, or engaged in, implementation. Findings show that it is challenging to recruit sufficient volunteers in a specific, small area/community, which may limit the ability to build health assets. Wider policy and organisational factors that should be understood prior to implementing a place-based volunteer-led health promotion programme are also identified.

Trial registration

https://www.isrctn.com/ISRCTN81942890.

•

Identifies factors affecting implementation of a place-based alcohol harm intervention.

•

Examines a locally driven asset-based community development volunteer approach.

•

Capacity was limited by small volunteer pool in the specific targeted intervention areas.

•

Demonstrates the need to cultivate a partnership-based approach at local level.

Conceptualising natural and quasi experiments in public health

Background

Natural or quasi experiments are appealing for public health research because they enable the evaluation of events or interventions that are difficult or impossible to manipulate experimentally, such as many policy and health system reforms. However, there remains ambiguity in the literature about their definition and how they differ from randomized controlled experiments and from other observational designs. We conceptualise natural experiments in the context of public health evaluations and align the study design to the Target Trial Framework.

Methods

A literature search was conducted, and key methodological papers were used to develop this work. Peer-reviewed papers were supplemented by grey literature.

Results

Natural experiment studies (NES) combine features of experiments and non-experiments. They differ from planned experiments, such as randomized controlled trials, in that exposure allocation is not controlled by researchers. They differ from other observational designs in that they evaluate the impact of events or process that leads to differences in exposure. As a result they are, in theory, less susceptible to bias than other observational study designs. Importantly, causal inference relies heavily on the assumption that exposure allocation can be considered ‘as-if randomized’. The target trial framework provides a systematic basis for evaluating this assumption and the other design elements that underpin the causal claims that can be made from NES.

Conclusions

NES should be considered a type of study design rather than a set of tools for analyses of non-randomized interventions. Alignment of NES to the Target Trial framework will clarify the strength of evidence underpinning claims about the effectiveness of public health interventions.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-021-01224-x.

'The opportunity to have their say'? Identifying mechanisms of community engagement in local alcohol decision-making

BACKGROUND

Engaging the community in decisions-making is recognised as important for improving public health, and is recommended in global alcohol strategies, and in national policies on controlling alcohol availability. Yet there is little understanding of how to engage communities to influence decision-making to help reduce alcohol-related harms. We sought to identify and understand mechanisms of community engagement in decision-making concerning the local alcohol environment in England.

METHODS

We conducted case studies in three local government areas in England in 2018, purposively selected for examples of community engagement in decisions affecting the local alcohol environment. We conducted 20 semi-structured interviews with residents, workers, local politicians and local government practitioners, and analysed documents linked to engagement and alcohol decision-making.

RESULTS

Four rationales for engaging the community in decision-making affecting the alcohol environment were identified: i) as part of statutory decision-making processes; ii) to develop new policies; iii) as representation on committees; and iv) occurring through relationship building. Many of the examples related to alcohol licensing processes, but also local economy and community safety decision-making. The impact of community inputs on decisions was often not clear, but there were a few instances of engagement influencing the process and outcome of decision-making relating to the alcohol environment.

CONCLUSIONS

While influencing statutory licensing decision-making is challenging, community experiences of alcohol-related harms can be valuable 'evidence' to support new licensing policies. Informal relationship-building between communities and local government is also beneficial for sharing information about alcohol-related harms and to facilitate future engagement. However, care must be taken to balance the different interests among diverse community actors relating to the local alcohol environment, and extra support is needed for those with least capacity to engage but who face more burden of alcohol-related harms, to avoid compounding existing inequalities.

Identifying opportunities for engaging the 'community' in local alcohol decision-making: A literature review and synthesis

INTRODUCTION

Engaging communities in actions to reduce alcohol harms has been identified as an international priority. While there exist recommendations for community engagement within alcohol licensing legislation, there is limited understanding of how to involve communities in local decision-making to reduce harms from the alcohol environment.

METHODS

A scoping literature review was conducted on community engagement in local government decision-making with relevance to the alcohol environment. Academic and grey literature databases were searched between April and June 2018 to identify examples of community engagement in local government in the UK, published since 2000. Texts were excluded if they did not describe in detail the mechanisms or rationale for community engagement. Information was extracted and synthesised through a narrative approach.

RESULTS

3030 texts were identified through the searches, and 30 texts were included in the final review. Only one text described community engagement in alcohol decision-making (licensing); other local government sectors included planning, regeneration and community safety. Four rationales for community engagement emerged: statutory consultation processes; non-statutory engagement; as part of broader participatory initiatives; and community-led activism. While not all texts reported outcomes, a few described direct community influence on decisions. Broader outcomes included improved relationships between community groups and local government. However, lack of influence over decisions was also common, with multiple barriers to effective engagement identified.

CONCLUSION

The lack of published examples of community engagement in local alcohol decision-making relevant to the UK suggests little priority has been placed on sharing learning about supporting engagement in this area. Taking a place-shaping perspective, useful lessons can be drawn from other areas of local government with relevance for the alcohol environment. Barriers to engagement must be considered carefully, particularly around how communities are defined, and how different interests toward the local alcohol environment are represented, or not.

Innovations in Mixed Methods Evaluations

Mixed methods research-i.e., research that draws on both qualitative and quantitative methods in varying configurations-is well suited to address the increasing complexity of public health problems and their solutions. This review focuses specifically on innovations in mixed methods evaluations of intervention, program or policy (i.e., practice) effectiveness, and implementation. The article begins with an overview of the structure, function, and process of different mixed methods designs and then provides illustrations of their use in effectiveness studies, implementation studies, and combined effectiveness-implementation hybrid studies. The article then examines four specific innovations: procedures for transforming (or "quantitizing") qualitative data, application of rapid assessment and analysis procedures in the context of mixed methods studies, development of measures to assess implementation outcomes, and strategies for conducting both random and purposive sampling, particularly in implementation-focused evaluation research. The article concludes with an assessment of challenges to integrating qualitative and quantitative data in evaluation research.