Framework for the development and evaluation of complex interventions: gap analysis, workshop and consultation-informed update

BACKGROUND

The Medical Research Council published the second edition of its framework in 2006 on developing and evaluating complex interventions. Since then, there have been considerable developments in the field of complex intervention research. The objective of this project was to update the framework in the light of these developments. The framework aims to help research teams prioritise research questions and design, and conduct research with an appropriate choice of methods, rather than to provide detailed guidance on the use of specific methods.

METHODS

There were four stages to the update: (1) gap analysis to identify developments in the methods and practice since the previous framework was published; (2) an expert workshop of 36 participants to discuss the topics identified in the gap analysis; (3) an open consultation process to seek comments on a first draft of the new framework; and (4) findings from the previous stages were used to redraft the framework, and final expert review was obtained. The process was overseen by a Scientific Advisory Group representing the range of relevant National Institute for Health Research and Medical Research Council research investments.

RESULTS

Key changes to the previous framework include (1) an updated definition of complex interventions, highlighting the dynamic relationship between the intervention and its context; (2) an emphasis on the use of diverse research perspectives: efficacy, effectiveness, theory-based and systems perspectives; (3) a focus on the usefulness of evidence as the basis for determining research perspective and questions; (4) an increased focus on interventions developed outside research teams, for example changes in policy or health services delivery; and (5) the identification of six 'core elements' that should guide all phases of complex intervention research: consider context; develop, refine and test programme theory; engage stakeholders; identify key uncertainties; refine the intervention; and economic considerations. We divide the research process into four phases: development, feasibility, evaluation and implementation. For each phase we provide a concise summary of recent developments, key points to address and signposts to further reading. We also present case studies to illustrate the points being made throughout.

LIMITATIONS

The framework aims to help research teams prioritise research questions and design and conduct research with an appropriate choice of methods, rather than to provide detailed guidance on the use of specific methods. In many of the areas of innovation that we highlight, such as the use of systems approaches, there are still only a few practical examples. We refer to more specific and detailed guidance where available and note where promising approaches require further development.

CONCLUSIONS

This new framework incorporates developments in complex intervention research published since the previous edition was written in 2006. As well as taking account of established practice and recent refinements, we draw attention to new approaches and place greater emphasis on economic considerations in complex intervention research. We have introduced a new emphasis on the importance of context and the value of understanding interventions as 'events in systems' that produce effects through interactions with features of the contexts in which they are implemented. The framework adopts a pluralist approach, encouraging researchers and research funders to adopt diverse research perspectives and to select research questions and methods pragmatically, with the aim of providing evidence that is useful to decision-makers.

FUTURE WORK

We call for further work to develop relevant methods and provide examples in practice. The use of this framework should be monitored and the move should be made to a more fluid resource in the future, for example a web-based format that can be frequently updated to incorporate new material and links to emerging resources.

FUNDING

This project was jointly funded by the Medical Research Council (MRC) and the National Institute for Health Research (Department of Health and Social Care 73514).

Explanation of context, mechanisms and outcomes in adult community mental health crisis care: the MH-CREST realist evidence synthesis

Background

Mental health crises cause significant disruption to individuals and families and can be life-threatening. The large number of community crisis services operating in an inter-agency landscape complicates access to help. It is unclear which underpinning mechanisms of crisis care work, for whom and in which circumstances.

Aim

The aim was to identify mechanisms to explain how, for whom and in what circumstances adult community crisis services work.

Objectives

The objectives were to develop, test and synthesise programme theories via (1) stakeholder expertise and current evidence; (2) a context, intervention, mechanism and outcome framework; (3) consultation with experts; (4) development of pen portraits; (5) synthesis and refinement of programme theories, including mid-range theory; and (6) identification and dissemination of mechanisms needed to trigger desired context-specific crisis outcomes.

Design

This study is a realist evidence synthesis, comprising (1) identification of initial programme theories; (2) prioritisation, testing and refinement of programme theories; (3) focused realist reviews of prioritised initial programme theories; and (4) synthesis to mid-range theory.

Main outcome

The main outcome was to explain context, mechanisms and outcomes in adult community mental health crisis care.

Data sources

Data were sourced via academic and grey literature searches, expert stakeholder group consultations and 20 individual realist interviews with experts.

Review methods

A realist evidence synthesis with primary data was conducted to test and refine three initial programme theories: (1) urgent and accessible crisis care, (2) compassionate and therapeutic crisis care and (3) inter-agency working.

Results

Community crisis services operate best within an inter-agency system. This requires compassionate leadership and shared values that enable staff to be supported; retain their compassion; and, in turn, facilitate compassionate interventions for people in crisis. The complex interface between agencies is best managed through greater clarity at the boundaries of services, making referral and transition seamless and timely. This would facilitate ease of access and guaranteed responses that are trusted by the communities they serve.

Strengths and limitations

Strengths include the identification of mechanisms for effective inter-agency community crisis care and meaningful stakeholder consultation that grounded the theories in real-life experience. Limitations include the evidence being heavily weighted towards England and the review scope excluding full analysis of ethnic and cultural diversity.

Conclusions

Multiple interpretations of crises and diverse population needs present challenges for improving the complex pathways to help in a crisis. Inter-agency working requires clear policy guidance with local commissioning. Seamless transitions between services generate trust through guaranteed responses and ease of navigation. This is best achieved where there is inter-agency affiliation that supports co-production. Compassionate leaders engender staff trust, and outcomes for people in crisis improve when staff are supported to retain their compassion.

Future work

Further work might explore inter-agency models of crisis delivery, particularly in rural communities. Future work could focus on evaluating outcomes across crisis care provider agencies and include evaluation of individual, as well as service-level, outcomes. The implementation and effect of mental health triage could be explored further, including via telehealth. Barriers to access for marginalised populations warrant a specific focus in future research.

Study registration

The study is registered as PROSPERO CRD42019141680.

Funding

This project was funded by the National Institute for Health and Care Research (NIHR) Health and Social Care Delivery Research programme and will be published in full in Health and Social Care Delivery Research; Vol. 11, No. 15. See the NIHR Journals Library website for further project information.