Introducing the revised framework for developing and evaluating complex interventions: A challenge and a resource for nursing research

This invited discussion paper highlights key updates in the MRC/NIHR's revised framework for the development and evaluation of complex nursing interventions and reflects on the implications for nursing research.

A new framework for developing and evaluating complex interventions: update of Medical Research Council guidance

The UK Medical Research Council's widely used guidance for developing and evaluating complex interventions has been replaced by a new framework, commissioned jointly by the Medical Research Council and the National Institute for Health Research, which takes account of recent developments in theory and methods and the need to maximise the efficiency, use, and impact of research.

Understanding the Impacts of Online Mental Health Peer Support Forums: Realist Synthesis

BACKGROUND

Online forums are widely used for mental health peer support. However, evidence of their safety and effectiveness is mixed. Further research focused on articulating the contexts in which positive and negative impacts emerge from forum use is required to inform innovations in implementation.

OBJECTIVE

This study aimed to develop a realist program theory to explain the impacts of online mental health peer support forums on users.

METHODS

We conducted a realist synthesis of literature published between 2019 and 2023 and 18 stakeholder interviews with forum staff.

RESULTS

Synthesis of 102 evidence sources and 18 interviews produced an overarching program theory comprising 22 context-mechanism-outcome configurations. Findings indicate that users' perceptions of psychological safety and the personal relevance of forum content are foundational to ongoing engagement. Safe and active forums that provide convenient access to information and advice can lead to improvements in mental health self-efficacy. Within the context of welcoming and nonjudgmental communities, users may benefit from the opportunity to explore personal difficulties with peers, experience reduced isolation and normalization of mental health experiences, and engage in mutual encouragement. The program theory highlights the vital role of moderators in creating facilitative online spaces, stimulating community engagement, and limiting access to distressing content. A key challenge for organizations that host mental health forums lies in balancing forum openness and anonymity with the need to enforce rules, such as restrictions on what users can discuss, to promote community safety.

CONCLUSIONS

This is the first realist synthesis of online mental health peer support forums. The novel program theory highlights how successful implementation depends on establishing protocols for enhancing safety and strategies for maintaining user engagement to promote forum sustainability.

TRIAL REGISTRATION

PROSPERO CRD42022352528; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=352528.

Designing a Library of Lived Experience for Mental Health: integrated realist synthesis and experience-based co-design study in UK mental health services

OBJECTIVE

Living Library events involve people being trained as living 'Books', who then discuss aspects of their personal experiences in direct conversation with attendees, referred to as 'Readers'. This study sought to generate a realist programme theory and a theory-informed implementation guide for a Library of Lived Experience for Mental Health (LoLEM).

DESIGN

Integrated realist synthesis and experience-based co-design.

SETTING

Ten online workshops with participants based in the North of England.

PARTICIPANTS

Thirty-one participants with a combination of personal experience of using mental health services, caring for someone with mental health difficulties and/or working in mental health support roles.

RESULTS

Database searches identified 30 published and grey literature evidence sources which were integrated with data from 10 online co-design workshops conducted over 12 months. The analysis generated a programme theory comprising five context-mechanism-outcome (CMO) configurations. Findings highlight how establishing psychological safety is foundational to productive Living Library events (CMO 1). For Readers, direct conversations humanise others' experiences (CMO 2) and provide the opportunity to flexibly explore new ways of living (CMO 3). Through participation in a Living Library, Books may experience personal empowerment (CMO 4), while the process of self-authoring and co-editing their story (CMO 5) can contribute to personal development. This programme theory informed the co-design of an implementation guide highlighting the importance of tailoring event design and participant support to the contexts in which LoLEM events are held.

CONCLUSIONS

The LoLEM has appeal across stakeholder groups and can be applied flexibly in a range of mental health-related settings. Implementation and evaluation are required to better understand the positive and negative impacts on Books and Readers.

TRIAL REGISTRATION NUMBER

PROSPERO CRD42022312789.

Realist synthesis protocol for understanding which strategies are effective to prevent urinary tract infection in older people in care homes

AIM

To identify and characterize strategies, which contribute to the prevention of urinary tract infection (UTI) in older people living in care homes.

DESIGN

The realist synthesis has four iterative stages to (1) develop initial programme theory; (2) search for evidence; (3) test and refine theory supported by relevant evidence and (4) formulate recommendations. Data from research articles and other sources will be used to explore the connection between interventions and the context in which they are applied in order to understand the mechanisms, which influence the outcomes to prevent UTI.

METHODS

A scoping search of the literature and workshops with stakeholders will identify initial programme theories. These theories will be tested and refined through a systematic search for evidence relating to mechanisms that trigger prevention and recognition of UTI in older people in care homes. Interviews with key stakeholders will establish practical relevance of the theories and their potential for implementation.

DISCUSSION

UTI is the most commonly diagnosed infection in care home residents. Evidence on the effectiveness of strategies to prevent UTI in long-term care facilities does not address the practicality of implementing these approaches in UK care homes. The realist synthesis is designed to examine this important gap in evidence.

IMPACT

Our evidence-informed programme theory will help inform programmes to improve practice to reduce the incidence of UTI in older people living in care homes and related research. Patient and public involvement will be crucial to ensuring that our findings reach carers and the public.

PATIENT AND PUBLIC CONTRIBUTION

Involvement of patient and public representatives is embedded throughout the study to ensure it is underpinned by multiple perspectives of importance to care home residents. Our co-investigator representing patient and public involvement is a lay member of the team and will chair the Project Advisory Group, which has two additional lay members. This will help to ensure that our findings and resources reach carers and the public and represent their voice in our publications and presentations to professional and lay audiences.

Facilitating cardiopulmonary resuscitation training in high-risk areas of England: A study protocol

Introduction

Bystanders' interventions improve chances of survival from out-of-hospital cardiac arrest (OHCA) before Emergency Medical Services arrive. Some areas in England are of concern. These high-risk areas have a higher incidence of cardiac arrest combined with lower-than-average bystander CPR rates and are characterised by higher proportions of minority ethnic group residents and deprivation.Collaborating with people from the Black African and Caribbean and South Asian minority communities in deprived areas of England, we aim to develop and evaluate the implementation of theoretically informed intervention(s) to address factors contributing to lower bystander intervention rates.

Methods

The study is a collaborative realist enquiry, informed by the Theoretical Domains Framework and associated Behaviour Change Wheel. It consists of 1) a realist evidence synthesis to produce initial program theories developed from primary workshop data and published evidence. It will include identifying factors contributing to the issue and potential interventions to address them; 2) theoretically informed intervention development, using the initial program theories and behaviour change theory and 3) a realist mixed methods implementation evaluation with embedded feasibility.Public involvement (PPI) as study team and public advisory group members is key to this study.We will conduct realist evidence synthesis, qualitative and statistical analyses appropriate to the various methods used.

Dissemination

We will develop a dissemination plan and materials targeted to members of the public in high-risk areas as well as academic outputs. We will hold an event for participating community groups and stakeholders to share findings and seek advice on next steps.

Study registration

ISRCTN90350842. Registration date 28.03.2023. The study was registered after its start date.

Improving Peer Online Forums (iPOF): protocol for a realist evaluation of peer online mental health forums to inform practice and policy

INTRODUCTION

Peer online mental health forums are commonly used and offer accessible support. Positive and negative impacts have been reported by forum members and moderators, but it is unclear why these impacts occur, for whom and in which forums. This multiple method realist study explores underlying mechanisms to understand how forums work for different people. The findings will inform codesign of best practice guidance and policy tools to enhance the uptake and effectiveness of peer online mental health forums.

METHODS AND ANALYSIS

In workstream 1, we will conduct a realist synthesis, based on existing literature and interviews with approximately 20 stakeholders, to generate initial programme theories about the impacts of forums on members and moderators and mechanisms driving these. Initial theories that are relevant for forum design and implementation will be prioritised for testing in workstream 2.Workstream 2 is a multiple case study design with mixed methods with several online mental health forums differing in contextual features. Quantitative surveys of forum members, qualitative interviews and Corpus-based Discourse Analysis and Natural Language Processing of forum posts will be used to test and refine programme theories. Final programme theories will be developed through novel triangulation of the data.Workstream 3 will run alongside workstreams 1 and 2. Key stakeholders from participating forums, including members and moderators, will be recruited to a Codesign group. They will inform the study design and materials, refine and prioritise theories, and codesign best policy and practice guidance.

ETHICS AND DISSEMINATION

Ethical approval was granted by Solihull Research Ethics Committee (IRAS 314029). Findings will be reported in accordance with RAMESES (Realist And MEta-narrative Evidence Syntheses: Evolving Standards) guidelines, published as open access and shared widely, along with codesigned tools.

TRIAL REGISTRATION NUMBER

ISRCTN 62469166; the protocol for the realist synthesis in workstream one is prospectively registered at PROSPERO CRD42022352528.

Connecting the science and practice of implementation - applying the lens of context to inform study design in implementation research

The saying "horses for courses" refers to the idea that different people and things possess different skills or qualities that are appropriate in different situations. In this paper, we apply the analogy of "horses for courses" to stimulate a debate about how and why we need to get better at selecting appropriate implementation research methods that take account of the context in which implementation occurs. To ensure that implementation research achieves its intended purpose of enhancing the uptake of research-informed evidence in policy and practice, we start from a position that implementation research should be explicitly connected to implementation practice. Building on our collective experience as implementation researchers, implementation practitioners (users of implementation research), implementation facilitators and implementation educators and subsequent deliberations with an international, inter-disciplinary group involved in practising and studying implementation, we present a discussion paper with practical suggestions that aim to inform more practice-relevant implementation research.

Designing a library of lived experience for mental health (LoLEM): protocol for integrating a realist synthesis and experience based codesign approach

INTRODUCTION

People with lived expertise in managing mental health challenges can be an important source of knowledge and support for other people facing similar challenges, and for carers to learn how best to help. However, opportunities for sharing lived expertise are limited. Living libraries support people with lived expertise to be 'living books', sharing their experiences in dialogue with 'readers' who can ask questions. Living libraries have been piloted worldwide in health-related contexts but without a clear model of how they work or rigorous evaluation of their impacts. We aim to develop a programme theory about how a living library could be used to improve mental health outcomes, using this theory to codesign an implementation guide that can be evaluated across different contexts.

METHODS AND ANALYSIS

We will use a novel integration of realist synthesis and experience-based codesign (EBCD) to produce a programme theory about how living libraries work and a theory and experience informed guide to establishing a library of lived experience for mental health (LoLEM). Two workstreams will run concurrently: (1) a realist synthesis of literature on living libraries, combined with stakeholder interviews, will produce several programme theories; theories will be developed collaboratively with an expert advisory group of stakeholders who have hosted or taken part in a living library and will form our initial analysis framework; a systematic search will identify literature about living libraries; data will be coded into our analysis framework, and we will use retroductive reasoning to explain living libraries' impacts across multiple contexts. Individual stakeholder interviews will help refine and test theories; (2) data from workstream 1 will inform 10 EBCD workshops with people with experience of managing mental health difficulties and health professionals to produce a LoLEM implementation guide; data from this process will also inform the theory in workstream 1.

ETHICS AND DISSEMINATION

Ethical approval was granted by Coventry and Warwick National Health Service Research Ethics Committee on 29 December 2021 (reference number 305975). The programme theory and implementation guide will be published as open access and shared widely through a knowledge exchange event, a study website, mental health provider and peer support networks, peer reviewed journals and a funders report.

PROSPERO REGISTRATION DETAILS

CRD42022312789.

Prioritising Responses Of Nurses To deteriorating patient Observations (PRONTO): a pragmatic cluster randomised controlled trial evaluating the effectiveness of a facilitation intervention on recognition and response to clinical deterioration

BACKGROUND

Most hospitals use physiological signs to trigger an urgent clinical review. We investigated whether facilitation could improve nurses' vital sign measurement, interpretation, treatment and escalation of care for deteriorating patients.

METHODS

In a pragmatic cluster randomised controlled trial, we randomised 36 inpatient wards at four acute hospitals to receive standard clinical practice guideline (CPG) dissemination to ward staff (n=18) or facilitated implementation for 6 months following standard dissemination (n=18). Expert, hospital and ward facilitators tailored facilitation techniques to promote nurses' CPG adherence. Patient records were audited pre-intervention, 6 and 12 months post-intervention on randomly selected days. Escalation of care as per hospital policy was the primary outcome at 6 and 12 months after implementation. Patients, nurses and assessors were blinded to group assignment. Analysis was by intention-to-treat.

RESULTS

From 10 383 audits, improved escalation as per hospital policy was evident in the intervention group at 6 months (OR 1.47, 95% CI (1.06 to 2.04)) with a complete set of vital sign measurements sustained at 12 months (OR 1.22, 95% CI (1.02 to 1.47)). There were no significant differences in escalation of care as per hospital policy between study groups at 6 or 12 months post-intervention. After adjusting for patient and hospital characteristics, a significant change from T0 in mean length of stay between groups at 12 months favoured the intervention group (-2.18 days, 95% CI (-3.53 to -0.82)).

CONCLUSION

Multi-level facilitation significantly improved escalation as per hospital policy at 6 months in the intervention group that was not sustained at 12 months. The intervention group had increased vital sign measurement by nurses, as well as shorter lengths of stay for patients at 12 months. Further research is required to understand the dose of facilitation required to impact clinical practice behaviours and patient outcomes.

TRIAL REGISTRATION NUMBER

ACTRN12616000544471p.

Optimising the Conceptualisation of Context Comment on "Stakeholder Perspectives of Attributes and Features of Context Relevant to Knowledge Translation in Health Settings: A Multi-country Analysis"

Context matters. Therefore, efforts to develop greater conceptual clarity are important for science and practice. In this commentary, we outline some key issues that were prompted by Squire's et al.'s contribution. Specifically, we reinforce context as an interactive concept and therefore something that is hard to 'pin down', the problematic nature of conceptualising context in implementation and de-implementation, and a requirement for the development of culturally sensitive understandings. Finally, we suggest it is vital that continued investment into providing a more comprehensive list of determinants needs to be accompanied by an equal effort in developing practical methods and tools to support use and application.

Framework for the development and evaluation of complex interventions: gap analysis, workshop and consultation-informed update

BACKGROUND

The Medical Research Council published the second edition of its framework in 2006 on developing and evaluating complex interventions. Since then, there have been considerable developments in the field of complex intervention research. The objective of this project was to update the framework in the light of these developments. The framework aims to help research teams prioritise research questions and design, and conduct research with an appropriate choice of methods, rather than to provide detailed guidance on the use of specific methods.

METHODS

There were four stages to the update: (1) gap analysis to identify developments in the methods and practice since the previous framework was published; (2) an expert workshop of 36 participants to discuss the topics identified in the gap analysis; (3) an open consultation process to seek comments on a first draft of the new framework; and (4) findings from the previous stages were used to redraft the framework, and final expert review was obtained. The process was overseen by a Scientific Advisory Group representing the range of relevant National Institute for Health Research and Medical Research Council research investments.

RESULTS

Key changes to the previous framework include (1) an updated definition of complex interventions, highlighting the dynamic relationship between the intervention and its context; (2) an emphasis on the use of diverse research perspectives: efficacy, effectiveness, theory-based and systems perspectives; (3) a focus on the usefulness of evidence as the basis for determining research perspective and questions; (4) an increased focus on interventions developed outside research teams, for example changes in policy or health services delivery; and (5) the identification of six 'core elements' that should guide all phases of complex intervention research: consider context; develop, refine and test programme theory; engage stakeholders; identify key uncertainties; refine the intervention; and economic considerations. We divide the research process into four phases: development, feasibility, evaluation and implementation. For each phase we provide a concise summary of recent developments, key points to address and signposts to further reading. We also present case studies to illustrate the points being made throughout.

LIMITATIONS

The framework aims to help research teams prioritise research questions and design and conduct research with an appropriate choice of methods, rather than to provide detailed guidance on the use of specific methods. In many of the areas of innovation that we highlight, such as the use of systems approaches, there are still only a few practical examples. We refer to more specific and detailed guidance where available and note where promising approaches require further development.

CONCLUSIONS

This new framework incorporates developments in complex intervention research published since the previous edition was written in 2006. As well as taking account of established practice and recent refinements, we draw attention to new approaches and place greater emphasis on economic considerations in complex intervention research. We have introduced a new emphasis on the importance of context and the value of understanding interventions as 'events in systems' that produce effects through interactions with features of the contexts in which they are implemented. The framework adopts a pluralist approach, encouraging researchers and research funders to adopt diverse research perspectives and to select research questions and methods pragmatically, with the aim of providing evidence that is useful to decision-makers.

FUTURE WORK

We call for further work to develop relevant methods and provide examples in practice. The use of this framework should be monitored and the move should be made to a more fluid resource in the future, for example a web-based format that can be frequently updated to incorporate new material and links to emerging resources.

FUNDING

This project was jointly funded by the Medical Research Council (MRC) and the National Institute for Health Research (Department of Health and Social Care 73514).

Realising the full potential of data-enabled trials in the UK: a call for action

RATIONALE

Clinical trials are the gold standard for testing interventions. COVID-19 has further raised their public profile and emphasised the need to deliver better, faster, more efficient trials for patient benefit. Considerable overlap exists between data required for trials and data already collected routinely in electronic healthcare records (EHRs). Opportunities exist to use these in innovative ways to decrease duplication of effort and speed trial recruitment, conduct and follow-up.

APPROACH

The National Institute of Health Research (NIHR), Health Data Research UK and Clinical Practice Research Datalink co-organised a national workshop to accelerate the agenda for 'data-enabled clinical trials'. Showcasing successful examples and imagining future possibilities, the plenary talks, panel discussions, group discussions and case studies covered: design/feasibility; recruitment; conduct/follow-up; collecting benefits/harms; and analysis/interpretation.

REFLECTION

Some notable studies have successfully accessed and used EHR to identify potential recruits, support randomised trials, deliver interventions and supplement/replace trial-specific follow-up. Some outcome measures are already reliably collected; others, like safety, need detailed work to meet regulatory reporting requirements. There is a clear need for system interoperability and a 'route map' to identify and access the necessary datasets. Researchers running regulatory-facing trials must carefully consider how data quality and integrity would be assessed. An experience-sharing forum could stimulate wider adoption of EHR-based methods in trial design and execution.

DISCUSSION

EHR offer opportunities to better plan clinical trials, assess patients and capture data more efficiently, reducing research waste and increasing focus on each trial's specific challenges. The short-term emphasis should be on facilitating patient recruitment and for postmarketing authorisation trials where research-relevant outcome measures are readily collectable. Sharing of case studies is encouraged. The workshop directly informed NIHR's funding call for ambitious data-enabled trials at scale. There is the opportunity for the UK to build upon existing data science capabilities to identify, recruit and monitor patients in trials at scale.

Theory and practical guidance for effective de-implementation of practices across health and care services: a realist synthesis

Background

Health-care systems across the globe are facing increased pressures to balance the efficient use of resources and at the same time provide high-quality care. There is greater requirement for services to be evidence based, but practices that are of limited clinical effectiveness or cost-effectiveness still occur.

Objectives

Our objectives included completing a concept analysis of de-implementation, surfacing decision-making processes associated with de-implementing through stakeholder engagement, and generating an evidence-based realist programme theory of ‘what works’ in de-implementation.

Design

A realist synthesis was conducted using an iterative stakeholder-driven four-stage approach. Phase 1 involved scoping the literature and conducting stakeholder interviews to develop the concept analysis and an initial programme theory. In Phase 2, systematic searches of the evidence were conducted to test and develop this theory, expressed in the form of contingent relationships. These are expressed as context–mechanism–outcomes to show how particular contexts or conditions trigger mechanisms to generate outcomes. Phase 3 consisted of validation and refinement of programme theories through stakeholder interviews. The final phase (i.e. Phase 4) formulated actionable recommendations for service leaders.

Participants

In total, 31 stakeholders (i.e. user/patient representatives, clinical managers, commissioners) took part in focus groups and telephone interviews.

Data sources

Using keywords identified during the scoping work and concept analysis, searches of bibliographic databases were conducted in May 2018. The databases searched were the Cochrane Library, Campbell Collaboration, MEDLINE (via EBSCOhost), the Cumulative Index to Nursing and Allied Health Literature (via EBSCOhost), the National Institute for Health Research Journals Library and the following databases via the ProQuest platform: Applied Social Sciences Index and Abstracts, Social Services Abstracts, International Bibliography of the Social Sciences, Social Sciences Database and Sociological Abstracts. Alerts were set up for the MEDLINE database from May 2018 to December 2018. Online sources were searched for grey literature and snowballing techniques were used to identify clusters of evidence.

Results

The concept analysis showed that de-implementation is associated with five main components in context and over time: (1) what is being de-implemented, (2) the issues driving de-implementation, (3) the action characterising de-implementation, (4) the extent that de-implementation is planned or opportunistic and (5) the consequences of de-implementation. Forty-two papers were synthesised to identify six context–mechanism–outcome configurations, which focused on issues ranging from individual behaviours to organisational procedures. Current systems can perpetuate habitual decision-making practices that include low-value treatments. Electronic health records can be designed to hide or remove low-value treatments from choice options, foregrounding best evidence. Professionals can be made aware of their decision-making strategies through increasing their attention to low-value practice behaviours. Uncertainty about diagnosis or patients’ expectations for certain treatments provide opportunities for ‘watchful waiting’ as an active strategy to reduce inappropriate investigations and prescribing. The emotional component of clinician–patient relationships can limit opportunities for de-implementation, requiring professional support through multimodal educational interventions. Sufficient alignment between policy, public and professional perspectives is required for de-implementation success.

Limitations

Some specific clinical issues (e.g. de-prescribing) dominate the de-implementation evidence base, which may limit the transferability of the synthesis findings. Any realist inquiry generates findings that are essentially cumulative and should be developed through further investigation that extends the range of sources into, for example, clinical research and further empirical studies.

Conclusions

This review contributes to our understanding of how de-implementation of low-value procedures and services can be improved within health-care services, through interventions that make professional decision-making more accountable and the prominence of a whole-system approach to de-implementation. Given the whole-system context of de-implementation, a range of different dissemination strategies will be required to engage with different stakeholders, in different ways, to change practice and policy in a timely manner.

Study registration

This study is registered as PROSPERO CRD42017081030.

Funding

This project was funded by the National Institute for Health Research (NIHR) Health Services and Delivery Research programme and will be published in full in Health Services and Delivery Research; Vol. 9, No. 2. See the NIHR Journals Library website for further project information.

Re-Framing the Knowledge to Action Challenge Through NIHR Knowledge Mobilisation Research Fellows Comment on "CIHR Health System Impact Fellows: Reflections on 'Driving Change' Within the Health System"

The ambition of the Canadian Institutes for Health Research Health System Impact (HSI) Fellowship initiative to modernise the health system is impressive. Embedded researchers who work between academia and non-academic settings offer an opportunity to reframe the problem of evidence uptake as a product of a gap between those who produce knowledge and those who use it. As such, there has been an increasing interest in the potential of people in embedded research roles to work with stakeholders in the co-production of knowledge to address service challenges. In this commentary, we draw on research and experiential evidence of an embedded researcher initiative, which has similar intentions to the HSI Fellowships programme: the National Institute for Health Research (NIHR) Knowledge Mobilisation Research Fellowship (KMRF) scheme. We outline the similarities and differences between the two schemes, and then consider the work, characteristics and skills, and organisational arrangements evident in operationalising these types of roles.

An online supported self-management toolkit for relatives of people with psychosis or bipolar experiences: the IMPART multiple case study

Background

Digital health interventions have the potential to improve the delivery of psychoeducation to people with mental health problems and their relatives. Despite substantial investment in the development of digital health interventions, successful implementation into routine clinical practice is rare.

Objectives

Use the implementation of the Relatives’ Education And Coping Toolkit (REACT) for psychosis/bipolar disorder to identify critical factors affecting uptake and use, and develop an implementation plan to support the delivery of REACT.

Design

This was an implementation study using a mixed-methods, theory-driven, multiple case study approach. A study-specific implementation theory for REACT based on normalisation process theory was developed and tested, and iterations of an implementation plan to address the key factors affecting implementation were developed.

Setting

Early-intervention teams in six NHS mental health trusts in England (three in the north and three in the south).

Participants

In total, 281 staff accounts and 159 relatives’ accounts were created, 129 staff and 23 relatives took part in qualitative interviews about their experiences, and 132 relatives provided demographic data, 56 provided baseline data, 21 provided data at 12 weeks’ follow-up and 20 provided data at 24 weeks’ follow-up.

Interventions

REACT is an online supported self-management toolkit, offering 12 evidence-based psychoeducation modules and support via a forum, and a confidential direct messaging service for relatives of people with psychosis or bipolar disorder. The implementation intervention was developed with staff and iteratively adapted to address identified barriers. Adaptations included modifications to the toolkit and how it was delivered by teams.

Main outcome measures

The main outcome was factors affecting implementation of REACT, assessed primarily through in-depth interviews with staff and relatives. We also assessed quantitative measures of delivery (staff accounts and relatives’ invitations), use of REACT (relatives’ logins and time spent on the website) and the impact of REACT [relatives’ distress (General Health Questionnaire-28), and carer well-being and support (Carer Well-being and Support Scale questionnaire)].

Results

Staff and relatives were generally positive about the content of REACT, seeing it as a valuable resource that could help services improve support and meet clinical targets, but only within a comprehensive service that included face-to-face support, and with some additional content. Barriers to implementation included high staff caseloads and difficulties with prioritising supporting relatives; technical difficulties of using REACT; poor interoperability with trust information technology systems and care pathways; lack of access to mobile technology and information technology training; restricted forum populations leading to low levels of use; staff fears of managing risk, online trolling, or replacement by technology; and uncertainty around REACT’s long-term availability. There was no evidence that REACT would reduce staff time supporting relatives (which was already very low), and might increase it by facilitating communication. In all, 281 staff accounts were created, but only 57 staff sent relatives invitations. In total, 355 relatives’ invitations were sent to 310 unique relatives, leading to the creation of 159 relatives’ accounts. The mean number of logins for relatives was 3.78 (standard deviation 4.43), but with wide variation from 0 to 31 (median 2, interquartile range 1–8). The mean total time spent on the website was 40.6 minutes (standard deviation 54.54 minutes), with a range of 0–298 minutes (median 20.1 minutes, interquartile range 4.9–57.5 minutes). There was a pattern of declining mean scores for distress, social dysfunction, depression, anxiety and insomnia, and increases in relatives’ well-being and eHealth literacy, but no changes were statistically significant.

Conclusions

Digital health interventions, such as REACT, should be iteratively developed, evaluated, adapted and implemented, with staff and service user input, as part of a long-term strategy to develop integrated technology-enabled services. Implementation strategies must instil a sense of ownership for staff and ensure that they have adequate training, risk protocols and resources to deliver the technology. Cost-effectiveness and impact on workload and inequalities in accessing health care need further testing, along with the generalisability of our findings to other digital health interventions.

Limitations

REACT was offered by the same team running the IMPlementation of A Relatives’ Toolkit (IMPART) study, and was perceived by staff and relatives as a time-limited research study rather than ongoing clinical service, which affected engagement. Access to observational data was limited.

Trial registration

Current Controlled Trials ISRCTN16267685.

Funding

This project was funded by the National Institute for Health Research (NIHR) Health Services and Delivery Research programme and will be published in full in Health Services and Delivery Research; Vol. 8, No. 37. See the NIHR Journals Library website for further project information.

The use of the PARIHS framework in implementation research and practice-a citation analysis of the literature

BACKGROUND

The Promoting Action on Research Implementation in Health Services (PARIHS) framework was developed two decades ago and conceptualizes successful implementation (SI) as a function (f) of the evidence (E) nature and type, context (C) quality, and the facilitation (F), [SI = f (E,C,F)]. Despite a growing number of citations of theoretical frameworks including PARIHS, details of how theoretical frameworks are used remains largely unknown. This review aimed to enhance the understanding of the breadth and depth of the use of the PARIHS framework.

METHODS

This citation analysis commenced from four core articles representing the key stages of the framework's development. The citation search was performed in Web of Science and Scopus. After exclusion, we undertook an initial assessment aimed to identify articles using PARIHS and not only referencing any of the core articles. To assess this, all articles were read in full. Further data extraction included capturing information about where (country/countries and setting/s) PARIHS had been used, as well as categorizing how the framework was applied. Also, strengths and weaknesses, as well as efforts to validate the framework, were explored in detail.

RESULTS

The citation search yielded 1613 articles. After applying exclusion criteria, 1475 articles were read in full, and the initial assessment yielded a total of 367 articles reported to have used the PARIHS framework. These articles were included for data extraction. The framework had been used in a variety of settings and in both high-, middle-, and low-income countries. With regard to types of use, 32% used PARIHS in planning and delivering an intervention, 50% in data analysis, 55% in the evaluation of study findings, and/or 37% in any other way. Further analysis showed that its actual application was frequently partial and generally not well elaborated.

CONCLUSIONS

In line with previous citation analysis of the use of theoretical frameworks in implementation science, we also found a rather superficial description of the use of PARIHS. Thus, we propose the development and adoption of reporting guidelines on how framework(s) are used in implementation studies, with the expectation that this will enhance the maturity of implementation science.

Using Nonparticipant Observation as a Method to Understand Implementation Context in Evidence-Based Practice

BACKGROUND

The uptake of evidence-based knowledge in practice is influenced by context. Observations are suggested as a valuable but under-used approach in implementation research for gaining a holistic understanding of contexts.

AIM

The aim of this paper is to demonstrate how data from observations can provide insights about context and evidence use in implementation research.

METHODS

Data were collected over 24 months in a randomised trial with an embedded realist evaluation in 24 nursing homes across four European countries; notes from 183 observations (representing 335 hours) were triangulated with interview transcripts and context survey data (from 357 staff interviews and 725 questionnaire responses, respectively).

RESULTS

Although there were similarities in several elements of context within survey, interview and observation data, the observations provided additional features of the implementation context. In particular, observations demonstrated if and how the resources (staffing and supplies) and leadership (formal and informal, teamwork, and professional autonomy) affected knowledge use and implementation. Further, the observations illuminated the influence of standards and the physical nursing environment on evidence-based practice, and the dynamic interaction between different aspects of context.

LINKING EVIDENCE TO ACTION

Although qualitative observations are resource-intensive, they add value when used with other data collection methods, further enlightening the understanding of the implementation context and how evidence use and sharing are influenced by context elements. Observations can enhance an understanding of the context, evidence use and knowledge-sharing triad in implementation research.

'Mind the gaps': the accessibility and implementation of an effective depression relapse prevention programme in UK NHS services: learning from mindfulness-based cognitive therapy through a mixed-methods study

OBJECTIVES

Mindfulness-based cognitive therapy (MBCT) is an evidence-based approach for people at risk of depressive relapse to support their long-term recovery. However, despite its inclusion in guidelines, there is an 'implementation cliff'. The study objective was to develop a better explanation of what facilitates MBCT implementation.

SETTING

UK primary and secondary care mental health services.

DESIGN, PARTICIPANTS AND METHODS

A national two-phase, multi-method qualitative study was conducted, which was conceptually underpinned by the Promoting Action on Research Implementation in Health Services framework. Phase I involved interviews with stakeholders from 40 service providers about current provision of MBCT. Phase II involved 10 purposively sampled case studies to obtain a more detailed understanding of MBCT implementation. Data were analysed using adapted framework analysis, refined through stakeholder consultation.

RESULTS

Access to MBCT is variable across the UK services. Where available, services have adapted MBCT to fit their context by integrating it into their care pathways. Evidence was often important to implementation but took different forms: the NICE depression guideline, audits, evaluations, first person accounts, experiential taster sessions and pilots. These were used to build a platform from which to develop MBCT services. The most important aspect of facilitation was the central role of the MBCT implementers. These were generally self-designated individuals who 'championed' grass-roots implementation. Our explanatory framework mapped out a prototypical implementation journey, often over many years with a balance of bottom-up and top-down factors influencing the fit of MBCT into service pathways. 'Pivot points' in the implementation journey provided windows of either challenge or opportunity.

CONCLUSIONS

This is one of the largest systematic studies of the implementation of a psychological therapy. While access to MBCT across the UK is improving, it remains patchy. The resultant explanatory framework about MBCT implementation provides a heuristic that informed an implementation resource.

Understanding how and why de-implementation works in health and care: research protocol for a realist synthesis of evidence

BACKGROUND

Strategies to improve the effectiveness and quality of health and care have predominantly emphasised the implementation of new research and evidence into service organisation and delivery. A parallel, but less understood issue is how clinicians and service leaders stop existing practices and interventions that are no longer evidence based, where new evidence supersedes old evidence, or interventions are replaced with those that are more cost effective. The aim of this evidence synthesis is to produce meaningful programme theory and practical guidance for policy makers, managers and clinicians to understand how and why de-implementation processes and procedures can work.

METHODS AND ANALYSIS

The synthesis will examine the attributes or characteristics that constitute the concept of de-implementation. The research team will then draw on the principles of realist inquiry to provide an explanatory account of how, in what context and for whom to explain the successful processes and impacts of de-implementation. The review will be conducted in four phases over 18 months. Phase 1: develop a framework to map the preliminary programme theories through an initial scoping of the literature and consultation with key stakeholders. Phase 2: systematic searches of the evidence to develop the theories identified in phase 1. Phase 3: validation and refinement of programme theories through stakeholder interviews. Phase 4: formulating actionable recommendations for managers, commissioners and service leaders about what works through different approaches to de-implementation.

DISCUSSION

This evidence synthesis will address gaps in knowledge about de-implementation across health and care services and ensure that guidance about strategies and approaches accounts for contextual factors, which may be operating at different organisational and decision-making levels. Through the development of the programme theory, which explains what works, how and under which circumstances, findings from the evidence synthesis will support managers and service leaders to make measured decisions about de-implementation.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO CRD42017081030.

The development and theoretical application of an implementation framework for dialectical behaviour therapy: a critical literature review

BACKGROUND

Dialectical behaviour therapy (DBT) is a third wave behaviour therapy combining behaviour based components with elements of mindfulness. Although DBT effectiveness has been explored, relatively little attention has been given to its implementation. Frameworks are often the basis for gathering information about implementation and can also direct how the implementation of an intervention is conducted. Using existing implementation frameworks, this critical literature review scoped the DBT implementation literature to develop and refine a bespoke DBT implementation framework.

METHOD AND RESULTS

The initial framework was developed by consolidating existing implementation frameworks and published guidance on DBT implementation. The critical literature review retrieved papers from Medline, CINAHL, PsycInfo, PubMed, and the reference lists of included papers. Framework elements were used as codes which were applied to the literature and guided the synthesis. Findings from the synthesis refined the framework.The critical literature review retrieved 60 papers but only 14 of these explicitly focused on implementation. The DBT implementation framework captured all the execution barriers and facilitators described in the literature. However, the evidence synthesis led to a more parsimonious framework as some elements (e.g., research and published guidance) were seldom discussed in DBT implementation.

CONCLUSION

To our knowledge this is the first published review exploring DBT implementation. The literature synthesis suggests some tentative recommendations which warrant further exploration. For instance, if DBT implementation is not pre-planned, having someone in the organisation who champions DBT can be advantageous. However, as the literature is limited and has methodological limitations, further prospective studies of DBT implementation are needed.

How Should We Evaluate and Use Evidence to Improve Population Oral Health?

Generating and implementing evidence-based policy is an important aim for many publicly funded health systems. In dentistry, this is based on the assumption that evidence-based health care increases the efficiency and effectiveness of interventions to improve oral health at a population level. This article argues that a linear logic model that links the generation of research evidence with its use is overly simplistic. It also challenges an uncritical interpretation of the evidence-based paradigm and explores approaches to the evaluation of complex interventions and how they can be embedded into policy and practice to improve oral health at a population level.

The generation of consensus guidelines for carrying out process evaluations in rehabilitation research

BACKGROUND

Although in recent years there has been a strong increase in published research on theories (e.g. realist evaluation, normalization process theory) driving and guiding process evaluations of complex interventions, there is limited guidance to help rehabilitation researchers design and carry out process evaluations. This can lead to the risk of process evaluations being unsystematic. This paper reports on the development of new consensus guidelines that address the specific challenges of conducting process evaluations alongside clinical trials of rehabilitation interventions.

METHODS

A formal consensus process was carried out based on a modified nominal group technique, which comprised two phases. Phase I was informed by the findings of a systematic review, and included a nominal group meeting with an expert panel of participants to rate and discuss the proposed statements. Phase II was an in depth semi-structured telephone interviews with expert panel participants in order to further discuss the structure and contents of the revised guidelines. Frequency of rating responses to each statement was calculated and thematic analysis was carried out on all qualitative data.

RESULTS

The guidelines for carrying out process evaluations within complex intervention rehabilitation research were produced by combining findings from Phase I and Phase II. The consensus guidelines include recommendations that are grouped in seven sections. These sections are theoretical work, design and methods, context, recruitment and retention, intervention staff, delivery of the intervention and results. These sections represent different aspects or stages of the evaluation process.

CONCLUSION

The consensus guidelines here presented can play a role at assisting rehabilitation researchers at the time of designing and conducting process evaluations alongside trials of complex interventions. The guidelines break new ground in terms of concepts and theory and works towards a consensus in regards to how rehabilitation researchers should go about carrying out process evaluations and how this evaluation should be linked into the proposed trials. These guidelines may be used, adapted and tested by rehabilitation researchers depending on the research stage or study design (e.g. feasibility trial, pilot trial, etc.).