A realist evaluation approach to explaining the role of context in the impact of a complex eHealth intervention for improving prevention of cardiovascular disease

Background

Reduction of cardiovascular disease (CVD) is a worldwide health priority and innovative uses of technology-based interventions may assist patients with improving prevention behaviours. Targeting these interventions to recipients most likely to benefit requires understanding how contexts of use influence responsiveness to the intervention, and how this interaction favours or discourages health behaviour. Using a realist evaluation approach, the aim of this study was to examine the contextual factors influencing behaviour change within a multi-feature eHealth intervention with personalised data integration from the primary care electronic health record (EHR).

Methods

Realist evaluation of qualitative data from the Consumer Navigation of Electronic Cardiovascular Tools (CONNECT) randomised trial (N = 934). Thirty-six participants from the intervention group (N = 486) who had completed 12 months of study follow-up were interviewed. Coding of transcripts was structured around configurations of contexts, mechanisms, and outcomes of intervention use. Contextual narratives were derived from thematic analysis of the interviews.

Results

Mechanisms favouring positive health behaviour occurred when participants responded to four interactive features of the intervention. Facilitating mechanisms included greater cognitive engagement whereby participants perceived value and benefit, and felt motivated, confident and incentivised. Participants moved from being unconcerned (or unaware) to more task-oriented engagement with personal CVD risk profile and prevention. Increased personalisation occurred when modifiable CVD risk factors became relatable to lifestyle behaviour; and experiences of feeling greater agency/self-efficacy emerged. Use and non-use of the intervention were influenced by four overarching narratives within the individual’s micro-level and meso-level environments: illness experiences; receptiveness to risk and prevention information; history of the doctor-patient relationship; and relationship with technology.

Conclusions

Intervention-context interactions are central to understanding how change mechanisms activate within complex interventions to exert their impact on recipients. Intervention use and non-use were context-dependent, underscoring the need for further research to target eHealth innovations to those most likely to benefit.

Assessing the impacts of the Agenda Gap intervention for youth mental health promotion through policy engagement: a study protocol

BACKGROUND

Mental health challenges are a leading health concern for youth globally, requiring a comprehensive approach incorporating promotion, prevention and treatment within a healthy public policy framework. However, the broad enactment of this vision has yet to be realized. Further, mental health promotion evidence specific to youth is still emerging and has not yet focused at a policy level. This is a critical gap, as policy is a key mental health promotion lever that can alter the social and structural conditions that contribute to shaping youth mental health outcomes for all youth, across the full spectrum of need. Responsive to this research and intervention priority, our prototype study intervention-the Agenda Gap-is comprised of an innovative, multi-media engagement intervention, developed in collaboration with youth. This intervention aims to equip youth and build capacity for them to lead meaningful policy change reflective of the mental health needs of diverse communities of youth, including those who experience structural vulnerability and who would not typically have had their voice represented in policymaking processes.

METHODS

This study will use a multiple case study design and mixed methods grounded in a realist approach and will be conducted in three sites across two Canadian provinces (British Columbia and Alberta). In an earlier phase of this research, we collaboratively designed the prototype intervention with youth, community and policy partners. In this phase of the study, the intervention will be implemented and further tested with new groups of youth collaborators (n = 10-15/site). Outcome data will be collected through realist qualitative interviews, validated questionnaires [i.e., Child and Youth Resilience Measure (CYRM-12), General Self-Efficacy (GSE) Scale, and the Critical Consiousness Scale (CCS)] and additional survey items developed by our study team. Analysis will focus on identification of key context-mechanism-outcome configurations to provide comprehensive insights into how this intervention works, for whom, and in what context.

DISCUSSION

This study is unique in its "upstream" focus on youth-engaged policymaking as a tool for improving the social and structural conditions that influence youth mental health across socioecological levels. Through the implementation and testing of the Agenda Gap intervention with diverse youth, this study will contribute to the evidence base on youth-engaged policymaking as a novel and innovative, mental health promotion strategy.

Housing as a social determinant of health and wellbeing: developing an empirically-informed realist theoretical framework

BACKGROUND

The role of housing as a social determinant of health is well-established, but the causal pathways are poorly understood beyond the direct effects of physical housing defects. For low-income, vulnerable households there are particular challenges in creating a sense of home in a new tenancy which may have substantial effects on health and wellbeing. This study examines the role of these less tangible aspects of the housing experience for tenants in the social and private rented sectors in west central Scotland.

METHODS

The paper analyses quantitative data from a mixed methods, longitudinal study of tenants from three housing organisations, collected across the first year of their tenancy. The paper postulates causal hypotheses on the basis of staff interviews and then uses a Realist Research approach to test and refine these into a theoretical framework for the connections between tenants' broader experience of housing and their health and wellbeing.

RESULTS

Housing service provision, tenants' experience of property quality and aspects of neighbourhood are all demonstrated to be significantly correlated with measures of of health and wellbeing. Analysis of contextual factors provides additional detail within the theoretical framework, offering a basis for further empirical work.

CONCLUSIONS

The findings provide an empirically-informed realist theoretical framework for causal pathways connecting less tangible aspects of the housing experience to health and wellbeing. Applying this within housing policy and practice would facilitate a focus on housing as a public health intervention, with potential for significant impacts on the lives of low-income and vulnerable tenants. The framework also offers a basis for further research to refine our understanding of housing as a social determinant of health.

A strategic approach to social accountability: Bwalo forums within the reproductive maternal and child health accountability ecosystem in Malawi

Background

The majority of documented social accountability initiatives to date have been ‘tactical’ in nature, employing single-tool, mostly community-based approaches. This article provides lessons from a ‘strategic’, multi-tool, multi-level social accountability project: UNICEF’s ‘Social Accountability for Every Woman Every Child’ intervention in Malawi.

Methods

The project targeted the national, district and community levels. Three Civil Society Organisations (CSOs) were engaged to carry out interventions using various tools to generate evidence and political advocacy at one or more levels. This article focuses on one of the social accountability methods – the bwalo forum (a meeting based on a traditional Malawian method of dialogue). A detailed political economy analysis was conducted by one of the co-authors using qualitative methods including interviews and group discussions. The authors conducted in-country consultations and analysed secondary data provided by the CSOs.

Results

The political economy analysis highlighted several ways in which CSO partners should modify their work plans to be more compatible with the project context. This included shifting the advocacy and support focus, as well as significantly expanding the bwalo forums. Bwalos were found to be an important platform for allowing citizens to engage with duty bearers at the community and district levels, and enabled a number of reproductive, maternal, newborn, child and adolescent health issues to be resolved at those levels. The project also enabled learning around participant responses as intermediate project outcomes.

Conclusions

The project utilised various tools to gather data, elevate community voices, and facilitate engagement between citizen and state actors at the community, district and national levels. This provided the scaffolding for numerous issues to be resolved at the community or district levels, or referred to the national level. Bwalo forums were found to be highly effective as a space for inter-level engagement between citizens and state; however, as they were not embedded in existing local structures, their potential for sustainability and scalability was tenuous. A key strength of the project was the political economy analysis, which provided direction for partners to shape their interventions according to local and national realities and be sensitive to the barriers and drivers to positive action.

Reflections on experiential learning in evaluation capacity building with a community organization, Dancing With Parkinson's

This paper discusses what was learned about evaluation capacity building with community organizations who deliver services to individuals with neurological disorders. Evaluation specialists engaged by the Ontario Brain Institute Evaluation Support Program were paired with community organizations, such as Dancing With Parkinson's. Some of the learning included: relationship building is key for this model of capacity building; community organizations often have had negative experiences with evaluation and the idea that evaluations can be friendly tools in implementing meaningful programs is one key mechanism by which such an initiative can work; community organizations often need evaluation most to be able to demonstrate their value; a strength of this initiative was that the focus was not just on creating products but mostly on developing a learning process in which capacities would remain; evaluation tools and skills that organizations found useful were developing a theory of change and the concept of heterogeneous mechanisms (informed by a realist evaluation lens).

Valuing and embracing complexity: How an understanding of complex interventions needs to shape our evaluation capacities building initiatives

This paper describes some of the main challenges of evaluating complex interventions, as well as the implications of such challenges for evaluation capacity building. It discusses lessons learned from a case study of an evaluation of Dancing with Parkinson's, an organization that provides dance classes to people with Parkinson's disease in Toronto, Canada. These implications are developed from a realist evaluation lens. Key lessons include the need to develop skills to understand program mechanisms and contexts, recognize multiple models of causality, apply mixed method designs, and ensure the successful scaling up and spread of an intervention.

Shifting care from hospital to community, a strategy to integrate care in Singapore: process evaluation of implementation fidelity

Background

Accessibility to efficient and person-centered healthcare delivery drives healthcare transformation in many countries. In Singapore, specialist outpatient clinics (SOCs) are commonly congested due to increasing demands for chronic care. To improve this situation, the National University Health System (NUHS) Regional Health System (RHS) started an integrated care initiative,the Right-Site Care (RSC) program in 2014. Through collaborations between SOCs at the National University Hospital and primary and community care (PCC) clinics in the western region of the county, the program was designed to facilitate timely discharge and appropriate transition of patients, who no longer required specialist care, to the community. The aim of this study was to evaluate the implementation fidelity of the NUHS RHS RSC program using the modified Conceptual Framework for Implementation Fidelity (CFIF), at three distinct levels; providers, organizational, and system levels to explain outcomes of the program and to inform further development of (similar) programs.

Methods

A convergent parallel mixed methods study using the realist evaluation approach was used. Data were collected between 2016 and 2018 through non-participatory observations, reviews of medical records and program database, together with semi-structured interviews with healthcare providers. Triangulation of data streams was applied guided by the modified CFIF.

Results

Our findings showed four out of six program components were implemented with low level of fidelity, and 9112 suitable patients were referred to the program while 3032 (33.3%) declined to be enrolled. Moderating factors found to influence fidelity included: (i) complexity of program, (ii) evolving providers’ responsiveness, (iii) facilitation through synergistic partnership, training of PCC providers by specialists and supportive structures: care coordinators, guiding protocols, shared electronic medical record and shared pharmacy, (iv) lack of organization reinforcement, and (v) mismatch between program goals, healthcare financing and providers’ reimbursement.

Conclusion

Functional integration alone is insufficient for a successful right-site care program implementation. Improvement in relationships between providers, organizations, and patients are also warranted for further development of the program.

What works, for whom and under what circumstances? Using realist methodology to evaluate complex interventions in nursing: A scoping review

BACKGROUND

The Medical Research Council's (MRC) framework recommends basing every evaluation of a complex intervention on a theory that explains WHY an intervention works instead of focusing exclusively on the identification IF an intervention works. Theory-based evaluation approaches such as realist evaluations are promising in this respect. The aim of this scoping review is to examine current scientific literature with regard to the use of realist methodology in studies that evaluate complex nursing interventions. We also seek to answer the question whether realist evaluation studies of complex nursing interventions are embedded in the Medical Research Council's framework.

METHODS

A systematic database search in MEDLINE and CINAHL supplemented with snowballing techniques and a hand search in relevant nursing journals were performed between January and February 2018 and updated in July 2019. Both authors independently screened the title / abstract of studies that were initially identified and appraised the full texts if inclusion criteria were fulfilled. Inclusion criteria were the explicit use of realist methodology and a focus on the development or evaluation of complex nursing interventions. Results were summarized narratively.

RESULTS

A total of 28 articles from 22 studies, published between 2010 and 2019, were included. The majority of the studies included originate in the UK. Realist methodology was used by a variety of nursing disciplines, including midwifery, community / public health, critical care, palliative care, gerontological nursing, psychiatric nursing / mental health, and infection control. Study designs included case studies and realist reviews; two studies were conducted alongside a randomized controlled trial. All studies included used more than one method to develop their results; the majority of methods were qualitative by nature. Thirteen of the studies included aimed to develop a theory about what works, for whom and under what circumstances regarding the investigated complex nursing intervention. One study was clearly embedded in the Medical Research Council's framework.

CONCLUSION

Realist evaluation broadens the methodologies that have traditionally been used in nursing. Conceptual clarity on its principal tenets is still needed, as well as the methods used to assess them. Nursing scientists should be more critical when using novel methodology and contribute to academic debates about limitations. Embedding realist evaluations into the Medical Research Council's framework is in its infancy in nursing and needs more contributions in order to develop a common academic position within the profession. Tweetable abstract Realist methodology in nursing - review of the use, methods, limitations and embedding into the Medical Research Council's framework.

How Can Autonomy Be Maintained and Informal Care Improved for People With Dementia Living in Residential Care Facilities: A Systematic Literature Review

Background and Objectives

For people with dementia living in residential care facilities, maintaining autonomy and receiving informal care are important. The objective of this review is to understand how caregiving approaches and physical environment, including technologies contribute to the maintenance of autonomy and informal care provision for this population.

Research Design and Methods

A literature review of peer-reviewed articles published between January 1995 and July 2017 was performed. Realist logic of analysis was used, involving context, mechanism and outcome configurations.

Results

Forty-nine articles were included. The improvement of the relationship between residents and formal/informal caregivers is important. This increases the knowledge (sharing) about the resident and contributes to their autonomy. A social, flexible, and welcoming attitude of the formal caregiver improves the provision of informal care. Specially designed spaces, for instance, therapeutic gardens, create activities for residents that remind them of themselves and contribute to their autonomy. Use of technologies reduces caregiver’s time for primary tasks and therefore enables secondary tasks such as interaction with the residents.

Discussion and Implications

The results revealed how residential care facilities could maintain autonomy of their residents and improve informal care delivery using caregiving approaches and the physical environment including technologies. The results are supporting toward each other in maintaining autonomy and also helped in enhancing informal care provision. For residential care facilities that want to maintain the autonomy of their residents and improve informal care delivery, it is important to pay attention to all aspects of living in a residential care facility.

Searching for new community engagement approaches in the Netherlands: a realist qualitative study

Background

Community engagement is increasingly seen as key to improving healthcare systems and to increasing communities’ involvement in the shaping of their own communities. This paper describes how ‘community engagement’ (CE) is understood and being operationalised in the Dutch healthcare system by investigating the CE approaches being implemented in six different regions and by examining engaged citizens’ and professionals’ experiences of those CE approaches.

Methods

For this realist study, interviews and focus groups were held with citizens (16) and professionals (42) involved in CE approaches in the six regions. Additionally, CE-related activities were observed to supplement interview data.

Results

This study shows that citizens and professionals defined and experienced CE differently and that they differed in who they felt had ownership of CE. The CE approaches implemented in community-led initiatives and organisationally-led initiatives varied accordingly. Furthermore, both citizens and professionals were searching for meaningful ways for citizens to have more control over healthcare in their own communities.

Conclusion

CE can be improved by, first of all, developing a shared and overarching vision of what CE should look like, establishing clear roles and remits for organisations and communities, and taking active measures to ensure CE is more inclusive and representative of harder-to-reach groups. At the same time, to help ensure such shared visions do not further entrench power imbalances between citizens and professionals, professionals require training in successful CE approaches.

Implementing population health management: an international comparative study

PURPOSE

The purpose of this paper is to gain insight into how population health management (PHM) strategies can successfully integrate and reorganize public health, health care, social care and community services to improve population health and quality of care while reducing costs growth, this study compared four large-scale transformation programs: Greater Manchester Devolution, Vancouver Healthy City Strategy, Gen-H Cincinnati and Gesundes Kinzigtal.

DESIGN/METHODOLOGY/APPROACH

Following the realist methodology, this explorative comparative case-study investigated PHM initiatives' key features and participants' experiences of developing such initiatives. A semi-structured interview guideline based on a theoretical framework for PHM guided the interviews with stakeholders (20) from different sectors.

FINDINGS

Five initial program theories important to the development of PHM were formulated: (1) create trust in a shared vision and understanding of the PHM rationale to establish stakeholders' commitment to the partnership; (2) create shared ownership for achieving the initiative's goals; (3) create shared financial interest that reduces perceived financial risks to provide financial sustainability; (4) create a learning environment to secure initiative's credibility and (5) create citizens' and professionals' awareness of the required attitudes and behaviours.

ORIGINALITY/VALUE

The study highlights initial program theories for the implementation of PHM including different strategies and structures underpinning the initiatives. These insights provide a deeper understanding of how large-scale transformation could be developed.

Examining school-level implementation of British Columbia, Canada's school food and beverage sales policy: a realist evaluation

OBJECTIVE

To identify key school-level contexts and mechanisms associated with implementing a provincial school food and beverage policy.

DESIGN

Realist evaluation. Data collection included semi-structured interviews (n 23), structured questionnaires (n 62), participant observation at public events (n 3) and scans of school, school district and health authority websites (n 67). The realist heuristic, context + mechanism → outcome configuration was used to conduct the analysis.

SETTING

Public schools in five British Columbia (BC), Canada school districts.

PARTICIPANTS

Provincial and regional health and education staff, private food vendors and school-level stakeholders.

RESULTS

We identified four mechanisms influencing the implementation of BC's school food and beverage sales policy. First, the mandatory nature of the policy triggered some actors' implementation efforts, influenced by their normative acceptance of the educational governance system. Second, some expected implementers had an opposite response to the mandate where they ignored or 'skirted' the policy, influenced by values and beliefs about the role of government and school food. A third mechanism related to economics demonstrated ways vendors' responses to school demand for compliance with nutritional Guidelines were mediated by beliefs about food preferences of children, health and food. The last mechanism demonstrated how resource constraints and lack of capacity led otherwise motivated stakeholders to not implement the mandatory policy.

CONCLUSION

Implementation of the food and beverage sales policy at the school level is shaped by interactions between administrators, staff, parent volunteers and vendors with contextual factors such as varied motivations, responsibilities and capacities.

Clarifying the role of belief-motive explanations in multi-stakeholder realist evaluation

In realist evaluation, where researchers aim to make program theories explicit, they can encounter competing explanations as to how programs work. Managing explanatory tensions from different sources of evidence in multi-stakeholder projects can challenge external evaluators, especially when access to pertinent data, like client records, is mediated by program stakeholders. In this article, we consider two central questions: how can program stakeholder motives shape a realist evaluation project; and how might realist evaluators respond to stakeholders' belief-motive explanations, including those about program effectiveness, based on factors such as supererogatory commitment or trying together in good faith? Drawing on our realist evaluation of a service reform initiative involving multiple agencies, we describe stakeholder motives at key phases, highlighting a need for tactics and skills that help to manage explanatory tensions. In conclusion, the relevance of stakeholders' belief-motive explanations ('we believe the program works') in realist evaluation is clarified and discussed.

The role of professional logics in quality register use: a realist evaluation

Background

Clinical practice improvements based on quality-register data are influenced by multiple factors. Although there is agreement that information from quality registers is valuable for quality improvement, practical ways of organising register use have been notoriously difficult to realise. The present study sought to investigate the mechanisms that lead various clinicians to use quality registers for improvement.

Methods

This research involves studying individuals’ decisions in response to a Swedish programme focusing on increasing the use of quality registers. Through a case study, we focused on heart failure care and its corresponding register: the Swedish Heart Failure Register. The empirical data consisted of a purposive sample collected longitudinally by qualitative methods between 2013 and 2015. In total, 18 semi-structured interviews were carried out. We used realist evaluation to identify contexts, mechanisms, and outcomes.

Results

We identified four contexts – registration, use of output data, governance, and improvement projects – that provide conditions for the initiation of specific mechanisms. Given a professional theoretical perspective, we further showed that mechanisms are based on the logics of either organisational improvement or clinical practice. The two logics offer insights into the ways in which clinicians choose to embrace or reject certain registers’ initiatives.

Conclusions

We identified a strong path dependence, as registers have historically been tightly linked to the medical profession’s competence. Few new initiatives in the studied programme reach the clinical context. We explain this through the lack of an organisational improvement logic and its corresponding mechanisms in the context of the medical profession. Implementation programmes must understand the logic of clinical practice; that is, be integrated with the ways in which work is carried out in everyday practice. Programmes need to be better at helping core health professionals to reach the highest standards of patient care.

"To mean something to someone": sport-for-development as a lever for social inclusion

Background

Socially excluded groups are at higher risk of low well-being and poor health. The link between social exclusion and health inequities is complex, and not being involved in society makes it difficult to be reached by standard prevention programs. Sport-for-development (SFD) programs are low-threshold and may be promising settings for inclusive actions. We explore the underlying mechanisms through which SFD might have an impact on social inclusion and examine the necessary conditions that work as a catalyst for these underlying mechanisms.

Methods

A realist evaluation approach was adopted. A non-profit SFD organization in a middle-large city in Flanders, Belgium, formed the setting for a single case study. Document analysis, participatory observations, interviews, and a focus group, were sources for identifying necessary context elements and essential mechanisms through which SFD could promote its participants’ health and wellbeing.

Results

Among the most efficient mechanisms triggered by the Foundation’s activities are learning by fun, connecting with peers (of whom some serve as role model) and engaging as a volunteer with some responsibilities. Building trust in oneself and in others is a necessary process throughout all these mechanisms. Facilitating context factors include the activities’ accessibility and unconditional approach (creating a sense of safety), the popularity of the first division football team the Foundation is associated with (leading to a sense of belonging), a steady network of social partners and a strongly positive relationship with the SFD coach(es).

Conclusions

Our findings demonstrate that a SFD setting may be a vehicle for engaging hard-to-reach population groups. It enhances socially vulnerable persons’ sense of competence and connectedness, leading to opportunities to improve life and work skills transferrable outside SFD settings. Based on these findings, suggestions are provided that may enhance the field and help to develop feasible (policy-led) interventions designed to promote social inclusion.

Can sexual health interventions make community-based health systems more responsive to adolescents? A realist informed study in rural Zambia

Introduction

Community-based sexual reproductive interventions are key in attaining universal health coverage for all by 2030, yet adolescents in many countries still lack health services that are responsive to their sexual reproductive health and rights’ needs. As the first step of realist evaluation, this study provides a programme theory that explains how, why and under what circumstances community-based sexual reproductive health interventions can transform (or not) ‘ordinary’ community-based health systems (CBHSs) into systems that are responsive to the sexual reproductive health of adolescents.

Methods

This realist approach adopted a case study design. We nested the study in the full intervention arm of the Research Initiative to Support the Empowerment of Girls trial in Zambia. Sixteen in-depth interviews were conducted with stakeholders involved in the development and/or implementation of the trial. All the interviews were recorded and analysed using NVIVO version 12.0. Thematic analysis was used guided by realist evaluation concepts. The findings were later synthesized using the Intervention−Context−Actors−Mechanism−Outcomes conceptualization tool. Using the retroduction approach, we summarized the findings into two programme theories.

Results

We identified two initial testable programme theories. The first theory presumes that adolescent sexual reproductive health and rights (SRHR) interventions that are supported by contextual factors, such as existing policies and guidelines related to SRHR, socio-cultural norms and CBHS structures are more likely to trigger mechanisms among the different actors that can encourage uptake of the interventions, and thus contribute to making the CBHS responsive to the SRHR needs of adolescents. The second and alternative theory suggests that SRHR interventions, if not supported by contextual factors, are less likely to transform the CBHSs in which they are implemented. At individual level the mechanisms, awareness and knowledge were expected to lead to value clarification’, which was also expected would lead to individuals developing a ‘supportive attitude towards adolescent SRHR. It was anticipated that these individual mechanisms would in turn trigger the collective mechanisms, communication, cohesion, social connection and linkages.

Conclusion

The two alternative programme theories describe how, why and under what circumstances SRHR interventions that target adolescents can transform ‘ordinary’ community-based health systems into systems that are responsive to adolescents.

Uptake of a primary care atrial fibrillation screening program (AF-SMART): a realist evaluation of implementation in metropolitan and rural general practice

Background

Screening for atrial fibrillation (AF) in people aged ≥65 years is recommended by international guidelines. The Atrial Fibrillation Screen, Management And guideline-Recommended Therapy (AF-SMART) studies of opportunistic AF screening in 16 metropolitan and rural general practices were conducted from November 2016–June 2019. These studies trialled custom-designed eHealth tools to support all stages of AF screening in general practice.

Methods

A realist evaluation of the AF-SMART studies, which aimed to explain the circumstances in which the program worked (or not) to increase the proportion of people screened for AF. The initial program theory was based on our previous research, policy documents and screening studies. To test this, we conducted 45 semi-structured interviews with general practitioners (GPs), nurses and practice managers across all participating practices, and collected observational and quantitative screening data. These data were analysed and interpreted to refine the program theory.

Results

GPs/nurses liked the eHealth tools, although technical problems sometimes disrupted screening. Time was the main barrier to screening for GPs/nurses, so systems need to be very efficient. Practices with leadership from a senior GP ‘screening champion’ had broader uptake, especially from the nursing team. Providing regular feedback on screening data was beneficial for quality improvement and motivation. Clear protocols for follow-up of abnormal results were required for successful nurse-led screening in a hierarchical system. Participation in the program had broader benefits of improving AF knowledge and raising the profile of cardiovascular health in the practice. Screening for a shorter, more intense period (eg during influenza vaccination) worked well for practices where sufficient staff time was allocated.

Conclusions

Introducing an AF screening program is likely to be successful in contexts where there is a senior GP ‘screening champion’, a clear protocol exists for abnormal results, and there is regular data reporting to staff. These contexts link to mechanisms around motivation, leadership, empowerment of nurses, and efficient screening systems. The contexts and mechanisms contribute to the longer-term outcomes of increasing the proportion of people screened and treated for AF, which is recommended by guidelines as a key strategy for the prevention of AF-related stroke.

Trial registrations

AF SMART (metropolitan): ACTRN12616000850471 (Australia New Zealand Clinical Trials Registry).

AF SMART II (rural): ACTRN12618000004268 (Australia New Zealand Clinical Trials Registry).

A realist evaluation to identify contexts and mechanisms that enabled and hindered implementation and had an effect on sustainability of a lean intervention in pediatric healthcare

BACKGROUND

In 2012, the Saskatchewan Ministry for Health mandated a system-wide Lean transformation. Research has been conducted on the implementation processes of this system-wide Lean implementation. However, no research has been done on the sustainability of these Lean efforts. We conducted a realist evaluation on the sustainability of Lean in pediatric healthcare. We used the context (C) + mechanism (M) = outcome (O) configurations (CMOcs) heuristic to explain under what contexts, for whom, how and why Lean efforts are sustained or not sustained in pediatric healthcare.

METHODS

We employed a case study research design. Guided by a realist evaluation framework, we conducted qualitative realist interviews with various stakeholder groups across four pediatric hospital units 'cases' at one acute hospital. Interview data was analyzed using an integrated approach of CMOc categorization coding, CMOc connecting and pattern matching.

RESULTS

We conducted thirty-two interviews across the four cases. Five CMOcs emerged from our realist interview data. These configurations illustrated a 'ripple-effect' from implementation outcomes to contexts for sustainability. Sense-making and staff engagement were prominent mechanisms to the sustainment of Lean efforts. Failure to trigger these mechanisms resulted in resistance. The implementation approach used influenced mechanisms and outcomes for sustainability, more so than Lean itself. Specifically, the language, messaging and training approaches used triggered mechanisms of innovation fatigue, poor 'sense-making' and a lack of engagement for frontline staff. The mandated, top-down, externally led nature of implementation and lack of customization to context served as potential pitfalls. Overall, there was variation between leadership and frontline staff's perceptions on how embedded Lean was in their contexts, and the degree to which participants supported Lean sustainability.

CONCLUSIONS

This research illuminates important contextual factors and mechanisms to the process of Lean sustainment that can be applicable to those implementing systems changes. Future work is needed to continue to develop the science on the sustainability of interventions for healthcare improvement.

A realist evaluation of the implementation of open visiting in an acute care setting for older people

BACKGROUND

Open visiting refers to the principle of unrestricted visiting hours in the hospital setting to enable relatives, families and carers to visit at any time. There has been recognition that open visiting supports the principle of patient and family supported care and improves communication. Despite this there has been difficulty in implementing open visiting and barriers identified. The aims of this study were therefore to evaluate the implementation of open visiting, the barriers to implementation, sustainability and the impact of open visiting on communication between health care professionals, families and carers.

METHODS

The study was conducted on two large acute wards for the older person. Realist evaluation methods were used to understand 'what works well, how, for whom and to what extent.' Mixed methods were employed including qualitative interviews and descriptive analyses of routine data sets. Following the methodology of realist evaluation, programme theories were identified a long with the context, mechanisms and outcomes of implementation, to better understand the implementation process.

RESULTS

The results of this study identified some key findings, demonstrating that open visiting does improve communication and can help to build trusting relationships between families/carers and health care professionals (HCP). Barriers to implementation were based on the belief that it would impinge on routines within the ward setting. To achieve the principles of patient and family/carer centred care, the key mechanisms are the confidence and skills of individual nurses and health care assistants to engage with relatives/carers, whilst retaining a sense of control, particularly when care is being delivered to other patients.

CONCLUSION

In summary, open visiting creates a positive culture which fosters better relationships between families/carers and HCPs. Involving families/carers as partners in care does not happen automatically in an environment where open visiting is the policy, but requires engagement with staff to encourage and support relatives/carers.

Transforming towards sustainable health and wellbeing systems: Eight guiding principles based on the experiences of nine Dutch Population Health Management initiatives

INTRODUCTION

Population Health Management initiatives are increasingly introduced, aiming to develop towards sustainable health and wellbeing systems. Yet, little is known about which strategies to implement during this development. This study provides insights into which strategies are used, why, and when, based on the experiences of nine Dutch Population Health Management initiatives.

METHODS

The realist evaluation approach was used to gain an understanding of the relationships between context, mechanisms and outcomes when Population Health Management strategies were implemented. Data were retrieved from three interview rounds (n = 207) in 2014, 2016 and 2017. Data was clustered into guiding principles, underpinned with strategy-context-mechanism-outcome configurations.

RESULTS

The Dutch initiatives experienced different developments, varying between immediate large-scale collaborations with eventual relapse, and incremental growth towards cross-sector collaboration. Eight guiding principles for development towards health and wellbeing systems were identified, focusing on: 1. Shared commitment for a Population Health Management-vision; 2. Mutual understanding and trust; 3. Accountability; 4. Aligning politics and policy; 5. Financial incentives; 6. A learning cycle based on a data-infrastructure; 7. Community input and involvement; and 8. Stakeholder representation and leadership.

CONCLUSION

Development towards a sustainable health and wellbeing system is complex and time-consuming. Its success not only depends on the implementation of all eight guiding principles, but is also influenced by applying the right strategies at the right moment in the development.

Collaborative governance in the Quebec Cancer Network: a realist evaluation of emerging mechanisms of institutionalization, multi-level governance, and value creation using a longitudinal multiple case study design

BACKGROUND

People living with and beyond cancer (PLC) receive various forms of specialty care at different locations and many interventions concurrently or over time. They are affected by the operation of professional and organizational silos. This results in undue delays in access, unmet needs, sub-optimal care experiences and clinical outcomes, and human and financial costs for PLCs and healthcare systems. National cancer control programs advocate organizing in a network to coordinate actions, solve fragmentation problems, and thus improve clinical outcomes and care experiences for every dollar invested. The variable outcomes of such networks and factors explaining them have been documented. Governance is the "missing link" for understanding outcomes. Governance refers to the coordination of collective action by a body in a position of authority in pursuit of a common goal. The Quebec Cancer Network (QCN) offers the opportunity to study in a natural environment how, why, by whom, for whom, and under what conditions collaborative governance contributes to practices that produce value-added outcomes for PLCs, healthcare providers, and the healthcare system.

METHODS/DESIGN

The study design consists of a longitudinal case study, with multiple nested cases (4 local networks nested in the QCN), mobilizing qualitative and quantitative data and mixed data from various sources and collected using different methods, using the realist evaluation approach. Qualitative data will be used for a thematic analysis of collaborative governance. Quantitative data from validated questionnaires will be analyzed to measure relational coordination and teamwork, care experience, clinical outcomes, and health-related health-related quality of life, as well as a cost analysis of service utilization. Associations between context, governance mechanisms, and outcomes will be sought. Robust data will be produced to support decision-makers to guide network governance towards optimized clinical outcomes and the reduction of the economic toxicity of cancer for PLCs and health systems.

Development of an evaluation tool for geriatric rehabilitation care

BACKGROUND

Geriatric rehabilitation care (GRC) is short-term and multidisciplinary rehabilitation care for older vulnerable clients. Studies were conducted about its effects. However, elements that influence the quality of GRC have not been studied previously.

METHODS

In this study realist evaluation is used to find out which are the mechanisms and outcomes and which (groups of) persons are the context for GRC, according to GRC professionals. The mechanisms, outcomes and context of GRC were explored in three consecutive phases of qualitative data gathering, i.e. individual interviews, expert meeting, and focus groups.

RESULTS

Eight mechanisms - client centeredness, client satisfaction during rehabilitation, therapeutic climate, information provision to client and informal care givers, consultation about the rehabilitation (process), cooperation within the MultiDisciplinary Team (MDT), professionalism of GRC professionals, and organizational aspects - were found. Four context groups-the client, his family and/or informal care giver(s), the individual GRC professional, and the MDT-were mentioned by the respondents. Last, two outcome factors were determined, i.e. client satisfaction at discharge and rehabilitation goals accomplished.

CONCLUSIONS

In order to translate these insights into a practical tool that can be used by MDTs in the practice of GRC, identified mechanisms, contexts, and outcomes were visualized in a GRC evaluation tool. A graphic designer developed an interactive PDF which is the GRC evaluation tool. This tool may enable MDTs to discuss, prioritize, evaluate, and improve the quality of their GRC practice.

In search for gold - The relevance of realist reviews and evaluations to pharmacy research and policy development

Pharmacy services and programs can be regarded as complex interventions which are developed and implemented within the open, complex system of overall healthcare. Realist research considers matters of complexity and provides insights into what programs and interventions work, why and in which contexts. Based on the philosophy of science of critical realism, realist evaluations and realist reviews generate causative explanations which inform pharmacy practitioners, educators and policy makers in which context programs and services achieve particular outcomes. This more nuanced understanding of how pharmacy services contribute to overall healthcare provides guidance for the refinement and targeting of programs, interventions and practice models. This article outlines key aspects of realist research approaches and provides insight into how realism can contribute to research in and the practice of pharmacy.

What is the impact of large-scale implementation of stroke Early Supported Discharge? A mixed methods realist evaluation study protocol

Background

Stroke Early Supported Discharge (ESD) is a service innovation that facilitates discharge from hospital and delivery of specialist rehabilitation in patients’ homes. There is currently widespread implementation of ESD services in many countries, driven by robust clinical trial evidence. In England, the type of ESD service patients receive on the ground is variable, and in some regions, ESD is still not offered at all. This protocol presents a study designed to investigate the mechanisms and outcomes of implementing ESD at scale in real-world conditions. This will help to establish which models of ESD are most effective and in what context.

Methods

A realist evaluation approach composed of two interlinking work packages will be adopted to investigate how and why ESD works, for whom and in what circumstances. Work package 1 (WP1) will begin with a rapid evidence synthesis to formulate preliminary realist hypotheses. Quantitative analyses of historical prospective Sentinel Stroke National Audit Programme (SSNAP) data will be performed to evaluate service outcomes based on the degree to which evidence-based ESD has been implemented. Work package 2 (WP2) will involve the qualitative investigation of purposively selected case study sites featuring in WP1 and covering different regions in England. The perspectives of clinicians, managers, commissioners, and service users will be explored qualitatively. Cost implications of ESD models will be examined using a cost-consequence analysis. Cross-case comparisons and triangulation of the data sources from both work packages will be performed to test, revise, and refine initial programme theories and address research aims.

Discussion

This study will investigate whether and how current large-scale implementation of ESD is achieving the outcomes suggested by the evidence base. The theory-driven evaluation approach will highlight key mechanisms and contextual conditions necessary to optimise outcomes and allow us to draw transferable lessons to inform the effective implementation and sustainability of ESD in clinical practice. In addition, the methodological framework will progress the theoretical understanding of implementation and evaluation of complex rehabilitation interventions in stroke care.

Trial registration

ISRCTN: 15568163, registration date: 26 October 2018.

A realist process evaluation of Enhanced Triple P for Baby and Mellow Bumps, within a Trial of Healthy Relationship Initiatives for the Very Early years (THRIVE): study protocol for a randomized controlled trial

Background

THRIVE is a three-arm randomised controlled trial (RCT) that aims to evaluate whether antenatal and early postnatal interventions, Enhanced Triple B for Baby (ETPB) plus care as usual (CAU) or Mellow Bumps (MB) plus CAU (versus CAU alone), can: 1) improve the mental health and well-being of pregnant women with complex health and social care needs; 2) improve mother-infant bonding and interaction; 3) reduce child maltreatment; and 4) improve child language acquisition. This paper focuses on THRIVE’s realist process evaluation, which is carefully monitoring what is happening in the RCT.

Methods

Realistic evaluation provides the theoretical rationale for the process evaluation. We question: 1) how faithfully are MB and ETPB implemented? 2) What are the mechanisms by which they work, if they do, and who do they work for and how? 3) What contextual factors are necessary for the programmes to function, or might prevent them functioning?

The mixed-methods design includes quantitative measures, which are pre- and post-training/intervention questionnaires for facilitators and mothers-to-be, and post-session evaluation forms. Qualitative data collection methods include participant observation of facilitator training and the delivery of a series of antenatal sessions in selected intervention groups (n = 3 for ETPB and n = 3 for MB), semi-structured interviews with facilitators, pregnant women, partners, and referring facilitators, and telephone interviews examining the content of the postnatal components of ETPB and MB.

Discussion

The findings of this process evaluation will help researchers and decision makers interpret the outcomes of THRIVE. It will provide a greater understanding of: how the interventions work (if they do); the extent and quality of their implementation; contextual factors facilitating and constraining intervention functioning; variations in response within and between subgroups of vulnerable parents; and benefits or unintended consequences of either intervention. Few studies to date have published detailed research protocols illustrating how realist process evaluation is designed and conducted as an integral part of a randomised controlled trial.

Trial registration

ISRCTN, ISRCTN21656568. Registered on 8 November 2013.

Electronic supplementary material

The online version of this article (10.1186/s13063-019-3395-3) contains supplementary material, which is available to authorized users.