No Mechanism Without Context: Strengthening the Analysis of Context in Realist Evaluations Using Causal Loop Diagramming

Scaling-Out Digitally Enabled Integrated Care in Europe Through Good Practices Transfer: The JADECARE Study

Introduction

The absence of a coordinated approach to health and social care compromises the ability of health systems to provide universal, equitable, high-quality, and financially sustainable care. Transferring evidence-based practices focused on digitally-enabled integrated care to new contexts can overcome this challenge if implementation is satisfactory. This paper presents the scaling-out methodology that JADECARE has designed to spread effective innovative practices across Europe.

Methodology

The scaling-out methodology pretends to guide the Next Adopters in the transfer and adoption of practices, whereas increasing their implementation capacity and providing an evaluation framework to assess impact and success.

Discussion

JADECARE scaling-out effort is based on guiding principles found in the literature such as the balance between fidelity to the original practice and the degree of adaptation required to fit the new context, the need for capacity building in implementation to bridge the gap between research and routine practice and the focus on explaining why, for whom and in what circumstances an intervention works.

Conclusion

The JADECARE scaling-out methodology is theory-driven and pragmatic and aims to facilitate the transfer of complex interventions across different contexts.

Protocol for an interview-based method for mapping mental models using causal-loop diagramming and realist interviewing

Causal-loop diagramming, a method from system dynamics, is increasingly used in evaluation to describe individuals' understanding of how policies or programs do or could work ("mental models"). The use of qualitative interviews to inform model development is common, but guidance for how to design and conduct these interviews to elicit causal information in participant mental models is scant. A key strength of semi-structured qualitative interviews is that they let participants speak freely; they are not, however, designed to elicit causal information. Moreover, much of human communication about mental models-particularly larger causal structures such as feedback loops-is implicit. In qualitative research, part of the skill and art of effective interviewing and analysis involves listening for information that is expressed implicitly. Similarly, a skilled facilitator can recognize and inquire about implied causal structures, as is commonly done in group model building. To standardize and make accessible these approaches, we have formalized a protocol for designing and conducting semi-structured interviews tailored to eliciting mental models using causal-loop diagramming. We build on qualitative research methods, system dynamics, and realist interviewing. This novel, integrative method is designed to increase transparency and rigor in the use of interviews for system dynamics and has a variety of potential applications.

How do urban green spaces influence heat-related mortality in elderly? A realist synthesis

BACKGROUND

An important consequence of climate change for urban health is heat-related mortality. Vulnerable groups, especially elderly, will be the most affected. A solution put forward in many reports and policy documents is the introduction or expansion of urban green spaces. While they have a proven effect in decreasing the ambient temperature and reducing heat related mortality, the causal pathways are far from clear. Moreover, results vary for different contexts, population types and characteristics of green spaces as they are 'complex systems thrusted into complex systems'. To our knowledge, there is no systematic synthesis of the literature that examines the mechanisms by which and the circumstances under which green spaces work to decrease heat-related mortality for elderly.

METHODS

We performed a realist synthesis- a theory-driven review method- to develop a complexity- and context-sensitive program theory. As a first step, a causal loop diagram was constructed which describes the possible pathways through which urban green spaces influence heat-related mortality in elderly. In a second step, one of the pathways - how they may lead to a reduction of heat-related mortality by increasing social capital - was further explored for underlying mechanisms, the context in which they work and the differentiated patterns of outcomes they generate. Literature was searched for evidence supporting or contradicting the initial programme theory, resulting in a refined theory.

RESULTS

Results show how urban green space can impact on heat-related mortality in elderly by its influence on their exposure to outdoor and indoor heat, by improving their resilience as well as by affecting their access to treatment. Urban green spaces and their interactions with social capital affect the access to health information, social support, and the capacity for effective lobbying. Several mechanisms help to explain these observed demi-regularities, among others perceived behavioural control, perceived usefulness, receptiveness, ontological security, and self-interest. If and how they are triggered depends on the characteristics of the urban green space, the population, and other contextual factors.

CONCLUSION

Looking into the impact of urban green spaces on heat-related mortality in elderly, researchers and policy makers should take interest in the role of social capital.

Understanding Whether and How a Digital Health Intervention Improves Transition Care for Emerging Adults Living With Type 1 Diabetes: Protocol for a Mixed Methods Realist Evaluation

BACKGROUND

Emerging adults living with type 1 diabetes (T1D) face a series of challenges with self-management and decreased health system engagement, leading to an increased risk of acute complications and hospital admissions. Effective and scalable strategies are needed to support this population to transfer seamlessly from pediatric to adult care with sufficient self-management capability. While digital health interventions for T1D self-management are a promising strategy, it remains unclear which elements work, how, and for which groups of individuals.

OBJECTIVE

This study aims to evaluate the design and implementation of a multicomponent SMS text message-based digital health intervention to support emerging adults living with T1D in real-world settings. The objectives are to identify the intervention components and associated mechanisms that support user engagement and T1D health care transition experiences and determine the individual characteristics that influence the implementation process.

METHODS

We used a realist evaluation embedded alongside a randomized controlled trial, which uses a sequential mixed methods design to analyze data from multiple sources, including intervention usage data, patient-reported outcomes, and realist interviews. In step 1, we conducted a document analysis to develop a program theory that outlines the hypothesized relationships among "individual-level contextual factors, intervention components and features, mechanisms, and outcomes," with special attention paid to user engagement. Among them, intervention components and features depict 10 core characteristics such as transition support information, problem-solving information, and real-time interactivity. The proximal outcomes of interest include user engagement, self-efficacy, and negative emotions, whereas the distal outcomes of interest include transition readiness, self-blood glucose monitoring behaviors, and blood glucose. In step 2, we plan to conduct semistructured realist interviews with the randomized controlled trial's intervention-arm participants to test the hypothesized "context-intervention-mechanism-outcome" configurations. In step 3, we plan to triangulate all sources of data using a coincidence analysis to identify the necessary combinations of factors that determine whether and how the desired outcomes are achieved and use these insights to consolidate the program theory.

RESULTS

For step 1 analysis, we have developed the initial program theory and the corresponding data collection plan. For step 2 analysis, participant enrollment for the randomized controlled trial started in January 2023. Participant enrollment for this realist evaluation was anticipated to start in July 2023 and continue until we reached thematic saturation or achieved informational power.

CONCLUSIONS

Beyond contributing to knowledge on the multiple pathways that lead to successful engagement with a digital health intervention as well as target outcomes in T1D care transitions, embedding the realist evaluation alongside the trial may inform real-time intervention refinement to improve user engagement and transition experiences. The knowledge gained from this study may inform the design, implementation, and evaluation of future digital health interventions that aim to improve transition experiences.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

PRR1-10.2196/46115.

Methods in realist evaluation: A mapping review

Realist evaluation is becoming increasingly popular as an evaluation methodology. Its main objective is to uncover the mechanisms that lead to observed outcomes following an intervention and the contextual conditions that enabled this. The focus is on explaining why, for whom and in what circumstances an intervention works. It is a theory-driven approach and is explicitly method neutral, meaning that both quantitative and qualitative data collection methods can be used to unearth the underlying mechanisms that cause the intervention outcomes. In this review, we aim to map the methods used in realist evaluation studies, to draw lessons from the findings and to reflect on ways forward. We found that qualitative methods and interviews specifically are most commonly used in realist evaluations; that theory is often absent behind the methods and sampling techniques used; and that more innovative methods remain underexplored. We conclude the review by proposing four ways forward: (1) developing realist surveys, (2) exploring the relevance of innovative methods, (3) increasing the attention paid to sampling procedures and (4) strengthening the theory-driven nature of method. We believe that these four action points can strengthen the practice of realist evaluation and its outcomes.

Person-centred care in primary care: What works for whom, how and in what circumstances?

This rapid realist review aims to explain how and why person-centred care (PCC) in primary care works (or not) among others for people with low health literacy skills and for people with a diverse ethnic and socioeconomic background, and to construct a middle-range programme theory (PT). Peered reviewed- and non-peer-reviewed literature (Jan 2013-Feb 2021) reporting on PCC in primary care was included. Selection and appraisal of documents were based on relevance and rigour according to the Realist And Meta-narrative Evidence Syntheses: Evolving Standards (RAMESES) criteria. Data on context, mechanisms and outcomes (CMO) were extracted. Based on the extracted data, CMO configurations were identified per source publication. Configurations containing all three constructs (CMO) were included in the PT. The middle-range PT demonstrates that healthcare professionals (HCPs) should be trained and equipped with the knowledge and skills to communicate effectively (i.e. in easy-to-understand words, emphatically, checking whether the patient understands everything, listening attentively) tailored to the wishes, needs and possibilities of the patient, which may lead to higher satisfaction. This way the patient will be more involved in the care process and in the shared decision-making process, which may result in improved concordance, and an improved treatment approach. A respectful and empathic attitude of the HCP plays an important role in establishing a strong therapeutic relationship and improved health (system) outcomes. Together with a good accessibility of care for patients, setting up a personalised care plan with all involved parties may positively affect the self-management skills of patients. Good collaboration within the team and between different domains is desirable to ensure good care coordination. The coherence of items related to PCC in primary care should be considered to better understand its effectiveness.

What are the mechanisms and contexts by which care groups achieve social and behavioural change in low- and middle-income countries? Group motivation findings from a realist synthesis

OBJECTIVE

Infant and under-five mortality rates in low- and middle-income countries (LMIC) can be reduced by encouraging behaviours such as sleeping under insecticide-treated bed nets, exclusive breast-feeding for the first 6 months, regular handwashing, etc. Community-based volunteer or peer-to-peer mechanisms are cost-effective ways of promoting these lifesaving practices. However, the sustainability and reach of community-based behaviour change promotion remains a challenge. Our inquiry focuses on the utilisation, by non-governmental organisations (NGO), of Care Groups, a peer-to-peer behaviour change intervention. We asked: What are the mechanisms and contexts by which Care Groups achieve social and behavioural change in nutrition, health and other sectors?

DESIGN

Realist synthesis reviewing forty-two texts that contained empirical evidence about Care Group interventions.

SETTING

LMIC.

PARTICIPANTS

We held consultations with a research reference group, which included Care Group and nutrition experts, and Care Group - implementing NGO staff in Malawi.

RESULTS

Different types of motivation drive the establishment and the sustainability of peer group interventions. A certain amount of motivation was derived from the resources provided by the NGO establishing the Care Groups. Subsequently, both volunteers and neighbourhood group members were motivated by the group dynamics and mutual support, as well as support from the wider community. Finally, volunteers and group members alike became self-motivated by their experience of being involved in group activities.

CONCLUSIONS

When designing and implementing community-based behaviour change interventions, awareness of the multi-directional nature of the motivating drivers that are experienced by peer- or community group members is important, to optimise these groups' reach and sustainability.

Understanding the maternal and child health system response to payment for performance in Tanzania using a causal loop diagram approach

Payment for performance (P4P) has been employed in low and middle-income (LMIC) countries to improve quality and coverage of maternal and child health (MCH) services. However, there is a lack of consensus on how P4P affects health systems. There is a need to evaluate P4P effects on health systems using methods suitable for evaluating complex systems. We developed a causal loop diagram (CLD) to further understand the pathways to impact of P4P on delivery and uptake of MCH services in Tanzania. The CLD was developed and validated using qualitative data from a process evaluation of a P4P scheme in Tanzania, with additional stakeholder dialogue sought to strengthen confidence in the diagram. The CLD maps the interacting mechanisms involved in provider achievement of targets, reporting of health information, and population care seeking, and identifies those mechanisms affected by P4P. For example, the availability of drugs and medical commodities impacts not only provider achievement of P4P targets but also demand of services and is impacted by P4P through the availability of additional facility resources and the incentivisation of district managers to reduce drug stock outs. The CLD also identifies mechanisms key to facility achievement of targets but are not within the scope of the programme; the activities of health facility governing committees and community health workers, for example, are key to demand stimulation and effective resource use at the facility level but both groups were omitted from the incentive system. P4P design considerations generated from this work include appropriately incentivising the availability of drugs and staffing in facilities and those responsible for demand creation in communities. Further research using CLDs to study heath systems in LMIC is urgently needed to further our understanding of how systems respond to interventions and how to strengthen systems to deliver better coverage and quality of care.

•

Holistic analysis key to avoiding suboptimal P4P performance.

•

Availability of drugs is influenced by P4P and affects success of P4P.

•

Need to incentivise groups outside facility that support service coverage.