Evaluating the implementation of a quality improvement process in General Practice using a realist evaluation framework

Realist Evaluation of the "Abiye" Safe Motherhood Initiative in Nigeria: Unveiling the Black-Box of Program Implementation and Health System Strengthening

Introduction

Realist evaluation studies have spanned different aspects of medicine, especially in the field of public health. However, very few of these studies explicitly detailed how program implementation triggered outcomes that could strengthen understanding of its effect on Health System Strengthening in specific settings. In low- and middle-income countries, like Nigeria, there is a paucity of realist evaluation studies, despite the implementation of multiple intervention programs and projects in these countries. This article is aimed at unveiling the black-box of program implementation and Health System Strengthening of the "Abiye" Safe Motherhood Program in Ondo State, Nigeria. Specifically, it identified the role of contextual factors in the "Abiye" program in Ondo State, determined the mechanisms that facilitated or constrained outcomes of the "Abiye" program, and developed a Context Mechanism Outcome (CMO) Configuration from which a Middle Range Theory (MRT) can be framed.

Methodology

This was qualitative research structured along with the realist domains (Context, Mechanism, and Outcome). The Initial Program Theory was validated by the qualitative study, after which a new MRT was developed. The study population comprised key stakeholders, secondary stakeholders, and primary stakeholders in the Abiye safe motherhood program. Data was collected through 10 key informant interviews, 28 in-depth interviews, and six focus group discussions sessions. Thematic analysis was used to analyze all the qualitative data collected, and seven themes with 19 subthemes emerged in the study.

Results

We identified 13 contextual factors under five principal areas, with most of the factors playing enabling roles, some playing inhibitory roles, while very few played both roles. We elicited eight mechanisms, and some of these facilitated the outcomes, while some constrained the outcomes of the program. Health system strengthening was a key feature of the outcome of the program. We developed a middle-range theory based on the 6 CMO configurations we elicited from the study.

Conclusion and Policy Implications

Realist evaluation is an iterative process that looks beyond the surface to generate evidence. By applying the realist approach, we generated pieces of evidence that can be adapted for policymaking in public health interventions in LMIC.

Uptake of a primary care atrial fibrillation screening program (AF-SMART): a realist evaluation of implementation in metropolitan and rural general practice

Background

Screening for atrial fibrillation (AF) in people aged ≥65 years is recommended by international guidelines. The Atrial Fibrillation Screen, Management And guideline-Recommended Therapy (AF-SMART) studies of opportunistic AF screening in 16 metropolitan and rural general practices were conducted from November 2016–June 2019. These studies trialled custom-designed eHealth tools to support all stages of AF screening in general practice.

Methods

A realist evaluation of the AF-SMART studies, which aimed to explain the circumstances in which the program worked (or not) to increase the proportion of people screened for AF. The initial program theory was based on our previous research, policy documents and screening studies. To test this, we conducted 45 semi-structured interviews with general practitioners (GPs), nurses and practice managers across all participating practices, and collected observational and quantitative screening data. These data were analysed and interpreted to refine the program theory.

Results

GPs/nurses liked the eHealth tools, although technical problems sometimes disrupted screening. Time was the main barrier to screening for GPs/nurses, so systems need to be very efficient. Practices with leadership from a senior GP ‘screening champion’ had broader uptake, especially from the nursing team. Providing regular feedback on screening data was beneficial for quality improvement and motivation. Clear protocols for follow-up of abnormal results were required for successful nurse-led screening in a hierarchical system. Participation in the program had broader benefits of improving AF knowledge and raising the profile of cardiovascular health in the practice. Screening for a shorter, more intense period (eg during influenza vaccination) worked well for practices where sufficient staff time was allocated.

Conclusions

Introducing an AF screening program is likely to be successful in contexts where there is a senior GP ‘screening champion’, a clear protocol exists for abnormal results, and there is regular data reporting to staff. These contexts link to mechanisms around motivation, leadership, empowerment of nurses, and efficient screening systems. The contexts and mechanisms contribute to the longer-term outcomes of increasing the proportion of people screened and treated for AF, which is recommended by guidelines as a key strategy for the prevention of AF-related stroke.

Trial registrations

AF SMART (metropolitan): ACTRN12616000850471 (Australia New Zealand Clinical Trials Registry).

AF SMART II (rural): ACTRN12618000004268 (Australia New Zealand Clinical Trials Registry).

Pragmatic trials and implementation science: grounds for divorce?

Background

The paper opens with a brief history of two of the major intellectual components of the recent utilitarian turn in clinical research, namely ‘pragmatic trials’ and ‘implementation science’. The two schools of thought developed independently and the paper scrutinises their mutual compatibilities and incompatibilities, asking: i) what do the leading advocates of pragmatic trials assume about the transfer of research findings to real-world practice and ii) what role pragmatic trials can and should play in the evaluation of implementation science strategies.

Methods

The paper utilises ‘explication de texte’: i) providing a close reading of the inferential logics contained in major published expositions of the two paradigms, and ii) interrogating the conclusions of a pragmatic trial of an intervention providing guidelines on retinal screening aimed at family practitioners.

Results

The paper is in two parts. Part 1 unearths some significant incommensurability – the pragmatic trial literature retains an antiquated view of knowledge transfer and is overly optimistic about the wide applicability the findings of pragmatic trials to ‘real world’ conditions. Part 2 of the paper outlines an empirical strategy to better penetrate the mechanisms of knowledge transfer and to tackle the issue of the generalisabilty of research findings in implementation science.

Conclusions

Pragmatism, classically, is about problem solving and the melding of perspectives. The core research requirement in implementation science is a fundamental shift from the narrow shoulders of pragmatic trials to a model of explanation building based upon a multi-case, multi-method body of evidence.