Process evaluation in the field: global learnings from seven implementation research hypertension projects in low-and middle-income countries

A guide to outcome evaluation of simulation-based education programmes in low and middle-income countries

Evaluation is a vital part of any learning activity and is essential to optimize and improve educational programmes. It should be considered and prioritized prior to the implementation of any learning activity. However, comprehensive programme evaluation is rarely conducted, and there are numerous barriers to high-quality evaluation. This review provides a framework for conducting outcome evaluation of simulation-based education programmes in low and middle-income countries (LMICs). The basis of evaluation, including core ideas of theory, purpose and structure are outlined, followed by an examination of the levels and healthcare applications of the Kirkpatrick model of evaluation. Then, methods of conducting evaluation of simulation-based education in LMICs are discussed through the lens of a successful surgical simulation programme in Myanmar, a lower-middle-income country. The programme involved the evaluation of 11 courses over 4 years in Myanmar and demonstrated evaluation at the highest level of the Kirkpatrick model. Reviewing this programme provides a bridge between evaluation theory and practical implementation. A range of evaluation methods are outlined, including surveys, interviews, and clinical outcome measurement. The importance of a mixed-methods approach, enabling triangulation of quantitative and qualitative analysis, is highlighted, as are methods of analysing data, including statistical and thematic analysis. Finally, issues and challenges of conducting evaluation are considered, as well as strategies to overcome these barriers. Ultimately, this review informs readers about evaluation theory and methods, grounded in a practical application, to enable other educators in low-resource settings to evaluate their own activities.

Group-based Education and monitoring program delivered by community health workers to improve control of high blood pressure in island districts of lake victoria, Uganda

BACKGROUND

Individuals living in communities with poor access to healthcare may be unaware of their high blood pressure (BP). While the use of community health workers (CHWs) can address gaps in human resources for health, CHWs in Uganda have not been used previously for BP screening and management. We report the results of an initiative to train CHWs to evaluate BP and to administer group-based education in Kalangala and Buvuma Island Districts of Lake Victoria, Uganda.

METHODS

We randomly selected 42 of 212 villages. We trained CHWs based in island districts on measuring BP. CHWs visited all households in the selected villages and invited all adults ≥ 18 years to be screened for high BP. We used the World Health Organization's STEPwise tool to collect data on demographic and behavioral characteristics and BP measurements. High blood pressure was defined as systolic BP (SBP) ≥ 140 mm Hg and/or diastolic BP (DBP) ≥ 90 mm Hg over three readings. CHWs created and led fortnight support groups for individuals identified with high blood pressure at baseline. At each group meeting, CHWs re-measured BP and administered an intervention package, which included self-management and lifestyle education to participants. The paired t-test was used to compare mean values of systolic blood pressure (SBP) and diastolic blood pressure (DBP) before and after the intervention. Generalized estimating equations (GEE) were used to model longitudinal changes in BP.

RESULTS

We trained 84 CHWs to measure BP and deliver the intervention package. Among 2,016 community members, 570 (28.3%) had high blood pressure; of these, 63 (11.1%) had a previous diagnosis of hypertension. The comparison of SBP and DBP before and after the intervention revealed significant reductions in mean SBP from 158mmHg (SD = 29.8) to 149 mmHg (SD = 29.8) (p < 0.001) and mean DBP from 97mmHg (SD = 14.3) to 92mmHg (p < 0.001). GEE showed decreases of -1.133 (SBP) and - 0.543 mmHg (DBP)/fortnight.

CONCLUSION

High BP was common but previously undiagnosed. The CHW-led group-based self-management and education for controlling high BP was effective in the island districts in Uganda. Scaling up the intervention in other hard-to-reach districts could improve control of high BP on a large scale.

Supported exercise TrAining for Men wIth prostate caNcer on Androgen deprivation therapy (STAMINA): study protocol for a randomised controlled trial of the clinical and cost-effectiveness of the STAMINA lifestyle intervention compared with optimised usual care, including internal pilot and parallel process evaluation

BACKGROUND

UK national clinical guidance recommends that men with prostate cancer on androgen deprivation therapy are offered twice weekly supervised aerobic and resistance exercise to address iatrogenic harm caused by treatment. Very few NHS trusts have established adequate provision of such services. Furthermore, interventions fail to demonstrate sustained behaviour change. The STAMINA lifestyle intervention offers a system-level change to clinical care delivery addressing barriers to long-term behaviour change and implementation of new prostate cancer care pathways. This trial aims to establish whether STAMINA is clinically and cost-effective in improving cancer-specific quality of life and/or reducing fatigue compared to optimised usual care. The process evaluation aims to inform the interpretation of results and, if the intervention is shown to benefit patients, to inform the implementation of the intervention into the NHS.

METHODS

Men with prostate cancer on androgen deprivation therapy (n = 697) will be identified from a minimum of 12 UK NHS trusts to participate in a multi-centre, two-arm, individually randomised controlled trial. Consenting men will have a 'safety to exercise' check and be randomly allocated (5:4) to the STAMINA lifestyle intervention (n = 384) or optimised usual care (n = 313). Outcomes will be collected at baseline, 3-, 6- and 12-month post-randomisation. The two primary outcomes are cancer-specific quality of life and fatigue. The parallel process evaluation will follow a mixed-methods approach to explore recruitment and aspects of the intervention including, reach, fidelity, acceptability, and implementation. An economic evaluation will estimate the cost-effectiveness of the STAMINA lifestyle intervention versus optimised usual care and a discrete choice experiment will explore patient preferences.

DISCUSSION

The STAMINA lifestyle intervention has the potential to improve quality of life and reduce fatigue in men on androgen deprivation therapy for prostate cancer. Embedding supervised exercise into prostate cancer care may also support long-term positive behaviour change and reduce adverse events caused by treatment. Findings will inform future clinical care and could provide a blueprint for the integration of supervised exercise and behavioural support into other cancer and/or clinical services.

TRIAL REGISTRATION

ISRCTN 46385239, registered on 30/07/2020. Cancer Research UK 17002, retrospectively registered on 24/08/2022.

Implementing and evaluating integrated care models for non-communicable diseases in fragile and humanitarian settings

In this commentary, we advocate for the wider implementation of integrated care models for NCDs within humanitarian preparedness, response, and resilience efforts. Since experience and evidence on integrated NCD care in humanitarian settings is limited, we discuss potential benefits, key lessons learned from other settings, and lessons from the integration of other conditions that may be useful for stakeholders considering an integrated model of NCD care. We also introduce our ongoing project in North Lebanon as a case example currently undergoing parallel tracks of program implementation and process evaluation that aims to strengthen the evidence base on implementing an integrated NCD care model in a crisis setting.

Using a theoretically informed process evaluation alongside a trial to improve oral health for care home residents

BACKGROUND

Poor oral health is common among older adults residing in care homes impacting their diet, quality of life, self-esteem, general health and well-being. The care home setting is complex and many factors may affect the successful implementation of oral care interventions. Exploring these factors and their embedded context is key to understanding how and why interventions may or may not be successfully implemented within their intended setting.

OBJECTIVES

This methodology paper describes the approach to a theoretically informed process evaluation alongside a pragmatic randomised controlled trial, so as to understand contextual factors, how the intervention was implemented and important elements that may influence the pathways to impact.

MATERIALS AND METHODS

SENIOR is a pragmatic randomised controlled trial designed to improve the oral health of care home residents in the United Kingdom. The trial uses a complex intervention to promote and provide oral care for residents, including education and training for staff.

RESULTS

An embedded, theoretically informed process evaluation, drawing on the PAHRIS framework and utilising a qualitative approach, will help to understand the important contextual factors within the care home that influence both the trial processes and the implementation of the intervention.

CONCLUSION

Utilising an implementation framework as the basis for a theoretically informed process evaluation provides an approach that specifically focuses on the contextual factors that may influence and shape the pathways to impact a given complex intervention a priori, while also providing an understanding of how and why an intervention may be effective. This contrasts with the more common post hoc approach that only focuses on implementation after the empirical results have emerged.

Implementing neurodevelopmental follow-up care for children with congenital heart disease: A scoping review with evidence mapping

AIM

To identify and map evidence describing components of neurodevelopmental follow-up care for children with congenital heart disease (CHD).

METHOD

This was a scoping review of studies reporting components of neurodevelopmental follow-up programmes/pathways for children with CHD. Eligible publications were identified through database searches, citation tracking, and expert recommendations. Two independent reviewers screened studies and extracted data. An evidence matrix was developed to visualize common characteristics of care pathways. Qualitative content analysis identified implementation barriers and enablers.

RESULTS

The review included 33 studies. Twenty-one described individual care pathways across the USA (n = 14), Canada (n = 4), Australia (n = 2), and France (n = 1). The remainder reported surveys of clinical practice across multiple geographical regions. While heterogeneity in care existed across studies, common attributes included enrolment of children at high-risk of neurodevelopmental delay; centralized clinics in children's hospitals; referral before discharge; periodic follow-up at fixed ages; standardized developmental assessment; and involvement of multidisciplinary teams. Implementation barriers included service cost/resourcing, patient burden, and lack of knowledge/awareness. Multi-level stakeholder engagement and integration with other services were key drivers of success.

INTERPRETATION

Defining components of effective neurodevelopmental follow-up programmes and care pathways, along with enhancing and expanding guideline-based care across regions and into new contexts, should continue to be priorities.

WHAT THIS PAPER ADDS

Twenty-two different neurodevelopmental follow-up care pathways/programmes were published, originating from four countries. Twelve additional publications described broad practices for neurodevelopmental follow-up across regions Common attributes across eligibility, service structure, assessment processes, and care providers were noted. Studies reported programme acceptability, uptake, cost, and effectiveness. Implementation barriers included service cost/resourcing, patient burden, and lack of knowledge/awareness.

The use of implementation science to close the research-to-treatment gap for cognitive impairment in psychosis

For people living with psychosis, cognitive impairment is common and can have significant impacts for functional recovery, impacting engagement with treatment and quality of life more broadly. There is now strong evidence for the effectiveness of cognition-focused treatments, such as cognitive remediation to improve clinical and functional outcomes for people with psychosis. However, engagement with treatment has been a long-standing issue in mental health care, including for people with psychosis, who often experience difficulties with motivation. While research on clinical effectiveness of cognition-focused treatment is growing, to date there has been little research focused on the implementation of such treatments and it is not clear how best to support uptake and engagement across diverse mental health settings. Implementation science is the study of methods and strategies to promote the adoption, application, and maintenance of evidence-based practices in routine care. To integrate cognition-focused treatments into routine practice, and improve engagement with treatment and the quality and effectiveness of care for people with psychosis, researchers need to embrace implementation science and research. This paper provides a succinct overview of the field of implementation science, current evidence for implementation of cognition-focused treatments for psychosis and practical guidance for using implementation science in clinical research. The future of psychosis research includes multidisciplinary teams of clinical researchers and implementation scientists, working together with providers and consumers to build the evidence that can improve the implementation of cognition-focused treatments.

What do we want to get out of this? a critical interpretive synthesis of the value of process evaluations, with a practical planning framework

BACKGROUND

Process evaluations aim to understand how complex interventions bring about outcomes by examining intervention mechanisms, implementation, and context. While much attention has been paid to the methodology of process evaluations in health research, the value of process evaluations has received less critical attention. We aimed to unpack how value is conceptualised in process evaluations by identifying and critically analysing 1) how process evaluations may create value and 2) what kind of value they may create.

METHODS

We systematically searched for and identified published literature on process evaluation, including guidance, opinion pieces, primary research, reviews, and discussion of methodological and practical issues. We conducted a critical interpretive synthesis and developed a practical planning framework.

RESULTS

We identified and included 147 literature items. From these we determined three ways in which process evaluations may create value or negative consequences: 1) through the socio-technical processes of 'doing' the process evaluation, 2) through the features/qualities of process evaluation knowledge, and 3) through using process evaluation knowledge. We identified 15 value themes. We also found that value varies according to the characteristics of individual process evaluations, and is subjective and context dependent.

CONCLUSION

The concept of value in process evaluations is complex and multi-faceted. Stakeholders in different contexts may have very different expectations of process evaluations and the value that can and should be obtained from them. We propose a planning framework to support an open and transparent process to plan and create value from process evaluations and negotiate trade-offs. This will support the development of joint solutions and, ultimately, generate more value from process evaluations to all.

Research protocol for impact assessment of a project to scale up food policies in the Pacific

Background

One of the challenges for countries implementing food policy measures has been the difficulty in demonstrating impact and retaining stakeholder support. Consequently, research funded to help countries overcome these challenges should assess impact and translation into practice, particularly in low-resource settings. However, there are still few attempts to prospectively, and comprehensively, assess research impact. This protocol describes a study co-created with project implementers, collaborative investigators and key stakeholders to optimize and monitor the impact of a research project on scaling up food policies in Fiji.

Methods

To develop this protocol, our team of researchers prospectively applied the Framework to Assess the Impact from Translational health research (FAIT). Activities included (i) developing a logic model to map the pathway to impact and establish domains of benefit; (ii) identifying process and impact indicators for each of these domains; (iii) identifying relevant data for impact indicators and a cost–consequence analysis; and (iv) establishing a process for collecting quantitative and qualitative data to measure progress. Impact assessment data will be collected between September 2022 and December 2024, through reports, routine monitoring activities, group discussions and semi-structured interviews with key implementers and stakeholders. The prospective application of the protocol, and interim and final research impact assessments of each project stream and the project as a whole, will optimize and enable robust measurement of research impact.

Discussion

By applying this protocol, we aim to increase understanding of pathways to impact and processes that need to be put in place to achieve this. This impact evaluation will inform future projects with a similar scope and will identify transferable and/or translatable lessons for other Pacific Island states and low- and middle-income countries.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12961-022-00927-x.

Process evaluation of a pragmatic implementation trial to support self-management for the prevention and management of type 2 diabetes in Uganda, South Africa and Sweden in the SMART2D project

INTRODUCTION

Type 2 diabetes (T2D) and its complications are increasing rapidly. Support for healthy lifestyle and self-management is paramount, but not adequately implemented in health systems. Process evaluations facilitate understanding why and how interventions work through analyzing the interaction between intervention theory, implementation and context. The Self-Management and Reciprocal Learning for Type 2 Diabetes project implemented and evaluated community-based interventions (peer support program; care companion; and link between facility care and community support) for persons at high risk of or having T2D in a rural community in Uganda, an urban township in South Africa, and socioeconomically disadvantaged urban communities in Sweden.

RESEARCH DESIGN AND METHODS

This paper reports implementation process outcomes across the three sites, guided by the Medical Research Council framework for complex intervention process evaluations. Data were collected through observations of peer support group meetings using a structured guide, and semistructured interviews with project managers, implementers, and participants.

RESULTS

The countries aligned implementation in accordance with the feasibility and relevance in the local context. In Uganda and Sweden, the implementation focused on peer support; in South Africa, it focused on the care companion part. The community-facility link received the least attention. Continuous capacity building received a lot of attention, but intervention reach, dose delivered, and fidelity varied substantially. Intervention-related and context-related barriers affected participation.

CONCLUSIONS

Identification of the key uncertainties and conditions facilitates focus and efficient use of resources in process evaluations, and context relevant findings. The use of an overarching framework allows to collect cross-contextual evidence and flexibility in evaluation design to adapt to the complex nature of the intervention. When designing interventions, it is crucial to consider aspects of the implementing organization or structure, its absorptive capacity, and to thoroughly assess and discuss implementation feasibility, capacity and organizational context with the implementation team and recipients. These recommendations are important for implementation and scale-up of complex interventions.

TRIAL REGISTRATION NUMBER

ISRCTN11913581.

Factors Influencing the Implementation of Remote Delivery Strategies for Non-Communicable Disease Care in Low- and Middle-Income Countries: A Narrative Review

Objectives: The COVID-19 pandemic has disrupted health care for non-communicable diseases (NCDs) and necessitated strategies to minimize contact with facilities. We aimed to examine factors influencing implementation of remote (non-facility-based) delivery approaches for people with hypertension and/or diabetes in low- and middle-income countries (LMICs), to inform NCD care delivery during health service disruption, including humanitarian crises.

Methods: Our narrative review used a hermeneutic and purposive approach, including primary studies conducted in LMICs, which assessed implementation factors influencing remote NCD care delivery. Results were analyzed using the Consolidated Framework for Implementation Research.

Results: Twenty-eight included studies revealed the strong influence of both internal organizational and broader contextual factors, such as community health worker policies or technological environment. Addressing patients’ specific characteristics, needs and resources was important for implementation success.

Conclusion: This review highlighted the multiple, complex, interdependent factors influencing implementation of remote NCD care in LMICs. Our findings may inform actors designing NCD care delivery in contexts where facility-based access is challenging. Implementation research is needed to evaluate context-adapted e-Health, community-based, and simplified clinical management strategies to facilitate remote NCD care.

Factors influencing the implementation of TB screening among PLHIV in selected HIV clinics in Ghana: a qualitative study

Background

Decreasing the burden of Tuberculosis (TB) among PLHIV through TB screening is an effective intervention recommended by the World Health Organization (WHO). However, after over a decade of implementation in Ghana, the intervention does not realize the expected outcomes. It is also not well understood whether this lack of success is due to implementation barriers. Our study, therefore, sought to examine the factors influencing the implementation of the intervention among people living with HIV (PLHIV) attending HIV clinics at district hospitals in Ghana.

Methods

This was a qualitative study conducted from 6th to 31 May 2019 in three regions of Ghana. We conducted 17 in-depth interviews (IDIs – comprising two regional, six districts and nine facility TB/HIV coordinators) and eight focus group discussions (FGD – consisting of a total of 65 participants) with HIV care providers. The Consolidated Framework for Implementation Research (CFIR) guided the design of interview guides, data collection and analysis. All responses were digitally audio-recorded and transcribed verbatim for coding and analysis using the Framework Approach. Participants consented to the interview and recording.

Results

The main barriers to TB screening relate to the low commitment of the implementers to screen for TB and limited facility infrastructure for the screening activities. Facilitators of TB screening include (1) ease in TB screening, (2) good communication and referral channels, (3) effective goals and feedback mechanisms, (4) health workers recognizing the need for the intervention and (5) the role of chemical sellers.

Conclusions

Key barriers and facilitators to the intervention are revealed. The study has shown that there is a need to increase HIV care providers and institutional commitment towards TB screening interventions. In addition, cost issues need to be assessed as they are drivers of sustainability. Our study also advances the field of implementation science through CFIR to better understand the factors influencing the implementation.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-022-08295-6.

Process Evaluation of an Implementation Trial: Design, Rationale, and Early Lessons Learnt From an International Cluster Clinical Trial in Intracerebral Hemorrhage

Background

The third INTEnsive care bundle with blood pressure Reduction in Acute Cerebral Hemorrhage Trial (INTERACT3) is an ongoing, international, multicenter, stepped-wedge cluster, prospective, randomized, open, blinded endpoint assessed trial evaluating the effectiveness of a quality improvement “care bundle” for the management of patients with acute spontaneous intracerebral hemorrhage (ICH) in low- and middle-income countries (LMICs). An embedded process evaluation aims to explore the uptake and implementation of the intervention, and understand the context and stakeholder perspectives, for interpreting the trial outcomes.

Methodology

The design was informed by Normalization Process Theory and the UK Medical Research Council process evaluation guidance. Mixed methods are used to evaluate the implementation outcomes of fidelity, reach, dose, acceptability, appropriateness, adoption, sustainability, and relevant contextual factors and mechanisms affecting delivery of the care bundle. Semi-structured interviews and non-participant observations are conducted with the primary implementers (physicians and nurses) and patients/carers to explore how the care bundle was integrated into routine care. Focus group discussions are conducted with investigators and project operational staff to understand challenges and possible solutions in the organization of the trial. Data from observational records, surveys, routine monitoring data, field notes and case report forms, inform contextual factors, and adoption of the intervention. Purposive sampling of sites according to pre-specified criteria is used to achieve sample representativeness.

Discussion

Implementation outcomes, and relevant barriers and facilitators to integrating the care bundle into routine practice, will be reported after completion of the process evaluation. The embedded process evaluation will aid understanding of the causal mechanisms between care bundle elements and clinical outcomes within complex health systems across diverse LMIC settings.

Trial Registration

The INTERACT3 study is registered at ClinicalTrials.gov (NCT03209258).

Evaluation of Electronic Service-Learning (e-Service-Learning) Projects in Mainland China under COVID-19

The use of electronic service-learning (e-Service-Learning or e-SL) is valuable under COVID-19 because we can provide the service without physical contact. Unfortunately, evaluation of e-SL is not widespread and there is no known study in different Chinese societies. Besides, there are many methodological limitations of the existing studies in the field. In this paper, we evaluated e-SL projects implemented in summer 2020 and 2021 in three sites in China. First, we examined service providers' changes based on pretest and posttest scores (i.e., objective outcome evaluation) and their perceptions of the e-SL projects (i.e., subjective outcome evaluation based on the service providers). Second, graduate student assessors in Chinese mainland universities and teachers of primary school students (i.e., service recipients) rated the SL program quality, service providers' performance and benefits to the service recipients after program completion (i.e., subjective outcome evaluation of SL projects based on other stakeholders). Third, trained graduate student assessors evaluated service quality during the implementation process (i.e., process evaluation). We found that university students (i.e., service providers) showed higher posttest scores in positive youth development attributes, leadership attributes and life satisfaction relative to pretest scores. Besides, service providers showed positive perceptions of their learning experience, own performance, benefits to the service recipients and themselves in the SL projects. Similarly, other stakeholders also had positive evaluation of the SL projects and related benefits. Finally, trained graduate student assessors had positive assessment of the quality of program implementation. The findings underscore the utility of e-SL involving both online teaching and learning as well as online service, particularly in a Chinese context.

A Protocol on Using the RE-AIM Framework in the Process Evaluation of the Primary Health Integrated Care Project for Four Chronic Conditions in Kenya

Background: There has been a rapid increase in morbidity and mortality arising from non-communicable diseases (NCDs). The Academic Model Providing Access to Healthcare (AMPATH) program has established a chronic disease management program in collaboration with the Ministry of Health (MoH) in Kenya at over 150 health facilities in western Kenya. The primary health integrated care for chronic (PIC4C) disease project seeks to deliver preventive, promotive, and curative care for diabetes, hypertension, cervical and breast cancers at the primary health care level. We apply the RE-AIM framework to conduct a process evaluation of the integrated PIC4C model. This paper describes the protocol we are using in the PIC4C process evaluation planning and activities.

Methods and Analysis: This evaluation utilizes clinic reports as well as primary data collected in two waves. Using mixed methods (secondary data, observation, semi-structured interviews, and focus group discussions), the process evaluation assesses the reach, effectiveness, adoption, implementation and maintenance of the PIC4C model in Busia and Trans Nzoia Kenya. The evaluation captures the PIC4C process, experiences of implementers and users, and the wishes of those using the PIC4C services. We will analyse our data across the RE-AIM dimensions using descriptive statistics and two-sample t-test to compare the mean scores for baseline and end line. Qualitative data will be analyzed thematically.

Discussion: The process evaluation of the PIC4C model in Kenya allows implementers and users to reflect and question its implementation, uptake and maintenance. Our experiences thus far suggest practicable strategies to facilitate primary health care can benefit extensively from deliberate process evaluation of the programs undertaken. Furthermore, integrating the RE-AIM framework in the process evaluation of health programs is valuable due to its pragmatic and reporting usefulness.

Is the Parwarish parenting intervention feasible and relevant for young people and parents in diverse settings in India? A mixed methods process evaluation

OBJECTIVE

To assess the feasibility, acceptability and relevance of the Parwarish, a positive parenting intervention (adapted from PLH-Teens) in three diverse settings in India.

DESIGN

This mixed methods study used the Medical Research Council framework for process evaluations of complex interventions.

SETTING

This study was set in disadvantaged communities in urban Agra, rural Uttar Pradesh and tribal Jharkhand in India.

PARTICIPANTS

Data were collected from 86 facilitators, implementers, parents and teens who participated in the Parwarish intervention among 239 families.

INTERVENTION

Couples from target communities facilitated groups of parents and teens over the 14-module structured, interactive Parwarish intervention which focused on building communication, reducing harsh parenting and building family budgeting skills.

OUTCOME MEASURES

We assessed relevance, acceptability and feasibility of the intervention using mixed methods. Qualitative data collected included semistructured interviews and focus group discussions with implementers, facilitators, parents and young people who were transcribed, translated and thematically analysed to develop themes inherent in the data. Quantitative data which assessed attendance, fidelity to the intervention and facilitator training and coaching were analysed descriptively.

RESULTS

Findings were grouped under the three domains of facilitation, community engagement and programme support with the following seven themes: (1) community-based facilitators increased contextual validity of the intervention; (2) gender relations were not only influenced by Parwarish implementation but were also influenced and transformed by Parwarish; (3) facilitator responsiveness to group concerns increased participation; (4) participation gathered momentum; (5) Parwarish's strong core and porous periphery allowed adaptations to local contexts; (6) technology that included Skype and WhatsApp enhanced implementation and (7) critical reflection with community trained coaches strengthened facilitation quality and programme fidelity.

CONCLUSION

This study found Parwarish engaging, feasible and acceptable in three diverse, low-income communities, although constrained by patriarchal gender relations. It paves the way for larger-scale implementation in other South Asian settings.

Process evaluations undertaken alongside randomised controlled trials in the hospital setting: A scoping review

Background

There is increasing recognition of the importance of undertaking process evaluations alongside implementation of health interventions by examining mechanisms of impact and contextual factors. However, a comprehensive synthesis of process evaluations undertaken alongside clinical trials in hospital settings is lacking. We undertook a scoping review to address this gap.

Methods

This review was guided by the methodological framework for scoping studies. Studies were identified using four databases; Ovid Medline, EBSCO CINAHL, EMBASE and Scopus. Two authors independently screened all titles and available abstracts, with a third author available to adjudicate. Studies were eligible for inclusion if they described a process evaluation undertaken alongside a randomised controlled trial in the hospital setting. Data were abstracted by one author and checked by two others and analysed both descriptively and using inductive content analysis.

Results

Data were extracted from 30 articles reporting on 15 trials, most of which were cluster randomised trials (c-RTs) (n = 12). The most common data collection methods used in process evaluations were interviews, questionnaires or surveys, and records or logs. Data analysis revealed three themes relative to how authors: use process data to interpret, understand and explain trial outcomes; evaluate responses to the intervention; and consider the implementation context.

Conclusions

Findings from this review demonstrate the complex nature of intervention implementation in the hospital setting. Overall, there is need for standardised reporting of process evaluations and more explicit descriptions of how authors use frameworks to guide their evaluation.

[Kinaesthetics competence in specialized palliative care: Process evaluation of a quasi-experimental study]

Kinaesthetics Competence in Specialized Palliative Care: Process Evaluation of a Quasi-Experimental Study Abstract. Background: To ensure professional symptom control in the context of palliative care, specific nursing skills are required. "Advanced Kinaesthetics in Palliative Care" (AdKinPal) is an education program intending to promote nurses' kinaesthetics competence in order to improve symptom management by means of nursing interventions. Aim: The aim of this article is to illuminate the process of the AdKinPal study in order to check the program's feasibility and to determine possible outcomes. Results may serve as a basis for revising the program with regard to further implementation. Methods: The process evaluation was part of a quasi-experimental study. We collected qualitative and quantitative data (interviews, observation protocols, questionnaire). For data interpretation, we used qualitative content analysis and descriptive quantitative analysis. Results: Overall, the participants valued the workshops and practical support as beneficial. Nevertheless, there were also inhibiting factors, such as workshop scheduling and location as well as working material. The participants questioned sustainable skills development. Conclusions: The first run of the education program was successful. To ensure sustainable competence development, adjustments are necessary for further implementation.

The Implementation of a Primary Care-Based Integrated Mobile Health Intervention for Stroke Management in Rural China: Mixed-Methods Process Evaluation

Background: There is a lack of evidence concerning the effective implementation of strategies for stroke prevention and management, particularly in resource-limited settings. A primary-care-based integrated mobile health intervention (SINEMA intervention) has been implemented and evaluated via a 1-year-long cluster-randomized controlled trial. This study reports the findings from the trial implementation and process evaluation that investigate the implementation of the intervention and inform factors that may influence the wider implementation of the intervention in the future.

Methods: We developed an evaluation framework by employing both the RE-AIM framework and the MRC process evaluation framework to describe the implementation indicators, related enablers and barriers, and illustrate some potential impact pathways that may influence the effectiveness of the intervention in the trial. Quantitative data were collected from surveys and extracted from digital health monitoring systems. In addition, we conducted quarterly in-depth interviews with stakeholders in order to understand barriers and enablers of program implementation and effectiveness. Quantitative data analysis and thematic qualitative data analysis were applied, and the findings were synthesized based on the evaluation framework.

Results: The SINEMA intervention was successfully implemented in 25 rural villages, reached 637 patients with stroke in rural Northern China during the 12 months of the trial. Almost 90% of the participants received all follow-up visits per protocol, and about half of the participants received daily voice messages. The majority of the intervention components were adopted by village doctors with some adaptation made. The interaction between human-delivered and technology-enabled components reinforced the program implementation and effectiveness. However, characteristics of the participants, doctor-patient relationships, and the healthcare system context attributed to the variation of program implementation and effectiveness.

Conclusion: A comprehensive evaluation of program implementation demonstrates that the SINEMA program was well implemented in rural China. Findings from this research provide additional information for program adaptation, which shed light on the future program scale-up. The study also demonstrates the feasibility of combining RE-AIM and MRC process evaluation frameworks in process and implementation evaluation in trials.

Clinical Trial Registration:www.ClinicalTrials.gov, identifier: NCT03185858.

ASHA-Led Community-Based Groups to Support Control of Hypertension in Rural India Are Feasible and Potentially Scalable

Background: To improve the control of hypertension in low- and middle-income countries, we trialed a community-based group program co-designed with local policy makers to fit within the framework of India's health system. Trained accredited social health activists (ASHAs), delivered the program, in three economically and developmentally diverse settings in rural India. We evaluated the program's implementation and scalability. Methods: Our mixed methods process evaluation was guided by the United Kingdom Medical Research Council guidelines for complex interventions. Meeting attendance reports, as well as blood pressure and weight measures of attendees and adherence to meeting content and use of meeting tools were used to evaluate the implementation process. Thematic analysis of separate focus group discussions with participants and ASHAs as well as meeting reports and participant evaluation were used to investigate the mechanisms of impact. Results: Fifteen ASHAs led 32 community-based groups in three rural settings in the states of Kerala and Andhra Pradesh, Southern India. Overall, the fidelity of intervention delivery was high. Six meetings were delivered over a 3-month period to each of the intervention groups. The mean number of meetings attended by participants at each site varied significantly, with participants in Rishi Valley attending fewer meetings [mean (SD) = 2.83 (1.68)] than participants in West Godavari (Tukeys test, p = 0.009) and Trivandrum (Tukeys test, p < 0.001) and participants in West Godavari [mean (SD) = 3.48 (1.72)] attending significantly fewer meetings than participants in Trivandrum [mean (SD) = 4.29 (1.76), Tukeys test, p < 0.001]. Culturally appropriate intervention resources and the training of ASHAs, and supportive supervision of them during the program were critical enablers to program implementation. Although highly motivated during the implementation of the program ASHA reported historical issues with timely remuneration and lack of supportive supervision. Conclusions: Culturally appropriate community-based group programs run by trained and supported ASHAs are a successful and potentially scalable model for improving the control of hypertension in rural India. However, consideration of issues related to unreliable/insufficient remuneration for ASHAs, supportive supervision and their formal role in the wider health workforce in India will be important to address in future program scale up. Trial Registration: Clinical Trial Registry of India [CTRI/2016/02/006678, Registered prospectively].

The Application of Systems Science in Nutrition-Related Behaviors and Outcomes Implementation Research: A Scoping Review

Use of systems science can improve the dissemination and implementation (D&I) process. However, little is known about use of systems science in nutrition D&I research. The purpose of this article is to synthesize the ways in which systems science methodology is applied in nutrition D&I research. Scoping review methodology involved searching 6 academic databases for full-text, peer-reviewed, English articles published between 1970 and 2020 that employed systems science within nutrition D&I research. Data extraction included intervention type, population, study aim, methods, theoretical approach, outcomes, and results. Descriptive statistics and qualitative thematic analysis followed. Thirty-four retained articles qualitatively identified benefits (successful planning and organization of complex interventions) and challenges (limited resources, trainings, and lack of knowledge) to utilizing systems science in nutrition D&I research. Future research should work toward building knowledge capacity among nutrition practitioners by increasing available trainings and resources to enhance the utilization of systems science in nutrition D&I research.

Fuzzy-set qualitative comparative analysis of implementation outcomes in an integrated mental healthcare trial in South Africa

BACKGROUND

Integrating mental health services into primary healthcare platforms is an established health systems strategy in low-to-middle-income countries. In South Africa, this was pursued through the Programme for Improving Mental Health Care (PRIME), a multi-country initiative that relied on task-sharing as a principle implementation strategy. Towards better describing the implementation processes, qualitative comparative analysis was adopted to explore causal pathways in the intervention.

OBJECTIVE

This study aimed to explore factors that could have influenced key outcomes of an integrated mental healthcare intervention in South Africa.

METHODS

Drawing from an embedded multiple case study design, the analysis used qualitative comparative analysis. Focusing on nine PHC clinics in the Dr Kenneth Kaunda District as cases, with depression reduction scores set as outcome measures, trial data variables were modelled in a hypothetical causal process. A fuzzy-set qualitative comparative analysis was performed by 1) developing the research questions, 2) developing the fuzzy set, 3) testing necessity and 4) testing sufficiency. These steps were undertaken collaboratively among the research team.

RESULTS

The data were calibrated during several meetings among team members to gain a degree of consensus. Necessity analyses suggested that none of the causal conditions exceeded the threshold of necessity and triviality, and confirmed the inclusion of relevant variables in line with the proposed models. Sufficiency analyses produced two configurations, which were subjected to standard and specific analyses. Ultimately, the results suggested that none of the causal conditions were necessary for a reduction in depression scores to occur, while programme fidelity was identified as a sufficient condition for a reduction in scores to occur.

CONCLUSIONS

The study highlights the importance of understanding implementation pathways to enable better integration of mental health services within primary healthcare in low-to-middle-income settings. It underlines the importance of programme fidelity in achieving the goals of implementation.

The perceived barriers and facilitators to implementation of early mobilisation within a multicentre, phase 3 randomised controlled trial: A qualitative process evaluation study

BACKGROUND

Process evaluation within clinical trials provides an assessment of the study implementation's accuracy and quality to explain causal mechanisms and highlight contextual factors associated with variation in outcomes.

OBJECTIVES

This study aimed to identify the barriers and facilitators of implementing early mobilisation (EM) within a trial.

METHODS

This is a qualitative process evaluation study within the Trial of Early Activity and Mobilisation (TEAM) phase 3 randomised controlled trial. Semistructured interviews were conducted remotely with multiprofessional clinicians (physiotherapists, medical staff, and nursing staff) involved in the delivery of the TEAM intervention at Australian hospitals participating in the TEAM study. Inductive coding was used to establish themes which were categorised into the Behaviour system involving domains of Capability, Opportunity, and Motivation (COM-B), which allowed barriers and enablers affecting EM to be identified.

FINDINGS

Semistructured interviews were conducted in three different states of Australia. There were 16 participants, including 10 physiotherapists, five physicians, and one nurse. The key themes that facilitated EM were mentoring, champions, additional staff, organisation of the environment, cultural changes, communication, and documented safety criteria. In contrast, the main factors that hindered EM were lack of expertise and confidence in delivering EM, heavy sedation, interdisciplinary conflicts, and perceived risks related to EM.

CONCLUSION

A wide range of barriers and facilitators that influenced EM within the TEAM study were identified using the COM-B framework. Many of these have been previously identified in the literature; however, participation in the study was viewed positively by multidisciplinary team members.

Can public sector community health workers deliver a nurturing care intervention in South Africa? The Amagugu Asakhula feasibility study

Background

Nurturing care interventions have the potential to promote health and development in early childhood. Amagugu Asakhula was designed to promote developmentally important dietary and movement behaviours among children of preschool age (3–5 years) in South Africa. An initial formative study in Cape Town found the intervention to be feasible and acceptable when delivered by community health workers (CHWs) linked to a community-based organisation. This study evaluated the delivery of the Amagugu Asakhula intervention by CHWs linked to a public sector primary health care facility in Soweto, as this mode of delivery could have more potential for sustainability and scalability.

Methods

A qualitative design was utilised to assess feasibility, acceptability, adoption, appropriateness, implementation, fidelity and context. CHWs (n = 14) delivered the intervention to caregivers (n = 23) of preschool-age children in Soweto over 6 weeks. Following the completion of the intervention, focus group discussions were held with CHWs and caregivers. Further data were obtained through observations, study records and key informant interviews (n = 5). Data were analysed using deductive thematic analysis guided by a process evaluation framework.

Results

The delivery of the Amagugu Asakhula intervention through CHWs linked to a primary health care facility in Soweto was not found to be feasible due to contextual challenges such as late payment of salaries influencing CHW performance and willingness to deliver the intervention. CHWs expressed dissatisfaction with their general working conditions and were thus reluctant to take on new tasks. Despite barriers to successful delivery, the intervention was well received by both CHWs and caregivers and was considered a good fit with the CHWs’ scope of work.

Conclusions

Based on these findings, delivery of the Amagugu Asakhula intervention is not recommended through public sector CHWs in South Africa. This feasibility study informs the optimisation of implementation and supports further testing of the intervention’s effectiveness when delivered by CHWs linked to community-based organisations. The present study further demonstrates how implementation challenges can be identified through qualitative feasibility studies and subsequently addressed prior to large-scale trials, avoiding the wasting of research and resources.

Supplementary Information

The online version contains supplementary material available at 10.1186/s40814-021-00802-6.

App-Based Salt Reduction Intervention in School Children and Their Families (AppSalt) in China: Protocol for a Mixed Methods Process Evaluation

Background

The app-based salt reduction intervention program in school children and their families (AppSalt) is a multicomponent mobile health (mHealth) intervention program, which involves multiple stakeholders, including students, parents, teachers, school heads, and local health and education authorities. The complexity of the AppSalt program highlights the need for process evaluation to investigate how the implementation will be achieved at different sites.

Objective

This paper presents a process evaluation protocol of the AppSalt program, which aims to monitor the implementation of the program, explain its causal mechanisms, and provide evidence for scaling up the program nationwide.

Methods

A mixed methods approach will be used to collect data relating to five process evaluation dimensions: fidelity, dose delivered, dose received, reach, and context. Quantitative data, including app use logs, activity logs, and routine monitoring data, will be collected alongside the intervention process to evaluate the quantity and quality of intervention activities. The quantitative data will be summarized as medians, means, and proportions as appropriate. Qualitative data will be collected through semistructured interviews of purposely selected intervention participants and key stakeholders from local health and education authorities. The thematic analysis technique will be used for analyzing the qualitative data with the support of NVivo 12. The qualitative data will be triangulated with the quantitative data during the interpretation phase to explain the 5 process evaluation dimensions.

Results

The intervention activities of the AppSalt program were initiated at 27 primary schools in three cities since October 2018. We have completed the 1-year intervention of this program. The quantitative data for this study, including app use log, activity logs, and the routine monitoring data, were collected and organized during the intervention process. After completing the intervention, we conducted semistructured interviews with 32 students, 32 parents, 9 teachers, 9 school heads, and 8 stakeholders from local health and education departments. Data analysis is currently underway.

Conclusions

Using mHealth technology for salt reduction among primary school students is an innovation in China. The findings of this study will help researchers understand the implementation of the AppSalt program and similar mHealth interventions in real-world settings. Furthermore, this process evaluation will be informative for other researchers and policy makers interested in replicating the AppSalt program and designing their salt reduction intervention.

International Registered Report Identifier (IRRID)

DERR1-10.2196/19430

A process evaluation of the improving wisely intervention: a peer-to-peer data intervention to reduce overuse in surgery

Background

The Improving Wisely intervention is a peer-to-peer audit and feedback intervention to reduce overuse of Mohs Micrographic Surgery (MMS). The objective of this study was to conduct a process evaluation to evaluate Mohs surgeons’ perceptions of the implementation quality and perceived impact of the Improving Wisely intervention.

Methods

Surgeons in the Improving Wisely intervention arm, comprised of members of the American College of Mohs Surgeons (ACMS) who co-led the intervention, were invited to complete surveys and key informant interviews. Participants described perceptions of implementation quality (evaluated via dose, quality of implementation, reach and participant responsiveness), perceived impact of the Improving Wisely intervention (evaluated on a 1–5 Likert and qualitatively), and barriers and facilitators to changing surgeons’ clinical practice patterns to reduce Mohs overuse.

Results

Seven hundred thirty-seven surgeons participated in the survey. 89% were supportive of the intervention. Participants agreed that the intervention would improve patient care and reduce the annual costs of Mohs surgery. Thirty surgeons participated in key informant interviews. 93% were interested in receiving additional data reports in the future. Participants recommended the reports be disseminated annually, that the reports be expanded to include appropriateness data, and that the intervention be extended to non ACMS members. Six themes identifying factors impacting potential MMS overuse were identified.

Conclusions

Participants were strongly supportive of the intervention. We present the template used to design and implement the Improving Wisely intervention and provide suggestions for specialty societies interested in leading similar quality improvement interventions among their members.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-020-06017-4.

Evaluating implementation strategies for essential newborn care interventions in low- and low middle-income countries: a systematic review

Neonatal mortality remains a significant health problem in low-income settings. Low-cost essential newborn care (ENC) interventions with proven efficacy and cost-effectiveness exist but have not reached high coverage (≥90%). Little is known about the strategies used to implement these interventions or how they relate to improved coverage. We conducted a systematic review of implementation strategies and implementation outcomes for ENC in low- and low middle-income countries capturing evidence from five medical and global health databases from 1990 to 2018. We included studies of implementation of delayed cord clamping, immediate drying, skin-to-skin contact (SSC) and/or early initiation of breastfeeding implemented in the first hour (facility-based studies) or the 1st day (community-based studies) of life. Implementation strategies and outcomes were categorized according to published frameworks: Expert Recommendations for Implementing Change and Outcomes for Implementation Research. The relationship between implementation strategies and outcomes was evaluated using standardized mean differences and correlation coefficients. Forty-three papers met inclusion criteria. Interventions included community-based care/health promotion and facility-based support and health care provider training. Included studies used 3-31 implementation strategies, though the consistency with which strategies were applied was variable. Conduct educational meetings was the most frequently used strategy. Included studies reported 1-4 implementation outcomes with coverage reported most frequently. Heterogeneity was high and no statistically significant association was found between the number of implementation strategies used and coverage of ENC. This review highlights several challenges in learning from implementation of ENC in low- and low middle-income countries, particularly poor description of interventions and implementation outcomes. We recommend use of UK Medical Research Council guidelines (2015) for process evaluations and checklists for reporting implementation studies. Improved reporting of implementation research in this setting is necessary to learn how to improve service delivery and outcomes and thereby reduce neonatal mortality.

Innovative Implementation Strategies for Hypertension Control in Low- and Middle-Income Countries: a Narrative Review

PURPOSE OF REVIEW

This review summarizes the most recent and innovative implementation strategies for hypertension control in low- and middle-income countries (LMICs).

RECENT FINDINGS

Implementation strategies from Latin America, Africa, and Asia were organized across three levels: community, health system, and policy/population. Multicomponent interventions involving task-shifting strategies, with or without mobile health tools, had the most supporting evidence, with policy or population-level interventions having the least, focused only on salt reduction with mixed results. More research is needed to better understand how context affects intervention implementation. There is an emerging evidence base for implementation strategies for hypertension control and CVD risk reduction in LMICs at the community and health system levels, but further research is needed to determine the most effective policy and population-level strategies. How to best account for local context in adapting and implementing these evidence-based interventions in LMICs still remains largely unknown. Accelerating the translation of this implementation research into policy and practice is imperative to improve health and save lives globally.