Understanding the implementation and effectiveness of a group-based early parenting intervention: a process evaluation protocol

Developing, implementing, and evaluating the visiting Neighbors' program in rural Appalachia: A quality improvement protocol

INTRODUCTION

Older adults living alone in rural areas frequently experience health declines, social isolation, and limited access to services. To address these challenges, our medical academic university supported a quality improvement project for developing and evaluating the Visiting Neighbors program in two rural Appalachian counties. Our Visiting Neighbors program trained local volunteers to visit and guide rural older adults in healthy activities. These age-appropriate activities (Mingle, Manage, and Move- 3M's) were designed to improve the functional health of older adults. The program includes four in-home visits and four follow-up telephone calls across three months.

PURPOSE

The purpose of this paper was to describe the 3M's Visiting Neighbors protocol steps guiding the quality improvement procedures relating to program development, implementation, and evaluation.

METHODS AND MATERIALS

This Visiting Neighbors study used a single-group exploratory quality improvement design. This program was tested using quality improvement standards, including collecting participant questionnaires and visit observations.

RESULTS

Older adults (> 65 years) living alone (N = 30) participants were female (79%) with a mean age of 82.96 (SD = 7.87) years. Volunteer visitor participants (N = 10) were older adult females. Two volunteer visitors implemented each visit, guided by the 3M's activities manual. All visits were verified as being consistently delivered (fidelity). Enrollment and retention data found the program was feasible to conduct. The older adult participants' total program helpfulness ratings (1 to 5) were high (M = 51.27, SD = 3.77). All volunteer visitor's program helpfulness ratings were also high (M = 51.78, SD = 3.73).

DISCUSSION

The Visiting Neighbors program consistently engaged older Appalachian adults living alone in the 3M's activities. The feasibility and fidelity of the 3M's home visits were verified. The quality improvement processes included engaging the expert advisory committee and rural county stakeholders to ensure the quality of the program development, implementation, and evaluation.

Research Ready Grant Program (RRGP) protocol: a model for collaborative multidisciplinary practice-research partnerships

Background

Little attention has been given to the process of implementing or evaluating a structured academic–clinician (university–health service) research capacity-building (RCB) model within healthcare settings. We have developed a model for collaborative multidisciplinary practice–research partnerships called the Research Ready Grant Program (RRGP). The RRGP is informed by Cooke’s (BMC Fam Pract 6:44, 2005) RCB framework and principles. The aim of the study outlined in this protocol is to conduct a process and outcome evaluation of the programme. We will explore how the RRGP's structured mentor model contributes to RCB of clinician-led multidisciplinary research teams. We will identify key factors at the organization, team and individual levels that affect research capacity of health professionals working in one regional health service district. This protocol describes the RRGP design and outlines the methods we will employ to evaluate an RCB programme, the RRGP, delivered in a regional health service in Australia.

Methods

The study will adopt an exploratory concurrent mixed-methods approach designed to evaluate the process of implementing an RCB model across one regional hospital and health service. Both quantitative and qualitative data collection methods over a 12-month period will be implemented. Data triangulation will be applied to capture the complex issues associated with implementing collaborative multidisciplinary practice–research partnerships.

Discussion

The RRGP is an innovative RCB model for clinicians in their workplace. It is expected that the programme will facilitate a culture of collaborative multidisciplinary research and strengthen hospital–university partnerships.

Barriers and facilitators in the delivery of a proportionate universal parenting program model (E-SEE Steps) in community family services

BACKGROUND

A proportionate universal (PU) approach to early years' service provision has been advocated to improve children's health and development and to reduce health inequality, by ensuring that services provide timely and high-quality parenting support commensurate with need. Process-oriented research is critical to examine the factors that contribute to, or hinder, the effective delivery/implementation of such a model in community-based family services. This study aimed to assess the delivery, acceptability and feasibility of a new PU parenting intervention model (called E-SEE Steps), using the Incredible Years® (IY) parent program, when delivered by trained health/family service staff in three "steps"-one universal step (the IY Babies Book), and two targeted steps (group-based IY Infant and Toddler programs).

METHODS

An embedded mixed-methods process evaluation within a pragmatic parallel two-arm, assessor blinded, randomized controlled trial was conducted in community services in four local authorities in England. The process evaluation used qualitative data gathered via interviews and focus groups with intervention arm parents who were offered the targeted steps (n = 29), practitioners (n = 50), service managers (n = 7) and IY program mentors (n = 3). This was supplemented by quantitative data collected using group leader pre-training (n = 50) and post-delivery (n = 39) questionnaires, and research notes of service design decisions.

RESULTS

The E-SEE Steps model was acceptable to most parents, particularly when it was accompanied by engagement strategies that supported attendance, such as providing childcare. Practitioners also highlighted the positive development opportunities provided by the IY training and supervision. However, participant views did not support the provision of the IY Babies book as a standalone universal component, and there were barriers to eligible parents-particularly those with low mood-taking up the targeted programs. Service providers struggled to align the PU model with their commissioned service contracts and with their staff capacity to engage appropriate parents, including tackling common barriers to attendance.

CONCLUSIONS

Despite general enthusiasm and support for delivering high-quality parenting programs in community services in the England, several barriers exist to successfully delivering IY in a proportionate universal model within current services/systems.

A Protocol on Using the RE-AIM Framework in the Process Evaluation of the Primary Health Integrated Care Project for Four Chronic Conditions in Kenya

Background: There has been a rapid increase in morbidity and mortality arising from non-communicable diseases (NCDs). The Academic Model Providing Access to Healthcare (AMPATH) program has established a chronic disease management program in collaboration with the Ministry of Health (MoH) in Kenya at over 150 health facilities in western Kenya. The primary health integrated care for chronic (PIC4C) disease project seeks to deliver preventive, promotive, and curative care for diabetes, hypertension, cervical and breast cancers at the primary health care level. We apply the RE-AIM framework to conduct a process evaluation of the integrated PIC4C model. This paper describes the protocol we are using in the PIC4C process evaluation planning and activities.

Methods and Analysis: This evaluation utilizes clinic reports as well as primary data collected in two waves. Using mixed methods (secondary data, observation, semi-structured interviews, and focus group discussions), the process evaluation assesses the reach, effectiveness, adoption, implementation and maintenance of the PIC4C model in Busia and Trans Nzoia Kenya. The evaluation captures the PIC4C process, experiences of implementers and users, and the wishes of those using the PIC4C services. We will analyse our data across the RE-AIM dimensions using descriptive statistics and two-sample t-test to compare the mean scores for baseline and end line. Qualitative data will be analyzed thematically.

Discussion: The process evaluation of the PIC4C model in Kenya allows implementers and users to reflect and question its implementation, uptake and maintenance. Our experiences thus far suggest practicable strategies to facilitate primary health care can benefit extensively from deliberate process evaluation of the programs undertaken. Furthermore, integrating the RE-AIM framework in the process evaluation of health programs is valuable due to its pragmatic and reporting usefulness.

Implementing the Dementia Early Stage Cognitive Aids New Trial (DESCANT) intervention: mixed-method process evaluation alongside a pragmatic randomised trial

OBJECTIVES

The DESCANT (Dementia Early Stage Cognitive Aids New Trial) intervention provided a personalised care package designed to improve the cognitive abilities, function and well-being of people with early-stage dementia and their carers, by providing a range of memory aids, together with appropriate training and support. This sub-study aimed to assess implementation and identify contextual factors potentially associated with participant outcomes.

METHOD

A mixed-methods approach was adopted alongside the pragmatic randomised trial. Data were obtained from intervention records and interviews with five dementia support practitioners across seven National Health Service Trusts in England and Wales. A reporting framework was constructed from available literature and data assessed by descriptive statistics and thematic analysis.

RESULTS

Participation and engagement was high with 126 out of 128 participants completing the intervention with packages tailored to individual participants. Misplacing items and poor orientation to date and time were common areas of need. Memory aids frequently supplied included orientation clocks (91%), whiteboards (60%), calendars (43%) and notebooks (32%), plus bespoke items. Intervention duration and timing were broadly consistent with expectations. Variation reflected participants' needs, circumstances and preferences. Qualitative findings suggested a potentially positive impact on the well-being of people with dementia and their carers. Issues associated with successful roll-out of the intervention are explored in the discussion.

CONCLUSION

Successful implementation increased confidence in future findings of the randomised trial. Depending on these, DESCANT may prove a scalable intervention with potential to improve the function and quality of life of people with dementia and their carers.

Family Talk versus usual services in improving child and family psychosocial functioning in families with parental mental illness (PRIMERA-Promoting Research and Innovation in Mental hEalth seRvices for fAmilies and children): study protocol for a randomised controlled trial

BACKGROUND

Parental mental illness is common and can lead to dependent children incurring a high risk of developing mental disorders, physical illness, and impaired educational and occupational outcomes. Family Talk is one of the better known interventions designed to prevent the intergenerational transmission of mental illness. However, its evidence base is small, with few robust independent randomised controlled trials, and no associated process or cost evaluations. The PRIMERA (Promoting Research and Innovation in Mental hEalth seRvices for fAmilies and children) research programme involves a mixed method evaluation of Family Talk which is being delivered in mental health settings in Ireland to improve child and family psychosocial functioning in families with parental mental illness.

METHODS

The study comprises a multi-centre, randomised controlled trial (RCT), with nested economic and process evaluations, to assess the clinical and cost-effectiveness and implementation mechanisms of Family Talk compared to usual services. The study is being conducted in 15 adult and child mental health settings in Ireland. Families with a parent with mental illness, and children aged 5-18 years (n = 144 families) will be randomised to either the 7-session Family Talk programme (n = 96) or to standard care (n = 48) using a 2:1 allocation ratio. The primary outcomes are child psychosocial functioning and family functioning. Secondary outcomes are as follows: understanding and experience of parental mental illness, parental mental health, child and parental resilience, partner wellbeing and service utilisation. Blind assessments will take place at pre-intervention and at 6- and 12-month follow-up.

DISCUSSION

Given the prevalence and burden of intergenerational mental illness, it is imperative that prevention through evidence-based interventions becomes a public health priority. The current study will provide an important contribution to the international evidence base for Family Talk whilst also helping to identify key implementation lessons in the scaling up of Family Talk, and other similar interventions, within routine mental health settings.

TRIAL REGISTRATION

ISRCTN Registry, ISRCTN13365858 . Registered 5th February 2019.

Dialogical Family Guidance (dfg)-Development and implementation of an intervention for families with a child with neurodevelopmental disorders

Aim

To describe the development and implementation of a Dialogical Family Guidance (DFG) intervention, aimed at families with a child with neurodevelopmental disorders (NDD).

Design

The DFG components are presented and the content of a DFG training course. Professionals' experiences after the DFG training were evaluated.

Methods

Dialogical Family Guidance development phases and implementation process are examined. The Revised Standards for Quality Improvement Reporting Excellence checklist (SQUIRE 2.0) was used to provide a framework for reporting new knowledge.

Results

The DFG training course seemed to increase possibilities of a more independent role as a nurse to deliver the DFG family intervention. The project showed that the use of dialogue can be difficult for some professionals. Analysis of the questionnaire completed after DFG training reported a high level of satisfaction. DFG training offered a new approach to deliver knowledge and understanding to families using dialogue, including tailored psychoeducation and emotional and practical guidance.

Enhancing national prevention and treatment services for sex workers in Zimbabwe: a process evaluation of the SAPPH-IRe trial

Targeted HIV interventions for female sex workers (FSW) combine biomedical technologies, behavioural change and community mobilization with the aim of empowering FSW and improving prevention and treatment. Understanding how to deliver combined interventions most effectively in sub-Saharan Africa is critical to the HIV response. The Sisters' Antiretroviral Programme for Prevention of HIV: an Integrated Response (SAPPH-Ire) randomized controlled trial in Zimbabwe tested an intervention to improve FSW engagement with HIV services. After 2 years, results of the trial showed no significant difference between study arms in proportion of FSW with HIV viral load ≥1000 copies/ml as steep declines occurred in both. We present the results of a process evaluation aiming to track the intervention's implementation, assess its feasibility and accessibility, and situate trial results within the national HIV policy context. We conducted a mixed methods study using data from routine programme statistics, qualitative interviews with participants and respondent driven surveys. The intervention proved feasible to deliver and was acceptable to FSW and providers. Intervention clinics saw more new FSW (4082 vs 2754), performed over twice as many HIV tests (2606 vs 1151) and nearly double the number of women were diagnosed with HIV (1042 vs 546). Community mobilization meetings in intervention sites also attracted higher numbers. We identified some gaps in programme fidelity: offering pre-exposure prophylaxis took time to engage FSW, viral load monitoring was not performed, and ratio of peer educators to FSW was lower than intended. During the trial, reaching FSW with HIV testing and treatment became a national priority, leading to increasing attendance at both intervention and control clinics. Throughout Zimbabwe, antiretroviral therapy coverage improved and HIV-stigma declined. Zimbabwe's changing HIV policy context appeared to contribute to positive improvements across the HIV care continuum for all FSW over the course of the trial. More intense community-based interventions for FSW may be needed to make further gains.

Implementation evaluation of multiple complex early years interventions: an evaluation framework and study protocol

INTRODUCTION

Implementation evaluations are integral to understanding whether, how and why interventions work. However, unpicking the mechanisms of complex interventions is often challenging in usual service settings where multiple services are delivered concurrently. Furthermore, many locally developed and/or adapted interventions have not undergone any evaluation, thus limiting the evidence base available. Born in Bradford's Better Start cohort is evaluating the impact of multiple early life interventions being delivered as part of the Big Lottery Fund's 'A Better Start' programme to improve the health and well-being of children living in one of the most socially and ethnically diverse areas of the UK. In this paper, we outline our evaluation framework and protocol for embedding pragmatic implementation evaluation across multiple early years interventions and services.

METHODS AND ANALYSIS

The evaluation framework is based on a modified version of The Conceptual Framework for Implementation Fidelity. Using qualitative and quantitative methods, our evaluation framework incorporates semistructured interviews, focus groups, routinely collected data and questionnaires. We will explore factors related to content, delivery and reach of interventions at both individual and wider community levels. Potential moderating factors impacting intervention success such as participants' satisfaction, strategies to facilitate implementation, quality of delivery and context will also be examined. Interview and focus guides will be based on the Theoretical Domains Framework to further explore the barriers and facilitators of implementation. Descriptive statistics will be employed to analyse the routinely collected quantitative data and thematic analysis will be used to analyse qualitative data.

ETHICS AND DISSEMINATION

The Health Research Authority (HRA) has confirmed our implementation evaluations do not require review by an NHS Research Ethics Committee (HRA decision 60/88/81). Findings will be shared widely to aid commissioning decisions and will also be disseminated through peer-reviewed journals, summary reports, conferences and community newsletters.

A Universal Early Parenting Education Intervention in Community-Based Primary Care Settings: Development and Installation Challenges

Prevention and early intervention programmes, which aim to educate and support parents and young children in the earliest stages of the family lifecycle, have become an increasingly popular policy strategy for tackling intergenerational disadvantage and developmental inequality. Evidence-based, joined-up services are recommended as best practice for achieving optimal outcomes for parents and their children; however, there are persistent challenges to the development, adoption and installation of these kinds of initiatives in community-based primary health care settings. In this paper, we present a description of the design and installation of a multi-stakeholder early parenting education and intervention service model called the Parent and Infant (PIN) programme. This new programme is delivered collaboratively on a universal, area-wide basis through routine primary care services and combines standardised parent-training with other group-based supports designed to educate parents, strengthen parenting skills and wellbeing and enhance developmental outcomes in children aged 0–2 years. The programme design was informed by local needs analysis and piloting to establish an in-depth understanding of the local context. The findings demonstrate that a hospitable environment is central to establishing interagency parenting education and supports. Partnership, relationship-building and strategic leadership are vital to building commitment and buy-in for this kind of innovation and programme implementation. A graduated approach to implementation which provides training/education and coaching as well as organisational and administrative supports for practice change, are also important in creating an environment conducive to collaboration. Further research into the impact, implementation and cost-effectiveness of the PIN programme will help to build an understanding of what works for parents and infants, as well as identifying lessons for the development and implementation of other similar complex prevention and intervention programmes elsewhere. This kind of research coupled with the establishment of effective partnerships involving service providers, parents, researchers and policy makers, is necessary to meeting the challenge of improving family education and enhancing the capacity of family services to help promote positive outcomes for children.

Exploration of the experiences, practices and needs of health promotion professionals when evaluating their interventions and programmes

INTRODUCTION

The aim of this study was to describe the practices of health promotion professionals when evaluating interventions and their transferability and to identify these professionals' needs in relation to a tool that will guide them during monitoring and evaluation.

METHOD

A survey was carried out among health promotion actors, which focussed on the interventions they had carried out.

RESULTS

Of the 1017 organisations (covering the whole of France) approached, 246 responded to the survey. More than 60% of the respondents had faced difficulties during the construction of the process evaluation, and more than 50% had faced difficulties during its implementation. These difficulties related to data collection, the conception or choice of indicators, criteria and relevant evaluation tools, and lack of time and personnel. Eighty per cent reported that they had carried out the same action in different places and had been faced with two main obstacles, namely budget and environment. Recognition, positive evaluation, existing collaborations and willingness to work in partnership plus the capacity to adapt to local contexts were identified as factors fostering transferability.

DISCUSSION

Despite the many recommendations available, constructing and carrying out evaluations is not systematic. Actors are confronted with many difficulties, which could be overcome with an IT tool that incorporates the actors' suggestions during its development.

Exploration of dynamics in a complex person-centred intervention process based on health professionals' perspectives

BACKGROUND

The assessment and evaluation of practical and sustainable development of health care has become a major focus of investigation in health services research. A key challenge for researchers as well as decision-makers in health care is to understand mechanisms influencing how complex interventions work and become embedded in practice, which is significant for both evaluation and later implementation. In this study, we explored nurses' and surgeons' perspectives on performing and participating in a complex multi-centre person-centred intervention process that aimed to support patients diagnosed with colorectal cancer to feel prepared for surgery, discharge and recovery.

METHOD

Data consisted of retrospective interviews with 20 professionals after the intervention, supplemented with prospective conversational data and field notes from workshops and follow-up meetings (n = 51). The data were analysed to construct patterns in line with interpretive description.

RESULTS

Although the participants highly valued components of the intervention, the results reveal influencing mechanisms underlying the functioning of the intervention, including multiple objectives, unclear mandates and competing professional logics. The results also reveal variations in processing the intervention focused on differences in using and talking about intervention components.

CONCLUSIONS

The study indicates there are significant areas of ambiguity in understanding how theory-based complex clinical interventions work and in how interventions are socially constructed and co-created by professionals' experiences, assumptions about own professional practice, contextual conditions and the researchers' intentions. This process evaluation reveals insights into reasons for success or failure and contextual aspects associated with variations in outcomes. Thus, there is a need for further interpretive inquiry, and not only descriptive studies, of the multifaceted characters of complex clinical interventions and how the intervention components are actually shaped in constantly shifting contexts.

Family-Based Maternal Sensitivity Model as a Strategy to Optimize Family Perception on the Role of Parents and the Growth of Infants

Introduction: Family perceptions on the role of parents and the growth of infants must be seriously considered by healthcare workers since they can affect the role of family development in taking care of the babies. A family that has optimally understood the role of their development as a parent will be able to give the appropriate care for the babies.Methods: The study aimed to find out the effects of a family-based Maternal Sensitivity Models (MSM) strategy for family perception optimization on the role of parents and the growth of infants. It used a quasi-experimental design with the samples of 50 families. The samples were obtained using purposive sampling technique. The data were analyzed using dependent t-test.Results: It confirmed that the Family-Based Maternal Sensitivity Models (MSM) strategy significantly affects family perceptions on the role of parents with t-value 5.915 and p-value 0.000. MSM also significantly affects family perceptions on the growth infants with the t-value -11.257 and p-value 0.000.Conclusions: Maternal Sensitivity Models (MSM) can be well applied as one of the health models provided by healthcare workers to optimize parents’ perceptions and infants’ growth as well as to develop a competent family in giving care for their babies.

Protocol for the mixed-methods process and context evaluation of the TB & Tobacco randomised controlled trial in Bangladesh and Pakistan: a hybrid effectiveness-implementation study

INTRODUCTION

Tuberculosis (TB) remains a significant public health problem in South Asia. Tobacco use increases the risks of TB infection and TB progression. The TB& Tobacco placebo-controlled randomised trial aims to (1) assess the effectiveness of the tobacco cessation medication cytisine versus placebo when combined with behavioural support and (2) implement tobacco cessation medication and behavioural support as part of general TB care in Bangladesh and Pakistan. This paper summarises the process and context evaluation protocol embedded in the effectiveness-implementation hybrid design.

METHODS AND ANALYSIS

We are conducting a mixed-methods process and context evaluation informed by an intervention logic model that draws on the UK Medical Research Council's Process Evaluation Guidance. Our approach includes quantitative and qualitative data collection on context, recruitment, reach, dose delivered, dose received and fidelity. Quantitative data include patient characteristics, reach of recruitment among eligible patients, routine trial data on dose delivered and dose received, and a COM-B ('capability', 'opportunity', 'motivation' and 'behaviour') questionnaire filled in by participating health workers. Qualitative data include semistructured interviews with TB health workers and patients, and with policy-makers at district and central levels in each country. Interviews will be analysed using the framework approach. The behavioural intervention delivery is audio recorded and assessed using a predefined fidelity coding index based on behavioural change technique taxonomy.

ETHICS AND DISSEMINATION

The study complies with the guidelines of the Declaration of Helsinki. Ethics approval for the study and process evaluation was granted by the University of Leeds (qualitative components), University of York (trial data and fidelity assessment), Bangladesh Medical Research Council and Bangladesh Drug Administration (trial data and qualitative components) and Pakistan Medical Research Council (trial data and qualitative components). Results of this research will be disseminated through reports to stakeholders and peer-reviewed publications and conference presentations.

TRIAL REGISTRATION NUMBER

ISRCTN43811467; Pre-results.