Small-scale implementation with pragmatic process evaluation: a model developed in primary health care

Examining the sustainability and effectiveness of co-created physical activity interventions in vocational education and training: a multimethod evaluation

Background

Co-creation approaches are increasingly used in physical activity promotion to develop interventions tailored to the target group and setting. The resulting complexity of such interventions raises challenges in evaluation. Accordingly, little is known about the effectiveness of co-created interventions and the underlying processes that impact their sustainable implementation. In this study, we attempt to fill this gap by evaluating co-created multi-component physical activity interventions in vocational education and training in nursing care and automotive mechatronics regarding (1) their sustainable implementation at the institutional level and (2) the effectiveness of single intervention components at the individual level.

Methods

Following a multimethod design, we conducted a questionnaire survey (n = 7) and semi-structured interviews (n = 4) to evaluate the sustainability of the interventions. Quantitative data were analyzed descriptively, and qualitative data were analyzed using qualitative content analysis. To examine the interventions’ effectiveness, we conducted two non-randomized controlled trials (n = 111). Analysis of variance was used to examine differences between groups.

Results

At the institutional level, long-term implementation of single intervention components in nursing care was observed; in contrast, long-term implementation in automotive mechatronics was not observed. In this context, various factors at the outer contextual (e.g., COVID-19 pandemic), inner contextual (e.g., health-promoting leadership), intervention (e.g., acceptance), and personal levels (e.g., champion) influenced sustainability. At the individual level, no significant intervention effects were found for changes in physical activity behavior and physical activity-related health competence.

Conclusion

The role of co-creation on the effectiveness and sustainability of physical activity promotion in vocational education and training cannot be answered conclusively. Only in the nursing care sector, a co-creation approach appeared promising for long-term intervention implementation. Sustainable implementation depends on various influencing factors that should be considered from the outset. Demonstrating effectiveness at the individual level was challenging. To conclusively clarify both the role and impact of co-creation, methodologically complex and elaborate evaluation designs will be required in future research projects.

Trial registration

This study was retrospectively registered at clinicaltrials.gov on 24/08/2021 (NCT05018559).

Supplementary Information

The online version contains supplementary material available at 10.1186/s12889-022-13133-9.

Implementing YouthCHAT – evaluation and development of an implementation framework

Purpose

A high proportion of indigenous Māori, a generally socioeconomically deprived population, live in Northland, New Zealand. Māori youth suffer a high rate of mental ill-health, substance misuse, and other risky behaviours. While evidence demonstrates that early detection and management of these issues leads to long-term positive health outcomes, implementation of systematic screening and intervention is challenging. YouthCHAT is a self-administered digital tool screening young people for mental health concerns and risky health behaviours. Our purpose was to implement YouthCHAT into Northland youth services using an iterative process of implementation, evaluation, and modification, and create a framework for national-level rollout.

Approach

A bicultural approach combined Kaupapa Māori research and co-design principles to inform implementation and evaluation. Informed by field notes, informal communications plus focus groups, and interviews with end-users, we identified and addressed barriers to uptake of YouthCHAT. Consolidated framework for implementation research and normalisation process theory was theoretical underpinnings of the project. We measured uptake by the number of YouthCHAT screens completed.

Discussion

It took three years to successfully embed YouthCHAT and implement it into usual practice. A change was needed at all levels, from the individual practitioner to the service team through to the over-arching health system. A bicultural co-design approach and programming remote functionality during coronavirus disease 2019 (COVID-19) lockdown contributed to YouthCHAT's ultimate acceptability and implementation. The cultural context in which an intervention is introduced is often overlooked. This partnership approach may be time-consuming and challenging. However, an intervention will only be adopted where there is local ownership and it is seen as fit for purpose. Our framework addresses the importance of communication and coordination in the acceptance and uptake of an innovation, with ongoing evaluation as an integral part of implementation. It is generalisable to other settings and augments other models such as the consolidated framework for implementation research currently available.

Plain language summary

The Northland region of New Zealand has a particularly socioeconomically deprived population with a high proportion of indigenous Māori. Māori youth suffer a high rate of mental ill-health, substance misuse, and other risky behaviours. Early detection of these issues means they can be addressed and youth are less likely to have future health and social problems. YouthCHAT is a self-administered digital tool screening young people for mental health concerns and risky health behaviours. We implemented YouthCHAT into Northland youth services using an iterative process of implementation, evaluation, and modification, and created a framework for national-level rollout. A bicultural approach combined Māori research and co-design principles, using information from many sources including focus groups, staff surveys, field notes, and informal communications to inform implementation and evaluation. It took three years to successfully embed YouthCHAT into usual practice. A change was needed at all levels, from the individual practitioner to the service team through to the over-arching health system. Programming remote functionality during COVID-19 lockdown contributed to YouthCHAT's ultimate acceptability and implementation. This bicultural partnership approach may be time-consuming and challenging. However, ultimately an intervention will only be enthusiastically adopted if there is local ownership and it is seen as fit for purpose. Our success is measured by the number of YouthCHAT screens now being completed. Our framework addresses the importance of communication and coordination in the acceptance and uptake of an innovation, with ongoing evaluation as an integral part of implementation. It is generalisable to other settings and augments other models currently available.

Adapting Strategies for Optimal Intervention Implementation in Nursing Homes: A Formative Evaluation

BACKGROUND AND OBJECTIVES

Nursing homes pose unique challenges for implementation of research and quality improvement (QI). We previously demonstrated successful implementation of a nursing home-led intervention to improve relationships between frontline staff and residents in 6 U.S. Department of Veterans Affairs (VA) Community Living Centers (CLCs). This article discusses early adaptations made to the intervention and its implementation to enhance frontline staff participation.

RESEARCH DESIGN AND METHODS

This is a formative evaluation of intervention implementation at the first 2 participating CLCs. Formative evidence-including site visitors' field notes, implementation facilitation records, and semistructured frontline staff interviews-were collected throughout the study period. Data analysis was informed by the Capability, Opportunity, Motivation, and Behavior model of behavior change.

RESULTS

Adaptations were made to 5 a priori intervention implementation strategies: (a) training leaders, (b) training frontline staff, (c) adapting the intervention to meet local needs, (d) auditing and providing feedback, and (e) implementation facilitation. On the basis of a 6-month implementation period at the first CLC, we identified elements of the intervention and aspects of the implementation strategies that could be adapted to facilitate frontline staff participation at the second CLC.

DISCUSSION AND IMPLICATIONS

Incremental implementation, paired with ongoing formative evaluation, proved critical to enhancing capability, opportunity, and motivation among frontline staff. In elucidating what was required to initiate and sustain the nursing home-led intervention, we provide a blueprint for responding to emergent challenges when performing research and QI in the nursing home setting.

Co-creating physical activity interventions: a mixed methods evaluation approach

BACKGROUND

Co-creation strategies, such as cooperative planning, are promising as a means to ensure that physical activity interventions address real-world problems and are tailored to the target group. This has already been validated in diverse settings. However, questions targeting the transferability of cooperative planning to new settings and the key factors influencing its success or failure remain unclear. At the same time, co-creation processes are complex, and evaluation can be challenging. Following calls for detailed reporting, this paper describes the programme activities, the underlying logic, and methodological design of a study that aims to evaluate the transfer of cooperative planning to new settings and to explore the associated key determinants.

METHODS

Cooperative planning was utilized as a strategy to target physical activity promotion in three real-world German settings in the nursing care and automotive mechatronics sectors. This involved researchers working alongside stakeholders from practice and policy to conjointly develop new interventions to promote physical activity in physically demanding jobs. A pragmatic approach is used to evaluate both the transferability and key determinants of this strategy. We developed a logic model for this co-creation process that describes the underlying assumptions and guides the evaluation. The evaluation outcomes of this study include planning meetings, newly developed interventions, and the determinants that are likely to affect cooperative planning. Quantitative and qualitative data will be collected using questionnaires, documents, and interviews. The quantitative data will be analysed descriptively, while the qualitative data will mainly be analysed using qualitative content analysis, split by settings. Subsequently, data triangulation will be used to integrate the quantitative and qualitative findings, which will then be compared across all three settings.

DISCUSSION

The study findings will contribute to a better understanding of co-creation strategies, their transferability, and key determinants. The practical implications can include a checklist for assessing key determinants and a guideline for transferring cooperative planning into new settings to benefit more people. Ultimately, this study will help to advance co-creation strategies and may be relevant for researchers, practitioners, and policy-makers targeting physical activity promotion in various contexts.

TRIAL REGISTRATION

Open Science Framework: https://osf.io/r6xnt/ (retrospectively registered).

Community nurses' experiences of the Swedish Dignity Care Intervention for older persons with palliative care needs - A qualitative feasibility study in municipal home health care

OBJECTIVES

The Swedish Dignity Care Intervention (DCI-SWE) is an intervention for people with palliative care needs to enhance their dignity. The original DCI was developed in Scotland, where it was tested by nurses in municipal care. In this study, the DCI has been tested for the first time in a Swedish home health care context. The aim was to describe experiences of the DCI-SWE from the perspectives of community nurses (CNs).

METHODS

This was a feasibility study with a qualitative design. Three focus group interviews and one individual interview were performed with CNs (n = 11). Reflective diaries and field notes were written by the CNs and researchers, respectively. Data were analysed using inductive content analysis.

RESULTS

Two main categories and six subcategories were identified. The first main category, 'Practising the palliative approach while responding to palliative care needs', consisted of the subcategories: gives structure while providing palliative care; gives older people opportunities to be confirmed; and responding to existential and sensitive needs. The second main category, 'Aspects influencing the use of the DCI-SWE' had two subcategories about facilitators and barriers to the use of the DCI-SWE, and another about how to establish the DCI-SWE in the context of home health care.

CONCLUSION

The DCI-SWE offers CNs an overview of older people's concerns while providing palliative care, and gives the older people opportunities to be listened to. Essential prerequisites for using the DCI-SWE in municipal home health care are that CNs are comfortable holding conversations and are given time and space for these by the organisation. Other aspects facilitating the use of the DCI-SWE are managers' engagement and support, continuing training for CNs and CNs' opportunities for reflection.

'Our village is dependent on us. That's why we can't leave our work'. Characterizing mechanisms of motivation to perform among Accredited Social Health Activists (ASHA) in Bihar

Community health workers (CHWs) play major roles in delivering primary healthcare services, linking communities to the formal health system and addressing the social determinants of health. Available evidence suggests that the performance of CHW programmes in low- and middle-income countries can be influenced by context-dependent causal mechanisms such as motivation to perform. There are gaps regarding what these mechanisms are, and what their contribution is to CHW performance. We used a theory-driven case study to characterize motivational mechanisms among Accredited Social Health Activists (ASHAs) in Bihar, India. Data were collected through semi-structured interviews with CHWs and focus group discussions with beneficiary women. Data were coded using a combined deductive and inductive approach. We found that ASHAs were motivated by a sense of autonomy and self-empowerment; a sense of competence, connection and community service; satisfaction of basic financial needs; social recognition; and feedback and answerability. Findings highlight the potential of ASHAs' intrinsic motivation to increase their commitment to communities and identification with the health system and of programme implementation and management challenges as sources of work dissatisfaction. Efforts to nurture and sustain ASHAs' intrinsic motivation while addressing these challenges are necessary for improving the performance of Bihar's ASHA programme. Further research is needed to characterize the dynamic interactions between ASHAs' motivation, commitment, job satisfaction and overall performance; also, to understand how work motivation is sustained or lost through time. This can inform policy and managerial reforms to improve ASHA programme's performance.

Designing and pilot testing of a leadership intervention to improve quality and safety in nursing homes and home care (the SAFE-LEAD intervention)

OBJECTIVE

To describe the design of a leadership intervention for nursing home and home care, including a leadership guide for managers to use in their quality and safety improvement work. The paper reports results from the pilot test of the intervention and describes the final intervention programme.

DESIGN

Qualitative design, using the participation of stakeholders.

METHODS

The leadership guide and intervention were designed in collaboration with researchers, coresearchers and managers in nursing homes and home care organisations, through workshops and focus group interviews. The pilot test consisted of three workshops with managers working on the leadership guide, facilitated and observed by researchers, and evaluated by means of observation and focus group interviews with the participants. The analysis combined the integration of data from interviews and observations with directed content analysis.

SETTING

Norwegian nursing homes and home care services.

PARTICIPANTS

Managers at different levels in three nursing homes and two home care services, coresearchers, and patient and next-of-kin representatives.

RESULTS

The managers and coresearchers suggested some revisions to the leadership guide, such as making it shorter, and tailoring the terminology to their setting. Based on their suggestions, we modified the intervention and developed learning resources, such as videos demonstrating the practical use of the guide. Evaluation of the pilot test study showed that all managers supported the use of the guide. They adapted the guide to their organisational needs, but found it difficult to involve patients in the intervention.

CONCLUSIONS

A participatory approach with stakeholders is useful in designing a leadership intervention to improve quality and safety in nursing homes and home care, although patient participation in its implementation remains difficult. The participatory approach made it easier for managers to adapt the intervention to their context and to everyday quality and safety work practice.

Curriculum development: a how to primer

Curriculum development is a topic everyone in the field of medical education will encounter. Due to the breadth of ages and types of care provided in Family Medicine, family medicine faculty in particular need to be facile in developing effective curricula for medical students, residents, fellows and for faculty development. In the area of medical education, changing and evolving learning environments, as well as changing requirements necessitate new and innovative curricula to address these evolving needs. The process of developing a medical education curriculum can seem daunting but when broken down into smaller components can become very straightforward and easy to accomplish. This paper focuses on the curriculum development process using a six-step approach: performing a needs assessment, determining content, writing goals and objectives, selecting the educational strategies, implementing the curriculum and, finally, evaluating the curriculum. This process may serve as a template for Family Medicine educators, and all medical educators looking to design (or redesign) their own medical education curriculum.