Healthy Living and Co-Production: Evaluation of Processes and Outcomes of a Health Promotion Initiative Co-Produced with Adolescents

Navigating process evaluation in co-creation: a Health CASCADE scoping review of used frameworks and assessed components

BACKGROUND

Co-creation is seen as a way to ensure all relevant needs and perspectives are included and to increase its potential for beneficial effects and uptake process evaluation is crucial. However, existing process evaluation frameworks have been built on practices characterised by top-down developed and implemented interventions and may be limited in capturing essential elements of co-creation. This study aims to provide a review of studies planning and/or conducting a process evaluation of public health interventions adopting a co-creation approach and aims to derive assessed process evaluation components, used frameworks and insights into formative and/or participatory evaluation.

METHODS

We searched for studies on Scopus and the Health CASCADE Co-Creation Database. Co-authors performed a concept-mapping exercise to create a set of overarching dimensions for clustering the identified process evaluation components.

RESULTS

54 studies were included. Conceptualisation of process evaluation included in studies concerned intervention implementation, outcome evaluation, mechanisms of impact, context and the co-creation process. 22 studies (40%) referenced ten existing process evaluation or evaluation frameworks and most referenced were the frameworks developed by Moore et al (14%), Saunders et al (5%), Steckler and Linnan (5%) and Nielsen and Randall (5%).38 process evaluation components were identified, with a focus on participation (48%), context (40%), the experience of co-creators (29%), impact (29%), satisfaction (25%) and fidelity (24%).13 studies (24%) conducted formative evaluation, 37 (68%) conducted summative evaluation and 2 studies (3%) conducted participatory evaluation.

CONCLUSION

The broad spectrum of process evaluation components addressed in co-creation studies, covering both the evaluation of the co-creation process and the intervention implementation, highlights the need for a process evaluation tailored to co-creation studies. This work provides an overview of process evaluation components, clustered in dimensions and reflections which researchers and practitioners can use to plan a process evaluation of a co-creation process and intervention.

Screening the health status of people working in a university

BACKGROUND

We aimed to evaluate the physical and mental well being of people working in our academic institution.

METHODS

This online survey targeted professors ( n  = 108), researchers ( n  = 78), technical and administrative staff ( n  = 279) working in the Scuola Superiore Sant'Anna (Pisa, Italy). Twenty-four multiple-choice questions explored the physical and mental health status, the main cardiovascular risk factors and levels of physical activity, the risk of cancer, and eating and drinking habits.

RESULTS

Over 1 week, 112 participants out of 465 (24%) completed the survey [69% women, median age 43 years (interquartile range 33-53)]. The physical and mental health were judged as 'poor' by 5% and 13%. Many individuals had at least one cardiovascular risk factor (diabetes, 4%; hypertension, 10%; family history of coronary artery disease before 40 years, 21%; hypercholesterolemia, 24%; current or former smoking habit, 39%), and 6% had all of them. Many participants were rather sedentary: for example, 44% never or hardly ever walked at a quick pace for ≥20 min. As for eating and drinking habits, 36% ate sweets five or six times a week or every day, 15% drank beer and/or wine at least five or six times a week, and 5% drank spirits three or four times a week.

CONCLUSIONS

A small but not negligeable proportion of responders complained of 'poor' health, and 65% had at least one cardiovascular risk factor. The global levels of physical activity and eating and drinking habits were globally suboptimal. Educational and screening activities to improve the wellbeing of people working in academia are advisable.

A virtuous cycle of co-production: Reflections from a community priority-setting exercise

INTRODUCTION

Co-production is gaining increasing recognition as a good way of facilitating collaboration among different stakeholders, including members of the public. However, it remains an ambiguous concept as there is no definitive or universal model of co-production or clarity on what constitutes a good co-production approach. This paper draws on the reflections of the academic researchers, practitioners and public advisors involved in co-producing a priority-setting exercise. The exercise was conducted by the Primary and Community Health Services (PCHS) Theme of the National Institute for Health and Care Research Applied Research Collaboration for Kent, Surrey and Sussex (NIHR ARC KSS).

METHODS

We collected data through written and verbal reflections from seven collaborators involved in the PCHS priority-setting exercise. We used Gibbs' model of reflection to guide the data collection. We then analysed the data through an inductive, reflexive thematic analysis.

RESULTS

A common thread through our reflections was the concept of 'sharing'. Although co-production is inherently shared, we used the virtuous cycle to illustrate a sequence of sharing concepts during the research cycle, which provides the underpinnings of positive co-production outcomes. We identified six themes to denote the iterative process of a shared approach within the virtuous cycle: shared values, shared understanding, shared power, shared responsibilities, shared ownership and positive outcomes.

CONCLUSION

Our results present a virtuous cycle of co-production, which furthers the conceptual underpinnings of co-production. Through our reflections, we propose that positive co-production outcomes require foundations of shared values and a shared understanding of co-production as a concept. These foundations facilitate a process of shared power, shared responsibilities and shared ownership. We argue that when these elements are present in a co-production exercise, there is a greater potential for implementable outcomes in the communities in which the research serves and the empowerment of collaborators involved in the co-production process.

PUBLIC MEMBERS' CONTRIBUTIONS

Three members of the public who are public advisors in the NIHR ARC KSS were involved in the priority-setting exercise that informed this paper. The public advisors were involved in the design of the priority-setting exercise and supported participants' recruitment. They also co-facilitated the focus groups during data collection and were involved in the data analysis, interpretation and preparation of the priority-setting report. For this current manuscript, two of them are co-authors. They provided reflections and contributed to the writing and reviewing of this manuscript.

Including patient-reported measures in performance evaluation systems: Patient contribution in assessing and improving the healthcare systems

In healthcare, the introduction of quality standards and indicators to assess performance triggered the development of multidimensional Performance Management Systems (PMSs). The concept of performance in healthcare has recently evolved and broadened its scope. One of the current challenges of PMSs is measuring and integrating the patient perspective into traditional measures. In the regional healthcare system of Tuscany (Italy), a PMS has been implemented and used since 2005. The PMS counts on the systematic involvement of clinicians and managers. Furthermore, the PMS also includes patients' perspective. Moreover, Tuscany has recently implemented the first regional permanent Patient-Reported Outcome and Experience Measures (PROMs and PREMs) Observatory in Italy. This paper presents the results of an action research aimed at analysing the integration of patient-reported outcome and experience indicators into a consolidated PMS. The study describes the process of identifying and discussing of patient-reported indicators with practitioners and categorising findings into three domains: design of patient-reported indicators, integration process into the PMS, and goal of adoption of the patient-related indicators. The paper also describes facilitators, challenges, and lessons learnt with respect to organisational, methodological, cultural, and responsibility-linked factors. The study discusses the ability to agree on how to transform patient data, especially PROMs, into effective performance indicators. The integration of patient-reported indicators into the PMS poses two main challenges: one in terms of sustainability of the performance system itself, and another regarding the attribution of the responsibility for patient outcomes between care settings and providers. This paper provides initial insights on how the integration of patient-reported indicators can make PMSs more inclusive and focussed on the patient-centred perspective.

A longitudinal assessment of chronic care pathways in real-life: self-care and outcomes of chronic heart failure patients in Tuscany

BACKGROUND

Worldwide healthcare systems face challenges in assessing and monitoring chronic care pathways and, even more, the value generated for patients. Patient-reported outcomes measures (PROMs) represent a valid Real-World Evidence (RWE) source to fully assess health systems' performance in managing chronic care pathways.

METHODS

The originality of the study consists in the chance of adopting PROMs, as a longitudinal assessment tool for continuous monitoring of patients' adherence to therapies and self-care behavior recommendations in clinical practice and as a chance to provide policy makers insights to improve chronic pathways adopting a patient perspective. The focus was on PROMs of patients with chronic heart failure (CHF) collected in the Gabriele Monasterio Tuscan Foundation (FTGM), a tertiary referral CHF centre in Pisa, Italy. During the hospital stay, CHF patients were enrolled and received a link (via SMS or email) to access to the first questionnaire. Follow-up questionnaires were sent 1, 7 and 12 months after the index hospitalisation. Professionals invited 200 patients to participate to PROMs surveys. 174 answers were digitally collected at baseline from 2018 to 2020 and analysed. Quantitative and qualitative analyses were conducted, using Chi2, t-tests and regression models together with narrative evidence from free text responses.

RESULTS

Both quantitative and qualitative results showed FTGM patients declared to strongly adhere to the pharmacological therapy across the entire pathway, while seemed less careful to adhere to self-care behavior recommendations (e.g., physical activity). CHF patients that performed adequate Self-Care Maintenance registered outcome improvements. Respondents declared to be supported by family members in managing their adherence.

CONCLUSIONS

The features of such PROMs collection model are relevant for researchers, policymakers and for managers to implement interventions aimed at improving pathway adherence dimensions. Among those, behavioral economics interventions could be implemented to increase physical activity among CHF patients since proven successful in Tuscany. Strategies to increase territorial care and support patients' caregivers in their daily support to patients' adherence should be further explored. Systematic PROMs collection would allow to monitor changes in the whole pathway organization. This study brings opportunities for extending such monitoring systems to other organizations to allow for reliable benchmarking opportunities.

Assessing the efficacy of coproduction to better understand the barriers to achieving sustainability in NHS chronic kidney services and create alternate pathways

CONTEXT

Too many people living with chronic kidney disease are opting for and starting on hospital-based dialysis compared to a home-based kidney replacement therapy. Dialysis services are becoming financially unsustainable.

OBJECTIVE

This study aimed to assess the efficacy of coproductive research in chronic kidney disease service improvement to achieve greater sustainability.

DESIGN

A 2-year coproductive service improvement study was conducted with multiple stakeholders with the specific intention of maximizing engagement with the national health kidney services, patients and public.

SETTING AND PARTICIPANTS

A national health kidney service (3 health boards, 18 dialysis units), patients and families (n = 50), multidisciplinary teams including doctors, nurses, psychologists, social workers, and so forth (n = 68), kidney charities, independent dialysis service providers and wider social services were part of this study.

FINDINGS

Coproductive research identified underutilized resources (e.g., patients on home dialysis and social services) and their potential, highlighted unmet social care needs for patients and families and informed service redesign. Education packages were reimagined to support the home dialysis agenda including opportunities for wider service input. The impacts of one size fits all approaches to dialysis on specialist workforce skills were made clearer and also professional, patient and public perceptions of key sustainability policies.

DISCUSSION AND CONCLUSIONS

Patient and key stakeholders mapped out new ways to link services to create more sustainable models of kidney health and social care. Maintaining principles of knowledge coproduction could help achieve financial sustainability and move towards more prudent adult chronic kidney disease services.

PATIENT OR PUBLIC CONTRIBUTION

Involved in developing research questions, study design, management and conduct, interpretation of evidence and dissemination.

Nurse’s Evaluation on Health Education in Portuguese Pediatric Hospitals and Primary Care for Children/Young and Parents

Aim: This study aimed to analyze the nurse’s evaluation of the health education practice to children and parents. Methods: This is a descriptive and transversal research with a quantitative approach. The selection took place by non-probabilistic convenience sampling, and was developed with nurses on health units for pediatric hospitalization and primary health care in northern Portugal (Trás-os-Montes and Alto Douro Hospital Center, EPE, Northern Regional Health Administration, Northern Local Health Unit, EPE.). Data were collected using a questionnaire with a sample of 311 nurses in the second semester of 2018. Results: 77.5% (n = 241) of nurses perform health education daily; 65% (n = 202) prepare according to the identified needs; the “Identification of children/young and parents’ health priorities” was considered to be the most facilitating element (n = 279; 89.7%); the most difficult element was the “Fear of confidentiality breach by the children/young and parents” (74.6%; n = 232); and 65.9% (n = 205) of nurses considered this practice to have equal importance compared to other nursing interventions. It was also found that academic/professional qualifications and the place of professional practice influence the importance that nurses attribute to HE practice. Conclusions: We can state that there is an appreciation of the binomial child and parents for a better identification of needs, and of the importance attributed to the current legal guidelines (letting themes leading to good health practices be addressed by nurses), which translates into a practice capable of influencing the determinants of health, which promotes health-enhancing behaviors and thus both leads to health gains and reinforces the nurse’s position as health-promoting agents.

Co-Produce, Co-Design, Co-Create, or Co-Construct—Who Does It and How Is It Done in Chronic Disease Prevention? A Scoping Review

Co-production in health literature has increased in recent years. Despite mounting interest, numerous terms are used to describe co-production. There is confusion regarding its use in health promotion and little evidence and guidance for using co-produced chronic disease prevention interventions in the general population. We conducted a scoping review to examine the research literature using co-production to develop and evaluate chronic disease prevention programs. We searched four electronic databases for articles using co-production for health behaviour change in smoking, physical activity, diet, and/or weight management. In 71 articles that reported using co-production, co-design, co-create, co-develop, and co-construct, these terms were used interchangeably to refer to a participatory process involving researchers, stakeholders, and end users of interventions. Overall, studies used co-production as a formative research process, including focus groups and interviews. Co-produced health promotion interventions were generally not well described or robustly evaluated, and the literature did not show whether co-produced interventions achieved better outcomes than those that were not. Uniform agreement on the meanings of these words would avoid confusion about their use, facilitating the development of a co-production framework for health promotion interventions. Doing so would allow practitioners and researchers to develop a shared understanding of the co-production process and how best to evaluate co-produced interventions.

Co-production of knowledge as part of a OneHealth approach to better control zoonotic diseases

There is increased global and national attention on the need for effective strategies to control zoonotic diseases. Quick, effective action is, however, hampered by poor evidence-bases and limited coordination between stakeholders from relevant sectors such as public and animal health, wildlife and forestry sectors at different scales, who may not usually work together. The OneHealth approach recognises the value of cross-sectoral evaluation of human, animal and environmental health questions in an integrated, holistic and transdisciplinary manner to reduce disease impacts and/or mitigate risks. Co-production of knowledge is also widely advocated to improve the quality and acceptability of decision-making across sectors and may be particularly important when it comes to zoonoses. This paper brings together OneHealth and knowledge co-production and reflects on lessons learned for future OneHealth co-production processes by describing a process implemented to understand spill-over and identify disease control and mitigation strategies for a zoonotic disease in Southern India (Kyasanur Forest Disease). The co-production process aimed to develop a joint decision-support tool with stakeholders, and we complemented our approach with a simple retrospective theory of change on researcher expectations of the system-level outcomes of the co-production process. Our results highlight that while co-production in OneHealth is a difficult and resource intensive process, requiring regular iterative adjustments and flexibility, the beneficial outcomes justify its adoption. A key future aim should be to improve and evaluate the degree of inter-sectoral collaboration required to achieve the aims of OneHealth. We conclude by providing guidelines based on our experience to help funders and decision-makers support future co-production processes.

The COVID-19 emergency as an opportunity to co-produce an innovative approach to health services provision: the women's antenatal classes move on the web

The COVID-19 pandemic has strongly affected healthcare organizations, leading to the need for reorganizing also maternal care services during pregnancy. The Regional Health Authorities in Tuscany (Italy) promoted the creation of online antenatal classes (ACs). This study illustrates the innovative approach to deliver ACs online and discusses how the collaborative approach in co-producing this innovative solution co-creates value in healthcare. The action research design was based, on one hand, on the indirect involvement of users by analyzing qualitative data collected through a continuous survey to pregnant women and, on the other one, on the direct involvement of managers and health professionals in meetings and workshops. The authors encompassed all necessary changes in organizational practices and facilitated the collaborative process implementation and analysis. The main findings are that moving ACs online has been a relevant choice, since the need of pregnant women to share information and receive emotional support increased in times of crisis. Additionally, in the perspective of health professionals, the new online ACs model emerged as a valuable solution not only for the contingent situation, but also in a long-term perspective to reach more women during pregnancy and to early support them throughout the maternal care pathway. This study shows that the collaborative approach to co-innovate healthcare services provision, such as with ACs online, facilitates the creation of, long-lasting, and integrated solutions in healthcare, to be used also after pandemic period. Finally, despite this action-research is context-specific, the findings presented in this paper may help other healthcare organizations innovate their own strategies in ACs’ provision.

A health and wellbeing programme for preadolescents in underserved Australian communities: child and stakeholder perspectives

Social inequities in childhood affect children's health and development. Active In-Betweens is a weekly, healthy lifestyle and outdoor activity after-school programme, tailor-made for preadolescent children (9-12 year olds), incorporating a strengths-based, trauma-informed, co-designed and place-based approach. This study evaluated the extent to which the programme strategies and activities met the anticipated short- to medium-term programme outcomes during its first year of operation in two socio-economically disadvantaged housing communities in New South Wales, Australia. A qualitative case study used data from semi-structured interviews with child participants (n = 11) and key stakeholders (n = 10). A broad range of positive outcomes were described. Children's feedback indicated they had experienced opportunities which facilitated the development of new physical activity skills; new healthy eating experiences and knowledge; positive relationships with peers and facilitators and new connections with neighbourhood programmes and outdoor environments. Stakeholders valued the programme for the positive outcomes they observed among the children and the strong organizational partnerships which resulted. The importance of skilled facilitators to deliver the programme, the engagement of local stakeholders and a long-term commitment to programme delivery, with secure funding to ensure continuity, were clearly identified as integral for effective, sustainable outcomes.

Co-Production Performance Evaluation in Healthcare. A Systematic Review of Methods, Tools and Metrics

Co-produced practices and publications in the healthcare sector are gaining momentum, since they can be a useful tool in addressing the sustainability and resilience challenges of health systems. However, the investigation of positive and, mainly, negative outcomes is still confused and fragmented, and above all, a comprehensive knowledge of the metrics used to assess these outcomes is lacking. To fill this gap, this study aims to systematically review the extant literature to map the methods, tools and metrics used to empirically evaluate co-production in health services. The search took place in six databases: Scopus, Web of Science, Psych INFO, PubMed, Cochrane and CINAHL. A total of 2311 articles were screened and 203 articles were included in the analysis, according to PRISMA guidelines. Findings show that outcomes are mainly investigated through qualitative methods and from the lay actor or provider perspective. Moreover, the detailed categorisation of the quantitative measures found offers a multidimensional performance measurement system and highlights the impact areas where research is needed to develop and test new measures. Findings should also promote improvements in empirical data collection on the multiple faceted co-produced activities and spur the consciousness of the adoption of sustainable co-productive initiatives.