Mainstreaming public involvement in a complex research collaboration: A theory-informed evaluation

Evaluating qualitative data analysis workshops from the perspective of public contributors

BACKGROUND

The aim of this project is to evaluate public contributors' experiences of their involvement in qualitative data analysis workshops during an on-going research project titled 'Personalised Primary care for Patients with Multiple long-term conditions'.

METHODS

Four qualitative data analysis workshops were designed and conducted between August and December 2023. We used the Cube evaluation framework (henceforth referred to as the Cube) to evaluate the workshops. The Cube suggests four domains for successful PPI (voice, agenda, change, contribute).Within Workshops One, Two and Three public contributors had to login to an account to access the Cube; this was modified in Workshop Four following feedback from public contributors.

FINDINGS

Across the four workshops the Cube was completed 11 times. Across all four workshops, public contributors thought that their voice was heard, that there were diverse ways to contribute and that they led the agenda. Public contributors thought that researchers responded to their questions and issues, when necessary.

CONCLUSION

This evaluation has shown that public contributors can gain new skills and lead qualitative data analysis discussions.

Collaborative evaluation of a pilot involvement opportunity: Cochrane Common Mental Disorders Voice of Experience College

BACKGROUND

Involving consumers in systematic reviews can make them more valuable and help achieve goals around transparency. Systematic reviews are technically complex and training can be needed to enable consumers to engage with them fully. The Cochrane Common Mental Disorders group sought to engage people with lived experience of mental health problems in the Voice of Experience College, three workshops introducing them to systematic review methods and to opportunities to contribute as Cochrane consumers. We aimed to collectively evaluate the College from the perspective of both facilitators and consumers, to critically reflect on the experience, and to identify how the College could be sustained and spread to other review groups.

METHODS

This study was a longitudinal qualitative and collaborative evaluation, structured around normalisation process theory. Both facilitators and consumers were involved in not only providing their perspectives but also reflecting on these together to identify key learning points.

RESULTS

The workshops were positively evaluated as being engaging and supportive, largely due to the relational skills of the facilitators, and their willingness to engage in joint or two-way learning. The College suffered from a lack of clarity over the role of consumers after the College itself, with a need for greater communication to check assumptions and clarify expectations. This was not achieved due to pandemic disruptions, which nevertheless demonstrated that resources for involvement were not prioritised as core business during this period.

CONCLUSIONS

Soft skills around communication and support are crucial to effective consumer engagement. Sustaining involvement requires sustained communication and opportunities to reflect together on opportunities and challenges. This requires committed resources to ensure involvement activity is prioritised. This is critical as negative experiences later in the involvement journey can undermine originally positive experiences if contributors are unclear as to what their involvement can lead to. Open discussions about this are necessary to avoid conflicting assumptions. The spread of the approach to other review groups could be achieved by flexibly adapting to group-specific resources and settings, but maintaining a core focus on collaborative relationships as the key mechanism of engagement.

PATIENT AND PUBLIC CONTRIBUTION

Public contributors were collaborators throughout the evaluation process and have co-authored the paper.

A Community of Practice to increase education and collaboration in dementia and ageing research and care: The Liverpool Dementia & Ageing Research Forum

BACKGROUND

Too often, dementia research is conducted in research silos without thorough integration and the involvement of people with lived experiences, care professionals and the Third Sector. Research can also get lost in academic publications, without reaching those benefiting most from the evidence. The aim of this methods and evaluation paper was to outline the aims, components and evaluation of the public-facing and -engaging Liverpool Dementia & Ageing Research Forum, to provide a blueprint for setting up similar communities of practice.

METHODS

The Forum was set up in 2019 with the aim to (a) connect different stakeholders in dementia and ageing and co-produce research and to (b) inform and educate. This paper provides an account of the Forum model and evaluates the following key elements: (1) engagement; (2) experiences of the Forum and its impact (via an online evaluation survey and three reflections). All Forum members and attendees were asked to complete a brief evaluation survey about their experiences from October to November 2022. Three regular Forum attendees provided a case study about their involvement and its impact.

FINDINGS

The Forum has reached out to diverse stakeholders and the general public, generating growing interest and engagement since its initiation. Forty-four members and attendees completed the survey. Most attendees completing the evaluation survey have so far engaged in between 5 and 20 activities (47.8%), and 91% felt the aims of the Forum have been met. Engaging in the Forum has produced various benefits for attendees, including increased research capacity and knowledge, as well as improved connectivity with other stakeholders. Eleven percent of respondents, 39% of lived experts, stated they experienced improved access to postdiagnostic care.

CONCLUSIONS

This is the first reported multistakeholder Community of Practice (CoP) on dementia and ageing. We make key recommendations for setting up and running similar dementia CoP, as they provide a noninterventional format for raising awareness, capacity and access to dementia care.

PATIENT AND PUBLIC INVOLVEMENT

This paper reports on the involvement and engagement of people with dementia, unpaid carers, health and social care providers and Third Sector organisations in a CoP.

Evaluating the benefit of early patient and public involvement for product development and testing with small companies

INTRODUCTION

There is a growing understanding of the benefits of patient and public involvement (PPI), and its evaluation, in research. An online version of the CUBE PPI evaluation framework has been developed. We sought to use the CUBE to evaluate the value of early PPI with two small healthcare companies during product development.

METHODS

Contributors were recruited online and had lived experience of either type 1 diabetes or obesity. Two 1-h sessions were run with a company developing a smartphone application to manage diabetes (DEE-EM): one with young people (YP; n = 5) and one with parents (n = 7). Two 1-h sessions were run with a company developing a weight-loss product, both with adults (n = 7 in each session). Sessions were facilitated by an independent University researcher and attended by company representatives, who presented their product. One facilitator led the evaluation of the session by giving a demonstration of the CUBE and asking simple questions in the YP session.

RESULTS

A high proportion of contributors completed the CUBE (80.5% DEE-EM; 93% Oxford Medical Products). Responses were positive to all four CUBE dimensions (in italics). Contributors felt there were diverse ways to contribute to the sessions, and that they had a strong voice to add to the discussion. Balance was achieved regarding whose concerns (public or company) led the agenda, and contributors felt that both companies would make changes based on the discussion. The supportive attitude of both companies resulted in most contributors feeling comfortable participating in PPI sessions with the industry, while recognising the profit-making aspect of their work.

CONCLUSIONS

PPI with small healthcare companies is both feasible and worthwhile. The CUBE framework facilitated the evaluation of the interaction between experts in different knowledge spaces. We provide recommendations for future projects, including considerations of who should participate and the level of implicit endorsement of the product that participation implies.

PATIENT OR PUBLIC CONTRIBUTION

People with lived experience of type 1 diabetes or obesity were invited to contribute to one of four PPI sessions, which they then evaluated. One contributor agreed to contribute to the analysis of the evaluation data and interpretation and preparation of the manuscript.

Researchers' and Research Users' Experiences With and Reasons for Working Together in Spinal Cord Injury Research Partnerships: A Qualitative Study

BACKGROUND

Research partnership approaches are becoming popular within spinal cord injury (SCI) health research system, providing opportunities to explore experiences of and learn from SCI research partnership champions. This study aimed to explore and describe SCI researchers' and research users' (RU') experiences with and reasons for conducting and/or disseminating (health) research in partnership in order to gain more insight into potentially ways to build capacity for and foster change to support research partnerships within a health research system.

METHODS

Underpinned by a pragmatic perspective, ten semi-structured timeline interviews were conducted with researchers and RU who have experiences with SCI research partnerships. Interviews focused on experiences in participants' lives that have led them to become a person who conducts and/or disseminates research in partnership. Data were analysed using narrative thematic analysis.

RESULTS

We identified three threads from participants' stories: (1) seeing and valuing different perspectives, (2) inspirational role models, and (3) relational and personal aspect of research partnerships. We identified sub-threads related to experiences that participants draw on how they came to be a person who engage in (health) research partnerships, and sub-threads related to participants' reasons for engaging in research partnerships. While most sub-threads were identified from both researchers' and RU' perspectives (eg, partnership successes and failures), some were unique for researchers (morally the right thing to do) or RU (advocating).

CONCLUSION

Using a narrative and pragmatic approach, this study provided a new understanding of SCI researchers' and RU' partnership experiences over time. We found that participants' research partnership experiences and motivations align with components of leadership theories. The findings from this study may be used to inform strategies and policy programs to build capacity for conducting and disseminating (health) research in partnership, within and beyond SCI research.

Core outcome set for preventive intervention trials in chronic and episodic migraine (COSMIG): an international, consensus-derived and multistakeholder initiative

OBJECTIVE

Typically, migraine prevention trials focus on reducing migraine days. This narrow focus may not capture all that is important to people with migraine. Inconsistency in outcome selection across trials limits the potential for data pooling and evidence synthesis. In response, we describe the development of core outcome set for migraine (COSMIG).

DESIGN

A two-stage approach sought to achieve international, multistakeholder consensus on both the core domain set and core measurement set. Following construction of a comprehensive list of outcomes, expert panellists (patients, healthcare professionals and researchers) completed a three-round electronic-Delphi study to support a reduction and prioritisation of core domains and outcomes. Participants in a consensus meeting finalised the core domains and methods of assessment. All stages were overseen by an international core team, including patient research partners.

RESULTS

There was a good representation of patients (episodic migraine (n=34) and chronic migraine (n=42)) and healthcare professionals (n=33) with high response and retention rates. The initial list of domains and outcomes was reduced from >50 to 7 core domains for consideration in the consensus meeting, during which a 2-domain core outcome set was agreed.

CONCLUSION

International and multistakeholder consensus emerged to describe a two-domain core outcome set for reporting research on preventive interventions for chronic and episodic migraine: migraine-specific pain and migraine-specific quality of life. Intensity of migraine pain assessed with an 11-point Numerical Rating Scale and the frequency as the number of headache/migraine days over a specified time period. Migraine-specific quality of life assessed using the Migraine Functional Impact Questionnaire.

Developing a programme theory for a transdisciplinary research collaboration: Complex Urban Systems for Sustainability and Health

Background: Environmental improvement is a priority for urban sustainability and health and achieving it requires transformative change in cities. An approach to achieving such change is to bring together researchers, decision-makers, and public groups in the creation of research and use of scientific evidence. Methods: This article describes the development of a programme theory for Complex Urban Systems for Sustainability and Health (CUSSH), a four-year Wellcome-funded research collaboration which aims to improve capacity to guide transformational health and environmental changes in cities. Results: Drawing on ideas about complex systems, programme evaluation, and transdisciplinary learning, we describe how the programme is understood to "work" in terms of its anticipated processes and resulting changes. The programme theory describes a chain of outputs that ultimately leads to improvement in city sustainability and health (described in an 'action model'), and the kinds of changes that we expect CUSSH should lead to in people, processes, policies, practices, and research (described in a 'change model'). Conclusions: Our paper adds to a growing body of research on the process of developing a comprehensive understanding of a transdisciplinary, multiagency, multi-context programme. The programme theory was developed collaboratively over two years. It involved a participatory process to ensure that a broad range of perspectives were included, to contribute to shared understanding across a multidisciplinary team. Examining our approach allowed an appreciation of the benefits and challenges of developing a programme theory for a complex, transdisciplinary research collaboration. Benefits included the development of teamworking and shared understanding and the use of programme theory in guiding evaluation. Challenges included changing membership within a large group, reaching agreement on what the theory would be 'about', and the inherent unpredictability of complex initiatives.

Developing a programme theory for a transdisciplinary research collaboration: Complex Urban Systems for Sustainability and Health

Background: Environmental improvement is a priority for urban sustainability and health and achieving it requires transformative change in cities. An approach to achieving such change is to bring together researchers, decision-makers, and public groups in the creation of research and use of scientific evidence.

Methods: This article describes the development of a programme theory for Complex Urban Systems for Sustainability and Health (CUSSH), a four-year Wellcome-funded research collaboration which aims to improve capacity to guide transformational health and environmental changes in cities.

Results: Drawing on ideas about complex systems, programme evaluation, and transdisciplinary learning, we describe how the programme is understood to “work” in terms of its anticipated processes and resulting changes. The programme theory describes a chain of outputs that ultimately leads to improvement in city sustainability and health (described in an ‘action model’), and the kinds of changes that we expect CUSSH should lead to in people, processes, policies, practices, and research (described in a ‘change model’).

Conclusions: Our paper adds to a growing body of research on the process of developing a comprehensive understanding of a transdisciplinary, multiagency, multi-context programme. The programme theory was developed collaboratively over two years. It involved a participatory process to ensure that a broad range of perspectives were included, to contribute to shared understanding across a multidisciplinary team. Examining our approach allowed an appreciation of the benefits and challenges of developing a programme theory for a complex, transdisciplinary research collaboration. Benefits included the development of teamworking and shared understanding and the use of programme theory in guiding evaluation. Challenges included changing membership within a large group, reaching agreement on what the theory would be ‘about’, and the inherent unpredictability of complex initiatives.

The value of involving patients and public in health services research and evaluation: a qualitative study

BACKGROUND

Public and Patient Involvement, Engagement and Participation research encompasses working with patients/service users (people with a medical condition receiving health service treatment), public members, caregivers and communities (who use services or care for patients). The Partner Priority Programme (PPP) was developed by the National Health Service [NHS] and National Institute for Health Research Collaboration for Leadership in Applied Health Research and Care [NIHR CLAHRC] NWC to share information and experience on evaluating new services being offered to patients that were seeking to reduce health inequalities, improve people's health and wellbeing and reduce emergency hospital admissions. This paper seeks to explore an approach developed for involving the public as equal partners within the evaluation and decision-making processes of health and social care services research. The aim of this study was to identify how public advisors were included, the impact of their involvement, and how change occurred within the organisations following their involvement.

METHODS

A qualitative approach using focus group discussions was adopted to explore the experiences of two cohorts of participants involved in PPP project teams. Focus groups were held with public advisors (n = 9), interns (n = 9; staff or public who received a funded internship for a PPP project), NHS and Local Authority initiative leads (n = 10), and academic facilitators (n = 14). These were transcribed verbatim and analysed using a thematic approach.

RESULTS

Thirty-two public advisors were recruited to support 25 PPP projects across the Collaboration for Leadership in Applied Health Research and CLAHRC North West Coast [NWC] partner organisations. Three inter-related themes were conceptualised: 1)"Where it all started - involving public advisors" identified the varying journeys to recruitment and experiences of becoming a public advisor; 2)"Steps toward active involvement and engagement" related to public advisors becoming core team members; and 3) "Collaborative working to enhance public and patient involvement" relayed how projects identified the benefits of working jointly with the public advisors, particularly for those who had not experienced this style of working before.

CONCLUSIONS

The findings indicate that the PPP model is effective for embedding Public and Patient Involvement [PPI] within health services research, and recommends that PPI is integrated at the earliest opportunity within research projects and service evaluations through the use of support-led and facilitative programmes.

Mainstreaming public involvement in a complex research collaboration: A theory-informed evaluation

INTRODUCTION

There is an extensive literature on public involvement (PI) in research, but this has focused primarily on experiences for researchers and public contributors and factors enabling or restricting successful involvement in specific projects. There has been less consideration of a 'whole system' approach to embedding PI across an organization from governance structures through to research projects.

OBJECTIVE

To investigate how a combination of two theoretical frameworks, one focused on mainstreaming and the other conceptualizing quality, can illuminate the embedding of positive and influential PI throughout a research organization.

METHODS

The study used data from the evaluation of a large UK research collaboration. Primary data were collected from 131 respondents (including Public Advisers, university, NHS and local government staff) via individual and group interviews/workshops. Secondary sources included monitoring data and internal documents.

FINDINGS

CLAHRC-NWC made real progress in mainstreaming PI. An organizational vision and infrastructure to embed PI at all levels were created, and the number and range of opportunities increased; PI roles became more clearly defined and increasingly public contributors felt able to influence decisions. However, the aspiration to mainstream PI throughout the collaboration was not fully achieved: a lack of staff 'buy-in' meant that in some areas, it was not experienced as positively or was absent.

CONCLUSION

The two theoretical frameworks brought a novel perspective, facilitating the investigation of the quality of PI in structures and processes across the whole organization. We propose that combining these frameworks can assist the evaluation of PI research.