Coproduction for feasibility and pilot randomised controlled trials: learning outcomes for community partners, service users and the research team

Developing research priorities for palliative care in Colombia: a priority setting partnership approach

BACKGROUND

A recent Lancet commission called for more research on palliative care in low- and middle-income (LMIC) countries such as Colombia. A research priority setting approach has been recommended by The Global Forum for Health Research to address the huge gap in research output between LMIC and high-income countries, with influential health service bodies recommending the active involvement of non-research expert stakeholders in establishing research priorities to address service user needs.

METHOD

Priority setting partnership (PSP) following the four stages of the James Lind Alliance methodology; establishing the partnership, identifying evidence uncertainties, refining questions and uncertainties, and prioritization. Data from MS forms were analysed using descriptive statistics.

RESULTS

A total of 33 stakeholders attended an online PSP workshop and completed the Mentimeter exercise in Microsoft Teams. A total of 48 attended the subsequent in person prioritisation exercise in urban Bogota (n = 22) and rural Popayan (n = 25). The stakeholders were a diverse group of health professionals (physicians, medical students, nurses, dentists, physiotherapists, nutritionist, occupational and speech therapists), financial and administrative staff and patients with life-limiting illness and caregivers. Top research priorities included patient and caregiver needs, service provider education and training, and better integration of palliative care with cancer and non-cancer services. The key challenges included a lack of interest in palliative care research, along with funding, time and resource constraints. Key solutions included collaboration across disciplines and settings, highlighting benefits of palliative research to help secure adequate resources, and multicentre, mixed method research, with patient involvement from the research development stage.

CONCLUSION

The findings of this PSP should be disseminated among palliative care associations worldwide to inform international multicentre studies, and among governmental and nongovernmental organisations that promote research in Colombia. A focus on patient and family caregiver palliative care needs in Colombia should be prioritised.

Reflections on co-producing an obesity-prevention toolkit for Islamic Religious Settings: a qualitative process evaluation

BACKGROUND

Islamic leaders, staff, and Muslim parents in the UK are supportive of healthy lifestyle intervention delivery through Islamic Religious Settings. Such interventions are necessary given high obesity rates in British South Asian (40%) compared to White British (32%) children of equivalent age. Co-production can facilitate the development of culturally appropriate health interventions, however it can be theoretically and practically challenging, and evaluation of co-production within an Islamic Religious Setting context is lacking. The aim of this study was to examine the feasibility and acceptability of taking a co-production approach to develop an obesity-prevention toolkit for Islamic Religious Settings.

METHODS

An obesity-prevention toolkit for use in Islamic Religious Settings, incorporating physical activity, healthy diet, and organisational change, has been co-produced to be evidence-informed and contextually relevant. A qualitative process evaluation was employed to examine experiences of co-production. Semi-structured interviews (n = 15) and a focus group (n = 5) were conducted with toolkit co-production stakeholders, e.g., subject experts, an Islamic scholar, and Islamic Religious Setting staff. Transcripts were analysed inductively using reflexive thematic analysis.

RESULTS

The analysis revealed four major themes regarding stakeholders' experiences of co-producing a childhood obesity-prevention toolkit for Islamic Religious Settings. These themes are: (1) attitudes towards obesity-prevention through Islamic Religious Settings, (2) benefits of co-production including capacity building and ownership (3) negotiating involvement, power, and perspectives within the co-production process, and (4) the complexities of effective communication in co-production.

CONCLUSION

This study adds to the evidence-base in support of delivering health promotion through faith settings. Taking a co-production approach to develop an obesity-prevention toolkit for Islamic Religious Settings provided benefit to the toolkit product and local stakeholders. The toolkit is currently being implemented across Bradford, UK and there is potential to adapt the toolkit to other geographical contexts, and for evaluating effectiveness for preventing obesity in British Muslim families.

Involving patients and the public In sTatistIcal Analysis pLans (INITIAL): A delphi survey

BACKGROUND

Patient and public involvement (PPI) in trials aims to enhance research by improving its relevance and transparency. Planning for statistical analysis begins at the design stage of a trial within the protocol and is refined and detailed in a Statistical Analysis Plan (SAP). While PPI is common in design and protocol development it is less common within SAPs. This study aimed to reach consensus on the most important and relevant statistical analysis items within an SAP to involve patients and the public.

METHODS

We developed a UK-based, two-round Delphi survey through an iterative consultation with public partners, statisticians, and trialists. The consultation process started with 55 items from international guidance for statistical analysis plans. We aimed to recruit at least 20 participants per key stakeholder group for inclusion in the final analysis of the Delphi survey. Participants were asked to vote on each item using a Likert scale from 1 to 9, where a rating of 1 to 3 was labelled as having 'limited importance'; 4 to 6 as 'important but not critical' and 7 to 9 as 'critical' to involve patients and the public. Results from the second round determined consensus on critical items for PPI.

RESULTS

The consultation exercise led to the inclusion of 15 statistical items in the Delphi survey. We recruited 179 participants, of whom 72% (129: 36 statisticians, 29 patients or public partners, 25 clinical researchers or methodologists, 27 trial managers, and 12 PPI coordinators) completed both rounds. Participants were on average 48 years old, 60% were female, 84% were White, 64% were based in England and 84% had at least five years' experience in trials. Four items reached consensus regarding critical importance for patient and public involvement: presentation of results to trial participants; summary and presentation of harms; interpretation and presentation of findings in an academic setting; factors impacting how well a treatment works. No consensus was reached for the remaining 11 items. In general, the results were consistent across stakeholder groups.

DISCUSSION

We identified four critical items to involve patients and the public in statistical analysis plans. The remaining 11 items did not reach consensus and need to be considered in a case-by-case basis with most responders considering patient and public involvement important (but not critical). Our research provides a platform to enable focused future efforts to improve patient and public involvement in trials and enhance the relevance of statistical analyses to patients and the public.

[Integrated Care for Patients with Somatic Symptom Disorder in Primary Care - A Qualitative Participatory Evaluation of a Randomised Feasibility Trial]

OBJECTIVE

The VISION intervention is a manualized short-term treatment for people with somatic symptom disorder, integrated into the primary care and delivered by psychotherapists via video consultations. As an innovative technology-based approach, the intervention was most recently piloted in a randomized feasibility trial. During the qualitative accompanying study presented here, the intervention was evaluated and optimised from the user perspective of patients who participated in the feasibility study.

METHODS

We interviewed a total of N=10 patients included in the intervention group in three semi structured focus group discussions focusing on how they (1) assess the intervention in terms of acceptance and individual benefit and (2) which adjustments are necessary from the user perspective for further optimization of the intervention. We performed a qualitative content analysis using MAXQDA Plus 2022.

RESULTS

Respondents reported initial reservations on their part regarding care via video consultation. After a brief period of habituation, the intervention was universally accepted. Main suggestions for improvement included a stronger focus on supporting patients find follow-up psychotherapy (if indicated) more flexible settings (e. g., location of video consultation), and a stronger integration of treatment into primary care.

DISCUSSION AND CONCLUSION

The study provides systematically elaborated comprehensive findings on participants' practical experiences with the intervention. The model is appropriate for treating people with somatic symptom disorder. Before implementation, important suggestions for improvement should be thoroughly considered with continued patient and public involvement.

Cocreators' experiences and effectiveness of cocreated interventions in improving health behaviours of adults with non-communicable diseases: a systematic review protocol

INTRODUCTION

Improved health behaviours and help-seeking behaviour reduce morbidity and mortality from non-communicable diseases (NCDs). Compliance with the recommendations of lifestyle changes for the management of NCDs has been challenging, as patients find it difficult to change and sustain lifestyle behaviours for a long period of time. Studies have reported that cocreated interventions are promising in addressing negative health behaviours and improving health outcomes in people with NCDs; however, no conclusive evidence exists. Therefore, this review aims to evaluate cocreators' experiences and the effectiveness of cocreated interventions in improving the health behaviours of individuals with NCDs.

METHODS AND ANALYSIS

This review will follow the recommendations described in the Preferred Reporting Items for Systematic Reviews and Meta-analyses guideline and the Enhancing Transparency in Reporting the Synthesis of Qualitative Research statement for the synthesis of qualitative data. The following databases: Co-creation Database (https://zenodo.org/record/6773028%23.Y9h2sezP1pg), MEDLINE (via OVID), Cumulative Index to Nursing and Allied Health Literature (via EBSCO Host), EMBASE (via OVID), PsycINFO (via OVID), Scopus, Web of Science, Cochrane Library and grey literature will be searched. The identified studies will be independently screened by two reviewers to determine their eligibility. The review will target to include studies that investigated the experiences of cocreators and/or the effectiveness of cocreated interventions on the health behaviour and/or health outcomes of adults with NCDs. Two independent reviewers will also appraise the quality of the included studies, as well as data extraction. A narrative synthesis will be used to summarise the findings. Thematic synthesis and meta-analysis will be conducted for the qualitative and quantitative data, respectively. The qualitative and quantitative findings will be integrated using the parallel result convergent synthesis.

ETHICS AND DISSEMINATION

Ethics approval is not applicable because the review will only use data from the published studies. The findings will be disseminated through publication in peer-reviewed journals and conference presentations.

PROSPERO REGISTRATION NUMBER

CRD42023391746.

Experiences of patients enrolled and staff involved in the prehabilitation of elective patients undergoing cardiac surgery trial: a nested qualitative study

The purpose of this study was to understand the views and experiences of patients enrolled and staff involved in the prehabilitation of elective patients undergoing cardiac surgery trial. This sub-study was informed by normalisation process theory, a framework for evaluating complex interventions, and used consecutive sampling to recruit patients assigned to both the intervention and control groups. Patients and all staff involved in delivering the trial were invited to participate in focus groups, which were recorded, transcribed verbatim and subjected to reflexive thematic analysis. Five focus groups were held comprising 24 participants in total (nine patients assigned to the prehabilitation; seven assigned to control; and eight staff). Five themes were identified. First, preparedness for surgery reduced fear, where participants described that knowing what to expect from surgery and preparing the body physically increased feelings of control and subsequently reduced apprehension regarding surgery. Second, staff were concerned but trusted in a safe environment, describing how, despite staff's concerns regarding the risks of exercise in this population, the patients felt safe in their care whilst participating in an exercise programme in hospital. Third, rushing for recovery and the curious carer, where patients from both groups wanted to mobilise quickly postoperatively whilst staff visited patients on the ward to observe their recovery progress. Fourth, to survive and thrive postoperatively, reflecting staff and patients' expectations from the trial and what motivated them to participate. Fifth, benefits are diluted by lengthy waiting periods, reflecting the frustration felt by patients waiting for their surgery after completing the intervention and the fear about continuing exercise at home before they had been 'fixed'. To conclude, functional exercise capacity may not have improved following prehabilitation in people before elective cardiac surgery due to concerns regarding the safety of exercise that may have hindered delivery and receipt of the intervention. Instead, numerous non-physical benefits were elicited. The information from this qualitative study offers valuable recommendations regarding refining a prehabilitation intervention and conducting a subsequent trial.

Starting a conversation about estimands with public partners involved in clinical trials: a co-developed tool

BACKGROUND

Clinical trials aim to draw conclusions about the effects of treatments, but a trial can address many different potential questions. For example, does the treatment work well for patients who take it as prescribed? Or does it work regardless of whether patients take it exactly as prescribed? Since different questions can lead to different conclusions on treatment benefit, it is important to clearly understand what treatment effect a trial aims to investigate-this is called the 'estimand'. Using estimands helps to ensure trials are designed and analysed to answer the questions of interest to different stakeholders, including patients and public. However, there is uncertainty about whether patients and public would like to be involved in defining estimands and how to do so. Public partners are patients and/or members of the public who are part of, or advise, the research team. We aimed to (i) co-develop a tool with public partners that helps explain what an estimand is and (ii) explore public partner's perspectives on the importance of discussing estimands during trial design.

METHODS

An online consultation meeting was held with 5 public partners of mixed age, gender and ethnicities, from various regions of the UK. Public partner opinions were collected and a practical tool describing estimands, drafted before the meeting by the research team, was developed. Afterwards, the tool was refined, and additional feedback sought via email.

RESULTS

Public partners want to be involved in estimand discussions. They found an introductory tool, to be presented and described to them by a researcher, helpful for starting a discussion about estimands in a trial design context. They recommended storytelling, analogies and visual aids within the tool. Four topics related to public partners' involvement in defining estimands were identified: (i) the importance of addressing questions that are relevant to patients and public in trials, (ii) involving public partners early on, (iii) a need for education and communication for all stakeholders and (iv) public partners and researchers working together.

CONCLUSIONS

We co-developed a tool for researchers and public partners to use to facilitate the involvement of public partners in estimand discussions.

Multi-level stakeholders' perspectives on implementation and scaling up community-based health promotion in Germany

Community-based health promotion has the potential to address existing health inequities, although such approaches are scarcely scaled up. For a successful scale up, various stakeholders at different levels and sectors need to be involved. The article's aims are to assess what kind of external support communities need for implementation and to identify facilitators and barriers for scaling up community-based health promotion. Two national digital workshops were conducted in Germany with stakeholders at the community level (n = 161) and with stakeholders at the federal and state levels (n = 84). Protocols were compiled and coded using qualitative content analysis. During the first workshop, we revealed 11 themes for external support needs ('Strategic approach', 'Define & compare indicators', 'International human resource', 'Tools & Aids', 'External conduction of the assessment', 'Involvement of people in difficult life situations', 'Overview of actors', 'Moderation', 'Obtain funding', 'Quality assurance/evaluation' and 'External support'). Eleven facilitators and barriers were identified for scaling up ('Assessment and evaluation', 'Intersectoral collaboration and partnerships', 'Communication', 'Characteristics of the program', 'Political and legal conditions', 'Political support', 'Local coordinator', 'Resources', 'Participation', 'Strategic planning/methods' and 'Intermediary organization'). The identified results provide practice-based evidence on support needed for scaling up, facilitators that promote scaling up and barriers that might hinder scaling up community-based health promotion in Germany. In a next step, this practice-based evidence needs to be systematically integrated with scientific-based evidence on key components for scaling up such approaches for the development of an effective scaling-up concept.

NGO-academia research co-production in humanitarian settings: opportunities and challenges

The term 'co-production' is increasingly used to describe varied forms of research partnerships, expanding from its application within the health sector to other areas. In humanitarian settings, alongside more calls for localisation and decolonising aid, research co-production is emerging as a means of tackling power dynamics within NGO (non-governmental organisation)-academia research partnerships. Based on semi-structured interviews with practitioners and academics with experience of co-producing research and participating in research partnerships, this paper presents the opportunities and challenges associated with co-producing research in humanitarian settings. The findings suggest that similar to other buzzwords in the humanitarian sector, the label of 'co-production' is sometimes uncritically applied to any kind of research partnership. The study emphasises the importance of centring power within co-produced research in humanitarian settings and suggests that while the term co-production is sometimes misappropriated, the principles underlying this concept remain essential to unravelling power hierarchies within the humanitarian sector.

Co-production of a feasibility trial of pacing interventions for Long COVID

BACKGROUND

The high incidence of COVID-19 globally has led to a large prevalence of Long COVID but there is a lack of evidence-based treatments. There is a need to evaluate existing treatments for symptoms associated with Long COVID. However, there is first a need to evaluate the feasibility of undertaking randomised controlled trials of interventions for the condition. We aimed to co-produce a feasibility study of non-pharmacological interventions to support people with Long COVID.

METHODS

A consensus workshop on research prioritisation was conducted with patients and other stakeholders. This was followed by the co-production of the feasibility trial with a group of patient partners, which included the design of the study, the selection of interventions, and the production of dissemination strategies.

RESULTS

The consensus workshop was attended by 23 stakeholders, including six patients. The consensus from the workshop was to develop a clinical trial platform that focused on testing different pacing interventions and resources. For the co-production of the feasibility trial, patient partners selected three pacing resources to evaluate (video, mobile application, and book) and co-designed feasibility study processes, study materials and undertook usability testing of the digital trial platform.

CONCLUSION

In conclusion, this paper reports the principles and process used to co-produce a feasibility study of pacing interventions for Long COVID. Co-production was effective and influenced important aspects of the study.

Experiences of caregivers of community-dwelling older persons with moderate to advanced dementia in adapting the Namaste Care program: a qualitative descriptive study

BACKGROUND

Globally many older persons with dementia are living at home to maintain independence within the community. As older persons with dementia transition from early to moderate or advanced stages of dementia they require more support from family members and friends to complete their daily activities. Family and friend caregivers, however, often report a lack of preparation for their caregiving role. There are few psychosocial programs that can be delivered by caregivers of community-dwelling older persons with moderate to advanced dementia. Namaste Care is a psychosocial intervention, predominantly used in long-term care, to improve the quality of life of persons with advanced dementia. Namaste Care provides multisensory stimulation for persons with dementia through meaningful activities such as music, massage, aromatherapy, and nutrition. There have been limited attempts at adapting Namaste Care for use by caregivers in the community.There is a need to involve caregivers in adapting programs and understanding their experiences in research involvement so that strategies can be put in place for a positive experience. The purpose of this study is to explore the experiences of caregivers who participated in workshop sessions to adapt Namaste Care for community-dwelling older persons with moderate to advanced dementia.

METHODS

A qualitative descriptive design was used. Six caregivers residing in Ontario, Canada attended virtual workshop sessions (i.e., by phone or videoconference) that were guided by the Strategy for Patient-Oriented Research (SPOR) Patient Engagement Framework. Caregivers completed individual post-workshop interviews. Experiential thematic analysis was used to analyze interviews and post-interview researcher notes.

RESULTS

Key findings were that caregivers had a positive experience in adapting Namaste Care by learning how to improve their caregiving skills and being supported to engage in research through multiple facilitators such as flexible scheduling and an inclusive and respectful environment. Having designated time for discussions between caregivers was perceived as important to forming partnerships within the group to support co-creation of knowledge.

CONCLUSION

Findings support the need to improve caregiver research engagement processes by ensuring that caregivers can benefit through learning opportunities and discussions and empowering caregivers to value their contributions in adapting interventions.

Co-creating physical activity interventions: Findings from a multiple case study using mixed methods

Introduction

In health and physical activity promotion, there is growing interest in co-creation approaches that involve researchers and non-academic stakeholders in developing new interventions. Previous research has shown the promising results of cooperative planning as a co-creation approach in building new capacities and implementing physical activity-promoting interventions in nursing care and automotive mechatronics. However, it remains unclear whether (1) cooperative planning for physical activity promotion can be successfully transferred to other settings in the nursing care and automotive mechatronic sectors and (2) what key factors influence its success or failure.

Methods

We conducted a multiple case study in three settings in the nursing care and automotive mechatronics sectors. Following a mixed methods approach, we collected, analyzed, and triangulated data from documents (n = 17), questionnaires (n = 66), and interviews (n = 6). Quantitative data were analyzed descriptively and through using nonparametric analyses of variance; qualitative data were analyzed using qualitative content analysis by extraction.

Results

The transfer of cooperative planning to new settings was realized, though the impact varied by setting. While the interventions were developed and implemented in nursing care settings, interventions were developed but not implemented in the automotive mechatronics setting. In this context, intervention implementation was influenced by 11 key factors: champion, commitment, embedment, empowerment, engagement, health-promoting leadership, ownership, relevance, resources, responsibility, and strategic planning. Furthermore, the transfer of cooperative planning was influenced by different activity characteristics, namely elaboration & reconsideration, group composition, number of meetings, participation, period, prioritization, and researchers' input & support.

Discussion

The present article contributes to a better understanding of a co-creation approach utilized for physical activity promotion and provides new insights into (1) the transferability of cooperative planning and (2) the associated key factors influencing intervention implementation. The success of cooperative planning varied by setting and was influenced by several activity characteristics and key factors, some of which showed complex relationships. This raises the question of whether some settings might benefit more from a co-creation approach than others. Therefore, future co-creation initiatives should carefully consider the specific characteristics of a setting to select and apply the most appropriate approach.

Designing a Health Care Delivery Innovation Lab: Reflections From The First Year

This article shares initial experiences designing and operating a new health care delivery innovation program at a rural academic medical center. The program was designed with the belief that dedicated team member time, senior leadership engagement, deliberate project/team selection, and robust, tailored project support would enable rapid and transformative health care redesign. Three teams were supported in the initial 1-year funding cycle; all 3 teams successfully designed, implemented, and tested new care models for different serious illness populations. Results demonstrated improved satisfaction, decreased length of stay, and a positive return on investment. Critical evaluation of current structures and processes will help identify refined strategies to support diverse teams that will challenge the norms of health care delivery and explore novel partnerships, approaches, and settings for care delivery. This article helps advance the conversation on how to think strategically and critically about current and future health care innovation efforts.

The relationship between the living lab approach and successful implementation of healthcare innovations: an integrative review

OBJECTIVES

The concept of living labs as a research method to enhance participation of end-users in the development and implementation process of an innovation, gained increasing attention over the past decade. A living lab can be characterised by five key components: user-centric, cocreation, real-life context, test innovation and open innovation. The purpose of this integrative literature review was to summarise the literature on the relationship between the living lab approach and successful implementation of healthcare innovations.

METHODS

An integrative literature review searching PubMed, EMBASE, PsycINFO and Cinahl databases between January 2000 and December 2019. Studies were included when a living lab approach was used to implement innovations in healthcare and implementation outcomes were reported. Included studies evaluated at least one of the following implementation outcomes: acceptability, adoption, appropriateness, feasibility, fidelity, implementation cost, penetration or sustainability. Quality was assessed based on a tool developed by Hawker et al.

RESULTS

Of the 1173 retrieved articles, 30 studies were included of which 11 of high quality. Most studies involved a combination of patients/public (N=23) and providers (N=17) as key stakeholders in the living lab approach. Living lab components were mostly applied in the development phase of innovations (N=21). The majority of studies reported on achievement of acceptability (N=22) and feasibility (N=17) in terms of implementation outcomes. A broader spectrum of implementation outcomes was only evaluated in one study. We found that in particular six success factors were mentioned for the added-value of using living lab components for healthcare innovations: leadership, involvement, timing, openness, organisational support and ownership.

CONCLUSIONS

The living lab approach showed to contribute to successful implementation outcomes. This integrative review suggests that using a living lab approach fosters collaboration and participation in the development and implementation of new healthcare innovations.

PROSPERO REGISTRATION NUMBER

CRD42020166895.

Creating interventions to transition long-lasting insecticide net distribution in Ghana

OBJECTIVE

Mass long-lasting insecticide net (LLIN) distribution campaigns are rolled out, as a part of the Ghana Malaria Strategic plan (2021-2025) which seeks to protect at least 80% of the population at risk with effective malaria prevention interventions. Although the mass LLIN distribution campaign indicates a comprehensive stakeholder engagement approach, it does not systematically transition into the basic primary healthcare structures within the Ghana Health Services. This paper presents the process and outcome of creating an innovative social intervention, which focuses on community mobilisation and capacity building of community health officers.

METHODS

This study employed a concurrent triangulation mixed methods approach conducted across six districts in Eastern and Volta regions, Ghana. Findings were synthesised, grouped and further distilled to guide the participatory cocreation workshops. Cocreation involved participatory learning in action technique which is a practical, adaptive research strategy which enabled diverse groups and individuals to learn, work and act together in a cooperative manner.

RESULTS

The results suggest the establishment of a Community Health Advocacy Team (CHAT). This would be necessary in efforts aimed at transitioning LLIN distribution campaign in communities. The role of the CHAT would be centred on key elements of community/social mobilisation and capacity building, all nested in a social and behaviour change communication strategies.

CONCLUSION

The research team is in the process of assessing the acceptability and feasibility of the CHAT intervention with all stakeholders in the various communities. Assessment of the effectiveness of the CHAT intervention would be done at a later time.

Development and modelling of a school-based mental health intervention: the co-production of the R.E.A.C.T. programme

The lack of effective school-based interventions for addressing mental health issues and psychological well-being in young people, particularly those with stakeholder involvement, for reducing test anxiety in adolescents has caused a call for interventions to be developed through the process of co-production with the key stakeholders, i.e. teachers and students. The purpose of this paper is to present the development and modelling of a coproduced school-based intervention to improve mental health and psychological well-being in adolescents in the post-primary setting. The intervention was developed through a six step co-production model. This included an extensive evidence review, interviews (n = 7), focus groups (n = 6), observations in three school settings and initial modelling of the intervention programme and resources in the co-research partner school. Findings were used to identify the preferred structure and content of the intervention. A six-week intervention for 12–14 year olds was co-produced along with relevant teacher resources and student work books. The intervention consisting of a psycho-educational component and physical activity component underpinned by cognitive, behavioural and self-regulation theories aimed to reduce test anxiety and improve psychological well-being. The co-production model was a successful series of six steps used to create and refine the intervention. The programme represents a theoretically informed intervention comprising multiple components. This study contributes to a better understanding of the determinants of mental health issues among young people and how an intervention can be effectively co-produced. The results suggest that a feasibility study is warranted with teachers delivering the programme.

'Getting involved in research': a co-created, co-delivered and co-analysed course for those with lived experience of health and social care services

BACKGROUND

'Getting Involved in Research' was co-created and delivered by a multi-organisational group to provide an accessible introduction to research for those with lived experience of health and social care services.

METHOD

The evaluation of participants' perceptions adopted an exploratory mixed method research design and aimed to gather data to provide an in-depth understanding of the participants' experience of 'Getting Involved in Research' through the co-researchers' analysis of qualitative data using Participatory Theme Elicitation (PTE). PTE was used with the qualitative data to promote co-analysis by the course development group; analyses from an independent academic was also used to further validate the method of PTE.

RESULTS

Thirty-five participants in total participated in 'Getting Involved in Research'. Age ranges varied from 19 to 73 years old. Participants were predominately female (n = 24), five males participated (n = 5) and there was one participant who identified as non-binary (n = 1). Six core themes were identified using the PTE approach: (1) A Meaningful Participatory Approach (2) Increasing the Confidence of Participants (3) Interactive Online Format (4) An Ambient Learning Environment (5) A Desire for Future Courses (6) A Balance of Course Content and Discussion. Participants in 'Getting Involved in Research' reported that the content of the training was applicable, relevant, fostered awareness of research methods and anticipated that it would support their involvement in research.

CONCLUSION

'Getting Involved in Research' has contributed innovatively to the evidence base for how to engage with and motivate those who have experience of health and social care to become actively involved in research. This study demonstrates that 'Getting Involved in Research' may be helpful to train those with lived experience and their care partners however, further research following up on the application of the course learning would be required to ascertain effectiveness.

FUTURE DIRECTIONS

Future research should explore methods to apply research skills in practice to further develop participants' confidence in using the skills gained through 'Getting Involved in Research'.

Examining the sustainability and effectiveness of co-created physical activity interventions in vocational education and training: a multimethod evaluation

Background

Co-creation approaches are increasingly used in physical activity promotion to develop interventions tailored to the target group and setting. The resulting complexity of such interventions raises challenges in evaluation. Accordingly, little is known about the effectiveness of co-created interventions and the underlying processes that impact their sustainable implementation. In this study, we attempt to fill this gap by evaluating co-created multi-component physical activity interventions in vocational education and training in nursing care and automotive mechatronics regarding (1) their sustainable implementation at the institutional level and (2) the effectiveness of single intervention components at the individual level.

Methods

Following a multimethod design, we conducted a questionnaire survey (n = 7) and semi-structured interviews (n = 4) to evaluate the sustainability of the interventions. Quantitative data were analyzed descriptively, and qualitative data were analyzed using qualitative content analysis. To examine the interventions’ effectiveness, we conducted two non-randomized controlled trials (n = 111). Analysis of variance was used to examine differences between groups.

Results

At the institutional level, long-term implementation of single intervention components in nursing care was observed; in contrast, long-term implementation in automotive mechatronics was not observed. In this context, various factors at the outer contextual (e.g., COVID-19 pandemic), inner contextual (e.g., health-promoting leadership), intervention (e.g., acceptance), and personal levels (e.g., champion) influenced sustainability. At the individual level, no significant intervention effects were found for changes in physical activity behavior and physical activity-related health competence.

Conclusion

The role of co-creation on the effectiveness and sustainability of physical activity promotion in vocational education and training cannot be answered conclusively. Only in the nursing care sector, a co-creation approach appeared promising for long-term intervention implementation. Sustainable implementation depends on various influencing factors that should be considered from the outset. Demonstrating effectiveness at the individual level was challenging. To conclusively clarify both the role and impact of co-creation, methodologically complex and elaborate evaluation designs will be required in future research projects.

Trial registration

This study was retrospectively registered at clinicaltrials.gov on 24/08/2021 (NCT05018559).

Supplementary Information

The online version contains supplementary material available at 10.1186/s12889-022-13133-9.

Smartphone-based alcohol interventions: A systematic review on the role of notifications in changing behaviors toward alcohol

Background: Smartphone-based interventions are increasingly being used to facilitate positive behavior change, including reducing alcohol consumption. However, less is known about the effects of notifications to support this change, including intervention engagement and adherence. The aim of this review was to assess the role of notifications in smartphone-based interventions designed to support, manage, or reduce alcohol consumption. Methods: Five electronic databases were searched to identify studies meeting inclusion criteria: (1) studies using a smartphone-based alcohol intervention, (2) the intervention used notifications, and (3) published between 1st January 2007 and 30th April 2021 in English. PROSPERO was searched to identify any completed, ongoing, or planned systematic reviews and meta-analyses of relevance. The reference lists of all included studies were searched. Results: Overall, 14 papers were identified, reporting on 10 different interventions. The strength of the evidence regarding the role and utility of notifications in changing behavior toward alcohol of the reviewed interventions was inconclusive. Only one study drew distinct conclusions about the relationships between notifications and app engagement, and notifications and behavior change. Conclusions: Although there are many smartphone-based interventions to support alcohol reduction, this review highlights a lack of evidence to support the use of notifications (such as push notifications, alerts, prompts, and nudges) used within smartphone interventions for alcohol management aiming to promote positive behavior change. Included studies were limited due to small sample sizes and insufficient follow-up. Evidence for the benefits of smartphone-based alcohol interventions remains promising, but the efficacy of using notifications, especially personalized notifications, within these interventions remain unproven.

Introducing the Practice Dive Approach: an extension of co-creation in physical activity promotion and health promotion

Recently, there has been increasing interest in co-creation utilized for physical activity (PA) promotion and health promotion. Co-creation involves researchers and non-academic stakeholders conjointly developing and implementing interventions. In addition to the frequently reported benefits of co-creation, critical voices highlight the associated challenges (e.g. different interests that inhibit interaction). So far, research has not identified concrete solutions to these challenges and the limitations of co-creation. This article aims to introduce the Practice Dive Approach as a potential way to strengthen cooperation between researchers and non-academic stakeholders. We build on real-life experiences from a German research project, in which researchers moved into practice to familiarize themselves with the settings and end-users. After conducting a literature search on related concepts in PA/health promotion, we developed a comprehensive approach to fostering multi-sectoral cooperation. The introduced Practice Dive Approach assumes that a significant contribution to better cooperation among co-creators is the temporal immersion of researchers in their setting of interest, which has the potential to improve the success of co-creation in the PA/health promotion field. A four-level typology characterizes the intensity of researcher interactions with the setting and the non-academic stakeholders. Potential beneficial effects for both researchers and non-academic stakeholders can be hypothesized (e.g. familiarity with the setting structures and increased understanding of the end-users), while simultaneously, some challenges need to be considered. Future research should aim to validate the concept and its postulated effects.

Collaboration among researchers and non-academic stakeholders is increasingly used to promote physical activity and health. For example, people involved in such collaborations jointly develop new interventions. Potential challenges include different interests or work routines that can complicate cooperation. This article aims to introduce the Practice Dive Approach as a potential way to improve cooperation between researchers and non-academic stakeholders. We developed the approach based on observations from a German research project and a literature search on related concepts. This approach assumes that the temporal involvement of researchers in their setting of interest can strengthen research-practice cooperation and improve its success. We describe different types of a Practice Dive and the requirements for conducting Practice Dive activities. Furthermore, we present the potential effects of a Practice Dive for the researchers and the non-academic stakeholders, such as increased familiarity between both groups. However, some challenges need to be considered when applying the Practice Dive Approach. Future research should test this approach and its potential effects.

An early intervention for adolescent depression targeting emotional mental images and memory specificity: a process evaluation

We have evaluated a novel early intervention for adolescent depression (age 16-18) in a feasibility randomised controlled trial. This nested process evaluation aimed to understand how this complex intervention worked. We sought to understand participants' views and experiences of receiving and interacting with the intervention to evaluate whether the underpinning theoretical basis of the intervention is justified and whether it contributes to valued outcomes for participants. Twelve participants were invited to take part in semi-structured interviews. Framework analysis was employed to identify important aspects of adolescents' experiences. The active ingredients identified by participants were consistent with and extended our understanding of the theoretical basis of the intervention. Four principle themes were identified: understanding how memory works and being able to remember memories in more detail; processing negative experiences and letting go; imagining positive future events; and understanding and being kinder to myself. The outcomes of the intervention were valued by participants. Six principle themes were identified: improving mood and well-being; reducing impact of negative memories; motivation and goal-directed behaviour; overcoming avoidance and rumination; relationships, communication and being open; and self-understanding and acceptance. A simplified logic model is also proposed to connect the intervention components, active ingredients, and valued outcomes. The findings provide an in-depth understanding of how participants interacted with the intervention and what they derived from it. For example, the findings establish processing negative experiences as a core intervention component, extend it to include letting go of these memories, and highlight that reducing the impact of negative memories is valued by participants. This richer understanding guides further intervention development and future implementation.

Research as usual in humanitarian settings? Equalising power in academic-NGO research partnerships through co-production

Background

Research partnerships in conflict-affected and humanitarian settings can reveal complex power hierarchies between academics and NGOs. During the process of research, decision-making may skew in favour of more powerful actors, who often direct the scope of the research, hold the budget and lead the analysis. Co-production is increasingly emerging as a helpful approach that attempts to equalise power dynamics during research. The aim of this paper is to draw attention to the main challenges associated with a “research as usual” approach to research partnerships in humanitarian settings, as power hierarchies may be particularly magnified in these settings.

Methods

This paper is based on a comprehensive literature review and 32 semi-structured interviews with academics and practitioners from non-government organisations. Participants were selected purposively based on their experience in co-producing research or working within research partnerships. Some participants had worked in humanitarian settings while others had experience co-producing research in non-humanitarian contexts. We used Nvivo to thematically code data.

Results

This paper documents the problems with “research as usual” partnerships in humanitarian settings, specifically: the burden on communities as merely sources of data, certain forms of knowledge being valued over others, lack of reflection on the power hierarchies structuring research partnerships, top-down decision-making and lack of transparency, one-way “capacity-building”, lack of mutual benefit, and rigid research processes and timeframes.

Conclusion

This paper highlights key challenges with standard research practices in humanitarian settings and identifies seven key principles of co-production that can be helpful in attempting to equalise power dynamics within research partnerships, specifically in conflict-affected and humanitarian settings.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13031-021-00399-w.

Integrating perspectives of patients, healthcare professionals, system developers and academics in the co-design of a digital information tool

Background

Patients diagnosed with cancer who are due to commence radiotherapy, often, despite the provision of a considerable amount of information, report a range of unmet information needs about the treatment process. Factors such as inadequate provision of information, or the stressful situation of having to deal with information about unfamiliar things, may influence the patient’s ability to comprehend the information. There is a need to further advance the format in which such information is presented. The composition of information should be tailored according to the patient’s individual needs and style of learning.

Method and findings

The PD methodology is frequently used when a technology designed artefact is the desired result of the process. This research is descriptive of its kind and provides a transparent description of the co-design process used to develop an innovative digital information tool employing PD methodology where several stakeholders participated as co-designers. Involving different stakeholders in the process in line with recommended PD activities enabled us to develop a digital information tool that has the potential to be relevant and user-friendly for the ultimate consumer.

Conclusions

Facilitating collaboration, structured PD activities can help researchers, healthcare professionals and patients to co-design patient information that meets the end users’ needs. Furthermore, it can enhance the rigor of the process, ensure the relevance of the information, and finally have a potential to employ a positive effect on the reach of the related digital information tool.

The value of involving patients and public in health services research and evaluation: a qualitative study

BACKGROUND

Public and Patient Involvement, Engagement and Participation research encompasses working with patients/service users (people with a medical condition receiving health service treatment), public members, caregivers and communities (who use services or care for patients). The Partner Priority Programme (PPP) was developed by the National Health Service [NHS] and National Institute for Health Research Collaboration for Leadership in Applied Health Research and Care [NIHR CLAHRC] NWC to share information and experience on evaluating new services being offered to patients that were seeking to reduce health inequalities, improve people's health and wellbeing and reduce emergency hospital admissions. This paper seeks to explore an approach developed for involving the public as equal partners within the evaluation and decision-making processes of health and social care services research. The aim of this study was to identify how public advisors were included, the impact of their involvement, and how change occurred within the organisations following their involvement.

METHODS

A qualitative approach using focus group discussions was adopted to explore the experiences of two cohorts of participants involved in PPP project teams. Focus groups were held with public advisors (n = 9), interns (n = 9; staff or public who received a funded internship for a PPP project), NHS and Local Authority initiative leads (n = 10), and academic facilitators (n = 14). These were transcribed verbatim and analysed using a thematic approach.

RESULTS

Thirty-two public advisors were recruited to support 25 PPP projects across the Collaboration for Leadership in Applied Health Research and CLAHRC North West Coast [NWC] partner organisations. Three inter-related themes were conceptualised: 1)"Where it all started - involving public advisors" identified the varying journeys to recruitment and experiences of becoming a public advisor; 2)"Steps toward active involvement and engagement" related to public advisors becoming core team members; and 3) "Collaborative working to enhance public and patient involvement" relayed how projects identified the benefits of working jointly with the public advisors, particularly for those who had not experienced this style of working before.

CONCLUSIONS

The findings indicate that the PPP model is effective for embedding Public and Patient Involvement [PPI] within health services research, and recommends that PPI is integrated at the earliest opportunity within research projects and service evaluations through the use of support-led and facilitative programmes.

Co-creating physical activity interventions: a mixed methods evaluation approach

BACKGROUND

Co-creation strategies, such as cooperative planning, are promising as a means to ensure that physical activity interventions address real-world problems and are tailored to the target group. This has already been validated in diverse settings. However, questions targeting the transferability of cooperative planning to new settings and the key factors influencing its success or failure remain unclear. At the same time, co-creation processes are complex, and evaluation can be challenging. Following calls for detailed reporting, this paper describes the programme activities, the underlying logic, and methodological design of a study that aims to evaluate the transfer of cooperative planning to new settings and to explore the associated key determinants.

METHODS

Cooperative planning was utilized as a strategy to target physical activity promotion in three real-world German settings in the nursing care and automotive mechatronics sectors. This involved researchers working alongside stakeholders from practice and policy to conjointly develop new interventions to promote physical activity in physically demanding jobs. A pragmatic approach is used to evaluate both the transferability and key determinants of this strategy. We developed a logic model for this co-creation process that describes the underlying assumptions and guides the evaluation. The evaluation outcomes of this study include planning meetings, newly developed interventions, and the determinants that are likely to affect cooperative planning. Quantitative and qualitative data will be collected using questionnaires, documents, and interviews. The quantitative data will be analysed descriptively, while the qualitative data will mainly be analysed using qualitative content analysis, split by settings. Subsequently, data triangulation will be used to integrate the quantitative and qualitative findings, which will then be compared across all three settings.

DISCUSSION

The study findings will contribute to a better understanding of co-creation strategies, their transferability, and key determinants. The practical implications can include a checklist for assessing key determinants and a guideline for transferring cooperative planning into new settings to benefit more people. Ultimately, this study will help to advance co-creation strategies and may be relevant for researchers, practitioners, and policy-makers targeting physical activity promotion in various contexts.

TRIAL REGISTRATION

Open Science Framework: https://osf.io/r6xnt/ (retrospectively registered).

Designing and implementing an intervention for returning citizens living with substance use disorder: discovering the benefits of peer recovery coach involvement in pilot clinical trial decision-making

Peer-facilitated services in behavioral health care remain underutilized within criminal justice-involved community organizations, and there is little guidance for how to best involve peer workers in behavioral health-focused research activities. This paper described lessons learned regarding implementation of peer recovery coaches (PRCs) as part of development and pilot research on Substance Use Programming for Person-Oriented Recovery and Treatment (SUPPORT), a peer-facilitated substance use disorder (SUD) intervention for returning citizens. Qualitative data were collected from learning meetings with key stakeholders and group interviews with SUPPORT clinical trial participants and PRCs. Transcripts were analyzed to identify key decisions made impacting the development, implementation, and/or revision of the SUPPORT intervention and pilot clinical trial protocols. Analysis demonstrated that PRC involvement drove many of the decisions made regarding modifications to the original intervention and trial protocols, while benefitting client-level interactions and by influencing the non-profit agency and its connection to stakeholders. Moreover, PRCs improved the research design by refining the incentive structure and data collection plans. PRC involvement also contributed to the development of more recovery-oriented resources and catering support services to the unique needs of justice-involved individuals. Discussed were the implications for the role of PRCs in justice-involved behavioral health research and treatment.

An Exercise Training and Healthy Eating Group Program (ATHENA) for Overweight and Obese Women with Urinary Incontinence: An Intervention Description

BACKGROUND

Despite strong evidence for supervised pelvic floor muscle training (PFMT) for women with urinary incontinence (UI), and weight loss and exercise for overweight and obese women with UI, implementation literature on these combined interventions is limited. This paper aimed to describe the rigorous and systematic processes involved in the collaborative development, implementation, refinement and evaluation of a novel, holistic 12 week exercise training and healthy eating group program (ATHENA) for overweight and obese women with UI.

METHODS/DESIGN

This intervention description paper is part of a larger mixed-methods feasibility study of implementing the ATHENA intervention within a physiotherapy service at a public hospital in Australia. The collaborative intervention design had input from clinicians, researchers and a consumer representative.

RESULTS

The intervention involved four evidence-based components-(1) supervised PFMT; (2) general exercise training; (3) pelvic health education; and (4) healthy eating education-delivered face to face over a 12 week period. Supporting resources developed included a Facilitator's Guide and Participant Workbook.

CONCLUSION

ATHENA is an evidence-based, multifaceted, group-based intervention targeting exercise training and healthy eating for management of UI for overweight and obese women. The structured development process and transparency of intervention content and resources aims to enhance practical application and success in future studies.

Exploring patient and public involvement (PPI) and co-production approaches in mental health research: learning from the PARTNERS2 research programme

BACKGROUND

Patient and Public Involvement (PPI) in research is a growing field of work, incorporating experiential knowledge within research processes. Co-production is a more recent PPI approach that emphasises the importance of power-sharing to promote inclusive research practices, valuing and respecting knowledge from different sources, and relationship building. Applying co-production principles in research trials can be difficult, and there are few detailed worked examples or toolkits. This paper explores the successes and challenges encountered by one research team.

METHODS

Our paper is written by a team of 21 people working on PARTNERS2, led by a smaller co-ordinating group. Using a co-operative style inquiry, the authors have reflected on and written about their experiences; analysis of the resulting 15 accounts provided examples of how PPI and co-production were delivered in practice.

RESULTS

We reveal varied and complicated experiences as we developed our collaborative approach across the entire research programme. Four main themes emerge from reflective accounts which describe aspects of this process: (1) recognising the importance of 'emotional work'; (2) developing safe spaces to create and share knowledge; (3) some challenges of using our personal identities in research work; and (4) acknowledging power-sharing within the research hierarchy. We also found continual relationship building, how different forms of expertise were valued, and stigma were central to shaping what work was possible together. Other important practices were transparency, particularly over decision making, and clear communication.

CONCLUSIONS

Our work provides one example of the 'messy' nature of collaborative research in practice. The learning we surface was contextual, generated within a large-scale research programme, but applicable to other studies. We found for success there needs to be an acknowledgement of the importance of emotional work, creating safe spaces to co-produce, transparency in decision making and reflection on the difficulties of using personal identities in research work including for service user researchers. These elements are more important than existing guidelines suggest. Implementation of actions to support emotional work, will require changes within individual teams as well as institutions. Introducing reflective practice in teams may be helpful in identifying further improvements to inclusive research practice.

Place4Carers: a mixed-method study protocol for engaging family caregivers in meaningful actions for successful ageing in place

INTRODUCTION

Engaging family caregivers could be a critical asset to make the 'ageing-in-place' imperative a reality. This is particularly evident in rural and remote areas, where caregivers can fill the gaps that exist due to the fragmentation of the welfare system. However, there is little knowledge about the expectations that family caregivers have from healthcare services in rural and remote areas.Place4Carers (P4C) project aims to co-produce an innovative organisational model of social and healthcare services for family caregivers of older citizens living in Vallecamonica (Italy). The project is expected to facilitate ageing-in-place for older citizens, thus helping caregivers in their daily care activities.

METHODS AND ANALYSIS

P4C is a community-based participatory research project featuring five work packages (WPs). WP1 consists of a survey of unmet needs of caregivers and older people receiving services in Vallecamonica. WP2 consists of a scoping literature review to map services that provide interventions of support to caregivers living in remote areas and promote engagement. WP3 organises co-creation workshops with caregivers to co-design, co-manage, and co-assess ideas and proposals for shaping caregiver-oriented services and organisational models. WP3 enriches the results of WP1 (survey) and WP2 (scoping literature review), and aims to co-create new ideas for intervention support with and for caregivers in relation to the objectives, features and characteristics of a new service able to address the caregivers' needs and expectations. WP4 tests the service ideas co-created in WP3 through piloting an intervention based on ideas co-created with caregivers. Finally, WP5 assesses the transferability of the intervention to other similar contexts.

ETHICS AND DISSEMINATION

The study has been approved by the Ethics Committees of the Department of Psychology of Università Cattolica del Sacro Cuore and Politecnico of Milan. Results will be disseminated through peer-reviewed journals, scientific meetings and meetings with the general population.

Partnering with patients to get better outcomes with chimeric antigen receptor T-cell therapy: towards engagement of patients in early phase trials

AIM

Though patient engagement in clinical research is growing, recent reports suggest few clinical trials report on such activities. To address this gap, we describe our approach to patient engagement in the development of a clinical trial protocol to assess a new immunotherapy for blood cancer (chimeric antigen receptor T-cell therapy, CAR-T cell therapy).

METHODS

Our team developed a clinical trial protocol by working with patient partners from inception. Two patient partners with lived blood cancer experience were identified through referrals from our team's professional network and patient organization contacts. Our patient partners were onboarded to the team and engaged in several studies conducted to develop the clinical trial protocol, including a systematic review of the existing literature on the therapy, patient interviews and a survey to obtain perspectives on barriers and enablers to participating in the trial, an early economic analysis, and a retrospective cohort study.

RESULTS

Engaging patient partners enhanced our research in ways that would not have otherwise occurred. By selecting patient important outcomes for data collection, our partners helped flag that quality of life and health utility measures have not been reported in previous CAR-T cell therapy trials for blood cancer. Our partners also co-developed a non-technical summary of the systematic review that summarized results in an accessible manner. Our patient partners reviewed interview and survey questions, to improve the language and appropriateness; provided recruitment suggestions; and provided a patient perspective on the results, thereby confirming the importance of findings. Input was also obtained on costs for the early economic analysis. Our patient partners identified costs that may be a burden to both patients and caregivers during a trial and helped to confirm that the overall structure of the economic model reflected the patient care pathway. Our patient partners also shared their diagnosis and treatment stories, which helped to provide the research team with insight into this experience.

CONCLUSIONS

Contributions by our patient partners were invaluable to each component study, as well as the overall development of the trial protocol. We plan to use this approach in the future in order to meaningfully engage patients in the development of other clinical trials; we also hope that by reporting our methods this will help other research teams to do the same.

TRIAL REGISTRATION

Affiliated with the development of NCT03765177.

Co-producing a shared understanding and definition of empowerment with people with dementia

BACKGROUND

Empowerment for people with dementia (PWD) is not well defined within the research literature and we feel that this is an important area for development. It is important to seek, consult, and co-produce such a definition with PWD who are more actively involved in their communities post diagnosis (e.g. no longer the 'long goodbye'). This study seeks to combine academic literature review methods with participatory/co-production methods in order to address this gap. We feel this approach also adds to developing methodologies in the field of co-production and user involvement.

METHODS

We use a unique approach toward a definition of empowerment for PWD. Phase 1 - A scoping review of medical/health, social care and social policy-based databases to identify any previous literature that may have defined empowerment exclusively for PWD. Based on this literature, we collected a list of terms relating to empowerment for PWD. Phase 2 - Using empowerment key terms set on cards formulated from Phase 1 across three co-production workshops, academic team members, and nine members of Dementia NI (an organisation founded and led by people with dementia) we reviewed the findings of this search and co-produced an agreed definition they felt best described empowerment for them.

RESULTS

Phase 1 and 2 led to a definition of empowerment relevant to PWD. This shared understanding of empowerment was defined by PWD as 'A confidence building process whereby PWD are respected, have a voice and are heard, are involved in making decisions about their lives and have the opportunity to create change through access to appropriate resources'.

CONCLUSIONS

The strength of this research lies in addressing the current confusion and arbitrariness of empowerment within the context of dementia. This coproduced work also provides evidence for not only the possibility, but also the added value of involving PWD in research in terms of unique insights afforded by their lived experiences.

Learning as an outcome of involvement in research: what are the implications for practice, reporting and evaluation?

Public involvement in research has evolved over the last two decades in a culture dominated by the principles of evidence-based medicine. It is therefore unsurprising that some researchers have applied the same thinking to involvement, particularly to involvement in research projects. This may explain why they tend to conceptualise involvement as an intervention, seek to evaluate its impact in the same way that treatments are tested, highlight the need for an evidence-base for involvement, and use the language of research to describe its practice and report its outcomes. In this article we explore why this thinking may be unhelpful. We suggest an alternative approach that conceptualises involvement as 'conversations that support two-way learning'. With this framing, there is no 'method' for involvement, but a wide range of approaches that need to be tailored to the context and the needs of the individuals involved. The quality of the interaction between researchers and the public becomes more important than the process. All parties need to be better prepared to offer and receive constructive criticism and to engage in constructive conflict that leads to the best ideas and decisions. The immediate outcomes of involvement in terms of what researchers learn are subjective (specific to the researcher) and unpredictable (because researchers don't know what they don't know at the start). This makes it challenging to quantify such outcomes, and to carry out comparisons of different approaches. On this basis, we believe obtaining 'robust evidence' of the outcomes of involvement in ways that are consistent with the values of evidence-based medicine, may not be possible or appropriate. We argue that researchers' subjective accounts of what they learnt through involvement represent an equally valid way of knowing whether involvement has made a difference. Different approaches to evaluating and reporting involvement need to be adopted, which describe the details of what was said and learnt by whom (short term outcomes), what changes were made as a result (medium term outcomes), and the long-term, wider impacts on the research culture and agenda. Sharing researchers' personal accounts may support wider learning about how involvement works, for whom and when.