Remote working in public involvement: findings from a mixed methods study

Patient and public involvement and engagement in the ASCEND PLUS trial: reflections from the design of a streamlined and decentralised clinical trial

INTRODUCTION

ASCEND PLUS is a randomised controlled trial assessing the effects of oral semaglutide on the primary prevention of cardiovascular events in around 20,000 individuals with type 2 diabetes in the UK. The trial's innovative design includes a decentralised direct-to-participant invitation, recruitment, and follow-up model, relying on self-completion of online forms or telephone or video calls with research nurses, with no physical sites. Extensive patient and public involvement and engagement (PPIE) was essential to the design and conduct of ASCEND PLUS.

AIM

To report the process and conduct of PPIE activity in ASCEND PLUS, evaluate effects on trial design, reflect critically on successes and aspects that could have been improved, and identify themes and learning relevant to implementation of PPIE in future trials.

METHODS

PPIE activity was coordinated centrally and included six PPIE focus groups and creation of an ASCEND PLUS public advisory group (PAG) during the design phase. Recruitment to these groups was carefully considered to ensure diversity and inclusion, largely consisting of adults living with type 2 diabetes from across the UK. Two members of the PAG also joined the trial Steering Committee. Steering Committee meetings, focus groups, and PAG meetings were conducted online, with two hybrid workshops to discuss PPIE activity and aspects of the trial.

RESULTS

PPIE activity was critical to shaping the design and conduct of ASCEND PLUS. Key examples included supporting choice for participants to either complete the screening/consent process independently online, or during a telephone or video call interview with a research nurse. A concise 'initial information leaflet' was developed to be sent with the initial invitations, with the 'full' information leaflet sent later to those interested in joining the trial. The PAG reviewed the content and format of participant- and public-facing materials, including written documents, online screening forms, animated videos, and the trial website, to aid clarity and accessibility, and provided input into the choice of instruments to assess quality of life.

CONCLUSIONS

PPIE is integral in ASCEND PLUS and will continue throughout the trial. This involvement has been critical to optimising the trial design, successfully obtaining regulatory and ethical approval, and conducting the trial.

Sustaining connections: feasibility and impact of long-term virtual patient engagement

BACKGROUND

Virtual patient engagement has become more common in recent years. Emerging research suggests virtual engagement can increase accessibility for patients managing long-term health conditions and those living in larger geographic areas, but it can also be challenging to establish relationships and maintain engagement over time. Little is known about virtual engagement lasting more than two years, nor about the specific contributions of patients to virtual engagement projects. Here we describe a project where virtual engagement was sustained over a long period of time (3.5 years), measure patients' contributions to the work, and describe the facilitators and challenges of the project using the Valuing All Voices (VAV) patient engagement framework.

METHODS

Five researchers recruited four patient partners living with persistent pain to work together virtually on a project to improve care for others with long-term pain. Researchers documented engagement activities and patient partner contributions and categorized them using Carman et al.'s 3 types of engagement. They also collected data via semi-structured group interviews with patient partners about the facilitators and challenges of the project using the VAV framework.

RESULTS

In 3.5 years, patient partners contributed 487 h to the project, averaging 3.0 h per month, and participated in 40 meetings. They contributed to 17 products for patients, health care teams, and researchers. Most products (12 of 17) were created using the more in-depth engagement approaches of involvement or partnership and shared leadership. The group identified facilitators of the project across the five VAV domains of relationship-building, trust, understanding & acceptance, education & communication, and self-awareness, as well as some specific challenges such as keeping track of products across virtual platforms and managing the high volume of project information.

CONCLUSIONS

Long-term virtual patient engagement is feasible and can use more in-depth engagement approaches. Additionally, it can result in substantial contributions from patients in terms of time, effort, and products. These findings can inform future long-term virtual patient engagement efforts and provide insight into how researchers can structure their activities to encourage and maintain deep engagement over time.

Evaluating the impact of engaging older adults and service providers as research partners in the co-design of a community mobility-promoting program: a mixed methods developmental evaluation study

BACKGROUND

Increasingly researchers are partnering with citizens and communities in research; less is known about research impacts of this engagement. EMBOLDEN is an evidence-informed, mobility-promoting intervention for older adults co-designed by a 26-person Strategic Guiding Council (SGC) of health/social service providers and older adult citizens. This study evaluated research partners' perceptions of engagement strategies, the engagement context, strengths, areas for improvement, as well as the impacts of the guiding council on older-adult identified priority areas.

METHODS

This study was guided by developmental evaluation, working in partnership with four older adult SGC members who helped to set evaluation priorities, decide methods, and adapt patient-centred evaluation tools. Data sources included a questionnaire, focus groups and document analysis of meeting notes from 16 SGC meetings that took place between December 2019 and February 2022. A thematic approach to analysis guided the coding of focus group transcripts and SGC meeting notes. Convergent mixed methods guided the integration and presentation of qualitative and quantitative data sources in a joint display of evaluation results.

RESULTS

Of 26 SGC members, nine completed the evaluation squestionnaire, and five participated in focus groups. Around two thirds of the SGC commonly attended each meeting. EMBOLDEN's SGC was structured to include a diverse group (across gender, ethnicity and discipline) of older adults and service providers, which was perceived as a strength. Engagement processes were perceived as inclusive and well-facilitated, which stimulated discussion at meetings. Advantages and disadvantages of engaging with the SGC virtually, as compared to in-person (as was the case for the first 3 SGC meetings) were also discussed. Impacts of the SGC were identified across preparatory, execution phase and translational stages of research. Impacts of SGC involvement on members were also described.

CONCLUSION

Older adult research partners played an important role designing, implementing, and evaluating co-design approaches in this study. Older adults and service providers can make important contributions to the design, delivery and sharing results of health research through their lived expertise and connections to community. This project contributes to the growing field of citizen and community engagement in research by offering a participatory approach to engagement evaluation that considers diversity, satisfaction, and impact.

Association Between the Characteristics of mHealth Apps and User Input During Development and Testing: Secondary Analysis of App Assessment Data

BACKGROUND

User involvement is increasingly acknowledged as a central part of health care innovation. However, meaningful user involvement during the development and testing of mobile health apps is often not fully realized.

OBJECTIVE

This study aims to examine in which areas user input is most prevalent and whether there is an association between user inclusion and compliance with best practices for mobile health apps.

METHODS

A secondary analysis was conducted on an assessment data set of 1595 health apps. The data set contained information on whether the apps had been developed or tested with user input and whether they followed best practices across several domains. Background information was also available regarding the apps' country of origin, targeted condition areas, subjective user ratings, download numbers, and risk (as per the National Institute for Health and Care Excellence Evidence Standards Framework [ESF]). Descriptive statistics, Mann-Whitney U tests, and Pearson chi-square analyses were applied to the data.

RESULTS

User involvement was reported by 8.71% (139/1595) of apps for only the development phase, by 33.67% (537/1595) of apps for only the testing phase, by 21.88% (349/1595) of apps for both phases, and by 35.74% (570/1595) of apps for neither phase. The highest percentage of health apps with reported user input during development was observed in Denmark (19/24, 79%); in the condition areas of diabetes (38/79, 48%), cardiology (15/32, 47%), pain management (20/43, 47%), and oncology (25/54, 46%); and for high app risk (ESF tier 3a; 105/263, 39.9%). The highest percentage of health apps with reported user input during testing was observed in Belgium (10/11, 91%), Sweden (29/34, 85%), and France (13/16, 81%); in the condition areas of neurodiversity (42/52, 81%), respiratory health (58/76, 76%), cardiology (23/32, 72%), and diabetes (56/79, 71%); and for high app risk (ESF tier 3a; 176/263, 66.9%). Notably, apps that reported seeking user input during testing demonstrated significantly more downloads than those that did not (P=.008), and user inclusion was associated with better compliance with best practices in clinical assurance, data privacy, risk management, and user experience.

CONCLUSIONS

The countries and condition areas in which the highest percentage of health apps with user involvement were observed tended to be those with higher digital maturity in health care and more funding availability, respectively. This suggests that there may be a trade-off between developers' willingness or ability to involve users and the need to meet challenges arising from infrastructure limitations and financial constraints. Moreover, the finding of a positive association between user inclusion and compliance with best practices indicates that, where no other guidance is available, users may benefit from prioritizing health apps developed with user input as the latter may be a proxy for broader app quality.

Test, evidence, transition projects in Scotland: developing the evidence needed for transition of effective interventions in cancer care from innovation into mainstream practice

BACKGROUND

A robust evidence base is required to assist healthcare commissioners and providers in selecting effective and sustainable approaches to improve cancer diagnosis and treatment. Such evidence can be difficult to build, given the fast-paced and highly pressured nature of healthcare delivery, the absence of incentives, and the presence of barriers in conducting pragmatic yet robust research evaluations. Cancer Research UK (CRUK) has played an active part in building the evidence base through its funding of programmes to identify, evaluate and scale-up innovative approaches across the UK. The aim of this paper is to describe and explain the research design and intended approach and activities for two cancer services improvement projects in Scotland funded by CRUK.

METHODS

A hybrid effectiveness-implementation study design will assess both the efficiency of the new pathways and their implementation strategies, with the aim of generating knowledge for scale-up. A range of implementation, service and clinical outcomes will be assessed as determined by the projects' Theories of Change (ToCs). A naturalistic case study approach will enable in-depth exploration of context and process, and the collection and synthesis of data from multiple sources including routine datasets, patient and staff surveys, in-depth interviews and observational and other data. The evaluations are informed throughout by a patient/public representatives' group, and by small group discussions with volunteer cancer patients.

DISCUSSION

Our approach has been designed to provide a holistic understanding of how (well) the improvement projects work (in relation to their anticipated outcomes), and how they interact with their wider contexts. The evaluations will help identify barriers, facilitators, and unanticipated consequences that can impact scalability, sustainability and spread. By opting for a pragmatic, participatory evaluation research design, we hope to inform strategies for scaling up successful innovations while addressing challenges in a targeted manner.

Participatory evaluation of the process of co-producing resources for the public on data science and artificial intelligence

BACKGROUND

The growth of data science and artificial intelligence offers novel healthcare applications and research possibilities. Patients should be able to make informed choices about using healthcare. Therefore, they must be provided with lay information about new technology. A team consisting of academic researchers, health professionals, and public contributors collaboratively co-designed and co-developed the new resource offering that information. In this paper, we evaluate this novel approach to co-production.

METHODS

We used participatory evaluation to understand the co-production process. This consisted of creative approaches and reflexivity over three stages. Firstly, everyone had an opportunity to participate in three online training sessions. The first one focused on the aims of evaluation, the second on photovoice (that included practical training on using photos as metaphors), and the third on being reflective (recognising one's biases and perspectives during analysis). During the second stage, using photovoice, everyone took photos that symbolised their experiences of being involved in the project. This included a session with a professional photographer. At the last stage, we met in person and, using data collected from photovoice, built the mandala as a representation of a joint experience of the project. This stage was supported by professional artists who summarised the mandala in the illustration.

RESULTS

The mandala is the artistic presentation of the findings from the evaluation. It is a shared journey between everyone involved. We divided it into six related layers. Starting from inside layers present the following experiences (1) public contributors had space to build confidence in a new topic, (2) relationships between individuals and within the project, (3) working remotely during the COVID-19 pandemic, (4) motivation that influenced people to become involved in this particular piece of work, (5) requirements that co-production needs to be inclusive and accessible to everyone, (6) expectations towards data science and artificial intelligence that researchers should follow to establish public support.

CONCLUSIONS

The participatory evaluation suggests that co-production around data science and artificial intelligence can be a meaningful process that is co-owned by everyone involved.

Adapting Patient and Public Involvement processes in response to the Covid-19 pandemic

BACKGROUND

The COVID-19 pandemic brought rapid and major changes to research, and those wishing to carry out Patient and Public Involvement (PPI) activities faced challenges, such as restrictions on movement and contact, illness, bereavement and risks to potential participants. Some researchers moved PPI to online settings during this time but remote consultations raise, as well as address, a number of challenges. It is important to learn from PPI undertaken in this period as face-to-face consultation may no longer be the dominant method for PPI.

METHODS

UK stay-at-home measures announced in March 2020 necessitated immediate revisions to the intended face-to-face methods of PPI consultation for the ESORT Study, which evaluated emergency surgery for patients with common acute conditions. PPI plans and methods were modified to all components being online. We describe and reflect on: initial plans and adaptation; recruitment; training and preparation; implementation, contextualisation and interpretation. Through first-hand accounts we show how the PPI processes were developed, experienced and viewed by different partners in the process.

DISCUSSION AND CONCLUSIONS

While concerns have been expressed about the possible limiting effects of forgoing face-to-face contact with PPI partners, we found important benefits from the altered dynamic of the online PPI environment. There were increased opportunities for participation which might encourage the involvement of a broader demographic, and unexpected benefits in that the online platform seemed to have a 'democratising' effect on the meetings, to the benefit of the PPI processes and outcomes. Other studies may however find that their particular research context raises particular challenges for the use of online methods, especially in relation to representation and inclusion, as new barriers to participation may be raised. It is important that methodological challenges are addressed, and researchers provide detailed examples of novel methods for discussion and empirical study.

PATIENT AND PUBLIC CONTRIBUTION

We report a process which involved people with lived experience of emergency conditions and members of the public. A patient member was involved in the design and implementation, and two patients with lived experience contributed to the manuscript.

Strategies for expanding patient and public involvement into under-served head and neck cancer communities

PURPOSE OF REVIEW

In this review, we discuss two case studies in which we summarize the process of expanding patient and public involvement (PPI) representation specific to the head and neck cancer population, highlighting the challenges and successes within each project. The first case study reports on the expansion of HaNC PPI membership, a long-established PPI forum which supports Liverpool Head and Neck Centre research. The second case study describes the establishment of a novel palliative care network in head and neck cancer in the North of England, in which PPI was central to the project's success.

RECENT FINDINGS

Whilst recognizing diversity is important, it is imperative to acknowledge the contribution of existing members. Engagement with clinicians to reduce issues of gatekeeping is essential. A central theme is that the development of sustainable relationships is critical.

SUMMARY

The case studies highlight the challenge of identifying and accessing such a diverse population, particularly in the context of palliative care. Successful PPI is dependent on building and maintaining relationship with PPI members, ensuring flexibility in terms of timing, platforms and venues. Relationships formation should not be restricted to the academic-PPI representative dyad but should consider both clinical-academic and community partnerships to ensure those who are part of under-served communities are afforded the opportunities to become involved in research.

Public contributors' preferences for the organization of remote public involvement meetings in health and social care: A discrete choice experiment study

INTRODUCTION

Covid-19 expanded the use of remote working to engage with public contributors in health and social care research. These changes have the potential to limit the ability to participate in patient and public involvement and engagement (PPIE) for some public contributors. It is therefore important to understand public contributors' preferences, so that remote working can be organized in an optimal way to encourage rather than discourage participation.

METHODS

We use an economic preference elicitation tool, a discrete choice experiment (DCE), via an online survey, to estimate public contributors' preferences for and trade-offs between different features of remote meetings. The features were informed by previous research to include aspects of remote meetings that were relevant to public contributors and amenable to change by PPIE organizers.

RESULTS

We found that public contributors are more likely to participate in a PPIE project involving remote meetings if they are given feedback about participation; allowed to switch their camera off during meetings and step away if/when needed; were under 2.5 h long; organized during working hours, and are chaired by a moderator who can ensure that everyone contributes. Different combinations of these features can cause estimated project participation to range from 23% to 94%. When planning PPIE and engaging public contributors, we suggest that resources are focused on training moderators and ensuring public contributors receive meeting feedback.

DISCUSSION AND CONCLUSION

Project resources should be allocated to maximize project participation. We provide recommendations for those who work in public involvement and organize meetings on how resources, such as time and financial support, should be allocated. These are based on the preferences of existing public contributors who have been involved in health and social care research.

PATIENT OR PUBLIC CONTRIBUTION

We had a public contributor (Naheed Tahir) as a funded coapplicant on the UKRI ESRC application and involved members of the North West Coast Applied Research Collaboration (NWC ARC) Public Advisor Forum at every stage of the project. The survey design was informed from three focus groups held with NWC ARC public contributors. The survey was further edited and improved based on the results of six one-to-one meetings with public contributors.