Stakeholder involvement in systematic reviews: a protocol for a systematic review of methods, outcomes and effects

Improving oral health and related health behaviours (substance use, smoking, diet) in people with severe and multiple disadvantage: A systematic review of effectiveness and cost-effectiveness of interventions

BACKGROUND

People experiencing homelessness co-occurring with substance use or offending ('severe and multiple disadvantage' SMD) often have high levels of poor oral health and related health behaviours (particularly, substance use, smoking, poor diet). This systematic review aimed to assess the effectiveness and cost-effectiveness of interventions in adults experiencing SMD to improve oral health and related health behaviours.

METHODS AND FINDINGS

From inception to February 2023, five bibliographic databases (MEDLINE, EMBASE, PsycINFO, CINAHL, and Scopus) and grey literature were searched. Two researchers independently screened the search results. Randomized controlled trials (RCTs), comparative studies and economic evaluations were included that reported outcomes on oral health and the related health behaviours. Risk of bias was assessed and results narratively synthesized. Meta-analyses were performed where appropriate. This review was registered with PROSPERO (reg. no: CRD42020202416). Thirty-eight studies were included (published between 1991 and 2023) with 34 studies reporting about effectiveness. Most studies reported on substance use (n = 30). Interventions with a combination of housing support with substance use and mental health support such as contingent work therapy appeared to show some reduction in substance use in SMD groups. However, meta-analyses showed no statistically significant results. Most studies had short periods of follow-up and high attrition rates. Only one study reported on oral health; none reported on diet. Three RCTs reported on smoking, of which one comprising nicotine replacement with contingency management showed improved smoking abstinence at 4 weeks compared to control. Five studies with economic evaluations provided some evidence that interventions such as Housing First and enhanced support could be cost-effective in reducing substance use.

CONCLUSION

This review found that services such as housing combined with other healthcare services could be effective in improving health behaviours, particularly substance use, among SMD groups. Gaps in evidence also remain on oral health improvement, smoking, and diet. High quality studies on effectiveness with adequate power and retention are needed to address these significant health challenges in SMD populations.

Developing a Comprehensive List of Criteria to Evaluate the Characteristics and Quality of eHealth Smartphone Apps: Systematic Review

BACKGROUND

The field of eHealth is growing rapidly and chaotically. Health care professionals need guidance on reviewing and assessing health-related smartphone apps to propose appropriate ones to their patients. However, to date, no framework or evaluation tool fulfills this purpose.

OBJECTIVE

Before developing a tool to help health care professionals assess and recommend apps to their patients, we aimed to create an overview of published criteria to describe and evaluate health apps.

METHODS

We conducted a systematic review to identify existing criteria for eHealth smartphone app evaluation. Relevant databases and trial registers were queried for articles. Articles were included that (1) described tools, guidelines, dimensions, or criteria to evaluate apps, (2) were available in full text, and (3) were written in English, French, German, Italian, Portuguese, or Spanish. We proposed a conceptual framework for app evaluation based on the dimensions reported in the selected articles. This was revised iteratively in discussion rounds with international stakeholders. The conceptual framework was used to synthesize the reported evaluation criteria. The list of criteria was discussed and refined by the research team.

RESULTS

Screening of 1258 articles yielded 128 (10.17%) that met the inclusion criteria. Of these 128 articles, 30 (23.4%) reported the use of self-developed criteria and described their development processes incompletely. Although 43 evaluation instruments were used only once, 6 were used in multiple studies. Most articles (83/128, 64.8%) did not report following theoretical guidelines; those that did noted 37 theoretical frameworks. On the basis of the selected articles, we proposed a conceptual framework to explore 6 app evaluation dimensions: context, stakeholder involvement, features and requirements, development processes, implementation, and evaluation. After standardizing the definitions, we identified 205 distinct criteria. Through consensus, the research team relabeled 12 of these and added 11 more-mainly related to ethical, legal, and social aspects-resulting in 216 evaluation criteria. No criteria had to be moved between dimensions.

CONCLUSIONS

This study provides a comprehensive overview of criteria currently used in clinical practice to describe and evaluate apps. This is necessary as no reviewed criteria sets were inclusive, and none included consistent definitions and terminology. Although the resulting overview is impractical for use in clinical practice in its current form, it confirms the need to craft it into a purpose-built, theory-driven tool. Therefore, in a subsequent step, based on our current criteria set, we plan to construct an app evaluation tool with 2 parts: a short section (including 1-3 questions/dimension) to quickly disqualify clearly unsuitable apps and a longer one to investigate more likely candidates in closer detail. We will use a Delphi consensus-building process and develop a user manual to prepare for this undertaking.

TRIAL REGISTRATION

PROSPERO International Prospective Register of Systematic Reviews CRD42021227064; https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42021227064.

Patients' perspectives on the relevance of biosimilars' outcomes in oncology: qualitative study with nominal group technique

PURPOSE

We aimed to rate the importance of outcomes from a systematic review about biosimilars in oncology from patients' perspective.

METHODS

This is a qualitative research with nominal group technique. Patients with cancer were selected by convenience sampling and invited for two mediated virtual meetings in 2022. Twelve outcomes from a systematic review on biosimilars for oncology developed following a protocol were explained in plain language to participants who classified them as critical, important, or not important according to the Grading of Recommendations Assessment, Development and Evaluation approach. We employed Iramuteq software for lexical categorization of the meeting transcripts, and content analysis for interpretation.

RESULTS

Five women participated (three had metastatic cancer, one non-metastatic, one recurrent). Six outcomes were classified as critical: duration of response, progression-free survival, pathological complete response, overall survival, severe adverse events, and quality of life; three as important: mortality, event-free survival, and objective response; and three as non-important: neutralizing anti-drug antibody, any adverse event, and non-neutralizing anti-drug antibody. Duration of response, pathological complete response, severe adverse events, and quality of life were considered secondary in the review protocol, but critical by the patients. The main themes influencing the importance classification were related to the disease (progression and control) and treatment (recognition and healthcare setting).

CONCLUSION

Patients rated most outcomes as critical or important, some of them previously regarded as secondary by the researchers, which reinforces the need to include stakeholders' perspectives in oncology research. Aspects of the disease progression and treatment effects influenced participants' judgment on outcomes' relevance.

Exercise therapy for tendinopathy: a mixed-methods evidence synthesis exploring feasibility, acceptability and effectiveness

Background

Tendinopathy is a common, painful and functionally limiting condition, primarily managed conservatively using exercise therapy.

Review questions

(i) What exercise interventions have been reported in the literature for which tendinopathies? (ii) What outcomes have been reported in studies investigating exercise interventions for tendinopathy? (iii) Which exercise interventions are most effective across all tendinopathies? (iv) Does type/location of tendinopathy or other specific covariates affect which are the most effective exercise therapies? (v) How feasible and acceptable are exercise interventions for tendinopathies?

Methods

A scoping review mapped exercise interventions for tendinopathies and outcomes reported to date (questions i and ii). Thereafter, two contingent systematic review workstreams were conducted. The first investigated a large number of studies and was split into three efficacy reviews that quantified and compared efficacy across different interventions (question iii), and investigated the influence of a range of potential moderators (question iv). The second was a convergent segregated mixed-method review (question v). Searches for studies published from 1998 were conducted in library databases (n = 9), trial registries (n = 6), grey literature databases (n = 5) and Google Scholar. Scoping review searches were completed on 28 April 2020 with efficacy and mixed-method search updates conducted on 19 January 2021 and 29 March 2021.

Results

Scoping review - 555 included studies identified a range of exercise interventions and outcomes across a range of tendinopathies, most commonly Achilles, patellar, lateral elbow and rotator cuff-related shoulder pain. Strengthening exercise was most common, with flexibility exercise used primarily in the upper limb. Disability was the most common outcome measured in Achilles, patellar and rotator cuff-related shoulder pain; physical function capacity was most common in lateral elbow tendinopathy. Efficacy reviews - 204 studies provided evidence that exercise therapy is safe and beneficial, and that patients are generally satisfied with treatment outcome and perceive the improvement to be substantial. In the context of generally low and very low-quality evidence, results identified that: (1) the shoulder may benefit more from flexibility (effect sizeResistance:Flexibility = 0.18 [95% CrI 0.07 to 0.29]) and proprioception (effect sizeResistance:Proprioception = 0.16 [95% CrI -1.8 to 0.32]); (2) when performing strengthening exercise it may be most beneficial to combine concentric and eccentric modes (effect sizeEccentricOnly:Concentric+Eccentric = 0.48 [95% CrI -0.13 to 1.1]; and (3) exercise may be most beneficial when combined with another conservative modality (e.g. injection or electro-therapy increasing effect size by ≈0.1 to 0.3). Mixed-method review - 94 studies (11 qualitative) provided evidence that exercise interventions for tendinopathy can largely be considered feasible and acceptable, and that several important factors should be considered when prescribing exercise for tendinopathy, including an awareness of potential barriers to and facilitators of engaging with exercise, patients' and providers' prior experience and beliefs, and the importance of patient education, self-management and the patient-healthcare professional relationship.

Limitations

Despite a large body of literature on exercise for tendinopathy, there are methodological and reporting limitations that influenced the recommendations that could be made.

Conclusion

The findings provide some support for the use of exercise combined with another conservative modality; flexibility and proprioception exercise for the shoulder; and a combination of eccentric and concentric strengthening exercise across tendinopathies. However, the findings must be interpreted within the context of the quality of the available evidence.

Future work

There is an urgent need for high-quality efficacy, effectiveness, cost-effectiveness and qualitative research that is adequately reported, using common terminology, definitions and outcomes.

Study registration

This project is registered as DOI: 10.11124/JBIES-20-00175 (scoping review); PROSPERO CRD 42020168187 (efficacy reviews); https://osf.io/preprints/sportrxiv/y7sk6/ (efficacy review 1); https://osf.io/preprints/sportrxiv/eyxgk/ (efficacy review 2); https://osf.io/preprints/sportrxiv/mx5pv/ (efficacy review 3); PROSPERO CRD42020164641 (mixed-method review).

Funding

This project was funded by the National Institute for Health and Care Research (NIHR) HTA programme and will be published in full in HTA Journal; Vol. 27, No. 24. See the NIHR Journals Library website for further project information.

Incorporating the Concept of Relevance in Clinical Rehabilitation Research and Its Reviews May Improve Uptake by Stakeholders

ABSTRACT

The "relevance" of research to stakeholders is an important factor in influencing the uptake of new knowledge into practice; however, this concept is neither well defined nor routinely incorporated in clinical rehabilitation research. Developing a uniform definition, measurement standards, stakeholder engagement strategies, and guiding frameworks that bolster relevance may help incorporate the concept as a key element in research planning and design. This article presents a conceptual argument for why relevance matters, proposes a working definition, and suggests strategies for operationalizing the construct in the context of clinical rehabilitation research. We place special emphasis on the importance of promoting relevance to patients, caregivers, and clinicians and provide preliminary frameworks and innovative study designs that can assist clinical rehabilitation researchers in doing so. We argue that researchers who include a direct statement regarding why and to whom a study is relevant and who incorporate considerations of relevance throughout all phases of study design produce more useful research for patients, caregivers, and clinicians, increasing its chance of uptake into practice. Consistent consideration of relevance, particularly to nonacademic audiences, during the conceptualization, study design, presentation, and dissemination of clinical rehabilitation research may promote the uptake of findings by patients, caregivers, and providers.

Evaluation of Social Impact Within Primary School Health Promotion: A Systematic Review

BACKGROUND

Health promotion programs and interventions are designed to encourage behavioral changes in children, encouraging them to make safe and healthy life choices. This systematic review seeks to examine how social impact is measured in primary school health promotion interventions.

METHOD

A systematic search and review process was used to identify and examine primary school health promotion interventions. The PRISMA guidelines were followed to source articles from 6 electronic databases reporting school health promotion programs or interventions in Australia, Canada, New Zealand, or the United Kingdom.

RESULTS

A total of 77 studies were located, representing 55 health promotion interventions delivered in primary school settings. Of these interventions, only 8 (15%) measured or attempted to measure social impact, whereas another 8 (15%) alluded to social impact. The predominant theories reported were social based theories (theories which examine the social influences on people, environments, and behaviors) (n = 17, 59%), with almost a third not informed by an overt health promotion framework or model (n = 34, 59%). A systematic rating system identified some level of stakeholder engagement (n = 30, 53%).

CONCLUSIONS

This systematic review highlights the need for social impact measurement within health promotion to illuminate the role of school programs in delivering lasting change.

Clinical ethics support services in paediatric practice: protocol for a mixed studies systematic review on structures, interventions and outcomes

INTRODUCTION

Clinical ethics support services (CESS) have been developing worldwide with growing interest in evaluating their quality. Paediatric-specific CESSs (p-CESS) have received little attention, and evidence from adult services might not be generalisable. Evidence on service models and practices is crucial to inform further research and debate on quality evaluation and minimum standards for p-CESSs. We aim to systematically identify, appraise and synthesise evidence for p-CESS structures, processes and outcomes.

METHODS AND ANALYSIS

We will conduct a mixed-studies systematic review including peer-reviewed empirical studies published in English or Spanish language providing data on the evaluation and/or impact on any aspect of p-CESS. We will search seven electronic databases: MEDLINE, Philosopher's Index, EMBASE, PsycINFO, LILACS, Web of Science and CINHAL, without filters applied. Search terms will be related to "clinical ethics support" AND "paediatrics" AND "structure/process/outcome". Reference and citation list of included studies will be handsearched. A 10% random sample of retrieved titles/abstracts and all full texts will be independently dual-screened. We will conduct narrative and thematic synthesis for quantitative and qualitative data, respectively, following sequential explanatory synthesis guided by Donabedian's framework of structure, process and outcomes. Quality will be assessed using the Mixed-Methods Appraisal Tool (2018). The review will be reported using the adapted Preferred Reporting Items for Systematic Reviews and Meta-Analyses for reporting systematic reviews of qualitative and quantitative evidence template. Stakeholders will be involved twice in the review process; prior to data extraction and synthesis and after preliminary results.

ETHICS AND DISSEMINATION

As a systematic review of published data, no ethical approval is necessary. Results will be published in a relevant academic peer-reviewed journal.

PROSPERO REGISTRATION NUMBER

CRD42021280978.

Moving from consultation to co-creation with knowledge users in scoping reviews: guidance from the JBI Scoping Review Methodology Group

ABSTRACT

Knowledge user consultation is often limited or omitted in the conduct of scoping reviews. Not including knowledge users within the conduct and reporting of scoping reviews could be due to a lack of guidance or understanding about what consultation requires and the subsequent benefits. Knowledge user engagement in evidence synthesis, including consultation approaches, has many associated benefits, including improved relevance of the research and better dissemination and implementation of research findings. Scoping reviews, however, have not been specifically focused on in terms of research into knowledge user consultation and evidence syntheses. In this paper, we will present JBI's guidance for knowledge user engagement in scoping reviews based on the expert opinion of the JBI Scoping Review Methodology Group. We offer specific guidance on how this can occur and provide information regarding how to report and evaluate knowledge user engagement within scoping reviews. We believe that scoping review authors should embed knowledge user engagement into all scoping reviews and strive towards a co-creation model.

Investigating the Effectiveness and Acceptability of Oral Health and Related Health Behaviour Interventions in Adults with Severe and Multiple Disadvantage: Protocol for a Mixed-Methods Systematic Review

Increasing numbers of people in England experience homelessness, substance use, and repeated offending (known as 'severe and multiple disadvantage'; SMD). Populations experiencing SMD often have extremely poor oral health, which is closely inter-linked with high levels of substance use, smoking, and poor diet. This study aims to undertake an evidence synthesis to identify the effectiveness, resource requirements, and factors influencing the implementation and acceptability of oral health and related health behaviour interventions in adults experiencing SMD. Two systematic reviews will be conducted using mixed-methods. Review 1 will investigate the effectiveness and resource implications of oral health and related health behaviours (substance use, smoking, diet) interventions; Review 2 will investigate factors influencing the implementation of such interventions. The population includes adults (≥18 years) experiencing SMD. Standard review methods in terms of searches, screening, data extraction, and quality appraisal will be conducted. Narrative syntheses will be conducted. If feasible, a meta-analysis will be conducted for Review 1 and a thematic synthesis for Review 2. Evidence from the two reviews will then be synthesised together. Input from people with experience of SMD will be sought throughout to inform the reviews. An initial logic model will be iteratively refined during the review.

Co-creation of practical "how-to guides" for patient engagement in key phases of medicines development-from theory to implementation

BACKGROUND

The effective impact of patient engagement (PE) across the medicines development continuum is widely acknowledged across diverse health stakeholder groups, including health authorities; however, the practical applications of how to implement meaningful and consistent PE are not always addressed. Guidance for the practical implementation of PE requires granularity, and the need for such guidance has been identified as a priority. We describe the co-production and summarize the content of how-to guides that focus on PE in the early stages of medicines development.

METHODS

Multi-stakeholder working groups (WGs) were established by Patient Focused Medicines Development (PFMD) for how-to guide development. How-to guides were co-produced with patients for PE activities identified as priorities through public consultation and by WGs. Guides were developed by applying PE quality guidance and associated quality criteria in an iterative process. How-to guides underwent internal review and validation by experts (ie, those with relevant experience in the particular PE activity or focus area) in specific focus groups and external review and validation through appropriate events and public consultation.

RESULTS

Overall, 103 individual contributors from 38 organizations (representing eight stakeholder groups, including patients/patient organizations) and from 14 countries were organized into WGs and workstreams. Each WG comprised 15-30 contributors with PE experience relevant to the specific how-to guide. How-to guides were developed for PE in the early discovery and preclinical phases; PE in the development of a clinical outcomes assessment strategy; and PE in clinical trial protocol design. The how-to guides have a standardized format and structure to promote user familiarity. They provide detailed guidance and examples that are relevant to the individual PE activity and aim to facilitate the practical implementation of PE.

CONCLUSIONS

The how-to guides form a comprehensive series of actionable and stepwise resources that build from and integrate the PE quality criteria across the medicines continuum. They will be made freely available through PFMD's Patient Engagement Management Suite ( pemsuite.org ) and shared widely to a variety of audiences in different settings, ensuring access to diverse patient populations. Implementation of these guides should advance the field of PE in bringing new medicines to the market and ultimately will benefit patients. Medicines are developed to help patients improve their health and lives. Many organizations and individuals want to ensure that medicines are developed to meet real patient needs and to address what is most important to patients. Finding out what patients need and what patients want requires good patient engagement, but knowing how to do patient engagement is not always clear. This is because medicines development is complicated, and a lot of different steps, people, and organizations are involved. Patient Focused Medicines Development (PFMD) was established in 2015 to connect individuals and organizations that are committed to making medicines not just for patients but with patients. To do this, PFMD brought together patients and other groups of people with relevant experience and good ideas on how to achieve patient engagement in the real-world setting. Together, PFMD has developed "how-to guides" for patient engagement that cover the main activities along the medicines development process. The guides are free to use and provide practical advice and examples that anyone can use in their patient engagement activities. The how-to guides will also help patients to understand medicines development and how best they can participate in this process to address their needs.

Effectiveness of vocational interventions for gaining paid work for people living with mild to moderate mental health conditions: systematic review and meta-analysis

OBJECTIVES

To evaluate the effectiveness of vocational interventions to help people living with mild to moderate mental health conditions gain paid work.

METHODS

Systematic review of international, peer-reviewed literature. Development of the prepublished protocol and search strategy was done in consultation with stakeholder reference groups consisting of people with lived experience of long-term conditions, advocates and clinicians. We searched academic databases MEDLINE, EMBASE, PsychINFO, AMED, CINAHL, Proquest Dissertations and Theses database, and Business Source Complete for controlled trials comparing a specific vocational intervention against a control intervention or usual care, published between 1 January 2004 and 1 August 2019. Two authors independently screened search results, extracted data and appraised studies using the Cochrane risk of bias tool.

RESULTS

Eleven studies met inclusion criteria. Seven studies investigated Individual Placement and Support (IPS) modified for people who were not in intensive mental health treatment services. These studies occurred settings such as community vocational rehabilitation services, a housing programme and community mental health services. The studies provided very low quality evidence that people who receive IPS-style vocational rehabilitation are more likely to gain competitive employment than people who receive usual care (risk ratio 1.70, 95% CI 1.23 to 2.34, seven studies, 1611 participants). The remaining four studies considered cognitive behavioural therapy or specific vocational rehabilitation interventions designed to fit a unique context. There was insufficient evidence from these studies to draw conclusions regarding the effectiveness of non-IPS forms of vocational rehabilitation for people with mild to moderate mental health conditions.

DISCUSSION

The meta-analysis showed a clear intervention effect but low precision, and more high-quality studies are needed in this field. There is currently very low quality evidence that IPS-style intervention results in more participants in competitive employment compared with 'usual care' control groups in populations with mild to moderate mental health conditions.

Protocol for assessing stakeholder engagement in the development and evaluation of the Informed Health Choices resources teaching secondary school students to think critically about health claims and choices

BACKGROUND

As part of a five year plan (2019-2023), the Informed Health Choices Project, is developing and evaluating resources for helping secondary school students learn to think critically about health claims and choices. We will bring together key stakeholders; such as secondary school teachers and students, our main target for the IHC secondary school resources, school administrators, policy makers, curriculum development specialists and parents, to enable us gain insight about the context.

OBJECTIVES

To ensure that stakeholders are effectively and appropriately engaged in the design, evaluation and dissemination of the learning resources.To evaluate the extent to which stakeholders were successfully engaged.

METHODS

Using a multi-stage stratified sampling method, we will identify a representative sample of secondary schools with varied characteristics that might modify the effects of the learning resources such as, the school location (rural, semi-urban or urban), ownership (private, public) and ICT facilities (under resourced, highly resourced). A sample of schools will be randomly selected from the schools in each stratum. We will aim to recruit a diverse sample of students and secondary school teachers from those schools. Other stakeholders will be purposively selected to ensure a diverse range of experience and expertise.

RESULTS

Together with the teacher and student networks and the advisory panels, we will establish measurable success criteria that reflect the objectives of engaging stakeholders at the start of the project and evaluate the extent to which those criteria were met at the end of the project.

CONCLUSION

We aim for an increase in research uptake, improve quality and appropriateness of research results, accountability and social justice.

Qualitative Exploration of Engaging Patients as Advisors in a Program of Evidence Synthesis: Cobuilding the Science to Enhance Impact

There is an increasing expectation for research to involve patient stakeholders. Yet little guidance exists regarding patient-engaged research in evidence synthesis. Embedded in a learning health care system, the Veteran Affairs Evidence Synthesis Program (ESP) provides an ideal environment for exploring patient-engaged research in a program of evidence synthesis.

Smartphone-based computer vision travelling aids for blind and visually impaired individuals: A systematic review

Given the growth in the numbers of visually impaired (VI) people in low-income countries, the development of affordable electronic travel aid (ETA) systems employing devices, sensors, and apps embedded in ordinary smartphones becomes a potentially cost-effective and reasonable all-in-one solution of utmost importance for the VI. This paper offers an overview of recent ETA research prototypes that employ smartphones for assisted orientation and navigation in indoor and outdoor spaces by providing additional information about the surrounding objects. Scientific achievements in the field were systematically reviewed using PRISMA methodology. Comparative meta-analysis showed how various smartphone-based ETA prototypes could assist with better orientation, navigation, and wayfinding in indoor and outdoor environments. The analysis found limited interest among researchers in combining haptic interfaces and computer vision capabilities in smartphone-based ETAs for the blind, few attempts to employ novel state-of-the-art computer vision methods based on deep neural networks, and no evaluations of existing off-the-shelf navigation solutions. These results were contrasted with findings from a survey of blind expert users on their problems in navigating in indoor and outdoor environments. This revealed a major mismatch between user needs and academic development in the field.

What are Systematic Reviews?

Systematic reviews are a type of review that uses repeatable analytical methods to collect secondary data and analyse it. Systematic reviews are a type of evidence synthesis which formulate research questions that are broad or narrow in scope, and identify and synthesize data that directly relate to the systematic review question. While some people might associate ‘systematic review’ with 'meta-analysis', there are multiple kinds of review which can be defined as ‘systematic’ which do not involve a meta-analysis. Some systematic reviews critically appraise research studies, and synthesize findings qualitatively or quantitatively. Systematic reviews are often designed to provide an exhaustive summary of current evidence relevant to a research question. For example, systematic reviews of randomized controlled trials are an important way of informing evidence-based medicine, and a review of existing studies is often quicker and cheaper than embarking on a new study. While systematic reviews are often applied in the biomedical or healthcare context, they can be used in other areas where an assessment of a precisely defined subject would be helpful. Systematic reviews may examine clinical tests, public health interventions, environmental interventions, social interventions, adverse effects, qualitative evidence syntheses, methodological reviews, policy reviews, and economic evaluations. An understanding of systematic reviews and how to implement them in practice is highly recommended for professionals involved in the delivery of health care, public health and public policy.

Characteristics of stakeholder involvement in systematic and rapid reviews: a methodological review in the area of health services research

OBJECTIVE

Engaging stakeholders in reviews is considered to generate more relevant evidence and to facilitate dissemination and use. As little is known about stakeholder involvement, we assessed the characteristics of their engagement in systematic and rapid reviews and the methodological quality of included studies. Stakeholders were people with a particular interest in the research topic.

DESIGN

Methodological review.

SEARCH STRATEGY

Four databases (Medline, Embase, Cochrane database of systematic reviews, databases of the University of York, Center for Reviews and Dissemination (CRD)) were searched based on an a priori protocol. Four types of reviews (Cochrane and non-Cochrane systematic reviews, rapid and CRD rapid reviews) were retrieved between January 2011 and October 2015, pooled by potential review type and duplicates excluded. Articles were randomly ordered and screened for inclusion and exclusion criteria until 30 reviews per group were reached. Their methodological quality was assessed using AMSTAR and stakeholder characteristics were collected.

RESULTS

In total, 57 822 deduplicated citations were detected with potential non-Cochrane systematic reviews being the biggest group (56 986 records). We found stakeholder involvement in 13% (4/30) of Cochrane, 20% (6/30) of non-Cochrane, 43% (13/30) of rapid and 93% (28/30) of CRD reviews. Overall, 33% (17/51) of the responding contact authors mentioned positive effects of stakeholder involvement. A conflict of interest statement remained unmentioned in 40% (12/30) of non-Cochrane and in 27% (8/30) of rapid reviews, but not in Cochrane or CRD reviews. At most, half of non-Cochrane and rapid reviews mentioned an a priori study protocol in contrast to all Cochrane reviews.

CONCLUSION

Stakeholder engagement was not general practice, except for CRD reviews, although it was more common in rapid reviews. Reporting factors, such as including an a priori study protocol and a conflict of interest statement should be considered in conjunction with involving stakeholders.

Patient, Family, and Community Advisory Councils in Health Care and Research: a Systematic Review

BACKGROUND

Patient-centeredness is a characteristic of high-quality medical care and requires engaging community members in health systems' decision-making. One key patient engagement strategy is patient, family, and community advisory boards/councils (PFACs), yet the evidence to guide PFACs is lacking. Systematic reviews on patient engagement may benefit from patient input, but feasibility is unclear.

METHODS

A team of physicians, researchers, and a PFAC member conducted a systematic review to examine the impact of PFACs on health systems and describe optimal strategies for PFAC conduct. We searched MEDLINE, Embase, PsycINFO, CINAHL, Scopus, and Social Science Citation Index from inception through September 2016, as well as pre-identified websites. Two reviewers independently screened and abstracted data from studies, then assessed randomized studies for risk of bias and observational studies for quality using standardized measures. We performed a realist synthesis-which asks what works, for whom, under what circumstances-of abstracted data via 12 monthly meetings between investigators and two feedback sessions with a hospital-based PFAC.

RESULTS

Eighteen articles describing 16 studies met study criteria. Randomized studies demonstrated moderate to high risk of bias and observational studies demonstrated poor to fair quality. Studies engaged patients at multiple levels of the health care system and suggested that in-person deliberation with health system leadership was most effective. Studies involving patient engagement in research focused on increasing study participation. PFAC recruitment was by nomination (n = 11) or not described (n = 5). No common measure of patient, family, or community engagement was identified. Realist synthesis was enriched by feedback from PFAC members.

DISCUSSION

PFACs engage communities through individual projects but evidence of their impact on outcomes is lacking. A paucity of randomized controlled trials or high-quality observational studies guide strategies for engagement through PFACs. Standardized measurement tools for engagement are needed. Strategies for PFAC recruitment should be investigated and reported. PFAC members can feasibly contribute to systematic reviews.

REGISTRATION AND FUNDING SOURCE

A protocol for record eligibility was developed a priori and was registered in the PROSPERO database of systematic reviews (registration number CRD42016052817). The Department of Veterans Affairs' Office of Academic Affiliations, through the National Clinician Scholars Program, funded this study.

Development of the ACTIVE framework to describe stakeholder involvement in systematic reviews

Objectives

Involvement of patients, health professionals, and the wider public (‘stakeholders’) is seen to be beneficial to the quality, relevance and impact of research and may enhance the usefulness and uptake of systematic reviews. However, there is a lack of evidence and resources to guide researchers in how to actively involve stakeholders in systematic reviews. In this paper, we report the development of the ACTIVE framework to describe how stakeholders are involved in systematic reviews.

Methods

We developed a framework using methods previously described in the development of conceptual frameworks relating to other areas of public involvement, including: literature searching, data extraction, analysis, and categorization. A draft ACTIVE framework was developed and then refined after presentation at a conference workshop, before being applied to a subset of 32 systematic reviews. Data extracted from these systematic reviews, identified in a systematic scoping review, were categorized against pre-defined constructs, including: who was involved, how stakeholders were recruited, the mode of involvement, at what stage there was involvement and the level of control or influence.

Results

The final ACTIVE framework described whether patients, carers and/or families, and/or other stakeholders (including health professionals, health decision makers and funders) were involved. We defined: recruitment as either open or closed; the approach to involvement as either one-time, continuous or combined; and the method of involvement as either direct or indirect. The stage of involvement in reviews was defined using the Cochrane Ecosystem stages of a review. The level of control or influence was defined according to the roles and activities of stakeholders in the review process, and described as the ACTIVE continuum of involvement.

Conclusions

The ACTIVE framework provides a structure with which to describe key components of stakeholder involvement within a systematic review, and we have used this to summarize how stakeholders have been involved in a subset of varied systematic reviews. The ACTIVE continuum of involvement provides a new model that uses tasks and roles to detail the level of stakeholder involvement. This work has contributed to the development of learning resources aimed at supporting systematic review authors and editors to involve stakeholders in their systematic reviews. The ACTIVE framework may support the decision-making of systematic review authors in planning how to involve stakeholders in future reviews.

Optimizing the real-world impact of rehabilitation reviews: increasing the relevance and usability of systematic reviews in rehabilitation

BACKGROUND

Despite a growing portfolio of rehabilitation reviews, uptake of review findings into practice remains slow, with review findings perceived to be lacking in relevance and usability for stakeholders. Key aspects of review design, production and dissemination have been identified to contribute to this knowledge translation (KT) gap.

AIM

The aim of this study is to identify strategies relevant to rehabilitation review design, production and dissemination which have the potential to optimize uptake of review findings into practice.

RESULTS

Two strategies are discussed, drawing on case examples of existing rehabilitation reviews, including: 1) involving stakeholders in review design, production and dissemination; and 2) moving towards theory-based, mixed methods review design. The merits of these strategies are discussed with reference to the unique and specific characteristics of the rehabilitation context, where there is complexity inherent in the multiple interacting components across population, intervention, context and implementation processes.

CONCLUSIONS

Moving towards theory-based, mixed methods reviews which involve stakeholders may be a critical first step in supporting uptake of review findings into rehabilitation practice. Doing so also has the potential to support advances in knowledge and practice in rehabilitation through theory development, as well as creating the context for evidence-based practice.

Public Involvement in Global Genomics Research: A Scoping Review

Public involvement in research occurs when the public, patients, or research participants are actively contributing to the research process. Public involvement has been acknowledged as a key priority for prominent human genomics research initiatives in many different countries. However, to date, there has been no detailed analysis or review of the features, methods, and impacts of public involvement occurring in human genomics research projects worldwide. Here, we review the reported public involvement in 96 human genomics projects (initiatives), based on a database of initiatives hosted by the Global Alliance for Genomics and Health, according to information reported on public domain websites. To conduct the scoping review, we applied a structured categorization of criteria to all information extracted from the search. We found that only a third of all initiatives reported public involvement in any capacity (32/96, 33%). In those reporting public involvement, we found considerable variation in both the methods and tasks of involvement. Some noteworthy initiatives reported diverse and comprehensive ways of involving the public, occurring through different stages of the research project cycle. Three notable initiatives reported a total of eight distinct impacts as a result of involving people. Our findings suggest there would be intrinsic value in having more public involvement occur in human genomics research worldwide. We also suggest that more systematic ways of reporting and evaluating involvement would be highly beneficial, to help develop best practices.

"Still learning and evolving in our approaches": patient and stakeholder engagement among Canadian community-based primary health care researchers

PLAIN ENGLISH SUMMARY

Increasingly, health researchers are conducting their research in partnership with non-researchers such as patients and caregivers, advocacy groups, clinicians, and policymakers. The idea behind this partnership is to make research more relevant and appropriate. However, so far there is not much evidence about how this partnership or engagement actually affects research. We conducted an online survey of 12 teams in Canada that have engaged patients and other stakeholders in community based health research, partly as a requirement to obtain funding. We found that in many cases, the teams have engaged a wide variety and large number of stakeholders, and have involved them in many different stages of their research. Teams reported that their overall experience of this approach to research has been positive, but some challenges have been encountered along the way. Some teams found that it was difficult to communicate appropriately with all the stakeholders, and to keep them informed when research was going slowly. Other teams had trouble finding government representatives to work with. Several teams noted that engagement is time-consuming, and requires a lot of effort. Nevertheless, all teams reported that they had learned from the experience, and found it valuable. As a result, Canadian health care researchers are better positioned to engage with patients and other stakeholders in the future.

ABSTRACT

Background Patient and other stakeholder engagement in research is increasingly important, but there is limited evidence of its impact. In 2013, the Canadian Institutes of Health Research launched a five-year Community Based Primary Health Care (CBPHC) initiative that funded 12 teams for innovative approaches to primary health care involving engagement with patients, communities, decision-makers, and clinicians across jurisdictions in Canada. The present study examines the extent of engagement by these teams, and the factors that affected it, either as challenges or opportunities. Methods We conducted a cross-sectional web-based survey across the 12 CBPHC Innovation Teams, in which we were also participants. We used a data collection tool developed by the Patient Centered Outcomes Research Institute that included both closed and open-ended questions. Results The quantitative data showed that the CBPHC Innovation teams have engaged with diverse stakeholders at different levels and in different stages of research. Almost all teams surveyed engaged with policymakers, most with clinicians and health system representatives, and more than half with patients, mostly at the level of consultation or collaboration. There were very few instances of stakeholder-led research reported. There was a near universal recognition of the importance of communications processes/tools in facilitating engagement, whereas time was the most commonly identified challenge. In almost all cases, challenges encountered were partially if not fully resolved.The qualitative findings showed that each team's engagement was contextualized by factors such as the jurisdictions and geographic scope of the project, the number and type of stakeholders engaged and their level of involvement. These intersected with the researchers' motivations for engagement, to give rise to diverse experiences, but ones that the CBPHC teams assessed positively as an approach to research. Conclusions Over the past five years, primary health care researchers in Canada have been actively engaging with patients and other stakeholders. The wide range, extent and nature of that engagement shows that these researchers have anticipated developments in this approach to research and are thus in a position to support and strengthen future efforts to understand the impact of this engagement on health care outcomes.

Stakeholder involvement in systematic reviews: a scoping review

BACKGROUND

There is increasing recognition that it is good practice to involve stakeholders (meaning patients, the public, health professionals and others) in systematic reviews, but limited evidence about how best to do this. We aimed to document the evidence-base relating to stakeholder involvement in systematic reviews and to use this evidence to describe how stakeholders have been involved in systematic reviews.

METHODS

We carried out a scoping review, following a published protocol. We searched multiple electronic databases (2010-2016), using a stepwise searching approach, supplemented with hand searching. Two authors independently screened and discussed the first 500 abstracts and, after clarifying selection criteria, screened a further 500. Agreement on screening decisions was 97%, so screening was done by one reviewer only. Pre-planned data extraction was completed, and the comprehensiveness of the description of methods of involvement judged. Additional data extraction was completed for papers judged to have most comprehensive descriptions. Three stakeholder representatives were co-authors for this systematic review.

RESULTS

We included 291 papers in which stakeholders were involved in a systematic review. Thirty percent involved patients and/or carers. Thirty-two percent were from the USA, 26% from the UK and 10% from Canada. Ten percent (32 reviews) were judged to provide a comprehensive description of methods of involving stakeholders. Sixty-nine percent (22/32) personally invited people to be involved; 22% (7/32) advertised opportunities to the general population. Eighty-one percent (26/32) had between 1 and 20 face-to-face meetings, with 83% of these holding ≤ 4 meetings. Meetings lasted 1 h to ½ day. Nineteen percent (6/32) used a Delphi method, most often involving three electronic rounds. Details of ethical approval were reported by 10/32. Expenses were reported to be paid to people involved in 8/32 systematic reviews.

DISCUSSION/CONCLUSION

We identified a relatively large number (291) of papers reporting stakeholder involvement in systematic reviews, but the quality of reporting was generally very poor. Information from a subset of papers judged to provide the best descriptions of stakeholder involvement in systematic reviews provide examples of different ways in which stakeholders have been involved in systematic reviews. These examples arguably currently provide the best available information to inform and guide decisions around the planning of stakeholder involvement within future systematic reviews. This evidence has been used to develop online learning resources.

SYSTEMATIC REVIEW REGISTRATION

The protocol for this systematic review was published on 21 April 2017. Publication reference: Pollock A, Campbell P, Struthers C, Synnot A, Nunn J, Hill S, Goodare H, Watts C, Morley R: Stakeholder involvement in systematic reviews: a protocol for a systematic review of methods, outcomes and effects. Research Involvement and Engagement 2017, 3:9. https://doi.org/10.1186/s40900-017-0060-4 .

How to engage stakeholders in research: design principles to support improvement

Background

Closing the gap between research production and research use is a key challenge for the health research system. Stakeholder engagement is being increasingly promoted across the board by health research funding organisations, and indeed by many researchers themselves, as an important pathway to achieving impact. This opinion piece draws on a study of stakeholder engagement in research and a systematic literature search conducted as part of the study.

Main body

This paper provides a short conceptualisation of stakeholder engagement, followed by ‘design principles’ that we put forward based on a combination of existing literature and new empirical insights from our recently completed longitudinal study of stakeholder engagement. The design principles for stakeholder engagement are organised into three groups, namely organisational, values and practices. The organisational principles are to clarify the objectives of stakeholder engagement; embed stakeholder engagement in a framework or model of research use; identify the necessary resources for stakeholder engagement; put in place plans for organisational learning and rewarding of effective stakeholder engagement; and to recognise that some stakeholders have the potential to play a key role. The principles relating to values are to foster shared commitment to the values and objectives of stakeholder engagement in the project team; share understanding that stakeholder engagement is often about more than individuals; encourage individual stakeholders and their organisations to value engagement; recognise potential tension between productivity and inclusion; and to generate a shared commitment to sustained and continuous stakeholder engagement. Finally, in terms of practices, the principles suggest that it is important to plan stakeholder engagement activity as part of the research programme of work; build flexibility within the research process to accommodate engagement and the outcomes of engagement; consider how input from stakeholders can be gathered systematically to meet objectives; consider how input from stakeholders can be collated, analysed and used; and to recognise that identification and involvement of stakeholders is an iterative and ongoing process.

Conclusion

It is anticipated that the principles will be useful in planning stakeholder engagement activity within research programmes and in monitoring and evaluating stakeholder engagement. A next step will be to address the remaining gap in the stakeholder engagement literature concerned with how we assess the impact of stakeholder engagement on research use.

Electronic supplementary material

The online version of this article (10.1186/s12961-018-0337-6) contains supplementary material, which is available to authorized users.

How to incorporate patient and public perspectives into the design and conduct of research

International government guidance recommends patient and public involvement (PPI) to improve the relevance and quality of research.  PPI is defined as research being carried out 'with' or 'by' patients and members of the public rather than 'to', 'about' or 'for' them ( http://www.invo.org.uk/). Patient involvement is different from collecting data from patients as participants.  Ethical considerations also differ.  PPI is about patients actively contributing through discussion to decisions about research design, acceptability, relevance, conduct and governance from study conception to dissemination.  Occasionally patients lead or do research.  The research methods of PPI range from informal discussions to partnership research approaches such as action research, co-production and co-learning. This article discusses how researchers can involve patients when they are applying for research funding and considers some opportunities and pitfalls.  It reviews research funder requirements, draws on the literature and our collective experiences as clinicians, patients, academics and members of UK funding panels.

Selecting and implementing overview methods: implications from five exemplar overviews

BACKGROUND

Overviews of systematic reviews are an increasingly popular method of evidence synthesis; there is a lack of clear guidance for completing overviews and a number of methodological challenges. At the UK Cochrane Symposium 2016, methodological challenges of five overviews were explored. Using data from these five overviews, practical implications to support methodological decision making of authors writing protocols for future overviews are proposed.

METHODS

Methods, and their justification, from the five exemplar overviews were tabulated and compared with areas of debate identified within current literature. Key methodological challenges and implications for development of overview protocols were generated and synthesised into a list, discussed and refined until there was consensus.

RESULTS

Methodological features of three Cochrane overviews, one overview of diagnostic test accuracy and one mixed methods overview have been summarised. Methods of selection of reviews and data extraction were similar. Either the AMSTAR or ROBIS tool was used to assess quality of included reviews. The GRADE approach was most commonly used to assess quality of evidence within the reviews. Eight key methodological challenges were identified from the exemplar overviews. There was good agreement between our findings and emerging areas of debate within a recent published synthesis. Implications for development of protocols for future overviews were identified.

CONCLUSIONS

Overviews are a relatively new methodological innovation, and there are currently substantial variations in the methodological approaches used within different overviews. There are considerable methodological challenges for which optimal solutions are not necessarily yet known. Lessons learnt from five exemplar overviews highlight a number of methodological decisions which may be beneficial to consider during the development of an overview protocol.