The SPARK Tool to prioritise questions for systematic reviews in health policy and systems research: development and initial validation

A Structured Approach to Involve Stakeholders in Prioritising Topics for Systematic Reviews in Public Health

Objectives

This study aimed to develop and apply a structured approach for prioritising topics for systematic reviews in public health, framed according to the readily applicable PICO format, which encourages the involvement of stakeholders' preferences in a transparent matter.

Methods

We developed a multi-stage process, consisting of a scoping and two Delphi stages with web-based surveys and invited public health stakeholders in Switzerland to participate: First, respondents specified topics for different public health domains, which were reformulated in a PICO format by content analysis. Second, respondents rated the topics using five stakeholder-refined assessment criteria. Overall rankings were calculated to assess differences between stakeholder groups and rating criteria.

Results

In total, 215 respondents suggested 728 topics altogether. The response rate in the two Delphi stages was 91.6% and 77.6%, respectively. Most top-rated review topics focused on the effectiveness of interventions providing education to different target groups, followed by interventions to increase access to specific healthcare services.

Conclusion

Our approach encourages involvement of stakeholders in identifying priorities for systematic reviews and highlights disparities between stakeholders and between individual criteria.

Research gaps identified in Iran's health technology assessment reports

INTRODUCTION

Identifying gaps in the evidence is a useful byproduct of conducting a health technology assessment (HTA). This study aims to identify research gaps in Iran's HTA reports.

METHOD

We reviewed the HTA reports published between 2014 and 2016. Then, we developed two separate questionnaires for principal investigators (PIs) and independent HTA researchers. The questionnaire for independent HTA researchers consisted of four main parts. However, the PIs' questionnaire consisted of two main parts. We also conducted a literature search in the PubMed database in November 2017 to find frameworks for prioritizing research gaps. We also conducted a semi-structured interview with the head of the Iran's HTA Office at that time and sought feedback based on his expert opinion about questionnaires, the priority-setting tool and our process for extracting research gaps.

RESULTS

A total of 11 HTA reports published between 2014 and 2016 by Iran's HTA Office were selected for the study. Of these 11 reports, 5 involved technologies related to medical equipment, while 6 involved medical and surgical interventions. Assessing the outcomes of technology use in various indications and updating HTAs when new evidence arises; evaluating the viewpoints of patients, clinicians and key technology users; conducting post-marketing evaluations of technology; comparing the impact of the technology in question to other treatments for the same condition; and requesting long-term clinical and cost-effectiveness data for technologies with limited follow-up periods were identified as the main gaps by independent HTA researchers and PIs.

CONCLUSIONS

The research gaps identified from Iran's HTAs could be utilized by research funding agencies.

Evidence synthesis to policy: development and implementation of an impact-oriented approach from the Eastern Mediterranean Region

BACKGROUND

Despite the importance of evidence syntheses in informing policymaking, their production and use remain limited in the Eastern Mediterranean region (EMR). There is a lack of empirical research on approaches to promote and use policy-relevant evidence syntheses to inform policymaking processes in the EMR.

OBJECTIVE

This study sought to describe the development of an impact-oriented approach to link evidence synthesis to policy, and its implementation through selected case studies in Lebanon, a middle-income country in the EMR.

METHODS

This study followed a multifaceted and iterative process that included (i) a review of the literature, (ii) input from international experts in evidence synthesis and evidence-informed health policymaking, and (iii) application in a real-world setting (implementation). We describe four selected case studies of implementation. Surveys were used to assess policy briefs, deliberative dialogues, and post-dialogue activities. Additionally, Kingdon's stream theory was adopted to further explain how and why the selected policy issues rose to the decision agenda.

RESULTS

The approach incorporates three interrelated phases: (1) priority setting, (2) evidence synthesis, and (3) uptake. Policy-relevant priorities are generated through formal priority setting exercises, direct requests by policymakers and stakeholders, or a focusing event. Identified priorities are translated into focused questions that can be addressed via evidence synthesis (phase 1). Next, a scoping of the literature is conducted to identify existing evidence syntheses addressing the question of interest. Unless the team identifies relevant, up-to-date and high-quality evidence syntheses, it proceeds to conducting SRs addressing the priority questions of interest (phase 2). Next, the team prepares knowledge translation products (e.g., policy briefs) for undertaking knowledge uptake activities, followed by monitoring and evaluation (phase 3). There are two prerequisites to the application of the approach: enhancing contextual awareness and capacity strengthening. The four case studies illustrate how evidence produced from the suites of activities was used to inform health policies and practices.

CONCLUSIONS

To our knowledge, this is the first study to describe both the development and implementation of an approach to link evidence synthesis to policy in the EMR. We believe the approach will be useful for researchers, knowledge translation platforms, governments, and funders seeking to promote evidence-informed policymaking and practice.

Prioritising Cochrane reviews to be updated with health equity focus

BACKGROUND

The prioritisation of updating published systematic reviews of interventions is vital to prevent research waste and ensure relevance to stakeholders. The consideration of health equity in reviews is also important to ensure interventions will not exacerbate the existing inequities of the disadvantaged if universally implemented. This study aimed to pilot a priority setting exercise based on systematic reviews of interventions published in the Cochrane Library, to identify and prioritise reviews to be updated with a focus on health equity.

METHODS

We conducted a priority setting exercise with a group of 13 international stakeholders. We identified Cochrane reviews of interventions that showed a reduction in mortality, had at least one Summary of Findings table and that focused on one of 42 conditions with a high global burden of disease from the 2019 WHO Global Burden of Disease report. This included 21 conditions used as indicators of success of the United Nations Universal Health Coverage in attaining the Sustainable Development Goals. Stakeholders prioritised reviews that were relevant to disadvantaged populations, or to characteristics of potential disadvantage within the general population.

RESULTS

After searching for Cochrane reviews of interventions within 42 conditions, we identified 359 reviews that assessed mortality and included at least one Summary of Findings table. These pertained to 29 of the 42 conditions; 13 priority conditions had no reviews with the outcome mortality. Reducing the list to only reviews showing a clinically important reduction in mortality left 33 reviews. Stakeholders ranked these reviews in order of priority to be updated with a focus on health equity.

CONCLUSIONS

This project developed and implemented a methodology to set priorities for updating systematic reviews spanning multiple health topics with a health equity focus. It prioritised reviews that reduce overall mortality, are relevant to disadvantaged populations, and focus on conditions with a high global burden of disease. This approach to the prioritisation of systematic reviews of interventions that reduce mortality provides a template that can be extended to reducing morbidity, and the combination of mortality and morbidity as represented in Disability-Adjusted Life Years and Quality-Adjusted Life Years.

A rapid priority setting exercise combining existing, emergent evidence with stakeholder knowledge identified broad topic uncertainties

OBJECTIVES

The project aimed to rapidly identify priority topic uncertainties as a first step to identify future systematic review questions of pertinence to key international fecal incontinence (FI) stakeholders (patients, carers, health care professionals, policy makers and voluntary, community, or social enterprise representatives). The paper's aim is to share our methods, experience, and learning with other groups planning to deliver a rapid priority setting exercise.

STUDY DESIGN AND SETTING

An evidence gap map incorporated three evidence streams: emerging evidence identified through horizon scanning; existing evidence identified through systematic searches of bibliographic databases; and FI stakeholder insights collected through an international survey. The evidence gap map was presented during an online workshop with stakeholders, where they shared their expertize to expand, refine, and rank topic uncertainties using ideation techniques, focus group discussions, consensus techniques, and online polling.

RESULTS

The multistep methods used to deliver this priority setting exercise resulted in identification of broad priority topic uncertainties. The methods appear to have high acceptability and engagement with participants but await full evaluation.

CONCLUSION

This project successfully followed robust methodology, building upon frameworks from published priority setting and evidence gap mapping projects while incorporating strong patient and public involvement components.

A Modelling Framework for Evidence-based Public Health Policy Making

It is widely recognised that the process of public health policy making (i.e., the analysis, action plan design, execution, monitoring and evaluation of public health policies) should be evidenced based, and supported by data analytics and decision- making tools tailored to it. This is because the management of health conditions and their consequences at a public health policy making level can benefit from such type of analysis of heterogeneous data, including health care devices usage, physiological, cognitive, clinical and medication, personal, behavioural, lifestyle data, occupational and environmental data. In this paper we present a novel approach to public health policy making in a form of an ontology, and an integrated platform for realising this approach. Our solution is model-driven and makes use of big data analytics technology. More specifically, it is based on public health policy decision making (PHPDM) models that steer the public health policy decision making process by defining the data that need to be collected, the ways in which they should be analysed in order to produce the evidence useful for public health policymaking, how this evidence may support or contradict various policy interventions (actions), and the stakeholders involved in the decision-making process. The resulted web-based platform has been implemented using Hadoop, Spark and HBASE, developed in the context of a research programme on public health policy making for the management of hearing loss called EVOTION, funded by the Horizon 2020.

A common framework of steps and criteria for prioritizing topics for evidence syntheses: a systematic review

OBJECTIVE

The objective of the study was to systematically review the literature for proposed approaches and exercises conducted to prioritize topics or questions for systematic reviews and other types of evidence syntheses in any health-related area.

STUDY DESIGN AND SETTING

A systematic review. We searched Medline and CINAHL databases in addition to Cochrane website and Google Scholar. Teams of two reviewers independently screened the studies and extracted data.

RESULTS

We included 31 articles reporting on 29 studies: seven proposed approaches for prioritization and 25 conducted prioritization exercises (three studies did both). The included studies addressed the following fields: clinical (n = 19; 66%), public health (n = 10; 34%), and health policy and systems (n = 8; 28%), with six studies (21%) addressing more than one field. We categorized prioritization into 11 steps clustered in 3 phases (preprioritization, prioritization, and postprioritization). Twenty-eight studies (97%) involved or proposed involving stakeholders in the priority-setting process. These 28 studies referred to twelve stakeholder categories, most frequently to health care providers (n = 24; 86%) and researchers (n = 21; 75%). A common framework of 25 prioritization criteria was derived, clustered in 10 domains.

CONCLUSION

We identified literature that addresses different aspects of prioritizing topics or questions for evidence syntheses, including prioritization steps and criteria. The identified steps and criteria can serve as a menu of options to select from, as judged appropriate to the context.

Quality, safety and performance management in primary health care: from scoping review to research priority setting and implementation plan in the Eastern Mediterranean Region

INTRODUCTION

Strong primary health care (PHC) leads to better health outcomes, improves health equity and accelerates progress towards universal health coverage (UHC). The Astana Declaration on PHC emphasised the importance of quality care to achieve UHC. A comprehensive understanding of the quality paradigm of PHC is critical, yet it remains elusive in countries of the Eastern Mediterranean Region (EMR). This study used a multistep approach to generate a policy-relevant research agenda for strengthening quality, safety and performance management in PHC in the EMR.

METHODS

A multistep approach was adopted, encompassing the following steps: scoping review and generation of evidence and gap maps, validation and ranking exercises, and development of an approach for research implementation. We followed Joanna Briggs Institute guidelines for conducting scoping reviews and a method review of the literature to build the evidence and gap maps. For the validation and ranking exercises, we purposively sampled 55 high-level policy-makers and stakeholders from selected EMR countries. We used explicit multicriteria for ranking the research questions emerging from the gap maps. The approach for research implementation was adapted from the literature and subsequently tailored to address the top ranked research question.

RESULTS

The evidence and gap maps revealed limited production of research evidence in the area of quality, safety and performance management in PHC by country and by topic. The priority setting exercises generated a ranked list of 34 policy-relevant research questions addressing quality, safety and performance management in PHC in the EMR. The proposed research implementation plan involves collaborative knowledge generation with policy-makers along with knowledge translation and impact assessment.

CONCLUSION

Study findings can help inform and direct future plans to generate, disseminate and use research evidence to enhance quality, safety and performance management in PHC in EMR and beyond. Study methodology can help bridge the gap between research and policy-making.

Study-based registers reduce waste in systematic reviewing: discussion and case report

BACKGROUND

Maintained study-based registers (SBRs) have, at their core, study records linked to, potentially, multiple other records such as references, data sets, standard texts and full-text reports. Such registers can minimise and refine searching, de-duplicating, screening and acquisition of full texts. SBRs can facilitate new review titles/updates and, within seconds, inform the team about the potential workload of each task.

METHODS

We discuss the advantages/disadvantages of SBRs and report a case of how such a register was used to develop a successful grant application and deliver results-reducing considerable redundancy of effort.

RESULTS

SBRs saved time in question-setting and scoping and made rapid production of nine Cochrane systematic reviews possible.

CONCLUSION

Whilst helping prioritise and conduct systematic reviews, SBRs improve quality. Those funding information specialists for literature reviewing could reasonably stipulate the resulting SBR to be delivered for dissemination and use beyond the life of the project.

Selecting, refining and identifying priority Cochrane Reviews in health communication and participation in partnership with consumers and other stakeholders

BACKGROUND

Priority-setting partnerships between researchers and stakeholders (meaning consumers, health professionals and health decision-makers) may improve research relevance and value. The Cochrane Consumers and Communication Group (CCCG) publishes systematic reviews in 'health communication and participation', which includes concepts such as shared decision-making, patient-centred care and health literacy. We aimed to select and refine priority topics for systematic reviews in health communication and participation, and use these to identify five priority CCCG Cochrane Reviews.

METHODS

Twenty-eight participants (14 consumers, 14 health professionals/decision-makers) attended a 1-day workshop in Australia. Using large-group activities and voting, participants discussed, revised and then selected 12 priority topics from a list of 21 previously identified topics. In mixed small groups, participants refined these topics, exploring underlying problems, who they affect and potential solutions. Thematic analysis identified cross-cutting themes, in addition to key populations and potential interventions for future Cochrane Reviews. We mapped these against CCCG's existing review portfolio to identify five priority reviews.

RESULTS

Priority topics included poor understanding and implementation of patient-centred care by health services, the fact that health information can be a low priority for health professionals, communication and coordination breakdowns in health services, and inadequate consumer involvement in health service design. The four themes underpinning the topics were culture and organisational structures, health professional attitudes and assumptions, inconsistent experiences of care, and lack of shared understanding in the sector. Key populations for future reviews were described in terms of social health characteristics (e.g. people from indigenous or culturally and linguistically diverse backgrounds, elderly people, and people experiencing socioeconomic disadvantage) more than individual health characteristics. Potential interventions included health professional education, interventions to change health service/health professional culture and attitudes, and health service policies and standards. The resulting five priority Cochrane Reviews identified were improving end-of-life care communication, patient/family involvement in patient safety, improving future doctors' communication skills, consumer engagement strategies, and promoting patient-centred care.

CONCLUSIONS

Stakeholders identified priority topics for systematic reviews associated with structural and cultural challenges underlying health communication and participation, and were concerned that issues of equity be addressed. Priority-setting with stakeholders presents opportunities and challenges for review producers.

Research priorities in health communication and participation: international survey of consumers and other stakeholders

OBJECTIVE

To identify research priorities of consumers and other stakeholders to inform Cochrane Reviews in 'health communication and participation' (including such concepts as patient experience, shared decision-making and health literacy).

SETTING

International.

PARTICIPANTS

We included anyone with an interest in health communication and participation. Up to 151 participants (18-80 years; 117 female) across 12 countries took part, including 48 consumers (patients, carers, consumer representatives) and 75 professionals (health professionals, policymakers, researchers) (plus 25 people who identified as both).

DESIGN

Survey.

METHODS

We invited people to submit their research ideas via an online survey open for 4 weeks. Using inductive thematic analysis, we generated priority research topics, then classified these into broader themes.

RESULTS

Participants submitted 200 research ideas, which we grouped into 21 priority topics. Key research priorities included: insufficient consumer involvement in research (19 responses), 'official' health information is contradictory and hard to understand (18 responses), communication/coordination breakdowns in health services (15 responses), health information provision a low priority for health professionals (15 responses), insufficient eliciting of patient preferences (14 responses), health services poorly understand/implement patient-centred care (14 responses), lack of holistic care impacting healthcare quality and safety (13 responses) and inadequate consumer involvement in service design (11 responses). These priorities encompassed acute and community health settings, with implications for policy and research. Priority populations of interest included people from diverse cultural and linguistic backgrounds, carers, and people with low educational attainment, or mental illness. Most frequently suggested interventions focused on training and cultural change activities for health services and health professionals.

CONCLUSIONS

Consumers and other stakeholders want research addressing structural and cultural challenges in health services (eg, lack of holistic, patient-centred, culturally safe care) and building health professionals' communication skills. Solutions should be devised in partnership with consumers, and focus on the needs of vulnerable groups.

Engaging policy-makers, health system managers, and policy analysts in the knowledge synthesis process: a scoping review

Background

It is unclear how to engage a wide range of knowledge users in research. We aimed to map the evidence on engaging knowledge users with an emphasis on policy-makers, health system managers, and policy analysts in the knowledge synthesis process through a scoping review.

Methods

We used the Joanna Briggs Institute guidance for scoping reviews. Nine electronic databases (e.g., MEDLINE), two grey literature sources (e.g., OpenSIGLE), and reference lists of relevant systematic reviews were searched from 1996 to August 2016. We included any type of study describing strategies, barriers and facilitators, or assessing the impact of engaging policy-makers, health system managers, and policy analysts in the knowledge synthesis process. Screening and data abstraction were conducted by two reviewers independently with a third reviewer resolving discrepancies. Frequency and thematic analyses were conducted.

Results

After screening 8395 titles and abstracts followed by 394 full-texts, 84 unique documents and 7 companion reports fulfilled our eligibility criteria. All 84 documents were published in the last 10 years, and half were prepared in North America. The most common type of knowledge synthesis with knowledge user engagement was a systematic review (36%). The knowledge synthesis most commonly addressed an issue at the level of national healthcare system (48%) and focused on health services delivery (17%) in high-income countries (86%).

Policy-makers were the most common (64%) knowledge users, followed by healthcare professionals (49%) and government agencies as well as patients and caregivers (34%). Knowledge users were engaged in conceptualization and design (49%), literature search and data collection (52%), data synthesis and interpretation (71%), and knowledge dissemination and application (44%). Knowledge users were most commonly engaged as key informants through meetings and workshops as well as surveys, focus groups, and interviews either in-person or by telephone and emails. Knowledge user content expertise/awareness was a common facilitator (18%), while lack of time or opportunity to participate was a common barrier (12%).

Conclusions

Knowledge users were most commonly engaged during the data synthesis and interpretation phases of the knowledge synthesis conduct. Researchers should document and evaluate knowledge user engagement in knowledge synthesis.

Registration details

Open Science Framework (https://osf.io/4dy53/).

Electronic supplementary material

The online version of this article (10.1186/s13012-018-0717-x) contains supplementary material, which is available to authorized users.