The SPIRIT Action Framework: A structured approach to selecting and testing strategies to increase the use of research in policy

Building bridges: evaluating policymakers' research capacities, engagement, and utilization in health policymaking within the Kuwaiti context: a cross-sectional study

BACKGROUND

Health policymaking is a critical aspect of governmental decision-making that shapes the well-being of populations. In the Middle East and North Africa, particularly in Kuwait, limited attention has been given to exploring the research capacities, engagement, and utilization among health policymakers. This study aims to bridge this gap by investigating how Kuwaiti health policymakers incorporate evidence-based research into the formulation of health-related policies.

METHODS

This cross-sectional study targeted health policymakers in leadership positions within the Kuwait Ministry of Health (MOH). Using the Seeking, Engaging with and Evaluating Research (SEER) questionnaire, participants' capacities, engagement, and use of research were assessed. The targeted sample was all health policymakers in leadership positions, starting from the head of departments and above. The questionnaire comprises four domains, 14 sections, and 50 questions and utilizes Likert and binary scales, with aggregate scores predicting engagement actions and research use. The data were collected between March and July 2023. All the statistical analyses were performed using SPSS v27, and the numerical and categorical variables were analyzed using appropriate statistical tests, including t-tests, ANOVA, and Pearson's correlation.

RESULTS

Out of 205 policymakers, 88 participated (42.9% response rate): predominantly male (51.1%) and married (78.4%). The mean age was 49.84 ± 7.28 years, with a mean MOH tenure of 24.39 ± 6.80 years. Participants demonstrated high value for research (mean score 4.29 ± 0.55) and expressed confidence in the research utilization. Organizational emphasis on research use exhibited nuanced perceptions, identifying areas where MOH support may be lacking. Access to research resources and processes for policy development guidance were highlighted as challenges.

CONCLUSIONS

This study provides crucial insights into the research capacities and engagement of Kuwaiti health policymakers. It emphasizes the need for targeted interventions to align individual perceptions with organizational expectations, address confidence disparities, and enhance collaborative efforts. Organizational investments are crucial for fostering a dynamic research ecosystem to improve evidence-based policy development in Kuwait's healthcare landscape.

Mindfulness-Based Stress Reduction Health Insurance Coverage: If, How, and When? An Integrated Knowledge Translation (iKT) Delphi Key Informant Analysis

Objectives

Hundreds of trials have evaluated Mindfulness-Based Stress Reduction (MBSR), but in the United States, it is generally not covered by health insurance. Consequently, the aims were to identify the following: (1) key questions to make decisions about if, how, and when MBSR should be covered by health insurance; (2a) barriers and (2b) facilitators to understand and resolve for MBSR to be covered by health insurance; and (3) highest priority evidence needed to inform health insurance coverage decisions.

Methods

Key informants (n = 26) included health insurers, healthcare administrators, policymakers, clinicians, MBSR instructors, and MBSR students. An initial pool of items related to the study aims was generated through qualitative interviews. Through the Delphi process, participants rated, discussed, and re-rated each item's relevance. Items were required to reach a consensus of ≥ 80% agreement to be retained for final inclusion.

Results

Of the original 149 items, 42 (28.2%) met the ≥ 80% agreement criterion and were retained for final inclusion. The most highly rated items informing whether MBSR should be covered by health insurance included research demonstrating that MBSR works and that it is not harmful. The most highly rated barriers to coverage were that MBSR is not a medical treatment and patient barriers to attendance. Highly rated facilitators included the potential of MBSR to address common mental health and psychosomatic problems. Finally, understanding what conditions are effectively treated with MBSR and the impact of MBSR on stress were rated as the highest priority evidence needed to inform health insurance coverage decisions.

Conclusions

Findings highlight priorities for future research and policy efforts to advance health insurance coverage of MBSR in the United States.

Supplementary Information

The online version contains supplementary material available at 10.1007/s12671-024-02366-x.

Exploring evidence use and capacity for health services management and planning in Swiss health administrations: A mixed-method interview study

BACKGROUND

Health administrations require evidence, meaning robust information, data, and research, on health services and systems. Little is known about the resources and processes available within administrations to support evidence-informed policymaking. This study assessed Swiss health administrations' capacity for evidence use and investigated civil servants' needs and perspectives regarding the role and use of evidence in health services management and planning.

METHODS

In this mixed-method study, we interviewed civil servants from Swiss German-speaking cantonal health administrations. We quantitatively assessed administrations' organization-level capacity by applying six structured interviews using an existing measurement tool (ORACLe). Individual-level needs and perspectives regarding evidence use and capacity were qualitatively explored with twelve in-depth interviews that were analyzed using the framework method.

FINDINGS

Respondents indicated moderate evidence-use capacity in all administrations. Administrations displayed a similar pattern of high and low capacity in specific capacity areas, generally with considerable variation within administrations. Most administrations indicated high capacity for producing or commissioning evidence and close relationships with research. They showed limited capacity in the documentation of processes and availability of tools, programs, or training opportunities. Administrations place the responsibility for engagement with evidence at the level of individual civil servants rather than at the organizational level. Although administrations highly value evidence-informed policymaking and consider it vital to effective health services management and planning, they face significant constraints in accessing evidence-specific resources and receive little organizational support. Administrations rely on external capacity to compensate for these limitations and engage with evidence pragmatically.

CONCLUSION

Our findings indicate moderate and improvable capacity for evidence use in Swiss health administrations that place limited value on organizational support. Besides strengthening organizational support, leadership buy-in, particular staff needs, and balancing the implementation of specific measures with the provision of more general resources should be considered to unlock the potential of strengthened engagement with evidence.

Could you give me a leg up …? Models, frameworks and support structures to help aspiring clinical academic speech and language therapists

BACKGROUND

Increasing research capacity and capability for the speech and language therapy (SLT) profession is a key national strategic driver, with many speech and language therapists (SLTs) aspiring to a clinical academic (CA) career. There are known benefits but also acknowledged challenges with this career path, including limited funding opportunities and a poorly established career trajectory.

AIMS

To present models, frameworks and support structures that can be used by aspiring SLT CAs to chart research knowledge and skills, and plan career development. Organisational models are also presented to facilitate SLT CA career development and research capacity-building.

METHODS & PROCEDURES

A narrative review was conducted using a literature search of published peer-reviewed journals across four electronic databases: Medline, CINAHL, AMED and Embase, with additional search for grey literature through internet searches. Search results were screened against eligibility criteria by two researchers, with full-text articles retrieved and reviewed by four researchers independently.

RESULTS & DISCUSSION

The database search and grey literature search combined identified 610 records. Full-text screening of 66 records resulted in 19 articles or grey literature sources being included within the narrative review.

MAIN CONTRIBUTION

This paper details models, frameworks and support structures pertinent to SLTs that can be used at an individual and organizational level to assist CA skill development and career paths.

CONCLUSION & IMPLICATIONS

The national climate is looking positive for aspiring SLT CAs. The time is now to take the initiative and use the support structures available to show our CA value and develop the necessary skills outlined within these resources to fulfil our ambitions.

WHAT THIS PAPER ADDS

What is already known on the subject Interest in CA careers within the SLT profession is increasing. Whilst there are known benefits to embedding research within clinical practice, barriers exist including the strategic and operational steps individuals can take to make the career path a reality. What this study adds This narrative review has searched the literature for CA models, frameworks and support structures created for nurses, midwives and allied health professionals. These tools are presented and discussed, with special consideration and focus to the SLT profession. What are the clinical implications of this work? This paper will provide SLTs with the tools to support their personal CA career development as well as advocate for CA roles within their teams and organisations. Organisational models are also presented to support SLT managers to foster a CA path for workforce development.

What is context in knowledge translation? Results of a systematic scoping review

Knowledge Translation (KT) aims to convey novel ideas to relevant stakeholders, motivating their response or action to improve people's health. Initially, the KT literature focused on evidence-based medicine, applying findings from laboratory and clinical research to disease diagnosis and treatment. Since the early 2000s, the scope of KT has expanded to include decision-making with health policy implications.This systematic scoping review aims to assess the evolving knowledge-to-policy concepts, that is, macro-level KT theories, models and frameworks (KT TMFs). While significant attention has been devoted to transferring knowledge to healthcare settings (i.e. implementing health policies, programmes or measures at the meso-level), the definition of 'context' in the realm of health policymaking at the macro-level remains underexplored in the KT literature. This study aims to close the gap.A total of 32 macro-level KT TMFs were identified, with only a limited subset of them offering detailed insights into contextual factors that matter in health policymaking. Notably, the majority of these studies prompt policy changes in low- and middle-income countries and received support from international organisations, the European Union, development agencies or philanthropic entities.

A modified action framework to develop and evaluate academic-policy engagement interventions

BACKGROUND

There has been a proliferation of frameworks with a common goal of bridging the gap between evidence, policy, and practice, but few aim to specifically guide evaluations of academic-policy engagement. We present the modification of an action framework for the purpose of selecting, developing and evaluating interventions for academic-policy engagement.

METHODS

We build on the conceptual work of an existing framework known as SPIRIT (Supporting Policy In Health with Research: an Intervention Trial), developed for the evaluation of strategies intended to increase the use of research in health policy. Our aim was to modify SPIRIT, (i) to be applicable beyond health policy contexts, for example encompassing social, environmental, and economic policy impacts and (ii) to address broader dynamics of academic-policy engagement. We used an iterative approach through literature reviews and consultation with multiple stakeholders from Higher Education Institutions (HEIs) and policy professionals working at different levels of government and across geographical contexts in England, alongside our evaluation activities in the Capabilities in Academic Policy Engagement (CAPE) programme.

RESULTS

Our modifications expand upon Redman et al.'s original framework, for example adding a domain of 'Impacts and Sustainability' to capture continued activities required in the achievement of desirable outcomes. The modified framework fulfils the criteria for a useful action framework, having a clear purpose, being informed by existing understandings, being capable of guiding targeted interventions, and providing a structure to build further knowledge.

CONCLUSION

The modified SPIRIT framework is designed to be meaningful and accessible for people working across varied contexts in the evidence-policy ecosystem. It has potential applications in how academic-policy engagement interventions might be developed, evaluated, facilitated and improved, to ultimately support the use of evidence in decision-making.

From bench to policy: a critical analysis of models for evidence-informed policymaking in healthcare

Background

The use of research evidence in policy making is a complex and challenging process that has a long history in various fields, especially in healthcare. Different terms and concepts have been used to describe the relationship between research and policy, but they often lack clarity and consensus. To address this gap, several strategies and models have been proposed to facilitate evidence informed policy making and to identify the key factors and mechanisms involved. This study aims to critically review the existing models of evidence informed policy making (EIPM) in healthcare and to assess their strengths and limitations.

Method

A systematic search and review conducted to identify and critically assess EIPM models in healthcare. We searched PubMed, Web of Science and Scopus databases as major electronic databases and applied predefined inclusion criteria to select the models. We also checked the citations of the included models to find other scholars' perspectives. Each model was described and critiqued each model in detail and discussed their features and limitations.

Result

Nine models of EIPM in healthcare were identified. While models had some strengths in comprehension, flexibility and theoretical foundations, analysis also identified limitations including: presupposing rational policymaking; lacking alternatives for time-sensitive situations; not capturing policy complexity; neglecting unintended effects; limited context considerations; inadequate complexity concepts; limited collaboration guidance; and unspecified evidence adaptations.

Conclusion

The reviewed models provide useful frameworks for EIPM but need further improvement to address their limitations. Concepts from sociology of knowledge, change theory and complexity science can enrich the models. Future EIPM models should better account for the complexity of research-policy relationships and provide tailored strategies based on the policy context.

Clinical research imperatives: principles and priorities from the perspective of Allied Health executives and managers

Objective It is widely understood that a key means of improving health systems and patient outcomes is through research - accessing, understanding, generating and applying research evidence-based practice. To promote more targeted and strategic research in Allied Health practice, this study sought to establish the principles, areas and priorities for clinical research as perceived by Allied Health leaders in the South Australian public health system. Methods The study used a mixed-methods design (full, sequential and equal model). Participants were recruited from theSouth Australian Department of Health and Wellbeing employment lists for Allied Health senior leaders. Consenting participants attended face-to-face focus groups; after an overview presentation, they discussed the principles for Allied Health research, followed by areas and priorities for research. Summaries of the responses were themed descriptively and circulated electronically so participants could confirm the research areas and ascribe priority ratings, clinical populations and services. Results A total of 28 people attended the stakeholder forum (5 online); 20 responded to the second-round electronic summary. Nine principles of research action were agreed. Fourteen research areas were identified and prioritised. There was a relatively consistent prioritisation of measuring Allied Health value, Allied Health workforce, hospital avoidance and closing the gap for Aboriginal health outcomes - whether the individuals were thinking about their own service or the state as a whole. Conclusions Allied Health leadership identified key principles and priorities for research to improve service delivery and patient outcomes. These priorities should generate further discussion and interest for novice and experienced researchers and leaders and can be used to inform granting and project plans.

Evidence use in the UK's COVID-19 Free School Meals Policy: a thematic content analysis

Free School Meals (FSM) are a well-recognised intervention for tackling food insecurity among school children. National school closures during the COVID-19 pandemic meant that there was a need to rapidly adapt the delivery of FSM. A range of food-assistance policies were implemented, but it is not clear if they were evidence-based. This study aimed to determine the transparency of evidence use and identify other competing influences in the UK's FSM policy decisions. Thematic content analysis was used to review 50 publicly available policy documents and debate transcripts on FSM policy published between March 2020-2021. This period covered the first national school closures (March 2020-July 2020), school holidays and the second national school closures (January 2021- March 2021). The Evidence Transparency Framework was used to assess the transparency of evidence use in policy documents. We found that overall transparency of evidence use was poor but was better for the Holiday Activities and Food (HAF) programme. The Government showed preference for replacing FSM with food parcels, rather than more agentic modes of food assistance such as cash-vouchers. This preference appeared to be closely aligned with ideological views on the welfare state. With an absence of evidence, value-based reasoning took precedent and was polarised by social media. This paper highlights the need for a formal review into FSM, one which includes a comparison of low and high agentic food assistance policies. Such a review would address the evidence gap, improve food assistance policy, and aid policymakers in future periods of uncertainty.

Where is "policy" in dissemination and implementation science? Recommendations to advance theories, models, and frameworks: EPIS as a case example

BACKGROUND

Implementation science aims to accelerate the public health impact of evidence-based interventions. However, implementation science has had too little focus on the role of health policy - and its inseparable politics, polity structures, and policymakers - in the implementation and sustainment of evidence-based healthcare. Policies can serve as determinants, implementation strategies, the evidence-based "thing" to be implemented, or another variable in the causal pathway to healthcare access, quality, and patient outcomes. Research describing the roles of policy in dissemination and implementation (D&I) efforts is needed to resolve persistent knowledge gaps about policymakers' evidence use, how evidence-based policies are implemented and sustained, and methods to de-implement policies that are ineffective or cause harm. Few D&I theories, models, or frameworks (TMF) explicitly guide researchers in conceptualizing where, how, and when policy should be empirically investigated. We conducted and reflected on the results of a scoping review to identify gaps of existing Exploration, Preparation, Implementation, and Sustainment (EPIS) framework-guided policy D&I studies. We argue that rather than creating new TMF, researchers should optimize existing TMF to examine policy's role in D&I. We describe six recommendations to help researchers optimize existing D&I TMF. Recommendations are applied to EPIS, as one example for advancing TMF for policy D&I.

RECOMMENDATIONS

(1) Specify dimensions of a policy's function (policy goals, type, contexts, capital exchanged). (2) Specify dimensions of a policy's form (origin, structure, dynamism, outcomes). (3) Identify and define the nonlinear phases of policy D&I across outer and inner contexts. (4) Describe the temporal roles that stakeholders play in policy D&I over time. (5) Consider policy-relevant outer and inner context adaptations. (6) Identify and describe bridging factors necessary for policy D&I success.

CONCLUSION

Researchers should use TMF to meaningfully conceptualize policy's role in D&I efforts to accelerate the public health impact of evidence-based policies or practices and de-implement ineffective and harmful policies. Applying these six recommendations to existing D&I TMF advances existing theoretical knowledge, especially EPIS application, rather than introducing new models. Using these recommendations will sensitize researchers to help them investigate the multifaceted roles policy can play within a causal pathway leading to D&I success.

Tensions and Paradoxes of Scaling Up: A Critical Reflection on Physical Activity Promotion

Achieving system-level, sustainable 'scale-up' of interventions is the epitome of successful translation of evidence-based approaches in population health. In physical activity promotion, few evidence-based interventions reach implementation at scale or become embedded within systems for sustainable health impact. This is despite the vast published literature describing efficacy studies of small-scale physical activity interventions. Research into physical activity scale-up (through case-study analysis; evaluations of scale-up processes in implementation trials; and mapping the processes, strategies, and principles for scale-up) has identified barriers and facilitators to intervention expansion. Many interventions are implemented at scale by governments but have not been evaluated or have unpublished evaluation information. Further, few public health interventions have evaluations that reveal the costs and benefits of scaled-up implementation. This lack of economic information introduces an additional element of risk for decision makers when deciding which physical activity interventions should be supported with scarce funding resources. Decision-makers face many other challenges when scaling interventions which do not relate to formal research trials of scale-up; Methods: To explore these issues, a multidisciplinary two-day workshop involving experts in physical activity scale-up was convened by the University of Newcastle, Australia, and the University of Ottawa, Canada (February 2019); Results: In this paper we discuss some of the scale-up tensions (challenges and conflicts) and paradoxes (things that are contrary to expectations) that emerged from this workshop in the context of the current literature and our own experiences in this field. We frame scale-up tensions according to epistemology, methodology, time, and partnerships; and paradoxes as 'reach without scale', 'planned serendipity' and 'simple complexity'. We reflect on the implications of these scale-up tensions and paradoxes, providing considerations for future scale-up research and practice moving forward; Conclusions: In this paper, we delve deeper into stakeholders' assumptions, processes and expectations of scaling up, and challenge in what ways as stakeholders, we all contribute to desired or undesired outcomes. Through a lens of 'tensions' and 'paradoxes', we make an original contribution to the scale-up literature that might influence current perspectives of scaling-up, provide future approaches for physical activity promotion, and contribute to understanding of dynamic of research-practice partnerships.

A scoping review of frameworks in empirical studies and a review of dissemination frameworks

BACKGROUND

The field of dissemination and implementation (D&I) research has grown immensely in recent years. However, the field of dissemination research has not coalesced to the same degree as the field of implementation research. To advance the field of dissemination research, this review aimed to (1) identify the extent to which dissemination frameworks are used in dissemination empirical studies, (2) examine how scholars define dissemination, and (3) identify key constructs from dissemination frameworks.

METHODS

To achieve aims 1 and 2, we conducted a scoping review of dissemination studies published in D&I science journals. The search strategy included manuscripts published from 1985 to 2020. Articles were included if they were empirical quantitative or mixed methods studies about the dissemination of information to a professional audience. Studies were excluded if they were systematic reviews, commentaries or conceptual papers, scale-up or scale-out studies, qualitative or case studies, or descriptions of programs. To achieve aim 1, we compiled the frameworks identified in the empirical studies. To achieve aim 2, we compiled the definitions from dissemination from frameworks identified in aim 1 and from dissemination frameworks identified in a 2021 review (Tabak RG, Am J Prev Med 43:337-350, 2012). To achieve aim 3, we compile the constructs and their definitions from the frameworks.

FINDINGS

Out of 6017 studies, 89 studies were included for full-text extraction. Of these, 45 (51%) used a framework to guide the study. Across the 45 studies, 34 distinct frameworks were identified, out of which 13 (38%) defined dissemination. There is a lack of consensus on the definition of dissemination. Altogether, we identified 48 constructs, divided into 4 categories: process, determinants, strategies, and outcomes. Constructs in the frameworks are not well defined.

IMPLICATION FOR D&I RESEARCH

This study provides a critical step in the dissemination research literature by offering suggestions on how to define dissemination research and by cataloging and defining dissemination constructs. Strengthening these definitions and distinctions between D&I research could enhance scientific reproducibility and advance the field of dissemination research.

Inter-agency collaboration is associated with increased frequency of research use in children's mental health policy making

OBJECTIVE

To determine whether the self-report frequency of inter-agency collaboration about children's mental health issues is associated with the self-report frequency of using research evidence in children's mental health policy and program decision making in mental health agencies (MHAs).

DATA SOURCES

Primary data were collected through web-based surveys of state (N = 221) and county (N = 117) MHA officials.

DESIGN

The primary independent variable was a composite score quantifying the frequency of collaboration about children's mental health issues between officials in MHAs and six other state agencies. The dependent variables were composite scores quantifying the frequency of research use in children's mental health policy and program decision making in general and for specific purposes (i.e., conceptual, instrumental, tactical, imposed). Covariates were composite scores quantifying well-established determinants of research use (e.g., agency leadership, research use skills) in agency policy and program decision making.

DATA METHODS

Separate multiple linear regression models estimated associations between frequency of inter-agency collaboration and research use scores, adjusting for other determinants of research use, respondent state, and other covariates. Data from state and county officials were analyzed separately.

PRINCIPAL FINDINGS

The frequency of inter-agency collaboration was positively and independently associated with the frequency of research use in children's mental health policy making among state (β = 0.22, p = 0.004) and county (β = 0.39, p < 0.0001) MHA officials. Inter-agency collaboration was also the only variable significantly associated with the frequency of research use for all four specific purposes among state MHA officials, and similar findings we observed among county MHA officials. The magnitudes of associations between inter-agency collaboration and frequency of research use were generally stronger than for more well-established determinants of research use in policy making.

CONCLUSIONS

Strategies that promote collaboration between MHA officials and external agencies could increase the use of research evidence in children's mental health policy and program decision making in MHAs.

Use of research evidence varied in efforts to expand specific pharmacist autonomous prescriptive authority: an evaluation and recommendations to increase research utilization

BACKGROUND

An expanding body of literature shows that pharmacists' interventions improve health outcomes and are cost-saving. However, diverse state regulations of pharmacists' scope of practice create a discrepancy between what pharmacists are trained to do and what they legally can do. This study investigated how stakeholders utilized research evidence when developing expanded scope of practice policies in their respective states.

METHODS

Using autonomous pharmacist prescriptive authority as a surrogate for general pharmacist scope of practice, a general policy document analysis was performed to understand the scope of practice landscape for pharmacists across the United States. Next, semi-structured interviews with policy-makers and pharmacy advocates were conducted to explore how the identified states in the policy document analysis utilized evidence during the policy-making process. Investigators analysed findings from the transcribed interviews through application of the SPIRIT Action Framework. Resulting codes were summarized across themes, and recommendations to researchers about increasing utilization of research evidence were crafted.

RESULTS

Sixteen states with 27 autonomous pharmacist prescriptive authority policies were identified. Public health need and safety considerations motivated evidence engagement, while key considerations dictating utilization of research included perceptions of research, access to resources and experts, and the successful implementation of similar policy. Research evidence helped to advocate for and set terms for pharmacist prescribing. Barriers to research utilization include stakeholder opposition to pharmacist prescribing, inability to interpret research, and a lack of relevant evidence. Recommendations for researchers include investigating specific metrics to evaluate scope of practice policy, developing relationships between policy-makers and researchers, and leveraging pharmacy practice stakeholders.

CONCLUSIONS

Overall, alignment of researcher goals and legislative priorities, coupled with timely communication, may help to increase research evidence engagement in pharmacist scope of practice policy. By addressing these factors regarding research engagement identified in this study, researchers can increase evidence-based scope of practice, which can help to improve patient outcomes, contain costs, and provide pharmacists with the legal infrastructure to practise at the top of their license.

Policymakers' Research Capacities, Engagement, and Use of Research in Public Health Policymaking

The use of research in public health policymaking is one of the prerequisites for successfully implemented health policies which have better population health as an outcome. This policy process is influenced by the actors involved under the policy umbrella, with inter-related contextual factors and specific structural and institutional circumstances. Our study investigates how policymakers’ research capacities influence the use of research in the health policy process and identify areas where capacity-building interventions give the most meaning and impact. Furthermore, we investigate policymakers’ research engagement and use this to inform public health policy in the public sector in Denmark. We collect and report data using Seeking, Engaging with, and Evaluation Research (SEER) methodology. Policymakers are reported to have research capacity, but it is questionable how those competences have actually been used in policymaking. Decision-makers were often not aware or did not know about the existing organizational tools and systems for research engagement and use and two third of respondents had not been part of any research activities or had any collaboration with researchers. Overall, research use in public health policymaking and evaluation was limited. As a conclusion, we propose that capacity-building interventions for increasing research use and collaboration in EIPM should be context-oriented, measurable, and sustainable in developing individual and organizational competences.

The integrated screening action model (I-SAM): A theory-based approach to inform intervention development

Screening can reduce deaths if the people invited participate. However, good uptake is hard to achieve, and our current approaches are failing to engage the most vulnerable. A coherent model of screening behaviour to guide our understanding and intervention development is yet to be established. The present aim was to propose an Integrated Screening Action Model (I-SAM) to improve screening access. The I-SAM synthesises existing models of health behaviour and empirical evidence. The I-SAM was developed following: i) an appraisal of the predominant models used within the screening literature; ii) the integration of the latest knowledge on behaviour change; with iii) the empirical literature, to inform the development of a theory-based approach to intervention development. There are three key aspects to the I-SAM: i) a sequence of stages that people pass through in engaging in screening behaviour (based on the Precaution Adoption Process Model); ii) screening behaviour is shaped by the interaction between participant and environmental influences (drawing from the Access Framework); and iii) targets for intervention should focus on the sources of behaviour - 'capability', 'opportunity', and 'motivation' (based on the COM-B Model). The I-SAM proposes an integrated model to support our understanding of screening behaviour and to identify targets for intervention. It will be an iterative process to test and refine the I-SAM and establish its value in supporting effective interventions to improve screening for all.

Exploring research impact models: A systematic scoping review

This article explores the models and frameworks developed on “research impact’. We aim to provide a comprehensive overview of related literature through scoping study method. The present research investigates the nature, objectives, approaches, and other main attributes of the research impact models. It examines to analyze and classify models based on their characteristics. Forty-seven studies and 10 reviews published between 1996 and 2020 were included in the analysis. The majority of models were developed for the impact assessment and evaluation purposes. We identified three approaches in the models, namely outcome-based, process-based, and those utilized both of them, among which the outcome-based approach was the most frequently used by impact models and evaluation was considered as the main objective of this group. The process-based ones were mainly adapted from the W.K. Kellogg Foundation logic model and were potentially eligible for impact improvement. We highlighted the scope of processes and other specific features for the recent models. Given the benefits of the process-based approach in enhancing and accelerating the research impact, it is important to consider such approach in the development of impact models. Effective interaction between researchers and stakeholders, knowledge translation, and evidence synthesis are the other possible driving forces contributing to achieve and improve impact.

Health research and knowledge translation for achieving the sustainable development goals: tackling the hurdles

We are far from reaching the sustainable development goals (SDGs) for health despite a wealth of novel insights in disease mechanisms and possible solutions. Why have we failed in knowledge translation and implementation? Starting from the case of cardiovascular diseases as one of the most prevalent non-communicable diseases, we examine barriers and hurdles, and perspectives for future health research. Health has multiple links with other SDGs. To accelerate the progress towards a healthy society, health research needs to take a broader view and become more cross-disciplinary and cross-sectoral. As one example, behavioural studies will underpin better prevention and treatment adherence. The next generation workforce in health and research needs an adapted education and training to implement more effective health approaches. As well, only effective dialogue and communication between researchers, practitioners, society and policymakers can lead to translation of evidence into policies, addressing the complexity of socioeconomic factors and commercial interests. Within Europe, health research needs a comprehensive vision and strategy that connects to achieving better health, as one of the interconnected SDGs.

[Establishing evidence-based prevention and health promotion: criteria for evidence-based interventions and necessary organizational requirements and capacities]

Für die Umsetzung des Präventionsgesetzes in einem sich entwickelnden System Prävention und Gesundheitsförderung (PGF) ist die Anforderung der Evidenzbasierung formuliert. Vor diesem Hintergrund stellt sich die Frage, welche Schritte, Prozesse und Vorgehensweisen in diesem System benötigt werden, um der Anforderung zunehmend gerecht zu werden. Dieser Übersichtsartikel diskutiert für Deutschland, wie evidenzbasierte Maßnahmen in der Praxis operationalisiert werden können und welche organisationalen Rahmenbedingungen und Kapazitäten für ein evidenzbasiertes Handeln von AkteurInnen notwendig sind.

Aufbauend auf internationalen wissenschaftlichen Erkenntnissen und dem Memorandum Evidenzbasierte Prävention und Gesundheitsförderung der Bundeszentrale für gesundheitliche Aufklärung (BZgA) wird zunächst das Verständnis von evidenzbasierten Maßnahmen erläutert und im Weiteren werden Elemente zur Umsetzung von mehr Evidenzbasierung skizziert.

Neben der transparenten und einheitlichen Darstellung in Datenbanken und Empfehlungen ist es notwendig, bei EntscheidungsträgerInnen in Praxis und Politik ein gemeinsames Verständnis von evidenzbasierten Interventionen und von Anforderungen für eine Evaluation, die Evidenzbasierung sichert, zu schaffen. Darüber hinaus kann evidenzbasiertes Handeln von EntscheidungsträgerInnen gefördert werden durch Wertschätzung gegenüber Evidenzbasierung in ihren Organisationen, durch Gewährleistung eines regelhaften Zugangs zu Evidenzdatenbanken, durch verbesserte Kompetenzen in Bezug auf Interpretation von Evidenz und durch eine systematische Zusammenarbeit mit der Wissenschaft.

Mehr Evidenzbasierung ist eine Voraussetzung für die nachhaltige Verankerung von PGF als fünfte Säule des Gesundheitssystems.

Assessing organisational capacity for evidence-informed health policy and planning: an adaptation of the ORACLe tool for Australian primary health care organizations

Background

Many nations have established primary health care (PHC) organizations that conduct PHC planning for defined geographical areas. The Australian Government established Primary Health Networks (PHNs) in 2015 to develop and commission PHC strategies to address local needs. There has been little written about the capacity of such organizations for evidence-informed planning, and no tools have been developed to assess this capacity, despite their potential to contribute to a comprehensive effective and efficient PHC sector.

Methods

We adapted the ORACLe tool, originally designed to examine evidence-informed policy-making capacity, to examine organizational capacity for evidence-informed planning in meso-level PHC organizations, using PHNs as an example. Semi-structured interviews were conducted with 14 participants from five PHNs, using the ORACLe tool, and scores assigned to responses, in seven domains of capacity.

Results

There was considerable variation between PHNs and capacity domains. Generally, higher capacity was demonstrated in regard to mechanisms which could inform planning through research, and support relationships with researchers. PHNs showed lower capacity for evaluating initiatives, tools and support for staff, and staff training.

Discussion and conclusions

We critique the importance of weightings and scope of some capacity domains in the ORACLe tool. Despite this, with some minor modifications, we conclude the ORACLe tool can identify capacity strengths and limitations in meso-level PHC organizations. Well-targeted capacity development enables PHC organizations’ strategies to be better informed by evidence, for optimal impact on PHC and population health outcomes.

Determinants of using children's mental health research in policymaking: variation by type of research use and phase of policy process

Background

Research use in policymaking is multi-faceted and has been the focus of extensive study. However, virtually no quantitative studies have examined whether the determinants of research use vary according to the type of research use or phase of policy process. Understanding such variation is important for selecting the targets of implementation strategies that aim to increase the frequency of research use in policymaking.

Methods

A web-based survey of US state agency officials involved with children’s mental health policymaking was conducted between December 2019 and February 2020 (n = 224, response rate = 33.7%, 49 states responding (98%), median respondents per state = 4). The dependent variables were composite scores of the frequency of using children’s mental health research in general, specific types of research use (i.e., conceptual, instrumental, tactical, imposed), and during different phases of the policy process (i.e., agenda setting, policy development, policy implementation). The independent variables were four composite scores of determinants of research use: agency leadership for research use, agency barriers to research use, research use skills, and dissemination barriers (e.g., lack of actionable messages/recommendations in research summaries, lack of interaction/collaboration with researchers). Separate multiple linear regression models estimated associations between determinant and frequency of research use scores.

Results

Determinants of research use varied significantly by type of research use and phase of policy process. For example, agency leadership for research use was the only determinant significantly associated with imposed research use (β = 0.31, p < 0.001). Skills for research use were the only determinant associated with tactical research use (β = 0.17, p = 0.03) and were only associated with research use in the agenda-setting phase (β = 0.16, p = 0.04). Dissemination barriers were the most universal determinants of research use, as they were significantly and inversely associated with frequency of conceptual (β = −0.21, p = 0.01) and instrumental (β = −0.22, p = 0.01) research use and during all three phases of policy process.

Conclusions

Decisions about the determinants to target with policy-focused implementation strategies—and the strategies that are selected to affect these targets—should reflect the specific types of research use that these strategies aim to influence.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13012-021-01081-8.

Lessons learned from descriptions and evaluations of knowledge translation platforms supporting evidence-informed policy-making in low- and middle-income countries: a systematic review

BACKGROUND

Knowledge translation (KT) platforms are organisations, initiatives and networks that focus on supporting evidence-informed policy-making at least in part about the health-system arrangements that determine whether the right programmes, services and products get to those who need them. Many descriptions and evaluations of KT platforms in low- and middle-income countries have been produced but, to date, they have not been systematically reviewed.

METHODS

We identified potentially relevant studies through a search of five electronic databases and a variety of approaches to identify grey literature. We used four criteria to select eligible empirical studies. We extracted data about seven characteristics of included studies and about key findings. We used explicit criteria to assess study quality. In synthesising the findings, we gave greater attention to themes that emerged from multiple studies, higher-quality studies and different contexts.

RESULTS

Country was the most common jurisdictional focus of KT platforms, EVIPNet the most common name and high turnover among staff a common infrastructural feature. Evidence briefs and deliberative dialogues were the activities/outputs that were the most extensively studied and viewed as helpful, while rapid evidence services were the next most studied but only in a single jurisdiction. None of the summative evaluations used a pre-post design or a control group and, with the exception of the evaluations of the influence of briefs and dialogues on intentions to act, none of the evaluations achieved a high quality score.

CONCLUSIONS

A large and growing volume of research evidence suggests that KT platforms offer promise in supporting evidence-informed policy-making in low- and middle-income countries. KT platforms should consider as next steps expanding their current, relatively limited portfolio of activities and outputs, building bridges to complementary groups, and planning for evaluations that examine 'what works' for 'what types of issues' in 'what types of contexts'.

Powerful numbers: Exemplary quantitative studies of science that had policy impact

Much scientometric research aims to be relevant to policy, but such research only rarely has a notable policy impact. In this paper, we examine four exemplary cases of policy impact from quantitative studies of science. The cases are analyzed in light of lessons learned about the use of evidence in policy making in health services, which provides very thorough explorations of the problems inherent in policy use of academic research. The analysis highlights key dimensions of the examples, which offer lessons for those aspiring to influence policy with quantitative studies of science.

Health system decision-makers at the helm of implementation research: development of a framework to evaluate the processes and effectiveness of embedded approaches

Background

Embedded approaches to implementation research (IR), whereby health system decision-makers participate actively in the research process, are gaining traction as effective approaches to optimise the delivery of health programmes and policies. However, the evidence base on the processes and effectiveness of such collaborative research remains inchoate. Standardised approaches to evaluate these initiatives are needed to identify core elements of ‘embeddedness’, unveil the underlying pathways of change, and assess contribution to evidence uptake in decision-making and overall outcomes of effect. The framework presented in this paper responds to this need, designed to guide the systematic evaluation of embedded IR.

Methods

This evaluation framework for embedded IR approaches is based on the experience of a joint initiative by the Pan American Health Organization/Alliance for Health Policy and Systems Research, which has supported 19 IR grants in 10 Latin American and Caribbean countries from 2014 to 2017. The conceptualisation of this framework drew on various sources of information, including empirical evidence and conceptual insights from the literature, interviews with content experts, and a prospective evaluation of the 2016 cohort that included semi-structured key informant interviews, document analysis, and a research team survey to examine key aspects of embedded research.

Results

We developed a widely applicable conceptual framework to guide the evaluation of embedded IR in various contexts. Focused on uncovering how this collaborative research approach influences programme improvement, it outlines expected processes and intermediate outcomes. It also highlights constructs with which to assess ‘embeddedness’ as well as critical contextual factors. The framework is intended to provide a structure by which to systematically examine such embedded research initiatives, proposing three key stages of evidence-informed decision-making – co-production of evidence, engagement with research, and enactment of programme changes.

Conclusion

Rigorous evaluation of embedded IR is needed to build the evidence on its processes and effectiveness in influencing decision-making. The evaluation framework presented here addresses this gap with consideration of the complexity of such efforts. Its applicability to similar initiatives is bolstered by virtue of being founded on real-world experience; its potential to contribute to a nuanced understanding of embedded IR is significant.

The EVITA framework for evidence-based mental health policy agenda setting in low- and middle-income countries

The burden of mental illness is excessive, but many countries lack evidence-based policies to improve practice. Mental health research evidence translation into policymaking is a 'wicked problem', often failing despite a robust evidence base. In a recent systematic review, we identified a gap in frameworks on agenda setting and actionability, and pragmatic, effective tools to guide action to link research and policy are needed. Responding to this gap, we developed the new EVITA 1.1 (EVIdence To Agenda setting) conceptual framework for mental health research-policy interrelationships in low- and middle-income countries (LMICs). We (1) drafted a provisional framework (EVITA 1.0); (2) validated it for specific applicability to mental health; (3) conducted expert in-depth interviews to (a) validate components and mechanisms and (b) assess intelligibility, functionality, relevance, applicability and effectiveness. To guide interview validation, we developed a simple evaluation framework. (4) Using deductive framework analysis, we coded and identified themes and finalized the framework (EVITA 1.1). Theoretical agenda-setting elements were added, as targeting the policy agenda-setting stage was found to lead to greater policy traction. The framework was validated through expert in-depth interviews (n = 13) and revised. EVITA 1.1 consists of six core components [advocacy coalitions, (en)actors, evidence generators, external influences, intermediaries and political context] and four mechanisms (capacity, catalysts, communication/relationship/partnership building and framing). EVITA 1.1 is novel and unique because it very specifically addresses the mental health research-policy process in LMICs and includes policy agenda setting as a novel, effective mechanism. Based on a thorough methodology, and through its specific design and mechanisms, EVITA has the potential to improve the challenging process of research evidence translation into policy and practice in LMICs and to increase the engagement and capacity of mental health researchers, policy agencies/planners, think tanks, NGOs and others within the mental health research-policy interface. Next, EVITA 1.1 will be empirically tested in a case study.

Investigating the process of evidence-informed health policymaking in Bangladesh: a systematic review

Over the last four decades, Bangladesh has made considerable improvements in population health, this is in part due to the use of evidence to inform policymaking. This systematic review aims to better understand critical factors that have facilitated the diffusion of scientific evidence into multiple phases of health policymaking in Bangladesh. To do this an existing policy framework designed by Shiffman and Smith in 2007, was used to extract and synthesize data from selected policy analyses. This framework was used to ensure the content, context and actors involved with evidence-informed policymaking were considered in each case where research had helped shape a health policy. The 'PRISMA Checklist' was employed to design pre-specified eligibility criteria for the selection of information sources, search strategy, inclusion and exclusion criteria, and process of data extraction and synthesis. Through our systematic search conducted from February to May 2017, we initially identified 1859 articles; after removal of duplicates, followed by the screening of titles, abstracts and full-texts, 24 articles were included in the analysis. Health policy issues included the following topics: maternal and child health, tobacco control, reproductive health, infectious disease control and the impact and sustainability of knowledge translation platforms. Findings suggested that research evidence that could be used to meet key targets associated with the Millennium Development Goals (MDGs) were more likely to be considered as a political (and therefore policy) priority. Furthermore, avenues of engagement between research organizations and the government as well as collective action from civil-society organizations were important for the diffusion of evidence into policies. Through this article, it is apparent that the interface between evidence and policy formulation occurs when evidence is, disseminated by a cohesive policy-network with strong leadership and framed to deliver solutions for problems on both the domestic and global development agenda.

"Alcohol During Pregnancy? Nobody Does That Anymore": State Legislators' Use of Evidence in Making Policy on Alcohol Use in Pregnancy

OBJECTIVE

In recent years, U.S. states have passed many laws addressing alcohol use in pregnancy, despite limited evidence on the impact of such policies. This study explores how state legislators use evidence when making policy on alcohol use in pregnancy.

METHOD

Study data are drawn from semistructured interviews with 29 state lawmakers and their aides in Maryland, North Carolina, and Virginia, conducted in March through July 2017. Interview transcripts were coded and analyzed by inductive and deductive methods.

RESULTS

Despite evidence on the harms of alcohol use in pregnancy, most lawmakers did not express concern about this topic. Instead, they expressed concern about opioid use in pregnancy. Personal experiences, anecdotes, and known contacts influenced legislators' views on substance use in pregnancy, whereas evidence, for the most part, did not. The intermediaries who typically bring evidence about problems and solutions to legislators did not appear to be raising the issue of alcohol use in pregnancy on legislators' agenda.

CONCLUSIONS

Basic evidence on the prevalence and harms of alcohol use in pregnancy did not appear to influence state lawmakers' policy priorities. Concern over opioid use in general may provide a window of opportunity to educate legislators on the relative scope and harms of alcohol and opioid use in pregnancy. It remains unclear why states are passing alcohol-in-pregnancy policies. More research is needed to explore how state lawmakers form their understanding of substance use in pregnancy and related policies.

Evaluating research investment and impact at a regional Australian Hospital and Health Service: a programme theory and conceptual framework

Background

Health systems in Australia and worldwide are increasingly expected to conduct research and quality improvement activities in addition to delivering clinical care and training health professionals. This study aims to inform a research impact evaluation at a regional Australian Hospital and Health Service by developing a programme theory showing how research investment is expected to have impact.

Methods

This qualitative study, representing the first phase of a larger mixed methods research impact evaluation at the Townsville Hospital and Health Service (THHS), adopts a realist-informed design involving the development of a programme theory. Data were obtained between February and May 2019 from strategic documentation and interviews with six current and former health service executives and senior employees. Inductive themes were integrated into a conceptual framework to visually represent the programme theory.

Results

Research at THHS has developed organically as the service has matured into a regional tertiary referral service serving a diverse rural and remote population across northern Queensland. Throughout this journey, individual THHS leaders often adopted a research development mantle despite disincentives arising from a performance-driven reporting and activity-based funding service context. Impact expectations from research investment at THHS were identified in the categories of enhanced research activity and capacity among clinicians, and improved clinical practice, health workforce capability and stability, and patient and population health. Seven contextual factors were identified as potential enablers or obstacles to these impact expectations and ambitions.

Conclusions

By identifying both relevant impact types and key contextual factors, this study offers programme theory to inform a planned research impact evaluation at THHS. The conceptual framework may be useful in other regionally based health service settings. More broadly, there are opportunities for future research to test and refine hybrid versions of linear and realist research impact evaluation models that combine resource-intensive, theory-driven approaches with policy practicality.

Identification and evaluation of risk of generalizability biases in pilot versus efficacy/effectiveness trials: a systematic review and meta-analysis

BACKGROUND

Preliminary evaluations of behavioral interventions, referred to as pilot studies, predate the conduct of many large-scale efficacy/effectiveness trial. The ability of a pilot study to inform an efficacy/effectiveness trial relies on careful considerations in the design, delivery, and interpretation of the pilot results to avoid exaggerated early discoveries that may lead to subsequent failed efficacy/effectiveness trials. "Risk of generalizability biases (RGB)" in pilot studies may reduce the probability of replicating results in a larger efficacy/effectiveness trial. We aimed to generate an operational list of potential RGBs and to evaluate their impact in pairs of published pilot studies and larger, more well-powered trial on the topic of childhood obesity.

METHODS

We conducted a systematic literature review to identify published pilot studies that had a published larger-scale trial of the same or similar intervention. Searches were updated and completed through December 31st, 2018. Eligible studies were behavioral interventions involving youth (≤18 yrs) on a topic related to childhood obesity (e.g., prevention/treatment, weight reduction, physical activity, diet, sleep, screen time/sedentary behavior). Extracted information included study characteristics and all outcomes. A list of 9 RGBs were defined and coded: intervention intensity bias, implementation support bias, delivery agent bias, target audience bias, duration bias, setting bias, measurement bias, directional conclusion bias, and outcome bias. Three reviewers independently coded for the presence of RGBs. Multi-level random effects meta-analyses were performed to investigate the association of the biases to study outcomes.

RESULTS

A total of 39 pilot and larger trial pairs were identified. The frequency of the biases varied: delivery agent bias (19/39 pairs), duration bias (15/39), implementation support bias (13/39), outcome bias (6/39), measurement bias (4/39), directional conclusion bias (3/39), target audience bias (3/39), intervention intensity bias (1/39), and setting bias (0/39). In meta-analyses, delivery agent, implementation support, duration, and measurement bias were associated with an attenuation of the effect size of - 0.325 (95CI - 0.556 to - 0.094), - 0.346 (- 0.640 to - 0.052), - 0.342 (- 0.498 to - 0.187), and - 0.360 (- 0.631 to - 0.089), respectively.

CONCLUSIONS

Pre-emptive avoidance of RGBs during the initial testing of an intervention may diminish the voltage drop between pilot and larger efficacy/effectiveness trials and enhance the odds of successful translation.

The transfer of knowledge on integrated care among five European regions: a qualitative multi-method study

BACKGROUND

To examine how the knowledge transfer processes unfolded within SCIROCCO, a EU funded project (3rd Health Programme (2014-2020)) that aimed to facilitate the process of knowledge sharing across five European regions, to speed up adoption and scaling-up of integrated care initiatives.

METHODS

A qualitative multi-method design was used. Data collection methods included focus groups, project documents and action plans of the regions. The data was analysed using a qualitative content-analysis procedure, which was guided by the frameworks of knowledge exchange and the why, whose, what, how framework for knowledge mobilisers.

RESULTS

All five components (including the themes) of knowledge exchange could be identified in the approach developed on the knowledge transfer processes. The four questions and accompanying categories of the framework of knowledge mobilisation were also identified to a large degree.

CONCLUSIONS

The observed incorporation of distinct forms of knowledge from multiple sources and the observed dynamic and fluid knowledge transfer processes both suggest that SCIROCCO developed a comprehensive knowledge transfer approach aiming to enable the adoption and scaling-up of integrated care. Overall, the multi-method qualitative nature of this research has allowed some new and practical insights in the knowledge transfer activities on integrated care between several European regions. To obtain a clear understanding of the content of the knowledge transfer approaches, which could assist the operationalising of models to support the evaluation of knowledge transfer activities, it is strongly recommended that further research of this type should be conducted in other research settings.

Organisational factors that facilitate research use in public health policy-making: a scoping review

Background

Although important syntheses and theoretical works exist in relation to understanding the organisational factors that facilitate research use, these contributions differ in their scope and object of study as well as their theoretical underpinnings. Therefore, from an exploratory angle, it may be useful to map out the current literature on organisational factors of research use in public health policy-making when revisiting existing theories and frameworks to gain further theoretical insights.

Methods

Herein, a scoping review technique and thematic content analysis were used to bring together findings from both synthesised and empirical studies of different types to map out the organisational factors that facilitate research use in public health policy-making.

Results

A total of 14 reviews and 40 empirical studies were included in the analysis. These were thematically coded and the intra-organisational factors reported as enabling research use were examined. Five main categories of organisational factors that advance research use in policy organisations – (1) individual factors, (2) the management of research integration, (3) organisational systems and infrastructures of research use, (4) institutional structures and rules for policy-making, and (5) organisational characteristics – were derived as well as 18 subcategories and a total of 64 specific factors, where 27 factors were well supported by research.

Conclusions

Using a scoping review methodology, the intra-organisational factors influencing research use in policy-making (including individual factors) were systematically mapped and the theories applied in this area of research were assessed. The review findings confirm the importance of an intra-organisational perspective when exploring research use, showing that many organisational factors are critical facilitators of research use but also that many factors and mechanisms are understudied. The synthesis shows a lack of studies on politicians and the need for more theoretically founded research. Despite increased efforts to update the existing evidential and theoretical basis of research use, we still need frameworks that combine different approaches and theories to help us grasp the complex organisational mechanisms that facilitate research use in policy settings.

How was research engaged with and used in the development of 131 policy documents? Findings and measurement implications from a mixed methods study

Background

Much has been written about the use of evidence in policy; however, there is still little known about whether and how research is engaged with and used in policy development or the impact of reported barriers and facilitators. This paper aims to (1) describe the characteristics of 131 policy documents, (2) describe the ways in which research was engaged with (e.g. was searched for, appraised or generated) and used (e.g. to clarify understanding, persuade others or inform a policy) in the development of these policy documents, and (3) identify the most commonly reported barriers and facilitators and describe their association with research engagement and use.

Methods

Six health policy and program development agencies based in Sydney, Australia, contributed four recently finalised policy documents for consideration over six measurement periods. Structured, qualitative interviews were conducted with the policymakers most heavily involved in developing each of the 131 policy documents. Interviews covered whether and how research was engaged with and used in the development of the policy product and any barriers or facilitators related to this. Interviews were scored using the empirically validated SAGE tool and thematically analysed. Descriptive statistics were calculated for all key variables and comparisons made between agencies. Multiple regression analyses were used to estimate the impact of specific barriers and facilitators on research engagement and use.

Results

Our data shows large variations between policy agencies in the types of policy documents produced and the characteristics of these documents. Nevertheless, research engagement and use was generally moderate across agencies. A number of barriers and facilitators to research use were identified. No barriers were significantly associated with any aspects of research engagement or use. Access to consultants and relationships with researchers were both associated with increased research engagement but not use. Thus, access to consultants and relationships with researchers may increase the extent and quality of the evidence considered in policy development.

Conclusions

Our findings suggest that those wishing to develop interventions and programs designed to improve the use of evidence in policy agencies might usefully target increasing access to consultants and relationships with researchers in order to increase the extent and quality of the research considered, but that a greater consideration of context might be required to develop strategies to increase evidence use.

Electronic supplementary material

The online version of this article (10.1186/s13012-019-0886-2) contains supplementary material, which is available to authorized users.

Improving the use of evidence in public health policy development, enactment and implementation: a multiple-case study

Many of the policies shown to benefit the public's health in research studies are never widely implemented. We conducted a qualitative and exploratory multiple-case study of three U.S. academic research-policy translation initiatives that resulted in the development, enactment and/or implementation of evidence-based policies to address public health issues: gun policy (Case 1); opioid policy (Case 2) and drug control policy (Case 3). We conducted semi-structured qualitative interviews with 25 key stakeholders involved in the three cases and analysed transcripts using a hybrid a priori and data-driven approach. Across cases, participants identified four key processes that should be included in research-policy translation initiatives: stakeholder engagement, consensus-building, long-term coalition engagement and use of effective knowledge brokers. Participants perceived differences in how the structure of and activities within each initiative facilitated these processes. For example, participants perceived the gun policy initiative, which included large-scale policy dissemination strategies such as state forums where research experts interacted with state policymakers and advocates, as more effective at long-term coalition engagement than the other two initiatives. Study results suggest concrete and actionable academic research-policy strategies that have the potential for widespread adoption by universities or other organizations.

Levels of research evidence in health policy assessment in Malawi

PURPOSE

The purpose of this study is to examine levels of health research evidence in health policies in Malawi.

DESIGN/METHODOLOGY/APPROACH

The study selected a typology of health policies in Malawi from 2002 to 2017. The study adopted the SPIRIT conceptual framework and assessed the levels of research evidence in health policy, systems and services research using the revised SAGE policy assessment tool. Documentary analysis was used to assess levels of health research evidence in health policies in Malawi.

FINDINGS

In 29 (96.7 per cent) of the health policies, policy formulators including healthcare directors and managers used generic search engines such as Google or Google Scholar to look for heath research evidence. In 28 (93.3 per cent) of the health policies, they searched for grey literature and other government documents. In only 6 (20 per cent) of the heath policy documents, they used academic literature in a form of journal articles and randomised controlled trials. No systematic reviews or policy briefs were consulted. Overall, in 23 (76.7 per cent) of the health policy documents, health research evidence played a minimal role and had very little influence on the policy documents or decision-making.

RESEARCH LIMITATIONS/IMPLICATIONS

The empirical evidence in the health policy documents are limited because of insufficient research citation, low retrievability of health research evidence in the policy documents and biased selectivity of what constitutes health research evidence.

PRACTICAL IMPLICATIONS

The study indicates that unfiltered information (data from policy evaluations and registries) constitutes majority of the research evidence in health policies both in health policy, systems and services research. The study seeks to advocate for the use of filtered information (peer reviewed, clinical trials and data from systematic reviews) in formulating health policies.

ORIGINALITY/VALUE

There is dearth of literature on the levels of health research evidence in health policy-making both in health policy, systems and services research. This study seeks to bridge the gap with empirical evidence from a developing country perspective.

Increasing the capacity of policy agencies to use research findings: a stepped-wedge trial

BACKGROUND

This paper describes the trial of a novel intervention, Supporting Policy In health with evidence from Research: an Intervention Trial (SPIRIT). It examines (1) the feasibility of delivering this kind of programme in practice; (2) its acceptability to participants; (3) the impact of the programme on the capacity of policy agencies to engage with research; and (4) the engagement with and use of research by policy agencies.

METHODS

SPIRIT was a multifaceted, highly tailored, stepped-wedge, cluster-randomised, trial involving six health policy agencies in Sydney, Australia. Agencies were randomly allocated to one of three start dates to receive the 1-year intervention programme. SPIRIT included audit, feedback and goal setting; a leadership programme; staff training; the opportunity to test systems to facilitate research use in policies; and exchange with researchers. Outcome measures were collected at each agency every 6 months for 30 months.

RESULTS

Participation in SPIRIT was associated with significant increases in research use capacity at staff and agency levels. Staff reported increased confidence in research use skills, and agency leaders reported more extensive systems and structures in place to support research use. Self-report data suggested there was also an increase in tactical research use among agency staff. Given the relatively small numbers of participating agencies and the complexity of their contexts, findings suggest it is possible to effect change in the way policy agencies approach the use of research. This is supported by the responses on the other trial measures; while these were not statistically significant, on 18 of the 20 different measures used, the changes observed were consistent with the hypothesised intervention effect (that is, positive impacts).

CONCLUSIONS

As an early test of an innovative approach, SPIRIT has demonstrated that it is possible to increase research engagement and use in policy agencies. While more work is needed to establish the replicability and generalisability of these findings, this trial suggests that building staff skills and organisational structures may be effective in increasing evidence use.

Knowledge mobilisation for chronic disease prevention: the case of the Australian Prevention Partnership Centre

Background

Cross-sectoral, multidisciplinary partnership research is considered one of the most effective means of facilitating research-informed policy and practice, particularly for addressing complex problems such as chronic disease. Successful research partnerships tend to be underpinned by a range of features that enable knowledge mobilisation (KMb), seeking to connect academic researchers with decision-makers and practitioners to improve the nature, quality and use of research. This paper contributes to the growing discourse on partnership approaches by illustrating how knowledge mobilisation strategies are operationalised within the Australian Prevention Partnership Centre (the Centre), a national collaboration of academics, policy-makers and practitioners established to develop systems approaches for the prevention of lifestyle-related chronic diseases.

Methods

We undertook interviews with key academics, policy, and practice partners and funding representatives at the mid-point of the Centre’s initial 5-year funding cycle. We aimed to explore how the Centre is functioning in practice, to develop a conceptual model of KMb within the Centre for use in further evaluation, and to identify ways of strengthening our approach to partnership research. Inductive and deductive thematic analysis was used to identify the key mechanisms underpinning the Centre’s KMb approach.

Results

Six key mechanisms appeared to facilitate KMb within our Centre, namely Engagement, Partnerships, Co-production, Capacity and Skills, Knowledge Integration, and Adaptive Learning and Improvement. We developed a conceptual model that articulated these mechanisms in relation to the structures and processes that support them, as well as the Centre’s goals. Findings also informed adaptations designed to strengthen the Centre.

Conclusions

Findings provide insights into the practical realities of operationalising KMb strategies within a research partnership. Overall, the centre is perceived to be progressing towards its KMb goals, but challenges include stakeholders from different settings understanding each other’s contexts and working together effectively, and ensuring knowledge generated across different projects within the Centre is integrated into a more comprehensive understanding of chronic disease prevention policy and practice. Our conceptual model is now informing ongoing developmental evaluation activities within the Centre, where it is being tested and refined.

Do evidence summaries increase health policy-makers' use of evidence from systematic reviews? A systematic review

This review summarizes the evidence from six randomized controlled trials that judged the effectiveness of systematic review summaries on policymakers' decision making, or the most effective ways to present evidence summaries to increase policymakers' use of the evidence. This review included six randomized controlled studies. A randomized controlled study is one in which the participants are divided randomly (by chance) into separate groups to compare different treatments or other interventions. This method of dividing people into groups means that the groups will be similar and that the effects of the treatments they receive will be compared more fairly. At the time the study is done, it is not known which treatment is the better one. The researchers who did these studies invited people from Europe, North America, South America, Africa, and Asia to take part in them. Two studies looked at "policy briefs," one study looked at an "evidence summary," two looked at a "summary of findings table," and one compared a "summary of findings table" to an evidence summary. None of these studies looked at how policymakers directly used evidence from systematic reviews in their decision making, but two studies found that there was little to no difference in how they used the summaries. The studies relied on reports from decision makers. These studies included questions such as, "Is this summary easy to understand?" Some of the studies looked at users' knowledge, understanding, beliefs, or how credible (trustworthy) they believed the summaries to be. There was little to no difference in the studies that looked at these outcomes. Study participants rated the graded entry format higher for usability than the full systematic review. The graded entry format allows the reader to select how much information they want to read. The study participants felt that all evidence summary formats were easier to understand than full systematic reviews.

Plain language summary

Policy briefs make systematic reviews easier to understand but little evidence of impact on use of study findings: It is likely that evidence summaries are easier to understand than complete systematic reviews. Whether these summaries increase the use of evidence from systematic reviews in policymaking is not clear.What is this review about?: Systematic reviews are long and technical documents that may be hard for policymakers to use when making decisions. Evidence summaries are short documents that describe research findings in systematic reviews. These summaries may simplify the use of systematic reviews.Other names for evidence reviews are policy briefs, evidence briefs, summaries of findings, or plain language summaries. The goal of this review was to learn whether evidence summaries help policymakers use evidence from systematic reviews. This review also aimed to identify the best ways to present the evidence summary to increase the use of evidence.What are the main findings of this review?: This review included six randomized controlled studies. A randomized controlled study is one in which the participants are divided randomly (by chance) into separate groups to compare different treatments or other interventions. This method of dividing people into groups means that the groups will be similar and that the effects of the treatments they receive will be compared more fairly. At the time the study is done, it is not known which treatment is the better one.The researchers who did these studies invited people from Europe, North America, South America, Africa, and Asia to take part in them. Two studies looked at "policy briefs," one study looked at an "evidence summary," two looked at a "summary of findings table," and one compared a "summary of findings table" to an evidence summary.None of these studies looked at how policymakers directly used evidence from systematic reviews in their decision making, but two studies found that there was little to no difference in how they used the summaries. The studies relied on reports from decision makers. These studies included questions such as, "Is this summary easy to understand?"Some of the studies looked at users' knowledge, understanding, beliefs, or how credible (trustworthy) they believed the summaries to be. There was little to no difference in the studies that looked at these outcomes. Study participants rated the graded entry format higher for usability than the full systematic review. The graded entry format allows the reader to select how much information they want to read.. The study participants felt that all evidence summary formats were easier to understand than full systematic reviews.What do the findings of this review mean?: Our review suggests that evidence summaries help policymakers to better understand the findings presented in systematic reviews. In short, evidence summaries should be developed to make it easier for policymakers to understand the evidence presented in systematic reviews. However, right now there is very little evidence on the best way to present systematic review evidence to policymakers.How up to date is this review?: The authors of this review searched for studies through June 2016.

Executive summary/Abstract

Background: Systematic reviews are important for decision makers. They offer many potential benefits but are often written in technical language, are too long, and do not contain contextual details which makes them hard to use for decision-making. Strategies to promote the use of evidence to decision makers are required, and evidence summaries have been suggested as a facilitator. Evidence summaries include policy briefs, briefing papers, briefing notes, evidence briefs, abstracts, summary of findings tables, and plain language summaries. There are many organizations developing and disseminating systematic review evidence summaries for different populations or subsets of decision makers. However, evidence on the usefulness and effectiveness of systematic review summaries is lacking. We present an overview of the available evidence on systematic review evidence summaries.Objectives: This systematic review aimed to 1) assess the effectiveness of evidence summaries on policy-makers' use of the evidence and 2) identify the most effective summary components for increasing policy-makers' use of the evidence.Search methods: We searched several online databases (Medline, EMBASE, CINAHL, Cochrane Central Register of Controlled Trials, Global Health Library, Popline, Africa-wide, Public Affairs Information Services, Worldwide Political Science Abstracts, Web of Science, and DfiD), websites of research groups and organizations which produce evidence summaries, and reference lists of included summaries and related systematic reviews. These databases were searched in March-April, 2016.Selection criteria: Eligible studies included randomised controlled trials (RCTs), non-randomised controlled trials (NRCTs), controlled before-after (CBA) studies, and interrupted time series (ITS) studies. We included studies of policymakers at all levels as well as health system managers. We included studies examining any type of "evidence summary", "policy brief", or other product derived from systematic reviews that presented evidence in a summarized form. These interventions could be compared to active comparators (e.g. other summary formats) or no intervention.The primary outcomes were: 1) use of systematic review summaries decision-making (e.g. self-reported use of the evidence in policy-making, decision-making) and 2) policymaker understanding, knowledge, and/or beliefs (e.g. changes in knowledge scores about the topic included in the summary). We also assessed perceived relevance, credibility, usefulness, understandability, and desirability (e.g. format) of the summaries.Results: Our database search combined with our grey literature search yielded 10,113 references after removal of duplicates. From these, 54 were reviewed in full text and we included 6 studies (reported in 7 papers, 1661 participants) as well as protocols from 2 ongoing studies. Two studies assessed the use of evidence summaries in decision-making and found little to no difference in effect. There was also little to no difference in effect for knowledge, understanding or beliefs (4 studies) and perceived usefulness or usability (3 studies). Summary of Findings tables and graded entry summaries were perceived as slightly easier to understand compared to complete systematic reviews. Two studies assessed formatting changes and found that for Summary of Findings tables, certain elements, such as reporting study event rates and absolute differences were preferred as well as avoiding the use of footnotes. No studies assessed adverse effects. The risks of bias in these studies were mainly assessed as unclear or low however, two studies were assessed as high risk of bias for incomplete outcome data due to very high rates of attrition.Authors' conclusions: Evidence summaries may be easier to understand than complete systematic reviews. However, their ability to increase the use of systematic review evidence in policymaking is unclear.

A systematic review of frameworks for the interrelationships of mental health evidence and policy in low- and middle-income countries

BACKGROUND

The interrelationships between research evidence and policy-making are complex. Different theoretical frameworks exist to explain general evidence-policy interactions. One largely unexplored element of these interrelationships is how evidence interrelates with, and influences, policy/political agenda-setting. This review aims to identify the elements and processes of theories, frameworks and models on interrelationships of research evidence and health policy-making, with a focus on actionability and agenda-setting in the context of mental health in low- and middle-income countries (LMICs).

METHODS

A systematic review of theories was conducted based on the BeHeMOTh search method, using a tested and refined search strategy. Nine electronic databases and other relevant sources were searched for peer-reviewed and grey literature. Two reviewers screened the abstracts, reviewed full-text articles, extracted data and performed quality assessments. Analysis was based on a thematic analysis. The included papers had to present an actionable theoretical framework/model on evidence and policy interrelationships, such as knowledge translation or evidence-based policy, specifically target the agenda-setting process, focus on mental health, be from LMICs and published in English.

RESULTS

From 236 publications included in the full text analysis, no studies fully complied with our inclusion criteria. Widening the focus by leaving out 'agenda-setting', we included ten studies, four of which had unique conceptual frameworks focusing on mental health and LMICs but not agenda-setting. The four analysed frameworks confirmed research gaps from LMICs and mental health, and a lack of focus on agenda-setting. Frameworks and models from other health and policy areas provide interesting conceptual approaches and lessons with regards to agenda-setting.

CONCLUSION

Our systematic review identified frameworks on evidence and policy interrelations that differ in their elements and processes. No framework fulfilled all inclusion criteria. Four actionable frameworks are applicable to mental health and LMICs, but none specifically target agenda-setting. We have identified agenda-setting as a research theory gap in the context of mental health knowledge translation in LMICs. Frameworks from other health/policy areas could offer lessons on agenda-setting and new approaches for creating policy impact for mental health and to tackle the translational gap in LMICs.

Sustainability in Health care by Allocating Resources Effectively (SHARE) 11: reporting outcomes of an evidence-driven approach to disinvestment in a local healthcare setting

Background

This is the final paper in a thematic series reporting a program of Sustainability in Health care by Allocating Resources Effectively (SHARE) in a local healthcare setting. The SHARE Program was established to explore a systematic, integrated, evidence-based organisation-wide approach to disinvestment in a large Australian health service network. This paper summarises the findings, discusses the contribution of the SHARE Program to the body of knowledge and understanding of disinvestment in the local healthcare setting, and considers implications for policy, practice and research.

Discussion

The SHARE program was conducted in three phases. Phase One was undertaken to understand concepts and practices related to disinvestment and the implications for a local health service and, based on this information, to identify potential settings and methods for decision-making about disinvestment. The aim of Phase Two was to implement and evaluate the proposed methods to determine which were sustainable, effective and appropriate in a local health service. A review of the current literature incorporating the SHARE findings was conducted in Phase Three to contribute to the understanding of systematic approaches to disinvestment in the local healthcare context.

SHARE differed from many other published examples of disinvestment in several ways: by seeking to identify and implement disinvestment opportunities within organisational infrastructure rather than as standalone projects; considering disinvestment in the context of all resource allocation decisions rather than in isolation; including allocation of non-monetary resources as well as financial decisions; and focusing on effective use of limited resources to optimise healthcare outcomes.

Conclusion

The SHARE findings provide a rich source of new information about local health service decision-making, in a level of detail not previously reported, to inform others in similar situations. Multiple innovations related to disinvestment were found to be acceptable and feasible in the local setting. Factors influencing decision-making, implementation processes and final outcomes were identified; and methods for further exploration, or avoidance, in attempting disinvestment in this context are proposed based on these findings. The settings, frameworks, models, methods and tools arising from the SHARE findings have potential to enhance health care and patient outcomes.

Electronic supplementary material

The online version of this article (10.1186/s12913-018-3172-0) contains supplementary material, which is available to authorized users.

Frameworks for embedding a research culture in allied health practice: a rapid review

BACKGROUND

Although allied health clinicians play a key role in the provision of healthcare, embedding a culture of research within public and private health systems remains a challenge. In this rapid review we critically evaluate frameworks for embedding research into routine allied health practice, as the basis for high quality, safe, efficient and consumer-focused care.

METHODS

A rapid review (PROSPERO: CRD42017075699) was conducted to evaluate frameworks designed to create and embed research in the health sector. Included were full-text, English-language, peer-reviewed publications or Government reports of frameworks that could inform the implementation of an allied health research framework. Eight electronic databases and four government websites were searched, using search terms such as models, frameworks and research capacity-building. Two independent researchers conducted all review stages and used content and thematic analysis to interpret the results.

RESULTS

Sixteen framework papers were finally included. Content analysis identified 44 system and regulatory level items that informed the research frameworks, 125 healthcare organisation items and 76 items relating to individual clinicians. Thematic analysis identified four major themes. Firstly, sustainable change requires allied health research policies, regulation, governance and organisational structures that support and value evidence-based practice. Secondly, research capability, receptivity, advocacy and literacy of healthcare leaders and managers are key to successful research implementation. Third, organisational factors that facilitate a research culture include dedicated staff research positions, time allocated to research, mentoring, professional education and research infrastructure. When healthcare agencies had strong partnerships with universities and co-located research leaders, research implementation was strengthened. Finally, individual attributes of clinicians, such as their research skills and capabilities, motivation, and participation in research teams, are essential to embedding research into practice.

CONCLUSION

Theoretical frameworks were identified that informed processes to embed a culture of allied health research into healthcare services. Research-led and evidence-informed allied health practice enables optimisation of workforce capability and high-quality care.

Organisational capacity and its relationship to research use in six Australian health policy agencies

There are calls for policymakers to make greater use of research when formulating policies. Therefore, it is important that policy organisations have a range of tools and systems to support their staff in using research in their work. The aim of the present study was to measure the extent to which a range of tools and systems to support research use were available within six Australian agencies with a role in health policy, and examine whether this was related to the extent of engagement with, and use of research in policymaking by their staff. The presence of relevant systems and tools was assessed via a structured interview called ORACLe which is conducted with a senior executive from the agency. To measure research use, four policymakers from each agency undertook a structured interview called SAGE, which assesses and scores the extent to which policymakers engaged with (i.e., searched for, appraised, and generated) research, and used research in the development of a specific policy document. The results showed that all agencies had at least a moderate range of tools and systems in place, in particular policy development processes; resources to access and use research (such as journals, databases, libraries, and access to research experts); processes to generate new research; and mechanisms to establish relationships with researchers. Agencies were less likely, however, to provide research training for staff and leaders, or to have evidence-based processes for evaluating existing policies. For the majority of agencies, the availability of tools and systems was related to the extent to which policymakers engaged with, and used research when developing policy documents. However, some agencies did not display this relationship, suggesting that other factors, namely the organisation's culture towards research use, must also be considered.

Does engagement predict research use? An analysis of The Conversation Annual Survey 2016

The impact of research on the world beyond academia has increasingly become an area of focus in research performance assessments internationally. Impact assessment is expected to incentivise researchers to increase engagement with industry, government and the public more broadly. Increased engagement is in turn expected to increase translation of research so decision-makers can use research to inform development of policies, programs, practices, processes, products, and other mechanisms, through which impact can be realised. However, research has shown that various factors affect research use, and evidence on 'what works' to increase decision-makers' use of research is limited. The Conversation is an open access research communication platform, published under Creative Commons licence, which translates research into news articles to engage a general audience, aiming to improve understanding of current issues and complex social problems. To identify factors that predict use of academic research and expertise reported in The Conversation, regression analyses were performed using The Conversation Australia 2016 Annual Survey data. A broad range of factors predicted use, with engagement actions being the most common. Interestingly, different types of engagement actions predicted different types of use. This suggests that to achieve impact through increased engagement, a deeper understanding of how and why different engagement actions elicit different types of use is needed. Findings also indicate The Conversation is overcoming some of the most commonly identified barriers to the use of research: access, relevance, actionable outcomes, and timeliness. As such, The Conversation offers an effective model for providing access to and communicating research in a way that enables use, a necessary precursor to achieving research impact.

Preliminary testing of the reliability and feasibility of SAGE: a system to measure and score engagement with and use of research in health policies and programs

Background

Few measures of research use in health policymaking are available, and the reliability of such measures has yet to be evaluated. A new measure called the Staff Assessment of Engagement with Evidence (SAGE) incorporates an interview that explores policymakers’ research use within discrete policy documents and a scoring tool that quantifies the extent of policymakers’ research use based on the interview transcript and analysis of the policy document itself. We aimed to conduct a preliminary investigation of the usability, sensitivity, and reliability of the scoring tool in measuring research use by policymakers.

Methods

Nine experts in health policy research and two independent coders were recruited. Each expert used the scoring tool to rate a random selection of 20 interview transcripts, and each independent coder rated 60 transcripts. The distribution of scores among experts was examined, and then, interrater reliability was tested within and between the experts and independent coders. Average- and single-measure reliability coefficients were computed for each SAGE subscales.

Results

Experts’ scores ranged from the limited to extensive scoring bracket for all subscales. Experts as a group also exhibited at least a fair level of interrater agreement across all subscales. Single-measure reliability was at least fair except for three subscales: Relevance Appraisal, Conceptual Use, and Instrumental Use. Average- and single-measure reliability among independent coders was good to excellent for all subscales. Finally, reliability between experts and independent coders was fair to excellent for all subscales.

Conclusions

Among experts, the scoring tool was comprehensible, usable, and sensitive to discriminate between documents with varying degrees of research use. Secondly, the scoring tool yielded scores with good reliability among the independent coders. There was greater variability among experts, although as a group, the tool was fairly reliable. The alignment between experts’ and independent coders’ ratings indicates that the independent coders were scoring in a manner comparable to health policy research experts. If the present findings are replicated in a larger sample, end users (e.g. policy agency staff) could potentially be trained to use SAGE to reliably score research use within their agencies, which would provide a cost-effective and time-efficient approach to utilising this measure in practice.

Electronic supplementary material

The online version of this article (10.1186/s13012-017-0676-7) contains supplementary material, which is available to authorized users.

Policymakers' experience of a capacity-building intervention designed to increase their use of research: a realist process evaluation

BACKGROUND

An intervention's success depends on how participants interact with it in local settings. Process evaluation examines these interactions, indicating why an intervention was or was not effective, and how it (and similar interventions) can be improved for better contextual fit. This is particularly important for innovative trials like Supporting Policy In health with Research: an Intervention Trial (SPIRIT), where causal mechanisms are poorly understood. SPIRIT was testing a multi-component intervention designed to increase the capacity of health policymakers to use research.

METHODS

Our mixed-methods process evaluation sought to explain variation in observed process effects across the six agencies that participated in SPIRIT. Data collection included observations of intervention workshops (n = 59), purposively sampled interviews (n = 76) and participant feedback forms (n = 553). Using a realist approach, data was coded for context-mechanism-process effect configurations (retroductive analysis) by two authors.

RESULTS

Intervention workshops were very well received. There was greater variation of views regarding other aspects of SPIRIT such as data collection, communication and the intervention's overall value. We identified nine inter-related mechanisms that were crucial for engaging participants in these policy settings: (1) Accepting the premise (agreeing with the study's assumptions); (2) Self-determination (participative choice); (3) The Value Proposition (seeing potential gain); (4) 'Getting good stuff' (identifying useful ideas, resources or connections); (5) Self-efficacy (believing 'we can do this!'); (6) Respect (feeling that SPIRIT understands and values one's work); (7) Confidence (believing in the study's integrity and validity); (8) Persuasive leadership (authentic and compelling advocacy from leaders); and (9) Strategic insider facilitation (local translation and mediation). These findings were used to develop tentative explanatory propositions and to revise the programme theory.

CONCLUSION

This paper describes how SPIRIT functioned in six policy agencies, including why strategies that worked well in one site were less effective in others. Findings indicate a complex interaction between participants' perception of the intervention, shifting contextual factors, and the form that the intervention took in each site. Our propositions provide transferable lessons about contextualised areas of strength and weakness that may be useful in the development and implementation of similar studies.

Building a bright, evidence-informed future: a conversation starter from the incoming editors

Health Research and Policy Systems (HARPS) has gone from strength to strength since it was established in 2003. As new Editors-in-Chief, we look forward to a bright future for HARPS, and we would like to start a conversation with you, HARPS readers, authors, editors and others, about how HARPS can best support ongoing progress and debate on evidence-informed health research policy and systems, particularly in developing countries. As a starting point for discussion, we would like to highlight three areas that we are passionate about, namely supporting an integrated community of researchers and policy-makers; building a focus on how health research and policy systems can support achievement of the Sustainable Development Goals; and strengthening our commitment to communicating and disseminating the work published in HARPS. We invite you to contribute your thoughts, ideas and suggestions on the future of HARPS, as we work together towards an evidence-informed future.

Sustainability in Health care by Allocating Resources Effectively (SHARE) 9: conceptualising disinvestment in the local healthcare setting

BACKGROUND

This is the ninth in a series of papers reporting a program of Sustainability in Health care by Allocating Resources Effectively (SHARE) in a local healthcare setting. The disinvestment literature has broadened considerably over the past decade; however there is a significant gap regarding systematic, integrated, organisation-wide approaches. This debate paper presents a discussion of the conceptual aspects of disinvestment from the local perspective.

DISCUSSION

Four themes are discussed: Terminology and concepts, Motivation and purpose, Relationships with other healthcare improvement paradigms, and Challenges to disinvestment. There are multiple definitions for disinvestment, multiple concepts underpin the definitions and multiple alternative terms convey these concepts; some definitions overlap and some are mutually exclusive; and there are systematic discrepancies in use between the research and practice settings. Many authors suggest that the term 'disinvestment' should be avoided due to perceived negative connotations and propose that the concept be considered alongside investment in the context of all resource allocation decisions and approached from the perspective of optimising health care. This may provide motivation for change, reduce disincentives and avoid some of the ethical dilemmas inherent in other disinvestment approaches. The impetus and rationale for disinvestment activities are likely to affect all aspects of the process from identification and prioritisation through to implementation and evaluation but have not been widely discussed. A need for mechanisms, frameworks, methods and tools for disinvestment is reported. However there are several health improvement paradigms with mature frameworks and validated methods and tools that are widely-used and well-accepted in local health services that already undertake disinvestment-type activities and could be expanded and built upon. The nature of disinvestment brings some particular challenges for policy-makers, managers, health professionals and researchers. There is little evidence of successful implementation of 'disinvestment' projects in the local setting, however initiatives to remove or replace technologies and practices have been successfully achieved through evidence-based practice, quality and safety activities, and health service improvement programs.

CONCLUSIONS

These findings suggest that the construct of 'disinvestment' may be problematic at the local level. A new definition and two potential approaches to disinvestment are proposed to stimulate further research and discussion.

Uses of Population Health Rankings in Local Policy Contexts: A Multisite Case Study

Population health rankings are a common strategy to spur evidence-informed health policy making, but little is known about their uses or impacts. The study aims were to (1) understand how and why the County Health Rankings (CH-Rankings) are used in local policy contexts, (2) identify factors that influence CH-Rankings utilization, and (3) explore potentially negative impacts of the CH-Rankings. Forty-four interviews were conducted with health organization officials and public policy makers in 15 purposively selected counties. The CH-Rankings were used instrumentally to inform internal planning decisions, conceptually to educate the public and policy makers about determinants of population health, and politically to advance organizational agendas. Factors related to organizational capacity, county political ideology, and county rank influenced if, how, and why the CH-Rankings were used. The CH-Rankings sometimes had the negative impacts of promoting potentially ineffective interventions in politically conservative counties and prompting negative media coverage in some counties with poor rank.

Validity and reliability of the Evidence Utilisation in Policymaking Measurement Tool (EUPMT)

Background

As a well-known theory in studying the effective factors on behaviour, the theory of planned behaviour (TPB) is frequently used in evaluating the health behaviour of people and healthcare providers, but rarely applied in studying the behaviour of health policymakers. The aim of the present study is to design and validate a TPB-based measurement tool for evidence utilisation in health policymaking (the EUPMT) through a mixed approach using confirmatory factor analysis.

Methods

The study population consisted of all the specialised units and their employees in the five deputies of Iran’s Ministry of Health and Medical Education in 2013. All those eligible were invited to participate in the study, which comprised 373 persons. The reliability of the EUPMT was determined through test-retest and internal consistency. Additionally, its validity was determined by face, content, convergent, discriminant and construct validities. SPSS-20 and LISREL-8.8 were employed to analyse the data. To assess the fitness of the measurement models, three groups of indices were used, i.e. absolute, relative and parsimonious.

Results

The content and face validities of the tool were 83% and 67%, respectively. Cronbach’s alpha of different constructs ranged from 0.7 to 0.9. In the test-retest method, the intra-class correlations were between 0.75 and 0.87. Confirmatory factor analysis showed that the penta-factorial structure of the experimental data had acceptable fitness with the TPB (GFI = 0.86, NFI = 0.94, RSMEA = 0.075).

Conclusion

TPB is able to explain the behaviour of evidence utilisation in health policymaking. The finalised TPB-based tool has relatively good reliability and validity to assess evidence utilisation in health policymaking. The EUPMT can be applied to determine the status quo of evidence utilisation in health policymaking, whilst designing interventions for its improvement and assessing their outcomes.