Indicators to evaluate organisational knowledge brokers: a scoping review

From bench to policy: a critical analysis of models for evidence-informed policymaking in healthcare

Background

The use of research evidence in policy making is a complex and challenging process that has a long history in various fields, especially in healthcare. Different terms and concepts have been used to describe the relationship between research and policy, but they often lack clarity and consensus. To address this gap, several strategies and models have been proposed to facilitate evidence informed policy making and to identify the key factors and mechanisms involved. This study aims to critically review the existing models of evidence informed policy making (EIPM) in healthcare and to assess their strengths and limitations.

Method

A systematic search and review conducted to identify and critically assess EIPM models in healthcare. We searched PubMed, Web of Science and Scopus databases as major electronic databases and applied predefined inclusion criteria to select the models. We also checked the citations of the included models to find other scholars' perspectives. Each model was described and critiqued each model in detail and discussed their features and limitations.

Result

Nine models of EIPM in healthcare were identified. While models had some strengths in comprehension, flexibility and theoretical foundations, analysis also identified limitations including: presupposing rational policymaking; lacking alternatives for time-sensitive situations; not capturing policy complexity; neglecting unintended effects; limited context considerations; inadequate complexity concepts; limited collaboration guidance; and unspecified evidence adaptations.

Conclusion

The reviewed models provide useful frameworks for EIPM but need further improvement to address their limitations. Concepts from sociology of knowledge, change theory and complexity science can enrich the models. Future EIPM models should better account for the complexity of research-policy relationships and provide tailored strategies based on the policy context.

A scoping review of theories, models and frameworks used or proposed to evaluate knowledge mobilization strategies

BACKGROUND

Evaluating knowledge mobilization strategies (KMb) presents challenges for organizations seeking to understand their impact to improve KMb effectiveness. Moreover, the large number of theories, models, and frameworks (TMFs) available can be confusing for users. Therefore, the purpose of this scoping review was to identify and describe the characteristics of TMFs that have been used or proposed in the literature to evaluate KMb strategies.

METHODS

A scoping review methodology was used. Articles were identified through searches in electronic databases, previous reviews and reference lists of included articles. Titles, abstracts and full texts were screened in duplicate. Data were charted using a piloted data charting form. Data extracted included study characteristics, KMb characteristics, and TMFs used or proposed for KMb evaluation. An adapted version of Nilsen (Implement Sci 10:53, 2015) taxonomy and the Expert Recommendations for Implementing Change (ERIC) taxonomy (Powell et al. in Implement Sci 10:21, 2015) guided data synthesis.

RESULTS

Of the 4763 search results, 505 were retrieved, and 88 articles were eligible for review. These consisted of 40 theoretical articles (45.5%), 44 empirical studies (50.0%) and four protocols (4.5%). The majority were published after 2010 (n = 70, 79.5%) and were health related (n = 71, 80.7%). Half of the studied KMb strategies were implemented in only four countries: Canada, Australia, the United States and the United Kingdom (n = 42, 47.7%). One-third used existing TMFs (n = 28, 31.8%). According to the adapted Nilsen taxonomy, process models (n = 34, 38.6%) and evaluation frameworks (n = 28, 31.8%) were the two most frequent types of TMFs used or proposed to evaluate KMb. According to the ERIC taxonomy, activities to "train and educate stakeholders" (n = 46, 52.3%) were the most common, followed by activities to "develop stakeholder interrelationships" (n = 23, 26.1%). Analysis of the TMFs identified revealed relevant factors of interest for the evaluation of KMb strategies, classified into four dimensions: context, process, effects and impacts.

CONCLUSIONS

This scoping review provides an overview of the many KMb TMFs used or proposed. The results provide insight into potential dimensions and components to be considered when assessing KMb strategies.

Can co-created knowledge mobilisation interventions alter and enhance mindlines to improve childhood eczema care? A UK-based Social Impact Framework evaluation

OBJECTIVE

To evaluate the impact of using knowledge mobilisation interventions to alter and enhance mindlines and improve childhood eczema care.

DESIGN

The eczema mindlines study involved three stages: (1) mapping and confirming eczema mindlines, (2) intervention development and delivery and (3) analysis of intervention impact. The focus of this paper is on stage 3. Data analysis was guided by the Social Impact Framework to address the questions: (1) what is the impact of this study on individuals and groups? (2) what changes in behaviour and practice have occurred due to their involvement? (3) what mechanisms have enabled these impacts or changes to occur? and (4) what are the recommendations and questions arising from this research?

SETTINGS

A deprived inner-city neighbourhood in central England and national/international settings.

PARTICIPANTS

Patients, practitioners and wider community members exposed to the interventions locally, nationally and internationally.

RESULTS

Data revealed tangible multi-level, relational and intellectual impacts. Mechanisms supporting impact included: simplicity and consistency of messages adapted to audience, flexibility, opportunism and perseverance, personal interconnectivity and acknowledgement of emotion. Co-created knowledge mobilisation strategies to alter and enhance mindlines mediated through knowledge brokering were effective in producing tangible changes in eczema care practice and self-management and in 'mainstreaming' childhood eczema in positive way across communities. These changes cannot be directly attributed to the knowledge mobilisation interventions, however, the evidence points to the significant contribution made.

CONCLUSION

Co-created knowledge mobilisation interventions offer a valuable method of altering and enhancing eczema mindlines across lay-practitioner-wider society boundaries. The Social Impact Framework provides comprehensive method of understanding and documenting the complex web of impact occurring as a result of knowledge mobilisation. This approach is transferable to managing other long-term conditions.

Barriers and Facilitators to Knowledge Brokering Activities: Perspectives from Knowledge Brokers Working in Canadian Rehabilitation Settings

INTRODUCTION

Knowledge translation experts advocate for employing knowledge brokers (KBs) to promote the uptake of research evidence in health care settings. Yet, no previous research has identified potential barriers for KBs to promote the uptake of research evidence in rehabilitation settings. This study aimed to identify the barriers and facilitators for KBs in Canadian rehabilitation settings as perceived by individuals serving as KBs.

METHODS

Qualitative study using semistructured telephone interviews with individuals performing KB activities in rehabilitation settings across Canada. The interview topic guide was informed by the Consolidated Framework for Implementation Research (CFIR) and consisted of 20 questions covering three domains (characteristics of individuals, inner setting, and outer settings). We conducted qualitative descriptive analysis combining deductive coding guided by the CFIR.

RESULTS

Characteristics of individuals included having communication skills, clinical experience, research skills, and interpersonal features, in addition to being confident and motivated and receiving training. The inner setting domain included having constant networking with stakeholders and being aware of stakeholders' needs, in addition to resources availability, leadership engagement, knowledge accessibility, prioritizing brokering activities, and monitoring KBs' performance. Finally, the outer setting domain showed that KBs need to be connected to a community of practice to promote information exchange and avoid work duplications.

DISCUSSION

Factors likely to hinder or promote the optimal use of KBs within Canadian rehabilitation settings include skill sets and networking abilities; organizational culture, resources, and leadership engagement; and the need for specific training for KBs and for evaluation tools to monitor their performance.

A comprehensive monitoring and evaluation framework for evidence to policy networks

OBJECTIVE

To describe the development of a framework for monitoring and evaluating knowledge translation (KT) networks.

METHOD

The framework was developed using mixed methods over four phases, including i) a targeted literature review of KT networks, activities and indicators, ii) two scoping reviews to further enhance the set of indicators, iii) peer-reviews by international KT experts and an online expert consultation, and iv) piloting.

RESULTS

A comprehensive theory of change (ToC) and indicators, both for the Network Secretariat and its participating member countries, were identified to develop the monitoring and evaluation framework. The framework includes (i) a ToC, including three key indicator domains across the results chain (outputs, short term outcomes, intermediate outcomes), and (ii) indicators for the three key domains, that can be selected depending on the stage of network maturity, along with suggested data collection methods. The three key indicator domains are 1) KT capacity and skill building; 2) network (structure, governance and leadership); and 3) KT/evidence-informed policy value and culture.

CONCLUSION

The monitoring and evaluation framework that links KT activities with policy and health outcomes fills an important gap in optimizing KT procedures, generating lessons learned and increasing accountability of major multipartner KT networks.

Policy options for strengthening evidence-informed health policy-making in Iran: overall SASHA project findings

Background

The institutionalization of evidence-informed health policy-making (EIHP) is complex and complicated. It is complex because it has many players and is complicated because its institutionalization will require many changes that will be challenging to make. Like many other issues, strengthening EIHP needs a road map, which should consider challenges and address them through effective, harmonized and contextualized strategies. This study aims to develop a road map for enhancing EIHP in Iran based on steps of planning.

Methods

This study consisted of three phases: (1) identifying barriers to EIHP, (2) recognizing interventions and (3) measuring the use of evidence in Iran's health policy-making. A set of activities was established for conducting these, including foresight, systematic review and policy dialogue, to identify the current and potential barriers for the first phase. For the second phase, an evidence synthesis was performed through a scoping review, by searching the websites of benchmark institutions which had good examples of EIHP practices in order to extract and identify interventions, and through eight policy dialogues and two broad opinion polls to contextualize the list of interventions. Simultaneously, two qualitative-quantitative studies were conducted to design and use a tool for assessing EIHP in the third phase.

Results

We identified 97 barriers to EIHP and categorized them into three groups, including 35 barriers on the “generation of evidence” (push side), 41 on the “use of evidence” (pull side) and 21 on the “interaction between these two” (exchange side). The list of 41 interventions identified through evidence synthesis and eight policy dialogues was reduced to 32 interventions after two expert opinion polling rounds. These interventions were classified into four main strategies for strengthening (1) the education and training system (6 interventions), (2) the incentives programmes (7 interventions), (3) the structure of policy support organizations (4 interventions) and (4) the enabling processes to support EIHP (15 interventions).

Conclusion

The policy options developed in the study provide a comprehensive framework to chart a path for strengthening the country’s EIHP considering both global practices and the context of Iran. It is recommended that operational plans be prepared for road map interventions, and the necessary resources provided for their implementation. The implementation of the road map will require attention to the principles of good governance, with a focus on transparency and accountability.

The Role of Law Enforcement Officers/Police in Drug Prevention within Educational Settings-Study Protocol for the Development of a Guiding Document Based on Experts' Opinions

The United Nations Office on Drugs and Crime—World Health Organization International Standards on Drug Use Prevention—reflects the value of safe, nurturing and supportive social institutions around the lives of youths to benefit from the prevention of risky behavior extending beyond individually-developed resilience for healthy adolescent development. Schools are valuable social institutions to this effect and school safety and adolescent health outcomes can be threatened by drug use and violence. As such, collaborative, multi-level, evidence-based, developmentally sensitive, substance use prevention programs are imperative. The International Standards, in their latest version, did not reflect specific evidence of law enforcement officer-based programs with effect on drug use prevention, including in school settings. Nevertheless, the collaboration between law enforcement agencies and school-based substance use prevention programs continue to be the focus of research and policy. In this project, we aim to explore in more detail the role of law enforcement in preventing substance use in schools. We use mixed methods, including three phases: (i) scoping review on the best practices for effective law enforcement in school-based drug and crime prevention; (ii) interviews with experts, using the Delphi method, in substance use prevention and training law enforcement in school-based drug prevention; and iii) developing guidelines for law enforcement based on the findings. Initially, we identified a total of 17 papers that were categorized in four categories based on their results (negative or null effect n = 11 studies, positive effect n = 1 study, mixed effects n = 4 studies and indefinite conclusion n = 1 study). However, the authors of the studies with negative or null effect did recommend being cautious about these results due to the respective studies’ methodological limitations. The actual and perceived roles of police are largely unclear and/or variable. Therefore, clear outlines regarding law enforcement’s role within schools are crucial as one study showed that an officer’s role influences how they respond to student conduct. A secondary emergent theme from this review indicates that there is potential for positively impacting a youth’s perceptions of police through collaborative and engaging school-based programs. Currently the project is gradually moving to Phase II, where we are identifying the key experts based on scientifically published peer reviewed and grey literature/guidelines to investigate elements that make the role of law enforcement officers in school-based prevention more effective. Given the frequency with which policy makers around the world request information about the role of law enforcement in effective prevention efforts, guidelines on their roles within schools is a gap that needs to be filled. Such efforts would improve drug prevention in schools and better orient law enforcement’s role in drug prevention within educational settings.