A systematic review of barriers to and facilitators of the use of evidence by policymakers

Research and the real world: Analyzing the short- and long-term impact of knowledge transfer

Knowledge exchange interventions are an important driver of research impact. This study examines 5 years of intensive knowledge-exchange ‘reform labs’ in the field of education policy. Using qualitative analysis and a regression analysis approach applied to rich case data and quantitative results, we find that the reform labs have a significant impact on both participants and their reform projects in the short-term immediately after the event and in the medium term up to 5 years later. We also find evidence that the impact on individuals and projects drives broader social impact. We take a holistic approach to the knowledge brokerage of designing the reform labs and find that across contexts and project phases, but highly costly. In comparing the reform labs to best practices identified in the literature, we find evidence that knowledge exchange interventions of longer duration, with case-focused teamwork, and involving intensive researcher–participant interaction are more impactful. Additionally, we observe that diverse participants can drive impact and their needs must be considered.

Policy options to increase motivation for improving evidence-informed health policy-making in Iran

BACKGROUND

Current incentive programmes are not sufficient to motivate researchers and policy-makers to use research evidence in policy-making. We conducted a mixed-methods design to identify context-based policy options for strengthening motivations among health researchers and policy-makers to support evidence-informed health policy-making (EIHP) in Iran.

METHODS

This study was conducted in 2019 in two phases. In the first phase, we conducted a scoping review to extract interventions implemented or proposed to strengthen motivations to support EIHP. Additionally, we employed a comparative case study design for reviewing the performance evaluation (PE) processes in Iran and other selected countries to determine the current individual and organizational incentives to encourage EIHP. In the second phase, we developed two policy briefs and then convened two policy dialogues, with 12 and 8 key informants, respectively, where the briefs were discussed. Data were analysed using manifest content analysis in order to propose contextualized policy options.

RESULTS

The policy options identified to motivate health researchers and policy-makers to support EIHP in Iran were: revising the criteria of academic PE; designing appropriate incentive programmes for nonacademic researchers; developing an indicator for the evaluation of research impact on policy-making or health outcomes; revising the current policies of scientific journals; revising existing funding mechanisms; presenting the knowledge translation plan when submitting a research proposal, as a mandatory condition; encouraging and supporting mechanisms for increasing interactions between policy-makers and researchers; and revising some administrative processes (e.g. managers and staff PEs; selection, appointment, and changing managers and reward mechanisms).

CONCLUSIONS

The current individual or organizational incentives are mainly focused on publications, rather than encouraging researchers and policy-makers to support EIHP. Relying more on incentives that consider the other impacts of research (e.g. impacts on health system and policy, or health outcomes) is recommended. These incentives may encourage individuals and organizations to be more involved in conducting research evidence, resulting in promoting EIHP.

TRIAL REGISTRATION

NA.

Healthcare planning across healthcare sectors in Baden-Wuerttemberg, Germany: a stakeholder online survey to identify indicators

Background

Stakeholders in the German state of Baden-Wuerttemberg agreed upon the central aims for healthcare planning. These include a focus on geographical districts; a comprehensive, cross-sectoral perspective on healthcare needs and services; and use of regional data for healthcare planning. Therefore, healthcare data at district level is needed. Nevertheless, decision makers face the challenge to make a selection from numerous indicators and frameworks, which all have limitations or do not well apply to the targeted setting. The aim of this study was to identify district level indicators to be used in Baden-Wuerttemberg for the purpose of cross-sectoral and needs-based healthcare planning involving stakeholders of the health system.

Methods

A conceptual framework for indicators was developed. A structured search for indicators identified 374 potential indicators in indicator sets of German and international institutions and agencies (n = 211), clinical practice guidelines (n = 50), data bases (n = 35), indicator databases (n = 25), published literature (n = 35), and other sources (n = 18). These indicators were categorised according to the developed framework dimensions. In an online survey, institutions of various stakeholders were invited to assess the relevance of these indicators from December 2016 until January 2017. Indicators were selected in terms of a median value of the assessed relevance.

Results

22 institutions selected 212 indicators for the five dimensions non-medical determinants of health (20 indicators), health status (25), utilisation of the health system (34), health system performance (87), and healthcare provision (46).

Conclusions

Stakeholders assessed a large number of indicators as relevant for use in healthcare planning on district level.

Trial registration

Not applicable.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-021-06514-0.

Strategies for engaging policy stakeholders to translate research knowledge into practice more effectively: Lessons learned from the Canadian Alcohol Policy Evaluation project

INTRODUCTION

Evidence-based alcohol policies have the potential to reduce a wide range of related harms. Yet, barriers to adoption and implementation within governments often exist. Engaging relevant stakeholders may be an effective way to identify and address potential challenges thereby increasing reach and uptake of policy evaluation research and strengthening jurisdictional responses to alcohol harms.

METHODS

As part of the 2019 Canadian Alcohol Policy Evaluation project, we conducted interviews with government stakeholders across alcohol-related sectors prior to a second round of researcher-led policy assessments in Canada's 13 provinces and territories. Stakeholders were asked for feedback on the design and impact of an earlier policy assessment in 2013 and for recommendations to improve the design and dissemination strategy for the next iteration. Content analysis was used to identify ways of improving stakeholder engagement.

RESULTS

We interviewed 25 stakeholders across 12 of Canada's 13 jurisdictions, including representatives from government health ministries and from alcohol regulation, distribution and finance departments. In providing feedback on our stakeholder engagement strategy, participants highlighted the importance of maintaining ongoing contact; presenting results in accessible online formats; providing advance notice of results; and offering jurisdiction-specific webinars.

DISCUSSION AND CONCLUSIONS

This study offers important insight into the engagement preferences of government stakeholders involved in the health, regulation, distribution and financial aspects of alcohol control policy. Findings suggest that seeking input from stakeholders as part of conducting evaluation research is warranted; increasing the relevance, reach and uptake of results. Specific stakeholder engagement strategies are outlined.

Work empowerment among cancer care professionals: a cross-sectional study

BACKGROUND

There is a growing understanding that empowerment of interprofessional personnel is linked to job satisfaction levels and quality of care, but little is known about empowerment in the context of cancer care. This study describes how interprofessional cancer care personnel perceive their performance and factors that promote work empowerment.

METHODS

This cross-sectional study enrolled 475 (45.2%) of the 1050 employees who work at a regional cancer centre. The participants used two self-administered questionnaires - the Performance of an Empowered Personnel (PEN) questionnaire and Work Empowerment Promoting Factors (WEP) questionnaire - to report perceptions of work empowerment. Both questionnaires' categories comprise moral principles, personal integrity, expertise, future orientation, and sociality. The data were analyzed using IBM SPSS Statistics, Versions 24 and 25.

RESULTS

Overall, the performance of work empowerment was evaluated as being rather high (overall sum score mean: 4.05; range: 3.51-4.41; scale: 1-5). The category that rated highest was moral principles (4.41), and the one rated lowest was the social category (3.51). The factors that promoted work empowerment also ranked high (3.93; range: 3.55-4.08; scale: 1-5), with personal integrity (4.08) the highest and future orientation (3.55) the lowest. Performance and factors that promoted work empowerment correlated positively, moderately, and highly statistically significantly (r = 0.531; p < 0.001). Statistically significant associations also were found between empowered performance of personnel and empowerment promoting factors (sex, education, leadership position, belonging to an interprofessional team, and time elapsed since training in interprofessional cooperation).

CONCLUSION

The personnel rated their performance and the factors perceived to promote work empowerment rather highly. Personal empowerment can be promoted through teamwork training and supportive management in interprofessional cancer care.

Communication vs evidence: What hinders the outreach of science during an infodemic? A narrative review

Background

The COVID-19 pandemic is placing significant pressure on national and international health organizations and the measures taken to combat it are having many impacts beyond health. At the same time, misleading communication practices and what has been called an “infodemic” by the World Health Organization have been hampering the uptake of coronavirus-related scientific information. Moreover, public awareness about the dangers of the infodemic remains poor, and misinformation may lead to hazardous behaviours. We therefore analysed factors potentially undermining communication of scientific evidence and proposed strategies to counteract this phenomenon.

Methods

We sought official academic and institutional publications of any type, published in English and analyzed their approaches to communication used during the pandemic.

Results

The factors that might undermine appropriate communication include but are not limited to (a) the exponential increase of COVID-19-related publications, often including biases in the peer-review and editorial process; (b) the role of traditional media; (c) politicization of the virus; and (d) the impact of social media. We argue that evidence synthesis and knowledge translation are useful tools to communicate accurate scientific evidence to decision-makers.

Conclusion

Clear and concise messages in this form can help decision-makers to interpret data correctly, take consequent actions, and avoid being compromised by low-quality or even misleading information.

Integrated knowledge translation to advance noncommunicable disease policy and practice in South Africa: application of the Exploration, Preparation, Implementation, and Sustainment (EPIS) framework

Background

In response to the “know–do” gap, several initiatives have been implemented to enhance evidence-informed decision-making (EIDM). These include individual training, organizational culture change management, and legislative changes. The importance of relationships and stakeholder engagement in EIDM has led to an evolution of models and approaches including integrated knowledge translation (IKT). IKT has emerged as a key strategy for ensuring that engagement is equitable, demand-driven, and responsive. As a result, the African-German Collaboration for Evidence-Based Healthcare and Public Health in Africa (CEBHA+) incorporated an IKT approach to influence noncommunicable diseases (NCD) policy and practice. We documented the phased process of developing, implementing, and monitoring the IKT approach in South Africa; and explored the appropriateness of using the exploration, preparation, implementation, and sustainment (EPIS) framework for this purpose.

Methods

We mapped the South Africa IKT approach onto the EPIS framework using a framework analysis approach. Notes of team meetings, stakeholder matrices, and engagement strategies were analysed and purposefully plotted against the four phases of the framework in order to populate the different constructs. We discussed and finalized the analysis in a series of online iterations until consensus was reached.

Results

The mapping exercise revealed an IKT approach that was much more iterative, dynamic, and engaging than initially thought. Several constructs (phase-agnostic) remained important and stable across EPIS phases: stable and supportive funding; committed and competent leadership; skilled and dedicated IKT champions; diverse and established personal networks; a conducive and enabling policy environment; and boundary-spanning intermediaries. Constructs such as “innovations” constantly evolved and adapted to the changing inner and outer contexts (phase-specific).

Conclusions

Using the EPIS framework to interrogate, reflect on, and document our IKT experiences proved extremely relevant and useful. Phase-agnostic constructs proved critical to ensure resilience and agility of NCD deliberations and policies in the face of highly dynamic and changing local contexts, particularly in view of the current coronavirus disease 2019 (COVID-19) pandemic. Bridging IKT with a framework from implementation science helps to reflect on this process and can guide the development and planning of similar interventions and strategies.

Systematic review and policy dialogue to determine challenges in evidence-informed health policy-making: findings of the SASHA study

Background

Various interventions have been undertaken in Iran to promote evidence-informed health policy-making (EIHP). Identifying the challenges in EIHP is the first step toward strengthening EIHP in each country through the design of tailored interventions. Therefore, the current study was conducted to synthesize the results of earlier studies and to finalize the list of barriers to EIHP in Iran.

Methods

To identify the barriers to EIHP in Iran, two steps were taken: a systematic review and policy dialogue. To conduct the systematic review, three Iranian databases and PubMed, Health Systems Evidence (HSE), Embase, and Scopus were searched. The reference lists of included papers and documentation from some local organizations were hand-searched. Upon conducting the systematic review, given the significance of stakeholders in clarifying the problem of EIHP, policy dialogue was used to complete the list previously extracted and to do advocacy. Selection criteria for the stakeholders included influential and informed individuals from knowledge-producing, knowledge-utilizing, and knowledge-brokering organizations. Semi-structured interviews were held with three important absent stakeholders.

Results

Challenges specific to Iran that were identified included the lack of integration of the health ministry and the medical universities, lack of ties between health knowledge utilization organizations, failure to establish long-term research plans, neglect of national research needs at the time of recruiting human resources in knowledge-producing organizations, and duplication and lack of coordination in routine data obtained from surveillance systems, disease registration systems, and censuses. It seems that some challenges are common across countries, including neglecting the importance of inter- and intra-disciplinary studies, the capacity of policy-makers and managers to utilize evidence, the criteria for evaluating the performance of policy-makers, managers, and academic members, the absence of long-term programmes in knowledge-utilizing organizations, the rapid replacement of policy-makers and managers, and lack of use of evaluation studies.

Conclusions

In this study, we tried to identify the challenges regarding EIHP in Iran using a systematic review and policy dialogue approach. This is the first step toward determining the best interventions to improve evidence-informed policy-making in each country, because these challenges are contextual and need to be investigated contextually.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12961-021-00717-x.

Implementation research approaches to promoting universal health coverage in Africa: a scoping review

BACKGROUND

Implementation research has emerged as part of evidence-based decision-making efforts to plug current gaps in the translation of research evidence into health policy and practice. While there has been a growing number of initiatives promoting the uptake of implementation research in Africa, its role and effectiveness remain unclear, particularly in the context of universal health coverage (UHC). Hence, this scoping review aimed to identify and characterise the use of implementation research initiatives for assessing UHC-related interventions or programmes in Africa.

METHODS

The review protocol was developed based on the methodological framework proposed by Arksey and O'Malley, as enhanced by the Joanna Briggs Institute. The review is reported in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews (PRISMA-ScR). MEDLINE, Scopus and the Cochrane Library were searched. The search also included a hand search of relevant grey literature and reference lists. Literature sources involving the application of implementation research in the context of UHC in Africa were eligible for inclusion.

RESULTS

The database search yielded 2153 records. We identified 12 additional records from hand search of reference lists. After the removal of duplicates, we had 2051 unique records, of which 26 studies were included in the review. Implementation research was used within ten distinct UHC-related contexts, including HIV; maternal and child health; voluntary male medical circumcision; healthcare financing; immunisation; healthcare data quality; malaria diagnosis; primary healthcare quality improvement; surgery and typhoid fever control. The consolidated framework for implementation research (CFIR) was the most frequently used framework. Qualitative and mixed-methods study designs were the commonest methods used. Implementation research was mostly used to guide post-implementation evaluation of health programmes and the contextualisation of findings to improve future implementation outcomes. The most commonly reported contextual facilitators were political support, funding, sustained collaboration and effective programme leadership. Reported barriers included inadequate human and other resources; lack of incentives; perception of implementation as additional work burden; and socio-cultural barriers.

CONCLUSIONS

This review demonstrates that implementation research can be used to achieve UHC-related outcomes in Africa. It has identified important facilitators and barriers to the use of implementation research for promoting UHC in the region.

[Promotion of physical activity among the elderly as an example of knowledge translation: How do scientific findings enter community practice?]

BACKGROUND

Scientific findings can be an important source of knowledge for public health stakeholders involved in promoting physical activity, but several barriers hinder their use. Knowledge translation can simplify this process, but it requires the understanding of the stakeholder's needs.

OBJECTIVES

This qualitative study aims to describe how public health stakeholders access information and scientific findings, identify possible barriers, and highlight the needs of stakeholders in terms of presentation and processing.

MATERIALS AND METHODS

Semi-structured interviews were conducted with twelve local- and state-level stakeholders from North Rhine-Westphalia, Saxony-Anhalt, and Thuringia working in the area of physical activity promotion. The interviewees were selected through purposive sampling. The interviews were evaluated using qualitative content analysis.

RESULTS

The benefits of scientific findings are emphasized by the interviewees, but a lack of resources in combination with a flood of information, high complexity, and technical jargon complicate their application. There is a need for tailored preparation in the form of summaries, filter functions, elaboration of practice-relevant elements, and ways of provision.

CONCLUSIONS

To achieve successful knowledge translation, collaboration and interactive exchange between researchers, policymakers, and practice as well as a demand-oriented processing of scientific findings are central. Networking and bundling of knowledge on a platform are important tasks for the future.

Examining knowledge translation in blood donor research: A review of vasovagal reaction literature

BACKGROUND

Knowledge translation focuses on the transfer of research findings into policy and practice. To provide insight into the state of knowledge translation in blood donor research, we undertook a rapid review of a key research area in the field with high potential for translation, vasovagal reactions (VVRs). We examined the number and nature of VVR-related studies to determine the availability of research evidence, and mapped the included articles along the research-to-practice trajectory using the Knowledge to Action framework.

STUDY DESIGN AND METHODS

PubMed, PsycINFO, CINAHL, and EMBASE were searched for peer-reviewed journal articles from inception to October 2019 using the terms blood don* AND vasovagal OR faint* OR syncope.

RESULTS

A total of 176 articles met our inclusion criteria. Studies relating to VVRs increased substantially from 1942 to 2019, with 84% published in the last 20 years. Articles were predominately observation (non-intervention) studies (117; 66%), followed by intervention (knowledge inquiry) studies (31; 18%) and review (knowledge synthesis) studies (20; 11%). The evidence from intervention research was limited, with 14 strategies tested in 31 studies and often by the same research groups. Only 5 (3%) implementation and evaluation studies were found; all focused on evaluating the effects of a newly introduced intervention on VVR rates through uncontrolled or cross-sectional study designs.

DISCUSSION

VVR research is in the early stages of knowledge translation. More intervention research is needed to provide a robust evidence base as well as more published implementation research to share knowledge of translating research into policy and practice.

Health research knowledge translation into policy in Zambia: policy-maker and researcher perspectives

Background

The translation of public health research evidence into policy is critical to strengthening the capacity of local health systems to respond to major health challenges. However, a limited amount of public health research evidence generated in developing countries is actually translated into policy because of various factors. This study sought to explore the process of health research knowledge translation into policy and to identify factors that facilitate or hinder the process in Zambia.

Methods

This work was an exploratory qualitative study comprising two phases. Firstly, a document review of health policies and strategic frameworks governing research was undertaken to understand the macro-environment for knowledge translation in Zambia. Secondly, key informant interviews were conducted with those responsible for health research and policy formulation. The study interviewed 15 key informants and a thematic analysis approach was used.

Results

The document review showed that there are policy efforts to promote knowledge translation through improvement of the research macro-environment. However, the interviews showed that coordination and linkage of the knowledge creation, translation and policy-making processes remains a challenge owing to lack of research knowledge translation capacity, limited resources and lack of knowledge hubs. Emerging local research leadership and the availability of existing stock of underutilized local health research data provide an opportunity to enhance knowledge translation to feed into policy processes in Zambia.

Conclusions

Public health research knowledge translation into policy remains a challenge in Zambia. To enhance the uptake of research evidence in policy-making, this study suggests the need for improved coordination, financing and capacity-building in knowledge translation processes for both health researchers and policy-makers.

The decision sampling framework: a methodological approach to investigate evidence use in policy and programmatic innovation

Background

Calls have been made for greater application of the decision sciences to investigate and improve use of research evidence in mental health policy and practice. This article proposes a novel method, “decision sampling,” to improve the study of decision-making and research evidence use in policy and programmatic innovation. An illustrative case study applies the decision sampling framework to investigate the decisions made by mid-level administrators when developing system-wide interventions to identify and treat the trauma of children entering foster care.

Methods

Decision sampling grounds qualitative inquiry in decision analysis to elicit information about the decision-making process. Our case study engaged mid-level managers in public sector agencies (n = 32) from 12 states, anchoring responses on a recent index decision regarding universal trauma screening for children entering foster care. Qualitative semi-structured interviews inquired on questions aligned with key components of decision analysis, systematically collecting information on the index decisions, choices considered, information synthesized, expertise accessed, and ultimately the values expressed when selecting among available alternatives.

Results

Findings resulted in identification of a case-specific decision set, gaps in available evidence across the decision set, and an understanding of the values that guided decision-making. Specifically, respondents described 14 inter-related decision points summarized in five domains for adoption of universal trauma screening protocols, including (1) reach of the screening protocol, (2) content of the screening tool, (3) threshold for referral, (4) resources for screening startup and sustainment, and (5) system capacity to respond to identified needs. Respondents engaged a continuum of information that ranged from anecdote to research evidence, synthesizing multiple types of knowledge with their expertise. Policy, clinical, and delivery system experts were consulted to help address gaps in available information, prioritize specific information, and assess “fit to context.” The role of values was revealed as participants evaluated potential trade-offs and selected among policy alternatives.

Conclusions

The decision sampling framework is a novel methodological approach to investigate the decision-making process and ultimately aims to inform the development of future dissemination and implementation strategies by identifying the evidence gaps and values expressed by the decision-makers, themselves.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13012-021-01084-5.

The how and why of producing policy relevant research: perspectives of Australian childhood obesity prevention researchers and policy makers

BACKGROUND

Understanding why research is conducted may help address the under-utilisation of research. This study examined the reasons for childhood obesity prevention knowledge production in New South Wales (NSW), Australia, and the factors influencing research choices from the perspective of the researchers and health policy agencies contributing to the research.

METHODS

A literature search of SCOPUS and ISI Web of Knowledge (affiliation and key word searches) was conducted to compile a database of NSW childhood obesity research outputs, published between 2000 and 2015 (n = 543). Descriptive statistics were used to quantify outputs by research type, differentiating measurement, descriptive, and intervention research, systematic reviews and other publications. Interviews were conducted with a sample of researchers drawn from the database (n = 13) and decision makers from health policy agencies who funded and contributed to childhood obesity research in NSW (n = 15). Researcher interviews examined views about societal impacts, why and under what circumstances the research was conducted. Decision-maker interviews examined policy agency research investment and how research was used in decision making. Content analysis and a thematic approach was used to analyse the interview transcripts.

RESULTS

The research in this case was conducted for mix of reasons including those traditionally associated with academic inquiry, as well as intentions to influence policy and practice. Differences in funding mechanisms, administrative and employment arrangements, and 'who' initiated the research, created differing incentives and perspectives for knowledge production. Factors associated with the characteristics and experience of the individuals involved also influenced goals, as did the type of research conducted. Policy agencies played a role in directing research to address policy needs.

CONCLUSIONS

The findings of this study confirm that researchers are strongly influenced by their working environment. Funding schemes and other incentives to support policy relevant knowledge production are important. Contextual factors such as policy priorities, policy-driven research funding and the embedded nature or strong connections between some researchers and the policy agencies involved, are likely to have influenced the extent to which policy goals were reported in this study.

EVITA 2.0, an updated framework for understanding evidence-based mental health policy agenda-setting: tested and informed by key informant interviews in a multilevel comparative case study

Background

Mental health remains a neglected issue on the global health policy agenda, particularly in low- and middle-income countries (LMIC), and the translation of research evidence into policy and practice is slow. The new EVITA framework was developed to improve mental health evidence uptake and policy agenda-setting in LMICs. In addition, behavioural science methods may be able to support knowledge translation to policy.

Methods

Using a mixed-methods study design, we applied and tested the newly developed EVITA 1.1 framework against three case studies related to South Africa at the district, national and international levels. In-depth interviews with 26 experts were conducted between August and November 2019, transcribed, coded and analysed in NVivo, using iterative categorization. The data were analysed against both the EVITA framework and the MINDSPACE framework for behavioural insights.

Results

In our case study comparison, we found that (1) research translation to the policy agenda occurs in a complex, fluid system which includes multiple “research clouds”, “policy spheres” and other networks; (2) mental health research policy agenda-setting is based on key individuals and intermediaries and their interrelationships; and (3) key challenges and strategies for successful research to policy agenda impact are known, but are frequently not strategically implemented, such as including all stakeholders to overcome the policy implementation gap. Our data also suggest that behavioural science methods can be strategically applied to support knowledge translation to policy agenda-setting.

Conclusion

We found that the EVITA framework is useful for understanding and improving mental health research policy interrelationships to support evidence uptake to the policy agenda, and that behavioural science methods are effective support mechanisms. The revised EVITA 2.0 framework therefore includes behavioural insights, for improved mental health policy agenda-setting in LMICs. More research is needed to understand whether EVITA can be applied to other LMICs and to high-income contexts.

Assessing organisational capacity for evidence-informed health policy and planning: an adaptation of the ORACLe tool for Australian primary health care organizations

Background

Many nations have established primary health care (PHC) organizations that conduct PHC planning for defined geographical areas. The Australian Government established Primary Health Networks (PHNs) in 2015 to develop and commission PHC strategies to address local needs. There has been little written about the capacity of such organizations for evidence-informed planning, and no tools have been developed to assess this capacity, despite their potential to contribute to a comprehensive effective and efficient PHC sector.

Methods

We adapted the ORACLe tool, originally designed to examine evidence-informed policy-making capacity, to examine organizational capacity for evidence-informed planning in meso-level PHC organizations, using PHNs as an example. Semi-structured interviews were conducted with 14 participants from five PHNs, using the ORACLe tool, and scores assigned to responses, in seven domains of capacity.

Results

There was considerable variation between PHNs and capacity domains. Generally, higher capacity was demonstrated in regard to mechanisms which could inform planning through research, and support relationships with researchers. PHNs showed lower capacity for evaluating initiatives, tools and support for staff, and staff training.

Discussion and conclusions

We critique the importance of weightings and scope of some capacity domains in the ORACLe tool. Despite this, with some minor modifications, we conclude the ORACLe tool can identify capacity strengths and limitations in meso-level PHC organizations. Well-targeted capacity development enables PHC organizations’ strategies to be better informed by evidence, for optimal impact on PHC and population health outcomes.

Research use at the Ministry of Health in Burkina Faso: the decision-makers' perspective

Background

Despite the increased emphasis placed on the use of evidence for policy development, relatively few initiatives have been developed to support evidence-informed decision-making, especially in West Africa. Moreover, studies examining the conditions under which policy-makers use research-based evidence are still scarce, but they show that their attitudes and opinions about research are one of the main determinants of such use. In February 2017, Burkina Faso’s Minister of Health planned to create a unit to promote evidence-informed decision-making within the ministry. Before the unit was set up, documenting the attitudes towards research at the highest levels of his Ministry appeared profitable to the unit’s planning.

Method

Individual interviews were conducted by the author with 14 actors positioned to consider evidence during decision-making from the Burkina Faso’s Minister of health cabinet. An interview grid was used to explore several themes such as attitudes towards research, obstacles and facilitators to research use, example of research use in decision-making and finally, ways to increase decision-makers’ participation in knowledge transfer activities. Interviews were partially transcribed and analysed by the author.

Results

The results show a mixed attitude towards research and relatively little indication of research use reported by respondents. Important obstacles were identified: evidence inaccessibility, lack of implementation guidelines, absence of clear communication strategy and studies’ lack of relevance for decision-making. Many suggestions were proposed such as raising awareness, improving access and research communication and prioritizing interactions with researchers. Respondents agree with the low participation of decision-makers in knowledge transfer activities: more leadership from the senior officials was suggested and greater awareness of the importance of their presence.

Conclusions

The conclusion presents avenues for reflection and action to increase the potential impact of the knowledge transfer unit planned within the Ministry of Health of Burkina Faso. This innovative initiative will be impactful if the obstacles identified in this study and policy-makers’ preferences and needs are taken into account during its development and implementation.

Protocol for a scoping review of implementation research approaches to universal health coverage in Africa

INTRODUCTION

Implementation research has emerged as part of evidence-based decision-making efforts to plug current gaps in the translation of research evidence into health policy and practice. While there has been a growing number of institutions and initiatives promoting the uptake of implementation research in Africa, their role and effectiveness remain unclear, particularly in the context of universal health coverage (UHC). This review aims to extensively identify and characterise the nature, facilitators and barriers to the use of implementation research for assessing or evaluating UHC-related interventions or programmes in Africa.

METHODS AND ANALYSIS

This scoping review will be developed based on the methodological framework proposed by Arksey and O'Malley and enhanced by the Joanna Briggs Institute. It will be reported in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews guidelines. A comprehensive search of the following electronic databases will be conducted: Medline (via PubMed), Scopus and the Cochrane Library. Relevant grey literature and reference lists will also be searched. All publications describing the application of implementation research in the context of UHC will be considered for inclusion. Findings will be narratively synthesised and analysed using a predefined conceptual framework. Where applicable, quantitative evidence will be aggregated using summary statistics. There will be consultation of stakeholders, including UHC-oriented health professionals, programme managers, implementation researchers and policy-makers; to provide methodological, conceptual and practical insights.

ETHICS AND DISSEMINATION

The data used in this review will be sourced from publicly available literature; hence, this study will not require ethical approval. Findings and recommendations will be disseminated to reach a diverse audience, including UHC advocates, implementation researchers and key health system stakeholders within the African region. Additionally, findings will be disseminated through an open-access publication in a relevant peer-reviewed journal.

Implementation of evidence-based practice and associated factors among nurses and midwives working in Amhara Region government hospitals: a cross-sectional study

Background

Implementation of evidence-based practice is crucial to enhance quality health care, professional development, and cost-effective health service. However, many factors influence the implementation of evidence-based practice. Therefore, this study aimed to assess the implementation of evidence-based practice and associated factors among nurses and midwives.

Methods

Institutional-based cross-sectional study design was conducted to assess the implementation of evidence-based practice and associated factors from February 15 to March 15, 2019, among 790 nurses and midwives. Data were entered into EpiData version 3.1 then exported to SPSS version 20 for statistical analysis. Categorical variables were presented as frequency tables. Continuous variables were presented as descriptive measures, expressed as mean and standard deviation. Cronbach’s alpha was used to measure reliability, mean, standard deviation, and inter-items correlation of the factors. Independent variables with a probability value (P-value) of less than 0.2 in the Chi-square analysis were entered in the multivariable logistic regression model. Statistically significant associated factors were identified at probability value (P-value) less than 0.05 and adjusted odds ratio with a 95% confidence interval.

Results

The mean age of participants was 28.35 (SD ± 4.5) years. This study revealed that 34.7% (95% CI 31.5–38%) of participants implemented evidence-based practice moderately or desirably. Age of participants (AOR = 5.98, CI 1.34–26.7), barriers of implementation of evidence-based practice (AOR = 4.8, CI 2.2–10.6), the attitude of participants (AOR = 5.02, CI 1.2–21.5), nursing/midwifery work index (AOR = 3.9, CI 1.4–10.87), self-efficacy of implementation of evidence-based practice skills (AOR = 12.5, CI 5.7–27.5) and knowledge of participants (AOR = 3.06, CI 1.6–5.77) were statistically significant associated factors of implementation of evidence-based practice

Concussion

Implementation of evidence-based practice of nurses and midwives was poor. Age of participants, barriers of implementation of evidence-based practice, the attitude of participants, self-efficacy of implementation of evidence-based practice skills, nursing/midwifery work index, and knowledge of participants were found to be predictors of implementation of evidence-based practice. Insufficient time and difficulty in judging the quality of research papers and reports were the most common barriers to the implementation of evidence-based practice.

Using a co-design process to develop an integrated model of care for delivering self-management intervention to multi-morbid COPD people in rural Nepal

BACKGROUND

People with chronic obstructive pulmonary disease (COPD) in Nepal are not receiving adequate support to self-manage their chronic conditions, and primary health care can play a key role in the effective management of these. In this study, we aimed to develop a model of care, using a co-design approach, for delivering evidence-based biomedical and psycho-social care to support self-management for people with multi-morbid COPD in rural Nepal.

METHODS

A co-design approach, guided by the five stages of the design thinking model, was used for this study. Layering on "empathize" and "define" phases, we ideated a model of care that was further refined in a "prototype" stage, which included a series of consultative meetings and a 1-day co-design workshop with stakeholders. This co-design process involved a wide range of stakeholders from Nepal, including people with COPD and their families, community representatives, local government representatives, primary care practitioners, community health workers, policymakers, state-level government representatives and academics.

RESULTS

Through our co-design approach, a model of integrated care for delivering evidence-based biomedical and psycho-social care to support self-management for people with multi-morbid COPD was designed. The integrated model of care included: screening of the community members aged > 40 years or exhibiting symptoms for COPD and management of symptomatic patients within primary health care, establishing referral pathways for severe cases to and from secondary/tertiary-level health care and establishing a community-based support system. It involved specific roles for community health workers, patients and their caregivers and community representatives. It was built on existing services and programmes linking primary health care centres and tertiary-level health facilities.

CONCLUSION

The co-design approach is different from the currently dominant approach of rolling out models of care, which were designed elsewhere with minimal community engagement. In our study, the co-design approach was found to be effective in engaging various stakeholders and in developing a model of care for rural Nepal. This grassroots approach is more likely to be acceptable, effective and sustainable in rural Nepal. Further research is required to test the effectiveness of an integrated model of care in delivering self-management support for people with multi-morbid COPD in rural Nepal.

A secondary analysis of the childhood obesity prevention Cochrane Review through a wider determinants of health lens: implications for research funders, researchers, policymakers and practitioners

BACKGROUND

Randomised controlled trials (RCTs) are often regarded as the gold standard of evidence, and subsequently go on to inform policymaking. Cochrane Reviews synthesise this type of evidence to create recommendations for practice, policy, and future research. Here, we critically appraise the RCTs included in the childhood obesity prevention Cochrane Review to understand the focus of these interventions when examined through a wider determinants of health (WDoH) lens.

METHODS

We conducted a secondary analysis of the interventions included in the Cochrane Review on "Interventions for Preventing Obesity in Children", published since 1993. All 153 RCTs were independently coded by two authors against the WDoH model using an adaptive framework synthesis approach. We used aspects of the Action Mapping Tool from Public Health England to facilitate our coding and to visualise our findings against the 226 perceived causes of obesity.

RESULTS

The proportion of interventions which targeted downstream (e.g. individual and family behaviours) as opposed to upstream (e.g. infrastructure, environmental, policy) determinants has not changed over time (from 1993 to 2015), with most intervention efforts (57.9%) aiming to change individual lifestyle factors via education-based approaches. Almost half of the interventions (45%) targeted two or more levels of the WDoH. Where interventions targeted some of the wider determinants, this was often achieved via upskilling teachers to deliver educational content to children. No notable difference in design or implementation was observed between interventions targeting children of varying ages (0-5 years, 6-12 years, 13-18 years).

CONCLUSIONS

This study highlights that interventions, evaluated via RCTs, have persisted to focus on downstream, individualistic determinants of obesity over the last 25 years, despite the step change in our understanding of its complex aetiology. We hope that the findings from our analysis will challenge research funders, researchers, policymakers and practitioners to reflect upon, and critique, the evidence-based paradigm in which we operate, and call for a shift in focus of new evidence which better accounts for the complexity of obesity.

Integrated community case management of childhood illness in low- and middle-income countries

BACKGROUND

The leading causes of mortality globally in children younger than five years of age (under-fives), and particularly in the regions of sub-Saharan Africa (SSA) and Southern Asia, in 2018 were infectious diseases, including pneumonia (15%), diarrhoea (8%), malaria (5%) and newborn sepsis (7%) (UNICEF 2019). Nutrition-related factors contributed to 45% of under-five deaths (UNICEF 2019). World Health Organization (WHO) and United Nations Children's Fund (UNICEF), in collaboration with other development partners, have developed an approach - now known as integrated community case management (iCCM) - to bring treatment services for children 'closer to home'. The iCCM approach provides integrated case management services for two or more illnesses - including diarrhoea, pneumonia, malaria, severe acute malnutrition or neonatal sepsis - among under-fives at community level (i.e. outside of healthcare facilities) by lay health workers where there is limited access to health facility-based case management services (WHO/UNICEF 2012).

OBJECTIVES

To assess the effects of the integrated community case management (iCCM) strategy on coverage of appropriate treatment for childhood illness by an appropriate provider, quality of care, case load or severity of illness at health facilities, mortality, adverse events and coverage of careseeking for children younger than five years of age in low- and middle-income countries.

SEARCH METHODS

We searched CENTRAL, MEDLINE, Embase and CINAHL on 7 November 2019, Virtual Health Library on 8 November 2019, and Popline on 5 December 2018, three other databases on 22 March 2019 and two trial registers on 8 November 2019. We performed reference checking, and citation searching, and contacted study authors to identify additional studies.

SELECTION CRITERIA

Randomized controlled trials (RCTs), cluster-RCTs, controlled before-after studies (CBAs), interrupted time series (ITS) studies and repeated measures studies comparing generic WHO/UNICEF iCCM (or local adaptation thereof) for at least two iCCM diseases with usual facility services (facility treatment services) with or without single disease community case management (CCM). We included studies reporting on coverage of appropriate treatment for childhood illness by an appropriate provider, quality of care, case load or severity of illness at health facilities, mortality, adverse events and coverage of careseeking for under-fives in low- and middle-income countries.

DATA COLLECTION AND ANALYSIS

At least two review authors independently screened abstracts, screened full texts and extracted data using a standardised data collection form adapted from the EPOC Good Practice Data Collection Form. We resolved any disagreements through discussion or, if required, we consulted a third review author not involved in the original screening. We contacted study authors for clarification or additional details when necessary. We reported risk ratios (RR) for dichotomous outcomes and hazard ratios (HR) for time to event outcomes, with 95% confidence intervals (CI), adjusted for clustering, where possible. We used estimates of effect from the primary analysis reported by the investigators, where possible. We analysed the effects of randomized trials and other study types separately. We used the GRADE approach to assess the certainty of evidence.

MAIN RESULTS

We included seven studies, of which three were cluster RCTs and four were CBAs. Six of the seven studies were in SSA and one study was in Southern Asia. The iCCM components and inputs were fairly consistent across the seven studies with notable variation for the training and deployment component (e.g. on payment of iCCM providers) and the system component (e.g. on improving information systems). When compared to usual facility services, we are uncertain of the effect of iCCM on coverage of appropriate treatment from an appropriate provider for any iCCM illness (RR 0.96, 95% CI 0.77 to 1.19; 2 CBA studies, 5898 children; very low-certainty evidence). iCCM may have little to no effect on neonatal mortality (HR 1.01, 95% 0.73 to 1.28; 2 trials, 65,209 children; low-certainty evidence). We are uncertain of the effect of iCCM on infant mortality (HR 1.02, 95% CI 0.83 to 1.26; 2 trials, 60,480 children; very low-certainty evidence) and under-five mortality (HR 1.18, 95% CI 1.01 to 1.37; 1 trial, 4729 children; very low-certainty evidence). iCCM probably increases coverage of careseeking to an appropriate provider for any iCCM illness by 68% (RR 1.68, 95% CI 1.24 to 2.27; 2 trials, 9853 children; moderate-certainty evidence). None of the studies reported quality of care, severity of illness or adverse events for this comparison. When compared to usual facility services plus CCM for malaria, we are uncertain of the effect of iCCM on coverage of appropriate treatment from an appropriate provider for any iCCM illness (very low-certainty evidence) and iCCM may have little or no effect on careseeking to an appropriate provider for any iCCM illness (RR 1.06, 95% CI 0.97 to 1.17; 1 trial, 811 children; low-certainty evidence). None of the studies reported quality of care, case load or severity of illness at health facilities, mortality or adverse events for this comparison.

AUTHORS' CONCLUSIONS

iCCM probably increases coverage of careseeking to an appropriate provider for any iCCM illness. However, the evidence presented here underscores the importance of moving beyond training and deployment to valuing iCCM providers, strengthening health systems and engaging community systems.

Transforming cities for sustainability: A health perspective

Transformational change is urgently needed to address planetary health challenges in cities. Through an interdisciplinary overview of the literature, we consider how to frame and unpack city-level transformation towards synergistic benefits for urban health and environmental sustainability. By describing the characteristics of a 'healthy sustainable city' and by bringing together the ideas underlying frameworks for health and sustainability, we develop a conceptual understanding of how cities may progress towards achieving significant improvements in health and the environment. We investigate how urban change works, and build a theoretical understanding of how urban change may be directed to integrate health and sustainability. We conclude that urban transformation needs to be a multi-scalar process across city sectors to meet the scale, speed and form of change required. We propose that this can best be achieved in practice through a composition of mechanisms, including strengthening city governance, enabling technological and social innovations, applying sustainable urban planning and infrastructure development, and impelling social behaviour change; supported by systems-driven policy and practice-focused scientific evidence.

Barriers and facilitators to clinical information seeking: a systematic review

OBJECTIVE

The study sought to identify barriers to and facilitators of point-of-care information seeking and use of knowledge resources.

MATERIALS AND METHODS

We searched MEDLINE, Embase, PsycINFO, and Cochrane Library from 1991 to February 2017. We included qualitative studies in any language exploring barriers to and facilitators of point-of-care information seeking or use of electronic knowledge resources. Two authors independently extracted data on users, study design, and study quality. We inductively identified specific barriers or facilitators and from these synthesized a model of key determinants of information-seeking behaviors.

RESULTS

Forty-five qualitative studies were included, reporting data derived from interviews (n = 26), focus groups (n = 21), ethnographies (n = 6), logs (n = 4), and usability studies (n = 2). Most studies were performed within the context of general medicine (n = 28) or medical specialties (n = 13). We inductively identified 58 specific barriers and facilitators and then created a model reflecting 5 key determinants of information-seeking behaviors: time includes subthemes of time availability, efficiency of information seeking, and urgency of information need; accessibility includes subthemes of hardware access, hardware speed, hardware portability, information restriction, and cost of resources; personal skills and attitudes includes subthemes of computer literacy, information-seeking skills, and contextual attitudes about information seeking; institutional attitudes, cultures, and policies includes subthemes describing external individual and institutional information-seeking influences; and knowledge resource features includes subthemes describing information-seeking efficiency, information content, information organization, resource familiarity, information credibility, information currency, workflow integration, compatibility of recommendations with local processes, and patient educational support.

CONCLUSIONS

Addressing these determinants of information-seeking behaviors may facilitate clinicians' question answering to improve patient care.

Leisure-time physical activity interventions for children and adults with cerebral palsy: a scoping review

AIM

To summarize current evidence on the effects and reach of leisure-time physical activity (LTPA) interventions among children and adults with cerebral palsy (CP).

METHOD

Systematic searches were conducted in PubMed, CINAHL, and Google Scholar to identify randomized controlled trials (RCTs) of LTPA interventions in CP. Data from eligible studies were extracted for qualitative synthesis.

RESULTS

Forty-nine studies enrolled a total of 1513 participants (mean [SD] age 13y [7y], range 5-43y; 818 males, 655 females, 40 not reported) and primarily included ambulatory children. RCTs underrepresented adults and people in Gross Motor Function Classification System (GMFCS) levels IV and V. Forty-one studies reported at least one favorable benefit from LTPA. Benefits included improvements to musculoskeletal strength, cardiorespiratory fitness, quality of life, spasticity, participation, and core aspects of physical function. Regarding reach, only 34% of people that were contacted to participate enrolled within a study. A smaller percentage of participants dropped out from intervention (8%) and follow-up periods (3%).

INTERPRETATION

Study findings highlight effective interventions to improve health, fitness, and function. To enhance the reach and generalizability of LTPA trials for CP, future studies should examine how to increase study sample sizes and aim to include a better representation of adults and people in GMFCS levels IV and V.

WHAT THIS PAPER ADDS

People with cerebral palsy (CP) may experience improvements in health, fitness, and physical function from leisure-time physical activity (LTPA) interventions. Effective interventions include exercise training, active video games, recreation activities, behavioral coaching, and motor skills training. Interventions that incorporate telehealth technology, behavioral coaching, and community resources may enhance LTPA. Interventions primarily include children in Gross Motor Functional Classification System (GMFCS) levels I to III. Adults, wheelchair users, and those in GMFCS levels IV and V are underrepresented.

The shift from inpatient care to outpatient care in Switzerland since 2017: Policy processes and the role of evidence

The shift from inpatient care to the ambulatory sector is a central aspiration of European health systems. Despite demonstrated benefits, health reforms have struggled to realize their potential. In this context, we discuss recent hospital sector reforms in Switzerland and analyze the content, process, and role of evidence in the recent introduction of policies to substitute inpatient care with ambulatory care. The prevailing payment system incentivized hospitals to provide unnecessary and costly inpatient services, but federal reform on tariff structures was deemed politically unfeasible. Instead, driven by the pressure to contain costs, cantonal and federal health authorities began to deny reimbursement for selected inpatient procedures in 2017. These regulatory measures were effective in reducing inpatient admissions and health care costs. This case study illustrates that clear, simple messages about hospital sector reform can raise awareness of the need for change. However, the evidence used in the policy process was limited and not critically reviewed. Stakeholders used long-standing international comparisons of inpatient substitution potential to legitimize policies, but not to develop them. The analysis restates the importance of inter- and intranational comparative analyses and institutions such as health observatories and suggests aligning health system governance more proactively with international developments.

Use of a quantitative data report in a hypothetical decision scenario for health policymaking: a computer-assisted laboratory study

BACKGROUND

Quantitative data reports are widely produced to inform health policy decisions. Policymakers are expected to critically assess provided information in order to incorporate the best available evidence into the decision-making process. Many other factors are known to influence this process, but little is known about how quantitative data reports are actually read. We explored the reading behavior of (future) health policy decision-makers, using innovative methods.

METHODS

We conducted a computer-assisted laboratory study, involving starting and advanced students in medicine and health sciences, and professionals as participants. They read a quantitative data report to inform a decision on the use of resources for long-term care in dementia in a hypothetical decision scenario. Data were collected through eye-tracking, questionnaires, and a brief interview. Eye-tracking data were used to generate 'heatmaps' and five measures of reading behavior. The questionnaires provided participants' perceptions of understandability and helpfulness as well as individual characteristics. Interviews documented reasons for attention to specific report sections. The quantitative analysis was largely descriptive, complemented by Pearson correlations. Interviews were analyzed by qualitative content analysis.

RESULTS

In total, 46 individuals participated [students (85%), professionals (15%)]. Eye-tracking observations showed that the participants spent equal time and attention for most parts of the presented report, but were less focused when reading the methods section. The qualitative content analysis identified 29 reasons for attention to a report section related to four topics. Eye-tracking measures were largely unrelated to participants' perceptions of understandability and helpfulness of the report.

CONCLUSIONS

Eye-tracking data added information on reading behaviors that were not captured by questionnaires or interviews with health decision-makers.

Determinants of using children's mental health research in policymaking: variation by type of research use and phase of policy process

Background

Research use in policymaking is multi-faceted and has been the focus of extensive study. However, virtually no quantitative studies have examined whether the determinants of research use vary according to the type of research use or phase of policy process. Understanding such variation is important for selecting the targets of implementation strategies that aim to increase the frequency of research use in policymaking.

Methods

A web-based survey of US state agency officials involved with children’s mental health policymaking was conducted between December 2019 and February 2020 (n = 224, response rate = 33.7%, 49 states responding (98%), median respondents per state = 4). The dependent variables were composite scores of the frequency of using children’s mental health research in general, specific types of research use (i.e., conceptual, instrumental, tactical, imposed), and during different phases of the policy process (i.e., agenda setting, policy development, policy implementation). The independent variables were four composite scores of determinants of research use: agency leadership for research use, agency barriers to research use, research use skills, and dissemination barriers (e.g., lack of actionable messages/recommendations in research summaries, lack of interaction/collaboration with researchers). Separate multiple linear regression models estimated associations between determinant and frequency of research use scores.

Results

Determinants of research use varied significantly by type of research use and phase of policy process. For example, agency leadership for research use was the only determinant significantly associated with imposed research use (β = 0.31, p < 0.001). Skills for research use were the only determinant associated with tactical research use (β = 0.17, p = 0.03) and were only associated with research use in the agenda-setting phase (β = 0.16, p = 0.04). Dissemination barriers were the most universal determinants of research use, as they were significantly and inversely associated with frequency of conceptual (β = −0.21, p = 0.01) and instrumental (β = −0.22, p = 0.01) research use and during all three phases of policy process.

Conclusions

Decisions about the determinants to target with policy-focused implementation strategies—and the strategies that are selected to affect these targets—should reflect the specific types of research use that these strategies aim to influence.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13012-021-01081-8.

Collaborative research and knowledge translation on road crashes in Burkina Faso: the police perspective 18 months on

In this commentary, we present a follow-up of two articles published in 2017 and 2018 about road traffic crashes, which is an important public health issue in Africa and Burkina Faso. The first article reported on a research project, conducted in partnership with local actors involved in road safety, carried out in Ouagadougou in 2015. Its aim was to test the effectiveness, acceptability, and capacity of a surveillance system to assess the number of road traffic crashes and their consequences on the health of crash victims. Several knowledge translation activities were carried out to maximize its impact and were reported in the 2018 article published in HRPS: monthly reports presenting the research data, large-format printed maps distributed to the city's police stations, and a deliberative workshop held at the end of the research project. The present commentary presents our efforts to deepen our understanding of the impacts of the knowledge translation strategy, based on follow-up interviews, 18 months after the workshop, with the heads of the road traffic crash units in Ouagadougou police stations (n = 5). Several benefits were reported by respondents. Their involvement in the process prompted them to broaden their knowledge of other ways of dealing with the issue of road crashes. This led them, sometimes with their colleagues, to intervene differently: more rapid response at collision sites, increased surveillance of dangerous intersections, user awareness-raising on the importance of the highway code, etc. However, sustaining these actions over the longer term has proven difficult. Several lessons were derived from this experience, regarding the importance of producing useful and locally applicable research data, of ensuring the acceptability of the technologies used for data collection, of using collaborative approaches in research and knowledge translation, of ensuring the visibility of actions undertaken by actors in the field, and of involving decision-makers in the research process to maximize its impacts.

The most important facilitators and barriers to the use of Health Technology Assessment in Canada: a best-worst scaling approach

BACKGROUND

Health Technology Assessment (HTA), which can support public drug reimbursement decisions will play a core function in the planned national Pharmacare program in Canada. To address existing barriers to the use of HTA, these must be ranked in order of priority. The goal of this study was to access the relative importance of known facilitators and barriers to the use of HTA in the context of the Canadian health care system, with attention to differences between regions and stakeholder groups.

METHODS

We used the best-worst scaling object case approach to elicit a quantitative ranking of a list of 20 facilitators and 22 barriers. A sample of 68 Canadian HTA stakeholders, including members of expert committees, decision/policymakers, researchers/academics, and others participated in the study. Their task was to identify the most important and the least important item in 12 sub-sets of five facilitators and 14 sub-sets of five barriers.

FINDINGS

Relative Importance Scores derived via hierarchical Bayes analysis revealed relations, engagement, and contact between stakeholders as most important on both the barrier and facilitator sides. Other top-ranked facilitators included the availably of credible and relevant research. Other top-ranked barriers included inconsistencies in the evidence and limited generalizability. The availability of HTA guidelines did not rank highly on either side. The main limitation of the study was the challenge with reaching the relevant respondents; this was mitigated by involving the national HTA agency in the research.

CONCLUSION

Canadian stakeholders consider the relationships within the HTA network among the most important. Policies should focus on strengthening these relationships. Future research should focus on the connectivity and distribution of knowledge and power within the HTA network.

Barriers to eHealth adoption in routine antenatal care practices: Perspectives of expectant mothers in Uganda - A qualitative study using the unified theory of acceptance and use of technology model

Current empirical evidence suggests that successful adoption of eHealth systems improves maternal health outcomes, yet there are still existing gaps in adopting such systems in Uganda. Service delivery in maternal health is operating in a spectrum of inadequacy, hence eHealth adoption cannot ensue. This study set out to explore the challenges that impede eHealth adoption in women's routine antenatal care practices in Uganda. A qualitative approach using semi-structured interviews was employed to document challenges. These challenges were classified based on a unified theory of acceptance and use of technology constructs. One hundred and fifteen expectant mothers, aged between 18 and 49 years, who spoke either English or Luganda were included in the study that took place between January to May 2019. Thematic analysis using template analysis was adopted to analyse qualitative responses. Challenges were categorised based on five principal unified theories of acceptance and use of technology constructs namely: performance expectancy, effort expectancy, social influence, facilitating conditions and behavioural intention. Facilitating conditions had more influence on technology acceptance and adoption than the other four constructs. Specifically, the lack of training prior to using the system, technical support, computers and smart phones had a downhill effect on adoption. Subsequently, the cost of data services, internet intermittency, and the lack of systems that bridge the gap between mothers and health providers further hindered technology uptake. In conclusion, strategies such as co-development, training end-users, garnering support at the national and hospital levels should be advocated to improve user acceptance of technology.

A research impact model for work and health

Research organizations, governments and funding agencies are increasingly interested in the impact of research beyond academia. While a growing literature describes research impacts in healthcare and health services, little has focused on occupational health and safety research. This article describes a research impact model that has been in use for over a decade. The model was developed to track and describe the impact of research conducted by a mid-sized institute that focuses on work and health. Model development was informed by existing models, with the goal of contextualizing the institute's case studies describing three types of research impact: evidence of the diffusion of research; evidence of research informing decision-making; and evidence of societal impact. A logic model describes research actions and outcomes, as well as key audiences and knowledge transfer approaches. A unique element is its indication of the level of difficulty in determining types of impact. The model compares well with current research impact models developed or used in healthcare and health services research, and it has been useful in guiding a mid-sized research organization's process for tracking and describing the impact of its research. It may be useful to other small and mid-sized research organizations that focus on workplace health and safety.

Public opinion about evidence-informed health policy development in U.S. Congress

Promoting evidence-informed health policymaking is a priority of the Society of Behavioral Medicine and other professional societies. However, politics often impede the translation of research into policy. Public opinion is an important feature of political context that influences policymakers' behaviors, but prior research has not examined public opinion about evidence-informed health policy development. This exploratory study sought to characterize public opinion about the influence that evidence should, and does, have on health policy development in U.S. Congress relative to other factors and examine differences by political party affiliation. A public opinion survey was conducted in 2018 using the SSRS Probability Panel (N = 532). Respondents separately rated the extent to which six factors (e.g., evidence, budget impact, industry interests) "should have" and "currently have" influence on U.S. congresspersons' health policy decisions. Evidence (59%) was the most frequently identified factor that should have "a lot of influence" on health policy development, but only 11% of respondents thought that evidence currently has "a lot of influence" (p < .001). Opinions about evidence did not vary significantly by political party. The interests of insurance and pharmaceutical companies were identified as factors that should have the least influence on policy development, but were perceived as having the most influence (p < .001). There is strong bipartisan public support for evidence to have much more influence on health policy development in U.S. Congress. Efforts that aim to improve evidence-informed health policymaking should consider harnessing the power of public opinion to change elected policymakers' behaviors.

Lessons Learned From Implementing Prospective, Multicountry Mixed-Methods Evaluations for Gavi and the Global Fund

Lessons learned from implementing evaluations for Gavi, the Vaccine Alliance and the Global Fund for AIDS, Tuberculosis, and Malaria can help inform the design and implementation of ongoing or future evaluations of complex interventions. We share 5 lessons distilled from over 7 years of experience implementing evaluations in 7 countries.

Introduction:

As global health programs have become increasingly complex, corresponding evaluations must be designed to assess the full complexity of these programs. Gavi and the Global Fund have commissioned 2 such evaluations to assess the full spectrum of their investments using a prospective mixed-methods approach. We aim to describe lessons learned from implementing these evaluations.

Methods:

This article presents a synthesis of lessons learned based on the Gavi and Global Fund prospective mixed-methods evaluations, with each evaluation considered a case study. The lessons are based on the evaluation team’s experience from over 7 years (2013–2020) implementing these evaluations. The Centers for Disease Control and Prevention Framework for Evaluation in Public Health was used to ground the identification of lessons learned.

Results:

We identified 5 lessons learned that build on existing evaluation best practices and include a mix of practical and conceptual considerations. The lessons cover the importance of (1) including an inception phase to engage stakeholders and inform a relevant, useful evaluation design; (2) aligning on the degree to which the evaluation is embedded in the program implementation; (3) monitoring programmatic, organizational, or contextual changes and adapting the evaluation accordingly; (4) hiring evaluators with mixed-methods expertise and using tools and approaches that facilitate mixing methods; and (5) contextualizing recommendations and clearly communicating their underlying strength of evidence.

Conclusion:

Global health initiatives, particularly those leveraging complex interventions, should consider embedding evaluations to understand how and why the programs are working. These initiatives can learn from the lessons presented here to inform the design and implementation of such evaluations.

Perspectives and Experiences of Policy Makers, Researchers, Health Information Technology Professionals, and the Public on Evidence-Based Health Policies: Protocol for a Qualitative Study

Background

Evidence-based health policy (EBHP) development is critical to the judicious use of public funds. EBHPs increase transparency, accountability, effectiveness, and efficiency of policies. Encouraging collaboration between researchers or knowledge producers and policy makers is important because both communities have distinct professional cultures, resulting in them working separately without understanding each other. Knowledge sharing is a complex process that requires understanding of cultural aspects that may reduce cultural differences and increase the use of common language. Health information technology (HIT) is a useful tool to increase knowledge translation, which may result in the transparent use of evidence and networking in developing EBHPs. Our vision is to leverage HIT tools for a better health system that includes digitalized, open source, evidence-based, and transparent ways for collaboration and development of robust mechanisms and for sharing of synthesized evidence with knowledge user–friendly forms.

Objective

The aim of this study is to develop a conceptual framework on Knowledge translation and health Information Technology for Transparency (KhITT) in policy making and EBHPs (ie, the KhITT framework). The framework will be informed by the views of four key stakeholder groups (ie, policy makers, knowledge producers, HIT professionals, and the public) toward EBHP. The informants may also describe practices that demonstrate the EBHP development process and suggest technology platforms to enable this process.

Methods

We propose an exploratory, descriptive qualitative study to take place in British Columbia, Canada, using in-depth semistructured interviews. To ensure data saturation and trustworthiness, we will use a nonprobability, purposive snowball sample of up to 15 eligible participants in each of the four stakeholder groups. We will analyze the data using content analysis.

Results

The KhITT framework focuses on various stakeholders’ perspectives to better understand their perceived needs and priorities in identifying issues with EBHP, in order to make informed recommendations. Ethics approval has been obtained by the harmonized Behavioural Research Ethics Board at the University of British Columbia. We anticipate that we will complete data collection and analysis by December 2020. Preliminary results will be published in summer 2021.

Conclusions

Our ultimate goal of this study is to develop a conceptual framework and describe the technology platforms that would enable the EBHP process. We anticipate that our rigorous content analysis will be able to produce insights and themes that are able to address our objectives, contribute to an in-depth understanding of the EBHP process within British Columbia, highlight all influential factors, explicitly disseminate and communicate the study results, identify issues with EBHP and provide informed recommendations to address them, and enhance efforts toward transparent EBHPs.

International Registered Report Identifier (IRRID)

PRR1-10.2196/16268

Knowledge translation in Africa: are the structures in place?

Background

Contextualised evidence to generate local solutions on the progressive path to universal health coverage is essential. However, this evidence must be translated into action. Knowledge translation (KT) experts have highlighted the plausible mechanisms to foster the uptake of evidence. The objective of this study was to assess the extent to which structures are in place to boost uptake of evidence, in countries of the WHO African Region.

Methods

Employing a cross-sectional survey, we collected data on the availability of structures to foster the uptake of evidence into policy in 35 out of the 47 member states of the WHO African Region. Data were analysed using a simple counting of the presence or absence of such structures.

Results

Less than half of the countries had evidence collation and synthesis mechanisms. The lack of such mechanisms presents a missed opportunity to identify comprehensive solutions that can respond to health sector challenges. Close to 50% of the countries had KT platforms in place. However, the availability of these was in several forms, as an institution-based platform, as an annual event to disseminate evidence and as a series of conferences at the national level. In some countries, KT was mainstreamed into routine health sector performance review processes. Several challenges impacted the functionality of the KT platforms including inadequate funding and lack of dedicated personnel. Regarding dissemination of evidence, sharing reports, scientific publications and one-off presentations in meetings were the main approaches employed.

Conclusion

The availability of KT platforms in the WHO African countries can be described as at best and non-existent at the worst. The current structures, where these exist, cannot adequately foster KT. Knowledge translation platforms need to be viewed as sector-wide platforms and mainstreamed in routine health sector performance reviews and policymaking processes. Funds for their functionality must be planned for as part of the health sector budget. Dissemination of evidence needs to be viewed differently to embrace the concept of “disseminate for impact”. Further, funding for dissemination activities needs to be planned for as part of the evidence generation plan.

Public perspectives on acquired brain injury rehabilitation and components of care: A Citizens' Jury

BACKGROUND

Brain injury rehabilitation is an expensive and long-term endeavour. Very little published information or debate has underpinned policy for service delivery in Australia. Within the context of finite health budgets and the challenges associated with providing optimal care to persons with brain injuries, members of the public were asked 'What considerations are important to include in a model of care of brain injury rehabilitation?'

METHODS

Qualitative study using the Citizen Jury method of participatory research. Twelve adult jurors from the community and seven witnesses participated including a health services funding model expert, peak body representative with lived experience of brain injury, carer of a person with a brain injury, and brain injury rehabilitation specialists. Witnesses were cross-examined by jurors over two days.

RESULTS

Key themes related to the need for a model of rehabilitation to: be consumer-focused and supporting the retention of hope; be long-term; provide equitable access to services irrespective of funding source; be inclusive of family; provide advocacy; raise public awareness; and be delivered by experts in a suitable environment. A set of eight recommendations were made.

CONCLUSION

Instigating the recommendations made requires careful consideration of the need for new models of care with flexible services; family involvement; recruitment and retention of highly skilled staff; and providing consumer-focused services that prepare individuals and their carers for the long term.

PATIENT AND PUBLIC CONTRIBUTION

As jury members, the public deliberated information provided by expert witnesses (including a person with a head injury) and wrote the key recommendations.

Challenges, opportunities and solutions for local physical activity stakeholders: an implementation case study from a cross-sectoral physical activity network in Northeast England

Background

Increasingly, national policy initiatives and programmes have been developed to increase physical activity (PA). However, challenges in implementing and translating these policies into effective local-level programmes have persisted, and change in population PA levels has been small. This may be due to insufficient attention given to the implementation context, and the limited interactions between local policy-makers, practitioners and researchers. In this paper we use a case study of a cross-sectoral network in Northeast England, to identify the local-level challenges and opportunities for implementing PA policies and programmes, particularly the updated 2019 UK PA guidelines.

Methods

Five focus groups (n = 59) were conducted with practice partners, local policy-makers and researchers during an initial workshop in April 2018. Through facilitated discussion, participants considered regional priorities for research and practice, along with barriers to implementing this agenda and how these may be overcome. During a second workshop in December 2018, overarching findings from workshop one were fedback to a similar group of stakeholders, along with national policy-makers, to stimulate feedback from delegates on experiences that may support the implementation of the UK PA guidelines locally, focusing on specific considerations for research, evidence and knowledge exchange.

Results

In workshop one, three overarching themes were developed to capture local challenges and needs: (i) understanding complexity and context; (ii) addressing the knowledge and skills gap; and (iii) mismatched timescales and practices. In workshop two, participants’ implementation plans encompassed: (i) exploring a systems approach to implementation; (ii) adapting policy to context; and (iii) local prioritising.

Conclusions

Our findings suggest that academics, practitioners and policy-makers understand the complexities of implementing PA strategies, and the challenges of knowledge exchange. The updated UK PA guidelines policy presented an opportunity for multiple agencies to consider context-specific implementation and address enduring tensions between stakeholders. An organically derived implementation plan that prioritises PA, maps links to relevant local policies and supports a context-appropriate communication strategy, within local policy, practice and research networks, will help address these. We present 10 guiding principles to support transferable knowledge exchange activities within networks to facilitate implementation of national PA policy in local contexts.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12889-020-09847-3.

How can we monitor the impact of national health information systems? Results from a scoping review

Background

National health information (HI) systems provide data on population health, the determinants of health and health system performance within countries. The evaluation of these systems has traditionally focused on statistical practices and procedures, and not on data use or reuse for policy and practice. This limits the capacity to assess the impact of HI systems on healthcare provision, management and policy-making. On the other hand, the field of Knowledge Translation (KT) has developed frameworks to guide evidence into practice.

Methods

A scoping review of the KT literature to identify the essential mechanisms and determinants of KT that could help monitor the impact of HI systems.

Results

We examined 79 publications and we identified over 100 different KT frameworks but none of these were focused on HI systems per se. There were specific recommendations on disseminating evidence to stakeholders at the institutional and organizational level, and on sustaining the use of evidence in practice and the broader community setting.

Conclusions

We developed a new model, the HI-Impact framework, in which four domains are essential for mapping the impact of national HI systems: (i) HI Evidence Quality, (ii) HI System Responsiveness, (iii) Stakeholder Engagement and (iv) Knowledge Integration. A comprehensive impact assessment of HI systems requires addressing the use of HI in public health decision-making, health service delivery and in other sectors which might have not been considered previously. Monitoring Stakeholder Engagement and Knowledge Integration certifies that the use of HI in all policies is an explicit point of assessment.

Dissemination Strategies to Accelerate the Policy Impact of Children's Mental Health Services Research

The United States is in the midst of a children's mental health crisis, with rates of depression, anxiety, and suicide increasing precipitously. Evidence produced by children's mental health services research can help address this crisis by informing public policy decisions about service delivery, system design, and investments in the social determinants of mental health. Unfortunately, the policy impact of children's mental health services research is limited because evidence often fails to reach policy makers, be responsive to their needs, resonate with their worldview, or reflect the contexts in which they make decisions. Dissemination strategies-defined as the development and targeted distribution of messages and materials about research evidence pertaining to a specific issue or intervention-can help address these challenges. Yet, limited integrated guidance exists to inform the design of such strategies. This article addresses this need by synthesizing the results of empirical studies to provide guidance about how to enhance the dissemination of children's mental health services research to policy makers. The article provides four recommendations about the content of policy maker-focused dissemination materials, discusses how strategic framing and message tailoring can increase the chances that evidence is persuasive to policy makers, and highlights strategies to ensure that evidence reaches policy makers.

The Evolving Field of Risk Communication

The 40th Anniversary of the Society for Risk Analysis presents an apt time to step back and review the field of risk communication. In this review, we first evaluate recent debates over the field's current state and future directions. Our takeaway is that efforts to settle on a single, generic version of what constitutes risk communication will be less productive than an open-minded exploration of the multiple forms that comprise today's vibrant interdisciplinary field. We then review a selection of prominent cognitive, cultural, and social risk communication scholarship appearing in the published literature since 2010. Studies on trust in risk communication messengers continued to figure prominently, while new research directions emerged on the opportunities and critical challenges of enhancing transparency and using social media. Research on message attributes explored how conceptual insights particularly relating to framing, affective and emotional responses, and uncertainty might be operationalized to improve message effectiveness. Studies consistently demonstrated the importance of evaluation and how varying single attributes alone is unlikely to achieve desired results. Research on risk communication audiences advanced on risk perception and multiway engagement with notable interest in personal factors such as gender, race, age, and political orientation. We conclude by arguing that the field's interdisciplinary tradition should be further nurtured to drive the next evolutionary phase of risk communication research.

Reflecting on knowledge translation strategies from global health research projects in Tunisia and the Republic of Côte d'Ivoire

OBJECTIVES

We describe the knowledge translation strategies in two projects and share lessons learned about knowledge sharing and uptake.

METHODS

To generate findings for dissemination: (1) the Republic of Côte d'Ivoire (RCI) project relied on a multiple case study design to document barriers and facilitators to implementing a community-led prevention strategy targeting Ebola virus disease; and (2) the Tunisia project used several designs to assess a mental health training's effectiveness, and a case study design to explore contextual factors that may influence anticipated outcomes.

RESULTS

To share findings with participants, the RCI project relied on workshops and a pamphlet, and the Tunisia project relied on a structured half-day dissemination workshop and research summary. Facilitators that may have encouraged sharing and using findings include involving champions in dissemination activities, ongoing collaboration, and developing/implementing context-specific knowledge sharing strategies. Barriers include omitting to assess strategies, limited consideration of a wider audience, and the exclusion of a knowledge translation training component.

CONCLUSIONS

Our experiences might be useful to contexts involved in global and public health research that wish to address the "know-do gap."

Strategies for effective dissemination of research to United States policymakers: a systematic review

BACKGROUND

Research has the potential to influence US social policy; however, existing research in this area lacks a coherent message. The Model for Dissemination of Research provides a framework through which to synthesize lessons learned from research to date on the process of translating research to US policymakers.

METHODS

The peer-reviewed and grey literature was systematically reviewed to understand common strategies for disseminating social policy research to policymakers in the United States. We searched Academic Search Premier, PolicyFile, SocINDEX, Social Work Abstracts, and Web of Science from January 1980 through December 2019. Articles were independently reviewed and thematically analyzed by two investigators and organized using the Model for Dissemination of Research.

RESULTS

The search resulted in 5225 titles and abstracts for inclusion consideration. 303 full-text articles were reviewed with 27 meeting inclusion criteria. Common sources of research dissemination included government, academic researchers, the peer reviewed literature, and independent organizations. The most frequently disseminated research topics were health-related, and legislators and executive branch administrators were the most common target audience. Print materials and personal communication were the most common channels for disseminating research to policymakers. There was variation in dissemination channels by level of government (e.g., a more formal legislative process at the federal level compared with other levesl). Findings from this work suggest that dissemination is most effective when it starts early, galvanizes support, uses champions and brokers, considers contextual factors, is timely, relevant, and accessible, and knows the players and process.

CONCLUSIONS

Effective dissemination of research to US policymakers exists; yet, rigorous quantitative evaluation is rare. A number of cross-cutting strategies appear to enhance the translation of research evidence into policy.

REGISTRATION

Not registered.

What prevents health policy being 'evidence-based'? New ways to think about evidence, policy and interventions in health

BACKGROUND

Evidence-based policy decision-making is a dominant paradigm in health but realizing this ideal has proven challenging.

SOURCES OF DATA

This paper conceptually maps health policy, policy studies and social science literature critically engaged with evidence and decision-making. No new data were generated or analysed in support of this review.

AREAS OF AGREEMENT

Barriers to evidence-based policy have been documented, with efforts made to increase the uptake of evidence.

AREAS OF CONTROVERSY

Evident complexities have been regarded as a problem of translation. However, this assumes that policy-making is a process of authoritative choice, and that 'evidence' is inherently valuable policy knowledge, which has been critiqued.

GROWING POINTS

Alternative accounts urge consideration of how evidence comes to bear on decisions made within complex systems, and what counts as evidence.

AREAS TIMELY FOR DEVELOPING RESEARCH

An 'evidence-making intervention' approach offers a framework for conceptualizing how evidence and interventions are made relationally in practices, thus working with the politics and contingencies of implementation and policy-making.

Evaluation of the performance and achievements of the WHO Evidence-informed Policy Network (EVIPNet) Europe

Background

The Evidence-informed Policy Network (EVIPNet) is one of the key mechanisms introduced by WHO to reduce the research-to-policy gap. EVIPNet Europe was launched in 2012. We evaluated the performance and achievements of EVIPNet Europe with the overall aims (1) to inform future developments and strategic planning of EVIPNet Europe and (2) to contribute to the evidence base for organisational knowledge translation activities by sharing the lessons learnt.

Methods

The evaluation covered the WHO Secretariat of EVIPNet Europe and its 21 member countries, from its inception to mid-2018. A mixed methods design was used to assess changes in three domains, including triangulation of quantitative and qualitative methods, based on the EVIPNet Europe Monitoring & Evaluation framework and theory of change. Data were collected between August and October 2018. Data collection comprised documentary review, social media analysis, online country evaluation, key informant interviews and validated tools. Two case studies were also developed.

Results

The evaluation showed promising results as well as lessons to guide the future development of EVIPNet in the WHO European Region and other regions of the world. EVIPNet Europe appears to be filling a niche in promoting the capacity of Network member countries for evidence-informed policy-making. There is evidence that EVIPNet Europe’s capacity-building programme of work is improving knowledge and skills at the individual level. There has been an increase in activity and outputs since its establishment and evidence has been used to inform new policies in some member countries. However, the speed at which member countries are developing or publishing products varies greatly and no formalised knowledge translation platforms have yet been created. Financial and human resources are limited and staff turnover is a cause for concern, both at the WHO Secretariat and country team levels.

Conclusions

Six years since the launch of EVIPNet Europe, the Network has grown quickly, is clearly valued and has had some successes. However, more work and support are needed if it is to achieve its vision of a Europe in which high-quality, context-sensitive evidence routinely informs health decision-making processes that ultimately serve to strengthen health outcomes across the Region.

Integrated mental health screening for obstetric fistula patients in Mali: From evidence to policy

BACKGROUND

Obstetric fistula significantly impacts women's mental health and well-being. Routine screening for mental health in fistula repair programs can be a gateway to link patients to services, and can produce routine data to inform programmatic investments. This study observed the integration of a mental health screening program into an obstetric fistula repair program in Mali, with two specific objectives: 1) to describe the social and mental health well-being of women presenting with obstetric fistulas in Mali, and 2) to document the impact of the mental health screening pilot on policy change in Mali.

METHODS

Seven fistula repair campaigns were conducted between June 2016 and May 2017. All individuals presenting for fistula repair completed a mental health assessment at intake, including a depression screener (PHQ-9) and an assessment of psycho-social impacts of fistula. The depression screener was repeated three months following inpatient discharge. Findings were shared with stakeholders in Mali and impacts on policy were documented.

RESULTS

Of 207 women who presented for fistula repair, 167 patients completed the mental health assessment at surgical intake, and 130 patients repeated the screener at 3-month follow-up. At intake, 36.5% of women had moderate or severe depression, decreasing to 16.9% at follow-up. The mean depression score differed significantly by timepoint (9.14 vs. 6.72, p <0.001). Results were shared in a report with stakeholders, and consultations with the Mali Ministry of Health. As a result of advocacy, mental health was a key component of Mali's National Fistula Prevention and Treatment Strategy (2018-2022).

CONCLUSION

The high prevalence of depression in Malian fistula patients underscores a need for more robust mental health support for patients after surgery. Data on mental health from routine screening informs community reintegration strategies for individual patients, elevates the overall quality of care of fistula repair programs by addressing patients' holistic health needs, and contributes to evidence-informed decision-making and data-driven policy change within the larger health system.