Towards systematic reviews that inform health care management and policy-making

A knowledge synthesis of health research reporting standards relevant to epilepsy surgery

OBJECTIVE

Numerous studies have examined epilepsy surgery outcomes, yet the variability in the level of detail reported hampers our ability to apply these findings broadly across patient groups. Established reporting standards in other clinical research fields enhance the quality and generalizability of results, ensuring that the insights gained from studying these surgeries can benefit future patients effectively. This study aims to assess current reporting standards for epilepsy surgery research and identify potential gaps and areas for enhancement.

METHODS

The Enhancing the Quality and Transparency of Health Research (EQUATOR) repository was accessed from inception to April 27, 2023, yielding 561 available reporting standards. Reporting standards were manually reviewed in duplicate independently for applicability to epilepsy and/or neurosurgery research. The reporting standards had to cover the following aspects in human studies: (1) reporting standards for epilepsy/epilepsy surgery and (2) reporting standards for neurosurgery. Disagreements were resolved by a third author. The top five neurosurgery, neurology, and medicine journals were also identified through Google Scholar's citation index and examined to determine the relevant reporting standards they recommended and whether those were registered with EQUATOR.

RESULTS

Of the 561 EQUATOR reporting standards, 181 were pertinent to epilepsy surgery. One was related to epilepsy, six were specific to surgical research, and nine were related to neurological/neurosurgical research. The remaining 165 reporting standards were applicable to research across various disciplines and included but were not limited to CONSORT (Consolidated Standards of Reporting Trails), STROBE (Strengthening the Reporting of Observational Studies in Epidemiology), and PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses). None of these required reporting factors associated with epilepsy surgery outcomes, such as duration of epilepsy or magnetic resonance imaging findings.

SIGNIFICANCE

Reporting standards specific to epilepsy surgery are lacking, reflecting a gap in standards that may affect the quality of publications. Improving this gap with a set of specific reporting standards would ensure that epilepsy surgery studies are more transparent and rigorous in their design.

Various application roles for Campbell systematic reviews: a citation analysis

OBJECTIVES

Systematic reviews (SRs) are becoming essential evidence in the decision-making process within the field of social sciences. This study aimed to investigate how Campbell SRs were cited and explore their specific application roles.

STUDY DESIGN AND SETTING

We included Campbell SRs published between 2016 and 2020 by searching the Wiley online library, and retrieved the articles and documents citing Campbell SRs from the Web of Science and Google Scholar by December 31, 2021. We described the characteristics of the SRs and citations, and formulated a set of application roles by analyzing the sentences or paragraphs where the SRs were cited.

RESULTS

Sixty nine Campbell SRs were published between 2016 and 2020; they were cited in 641 articles or documents a total of 1,289 times. The primary types of articles that cited Campbell SRs were cross-sectional studies (n = 226, 35.3%), SRs (n = 112, 17.5%), randomized controlled trials (n = 77, 12.0%), and policy reports (n = 57, 8.8%). Articles utilizing Campbell SRs were predominantly led by authors from the United States (n = 184, 28.7%), the United Kingdom (n = 98, 15.3%), and Australia (n = 51, 8.0%). We formulated a set of 10 application roles for the Campbell SRs, of which the most frequent were: describing the current status in the field of interest (n = 691, 53.6%), corroboration of the results (n = 140, 10.9%), identifying research gaps (n = 130, 10.1%), and providing methodological references (n = 126, 9.8%); the role of supporting policy recommendations or decisions accounted for 6.0% (n = 77) of the citations. Approximately 12% of Campbell SRs were used to support policy recommendations or decisions.

CONCLUSION

Campbell SRs are widely applied, particularly in scientific research, to describe the current status in the field of interest. Although the current application of Campbell SRs in supporting policy recommendations and decisions may not be predominant, there is a growing recognition of their value in using Campbell SRs to inform decision-making.

Challenges of using evidence in managerial decision-making of the primary health care system

BACKGROUND

Managerial Evidence-Based Decision-Making [EBDM] in the primary is a systematic approach that directs the decision-maker in a conscientious, explicit, and judicious utilization of reliable and best evidence based on the professional experiences and preferences of stakeholders and patients from various sources. This study aimed to investigate the challenges primary healthcare managers encounter while undertaking decision-making processes.

METHOD

A systematic review was conducted in 2022 with the aim of identifying and collecting all qualitative articles pertaining to evidence-based decision-making in the primary healthcare system. To achieve this, a meticulous search was conducted using the relevant keywords, including primary health care and evidence-based decision making, as well as their corresponding synonyms, across the databases Web of Science, Scopus, and Pubmed. Importantly, there were no limitations imposed on the timeframe for the search. To carefully analyze and consolidate the findings of this systematic review, the meta-synthesis approach was employed.

RESULTS

A total of 22 articles were assessed in this systematic review study. The results revealed the main categories including evidence nature, EBDM barriers, utilizing evidence, decision-makers ability, organizational structure, evidence-based, EBDM support, communication for EBDM, evidence sides, EBDM skill development, public health promotion, and health system performance improvement.

CONCLUSION

The primary healthcare system is crucial in improving health outcomes and ensuring access to healthcare services for all individuals. This study explored the utilization of evidence-based EBDM within the primary healthcare system. We identified five key dimensions: causal, contextual, and intervening conditions, strategies, and consequences of EBDM as a core phenomenon. The findings will help policymakers and administrators comprehend the importance of evidence-based decision-making, ultimately leading to enhanced decision quality, community well-being, and efficiency within the healthcare system. EBDM entails considering the best reliable evidence, and incorporating community preferences while also exploiting the professional expertise and experiences of decision-makers. This systematic review has the potential to provide guidance for future reforms and enhance the quality of decision-making at the managerial level in primary healthcare.

Rapid Reviews to Support Practice: A Guide for Professional Organization Practice Networks

Background. Occupational Therapists, among other healthcare decision makers, often need to make decisions within limited timeframes and cannot wait for the completion of large rigorous systematic reviews and meta-analyses. Rapid reviews are one method to increase the integration of research evidence into clinical decision making. Rapid reviews streamline the systematic review process to allow for the timely synthesis of evidence; however, there does not exist a single agreed upon guide for the methodology and reporting of rapid reviews. Purpose. This paper proposes a rapid review methodology that is customized to a professional organization practice which can feasibly be used by practice networks such as those of the Canadian Association for Occupational Therapy to conduct reviews. Implications. Practice networks provide a sustainable mechanism to integrate research evidence and foster communication amongst practitioners. This guide for conducting and reporting rapid reviews can be used across Occupational Therapy practice networks and similar groups to support the consistent and timely synthesis of evidence necessary to improve evidence-informed clinical decision making.

Research evidence use in local government-led public health interventions: a systematic review

BACKGROUND

Local governments play an important role in improving public health outcomes globally, critical to this work is applying the best-available research evidence. Despite considerable exploration of research use in knowledge translation literature, how research is practically applied by local governments remains poorly understood. This systematic review examined research evidence use in local government-led public health interventions. It focused on how research was used and the type of intervention being actioned.

METHODS

Quantitative and qualitative literature published between 2000 and 2020 was searched for studies that described research evidence use by local governments in public health interventions. Studies reporting interventions developed outside of local government, including knowledge translation interventions, were excluded. Studies were categorised by intervention type and their level of description of research evidence use (where 'level 1' was the highest and 'level 3' was the lowest level of detail).

FINDINGS

The search identified 5922 articles for screening. A final 34 studies across ten countries were included. Experiences of research use varied across different types of interventions. However, common themes emerged including the demand for localised research evidence, the legitimising role of research in framing public health issues, and the need for integration of different evidence sources.

CONCLUSIONS

Differences in how research was used were observed across different local government public health interventions. Knowledge translation interventions aiming to increase research use in local government settings should consider known barriers and facilitators and consider contextual factors associated with different localities and interventions.

Health technology assessment for oral health in the past decade: a scoping review

BACKGROUND

Health technology assessment (HTA) is the systematic evaluation of various properties and effects of a health technology. HTA can serve as a bridge between the world of knowledge and that of decision making, offering decision makers the best summary of scientific evidence. Scoping HTA reports in the context of dentistry can help researchers identify grey areas; help practitioners make evidence-based decisions and further initiate better policy making.

AIM

To provide an overview on HTAs pertaining to oral health and dentistry in the past decade, map the extension and scope of the methodological practices, key findings, and limitations.

METHODOLOGY

A scoping review was conducted using the Joanna Briggs Institute framework. A comprehensive search for HTA reports was done through the International Network of Agencies for Health Technology Assessment Database from January 2010 to December 2020. Consecutively, electronic databases (PubMed and Google Scholar) were searched. Finally, thirty-six reports were included in this review and analyzed.

RESULTS

A total of 709 articles were initially identified, of which thirty-six met the inclusion criteria. Reviewed HTAs focused on various specialties of dentistry worldwide. Maximum number of reports (N = 5) were related to "prosthodontics and dental implants" and technologies related to preventive dentistry were most commonly assessed (N = 4).

CONCLUSION

Functional, appropriate, and evidence-based information provided through HTA pertaining to oral health on a regular basis will enable decision makers to have enough data to make decisions on the future use of new technology, modify existing policies, accelerate its translation into practice, and ensure provision of robust dental healthcare services.

Knowledge user survey and Delphi process to inform development of a new risk of bias tool to assess systematic reviews with network meta-analysis (RoB NMA tool)

BACKGROUND

Network meta-analysis (NMA) is increasingly used in guideline development and other aspects of evidence-based decision-making. We aimed to develop a risk of bias (RoB) tool to assess NMAs (RoB NMA tool). An international steering committee recommended that the RoB NMA tool to be used in combination with the Risk of Bias in Systematic reviews (ROBIS) tool (i.e. because it was designed to assess biases only) or other similar quality appraisal tools (eg, A MeaSurement Tool to Assess systematic Reviews 2 [AMSTAR 2]) to assess quality of systematic reviews. The RoB NMA tool will assess NMA biases and limitations regarding how the analysis was planned, data were analysed and results were presented, including the way in which the evidence was assembled and interpreted.

OBJECTIVES

Conduct (a) a Delphi process to determine expert opinion on an item's inclusion and (b) a knowledge user survey to widen its impact.

DESIGN

Cross-sectional survey and Delphi process.

METHODS

Delphi panellists were asked to rate whether items should be included. All agreed-upon item were included in a second round of the survey (defined as 70% agreement). We surveyed knowledge users' views and preferences about the importance, utility and willingness to use the RoB NMA tool to evaluate evidence in practice and in policymaking. We included 12 closed and 10 open-ended questions, and we followed a knowledge translation plan to disseminate the survey through social media and professional networks.

RESULTS

22 items were entered into a Delphi survey of which 28 respondents completed round 1, and 22 completed round 2. Seven items did not reach consensus in round 2. A total of 298 knowledge users participated in the survey (14% respondent rate). 75% indicated that their organisation produced NMAs, and 78% showed high interest in the tool, especially if they had received adequate training (84%). Most knowledge users and Delphi panellists preferred a tool to assess both bias in individual NMA results and authors' conclusions. Response bias in our sample is a major limitation as knowledge users working in high-income countries were more represented. One of the limitations of the Delphi process is that it depends on the purposive selection of experts and their availability, thus limiting the variability in perspectives and scientific disciplines.

CONCLUSIONS

This Delphi process and knowledge user survey informs the development of the RoB NMA tool.

Strategies to approach the judicialization of health in Brazil: an evidence brief

Abstract This article seeks to identify and discuss evidence-informed options to address the judicialization of health. The Supporting Policy Relevant Reviews and Trials Tools were used to define the problem and the search strategy, which was carried out in the following databases: PubMed, Health Systems Evidence, Campbell, Cochrane Collaboration, Rx for Change Database, and PDQ-Evidence. Selection and assessment of methodological quality was performed by two independent reviewers. The results were presented in a narrative synthesis. This study selected 19 systematic reviews that pointed out four strategies to address the judicialization of health in Brazil: 1) Rapid response service, 2) Continuous education program, 3) Mediation service between the parties involved, and 4) Adoption of a computer-based, online decision-making support tool and patient-mediated interventions. This study therefore presented and characterized four options that can be considered to address the judicialization of health. The implementation of these options must ensure the participation of different actors, reflecting on different contexts and the impact on the health system. The availability of human and financial resources and the training of teams are critical points for the successful implementation of the options.

Strategies to approach the judicialization of health in Brazil: an evidence brief

This article seeks to identify and discuss evidence-informed options to address the judicialization of health. The Supporting Policy Relevant Reviews and Trials Tools were used to define the problem and the search strategy, which was carried out in the following databases: PubMed, Health Systems Evidence, Campbell, Cochrane Collaboration, Rx for Change Database, and PDQ-Evidence. Selection and assessment of methodological quality was performed by two independent reviewers. The results were presented in a narrative synthesis. This study selected 19 systematic reviews that pointed out four strategies to address the judicialization of health in Brazil: 1) Rapid response service, 2) Continuous education program, 3) Mediation service between the parties involved, and 4) Adoption of a computer-based, online decision-making support tool and patient-mediated interventions. This study therefore presented and characterized four options that can be considered to address the judicialization of health. The implementation of these options must ensure the participation of different actors, reflecting on different contexts and the impact on the health system. The availability of human and financial resources and the training of teams are critical points for the successful implementation of the options.

Is equity considered in systematic reviews of interventions for mitigating social isolation and loneliness in older adults?

BACKGROUND

Social isolation and loneliness affect one in four older adults in many regions around the world. Social isolation and loneliness are shown to be associated with declines in physical and mental health. Intersecting social determinants of health influence both the risk of being socially isolated and lonely as well as the access and uptake of interventions. Our objective is to evaluate what evidence is available within systematic reviews on how to mitigate inequities in access to and effectiveness of interventions.

METHODS

We performed an overview of reviews following methods of the Cochrane Handbook for Overviews of Reviews. We selected systematic reviews of effectiveness of interventions aimed at mitigating social isolation and loneliness in older adults (aged 60 or above) published in the last 10 years. In addition, we assessed all primary studies from the most recent systematic review with a broad intervention focus. We searched MEDLINE, EMBASE, PsycINFO, CINAHL, and Scopus in collaboration with a librarian scientist. We used a structured framework called PROGRESS-Plus to assess the reporting and consideration of equity. PROGRESS-Plus stands for place of residence, race/ethnicity/culture/language, occupation, gender or sex, religion, education, socioeconomic status (SES), social capital, while "plus" stands for additional factors associated with discrimination and exclusion such as age, disability, and sexual orientation. We assessed whether PROGRESS-Plus factors were reported in description of the population, examination of differential effects, or discussion of applicability or limitations.

RESULTS

We identified and assessed 17 eligible systematic reviews. We assessed all 23 primary studies from the most recent systematic review with a broad intervention focus. All systematic reviews and primary studies described the population by one or more PROGRESS-Plus factor, most commonly across place of residence and age, respectively. None of the reviews and five primary studies examined differential effects across one or more PROGRESS-Plus dimension. Nine reviews and four primary studies discussed applicability or limitations of their findings by at least one PROGRESS-Plus factor.

CONCLUSIONS

Although we know that social isolation and loneliness are worse for the poorest and most socially disadvantaged older adults, the existing evidence base lacks details on how to tailor interventions for these socially disadvantaged older people.

User experiences of structured stakeholder engagement to consider transferability: The TRANSFER approach

BACKGROUND

Systematic reviews are increasingly used to inform decision-making in health, education, social care and environmental protection. However, decision makers still experience barriers to using reviews, including not knowing how findings might translate to their own contexts, and lack of collaboration with systematic review authors. The TRANSFER approach is a novel method that aims to support review authors to systematically and transparently collaborate with stakeholders to consider context and the transferability of review findings from the beginning of the review process. Such collaboration is intended to improve the usefulness and relevance of review findings for decision makers.

OBJECTIVES

We aim to explore the user experience of the TRANSFER approach conversation guide, and in doing so gain a better understanding of the role and perceived value of stakeholder engagement in systematic reviews for informed decision-making.

METHODS

We conducted four user tests of groups using the guide, organized around simulated meetings between review authors and stakeholders. Review authors led the meeting using the TRANSFER approach conversation guide. We audio-recorded and observed the meetings, collected feedback forms and conducted semi-structured interviews with review authors following the meeting. We analysed the data using framework analysis to examine the user experience of the TRANSFER approach conversation guide and of stakeholder engagement more generally.

RESULTS

Seventeen participants in four user groups participated in the user tests. Most participants were generally positive toward the structured approach using the conversation guide, and felt it would be useful in systematic review projects. We observed examples of misunderstanding of the terminology included in the guide, and received multiple suggestions for how to make the conversation guide more user friendly. We observed numerous challenges related to the hypothetical nature of a user test, including lack of familiarity with the review question/topic among participants and lack of preparation for the meeting.

CONCLUSIONS

Review authors and stakeholders are positive toward using a structured approach to guide collaboration within the context of a systematic review. The TRANSFER conversation guide helps participants to discuss the review question and context in a structured way. Such structured collaboration could help to improve the usefulness and relevance of systematic reviews for decision making by improving the review question, inclusion criteria and consideration of transferability of review findings. The conversation guide needs to be modified to improve user experience. Further research is needed to explore stakeholder collaboration and the use of the TRANSFER conversation guide in systematic review processes.

UK nurses' and midwives' experiences of healthful leadership practices during the COVID-19 pandemic: A rapid realist review

AIM

We aim to explore healthful leadership practices in nursing and midwifery evident within the COVID-19 pandemic in the United Kingdom, the contextual facilitators, barriers and outcomes.

BACKGROUND

Globally, the health and care sectors are under pressure and despite nurses and other professionals, demonstrating resilience and resourcefulness in the COVID-19 pandemic; this has negatively impacted on their health and wellbeing and on patient care.

EVALUATION

Two searches were conducted in July 2021 and December 2021. Inclusion/exclusion criteria were identified to refine the search, including papers written since the beginning of the pandemic in 2020. A total of 38 papers were included principally from the United States and United Kingdom. Ten were research papers; the others were commentaries, opinion pieces and editorials. MS Teams literature repository was created. A unique critical appraisal tool was devised to capture contexts, mechanisms and outcomes whilst reflecting more standardized tools, that is, the Critical Appraisal Skills Programme and the Authority, Accuracy, Coverage, Objectivity and Date tool for reviewing grey literature to refine the search further.

KEY ISSUES

Six tentative theories of healthful leadership emerged from the literature around leadership strategies, which are relational, being visible and present; being open and engaging; caring for self and others; embodying values; being prepared and preparing others; and using available information and support. Contextual factors that enable healthful leadership practices are in the main, created by leaders' values, attributes and style. The literature suggests that leaders who embody values of compassion, empathy, courage and authenticity create conditions for positive and healthful relations between leaders and others. Nurse and midwives' voices are however absent from the literature in this review.

CONCLUSION

Current available literature would suggest healthful leadership practices are not prioritized by nurse leaders. Perspectives of nurses' and midwives' about the impact of such practices on their wellbeing is also missing. Tentative theories are offered as a means of identifying healthful leadership strategies, the context that enable these and potential outcomes for nurses and midwives. These will be explored in phase two of this study.

IMPLICATIONS FOR NURSING MANAGEMENT

Nurse leaders must be adequately prepared to create working environments that support nurses' and midwives' wellbeing, so that they may be able to provide high-quality care. Ensuring a supportive organizational culture, which embodies the values of healthfulness, may help to mitigate the impact of the COVID-19 pandemic on nurses' and midwives' wellbeing in the immediate aftermath and going forward.

Mapping the Qualitative Evidence Base on the Use of Research Evidence in Health Policy-Making: A Systematic Review

BACKGROUND

The use of research evidence in health policy-making is a popular line of inquiry for scholars of public health and policy studies, with qualitative methods constituting the dominant strategy in this area. Research on this subject has been criticized for, among other things, disproportionately focusing on high-income countries; overemphasizing 'barriers and facilitators' related to evidence use to the neglect of other, less descriptive concerns; relying on descriptive, rather than in-depth explanatory designs; and failing to draw on insights from political/policy studies theories and concepts. We aimed to comprehensively map the global, peer-reviewed qualitative literature on the use of research evidence in health policy-making and to provide a descriptive overview of the geographic, temporal, methodological, and theoretical characteristics of this body of literature.

METHODS

We conducted a systematic review following PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. We searched nine electronic databases, hand-searched 11 health- and policy-related journals, and systematically scanned the reference lists of included studies and previous reviews. No language, date or geographic limitations were imposed.

RESULTS

The review identified 319 qualitative studies on a diverse array of topics related to the use of evidence in health policy-making, spanning 72 countries and published over a nearly 40 year period. A majority of these studies were conducted in high-income countries, but a growing proportion of the research output in this area is now coming from low- and middle-income countries, especially from sub-Saharan Africa. While over half of all studies did not use an identifiable theory or framework, and only one fifth of studies used a theory or conceptual framework drawn from policy studies or political science, we found some evidence that theory-driven and explanatory (eg, comparative case study) designs are becoming more common in this literature. Investigations of the barriers and facilitators related to evidence use constitute a large proportion but by no means a majority of the work in this area.

CONCLUSION

This review provides a bird's eye mapping of the peer reviewed qualitative research on evidence-to-policy processes, and has identified key features of - and gaps within - this body of literature that will hopefully inform, and improve, research in this area moving forward.

Non-pharmacological interventions for autism spectrum disorder in children: an overview of systematic reviews

OBJECTIVE

To assess the effectiveness of non-pharmacological interventions for the treatment of autism spectrum disorder (ASD) in children.

DESIGN

Overview of systematic reviews (SRs).

PARTICIPANTS

Children aged 12 years and under with ASD.

SEARCH METHODS

In October 2021, we searched Cochrane Central Register of Controlled Trials, MEDLINE, Embase, PsycINFO and Epistemonikos placing no restrictions on language or date of publication.

INTERVENTIONS

17 non-pharmacological interventions compared with placebo, no-treatment (including waiting list) or other interventions (ie, usual care, as defined by the authors of each study).

DATA COLLECTION AND ANALYSIS

We rated the methodological quality of the included SRs using A Measurement Tool to Assess Systematic Reviews (AMSTAR 2). We reported the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) certainty of the evidence (CoE) according to the analysis conducted by the authors of the included SRs.

MAIN OUTCOME MEASURES

A multidisciplinary group of experts agreed on analysing nine critical outcomes evolving core and non-core ASD symptoms.

PUBLIC AND PATIENT INVOLVEMENT STATEMENT

Organisations of parents of children with ASD participated in external revision of the final version of the report.

RESULTS

We identified 52 reports that were within our scope, of which 48 were excluded for various reasons. After excluding less reliable SRs, we included four SRs. Non-pharmacological interventions (ie, Early Intensive Behavioural Intervention, Applied Behaviour Analysis, Picture Exchange Communication System and Naturalistic Developmental Behavioural Interventions) may have favourable effects on some core outcomes including language, social and functioning, play or daily living skills in children with ASD (with either no GRADE assessment, very low or low CoE). In addition, we identified a lack of report for other key outcomes in the included SRs (ie, restricted, repetitive behaviour; play and sensory processing).

CONCLUSIONS

Synthesised evidence regarding the efficacy of non-pharmacological interventions for children with ASD is scarce. High-quality SRs addressing the variety of both non-pharmacological interventions and relevant outcomes are needed.

PROSPERO REGISTRATION NUMBER

CRD42020206535.

Individual and institutional capacity-building for evidence-informed health policy-making in Iran: a mix of local and global evidence

Background

Providing valid evidence to policy-makers is a key factor in the development of evidence-informed policy-making (EIPM). This study aims to review interventions used to promote researchers’ and knowledge-producing organizations’ knowledge and skills in the production and translation of evidence to policy-making and explore the interventions at the individual and institutional level in the Iranian health system to strengthen EIPM.

Methods

The study was conducted in two main phases: a systematic review and a qualitative study. First, to conduct the systematic review, the PubMed and Scopus databases were searched. Quality appraisal was done using the Joanna Briggs Institute checklists. Second, semi-structured interviews and document review were used to collect local data. Purposive sampling was used and continued until data saturation. A qualitative content analysis approach was used for data analysis.

Results

From a total of 11,514 retrieved articles, 18 papers were eligible for the analysis. Based on the global evidence, face-to-face training workshops for researchers was the most widely used intervention for strengthening researchers’ capacity regarding EIPM. Target audiences in almost all of the training programmes were researchers. Setting up joint training sessions that helped empower researchers in understanding the needs of health policy-makers had a considerable effect on strengthening EIPM. Based on the local collected evidence, the main interventions for individual and institutional capacity-building were educational and training programmes or courses related to the health system, policy-making and policy analysis, and research cycle management. To implement the individual and institutional interventions, health system planners and authorities and the community were found to have a key role as facilitating factors.

Conclusion

The use of evidence-based interventions for strengthening research centres, such as training health researchers on knowledge translation and tackling institutional barriers that can prevent well-trained researchers from translating their knowledge, as well as the use of mechanisms and networks for effective interactions among policy-makers at the macro and meso (organizational) level and the research centre, will be constructive for individual and institutional capacity-building. The health system needs to strengthen its strategic capacity to facilitate an educational and training culture in order to motivate researchers in producing appropriate evidence for policy-makers.

How effects on health equity are assessed in systematic reviews of interventions

BACKGROUND

Enhancing health equity is endorsed in the Sustainable Development Goals. The failure of systematic reviews to consider potential differences in effects across equity factors is cited by decision-makers as a limitation to their ability to inform policy and program decisions.  OBJECTIVES: To explore what methods systematic reviewers use to consider health equity in systematic reviews of effectiveness.

SEARCH METHODS

We searched the following databases up to 26 February 2021: MEDLINE, PsycINFO, the Cochrane Methodology Register, CINAHL, Education Resources Information Center, Education Abstracts, Criminal Justice Abstracts, Hein Index to Foreign Legal Periodicals, PAIS International, Social Services Abstracts, Sociological Abstracts, Digital Dissertations and the Health Technology Assessment Database. We searched SCOPUS to identify articles that cited any of the included studies on 10 June 10 2021. We contacted authors and searched the reference lists of included studies to identify additional potentially relevant studies.

SELECTION CRITERIA

We included empirical studies of cohorts of systematic reviews that assessed methods for measuring effects on health inequalities. We define health inequalities as unfair and avoidable differences across socially stratifying factors that limit opportunities for health. We operationalised this by assessing studies which evaluated differences in health across any component of the PROGRESS-Plus acronym, which stands for Place of residence, Race/ethnicity/culture/language, Occupation, Gender or sex, Religion, Education, Socioeconomic status, Social capital. "Plus" stands for other factors associated with discrimination, exclusion, marginalisation or vulnerability such as personal characteristics (e.g. age, disability), relationships that limit opportunities for health (e.g. children in a household with parents who smoke) or environmental situations which provide limited control of opportunities for health (e.g. school food environment).

DATA COLLECTION AND ANALYSIS

Two review authors independently extracted data using a pre-tested form. Risk of bias was appraised for included studies according to the potential for bias in selection and detection of systematic reviews.  MAIN RESULTS: In total, 48,814 studies were identified and the titles and abstracts were screened in duplicate. In this updated review, we identified an additional 124 methodological studies published in the 10 years since the first version of this review, which included 34 studies. Thus, 158 methodological studies met our criteria for inclusion. The methods used by these studies focused on evidence relevant to populations experiencing health inequity (108 out of 158 studies), assess subgroup analysis across PROGRESS-Plus (26 out of 158 studies), assess analysis of a gradient in effect across PROGRESS-Plus (2 out of 158 studies) or use a combination of subgroup analysis and focused approaches (20 out of 158 studies). The most common PROGRESS-Plus factors assessed were age (43 studies), socioeconomic status in 35 studies, low- and middle-income countries in 24 studies, gender or sex in 22 studies, race or ethnicity in 17 studies, and four studies assessed multiple factors across which health inequity may exist. Only 16 studies provided a definition of health inequity. Five methodological approaches to consider health equity in systematic reviews of effectiveness were identified: 1) descriptive assessment of reporting and analysis in systematic reviews (140 of 158 studies used a type of descriptive method); 2) descriptive assessment of reporting and analysis in original trials (50 studies); 3) analytic approaches which assessed differential effects across one or more PROGRESS-Plus factors (16 studies); 4) applicability assessment (25 studies) and 5) stakeholder engagement (28 studies), which is a new finding in this update and examines the appraisal of whether relevant stakeholders with lived experience of health inequity were included in the design of systematic reviews or design and delivery of interventions. Reporting for both approaches (analytic and applicability) lacked transparency and was insufficiently detailed to enable the assessment of credibility.

AUTHORS' CONCLUSIONS

There is a need for improvement in conceptual clarity about the definition of health equity, describing sufficient detail about analytic approaches (including subgroup analyses) and transparent reporting of judgments required for applicability assessments in order to consider health equity in systematic reviews of effectiveness.

Supporting Informal Caregivers of People With Dementia in Cost-Effective Ways: A Systematic Review and Meta-Analysis

OBJECTIVES

Caring for persons with dementia is a heavy burden for informal caregivers. This study aimed to appraise the economic evidence of interventions supporting informal caregivers of people with dementia.

METHODS

Literature was searched, and trial-based studies evaluating the costs and effects of interventions supporting informal caregivers of people with dementia were included. Cost data were analyzed from both healthcare and societal perspectives. Random-effects models were used to synthesize cost and effect data, based on mean differences (MDs) or standardized MDs.

RESULTS

Of 33 eligible studies identified from 48 588 records, 14 (42.4%) showed net savings in total cost regardless of analytical perspectives. Among 22 studies included in meta-analyses, caregiver-focused psychosocial interventions showed improvements in caregivers' psychological health (n = 4; standardized MD 0.240; 95% confidence interval 0.094-0.387); nevertheless, the increases in societal cost were significant (n = 5; MD 3144; 95% confidence interval 922-5366). Psychological intervention and behavioral management engaging patient-caregiver dyads showed positive effects on caregivers' subjective burden, also with increases in total cost. Subgroup analyses indicated that the inclusion of different intervention components, the caregiver characteristics, and the follow-up periods could affect the costs and effects of interventions supporting informal caregivers.

CONCLUSIONS

Psychosocial interventions directed at informal caregivers and dyad-based psychological and behavioral interventions are effective but also expensive. The use of these interventions depends on the society's willingness to pay. More comprehensive economic evidence of interventions supporting informal caregivers is required, and the design of intervention should focus more on different intervention components, characteristics of patients and caregivers, and healthcare systems.

Policymakers' Research Capacities, Engagement, and Use of Research in Public Health Policymaking

The use of research in public health policymaking is one of the prerequisites for successfully implemented health policies which have better population health as an outcome. This policy process is influenced by the actors involved under the policy umbrella, with inter-related contextual factors and specific structural and institutional circumstances. Our study investigates how policymakers’ research capacities influence the use of research in the health policy process and identify areas where capacity-building interventions give the most meaning and impact. Furthermore, we investigate policymakers’ research engagement and use this to inform public health policy in the public sector in Denmark. We collect and report data using Seeking, Engaging with, and Evaluation Research (SEER) methodology. Policymakers are reported to have research capacity, but it is questionable how those competences have actually been used in policymaking. Decision-makers were often not aware or did not know about the existing organizational tools and systems for research engagement and use and two third of respondents had not been part of any research activities or had any collaboration with researchers. Overall, research use in public health policymaking and evaluation was limited. As a conclusion, we propose that capacity-building interventions for increasing research use and collaboration in EIPM should be context-oriented, measurable, and sustainable in developing individual and organizational competences.

Epidemiology and reporting characteristics of preclinical systematic reviews

In an effort to better utilize published evidence obtained from animal experiments, systematic reviews of preclinical studies are increasingly more common-along with the methods and tools to appraise them (e.g., SYstematic Review Center for Laboratory animal Experimentation [SYRCLE's] risk of bias tool). We performed a cross-sectional study of a sample of recent preclinical systematic reviews (2015-2018) and examined a range of epidemiological characteristics and used a 46-item checklist to assess reporting details. We identified 442 reviews published across 43 countries in 23 different disease domains that used 26 animal species. Reporting of key details to ensure transparency and reproducibility was inconsistent across reviews and within article sections. Items were most completely reported in the title, introduction, and results sections of the reviews, while least reported in the methods and discussion sections. Less than half of reviews reported that a risk of bias assessment for internal and external validity was undertaken, and none reported methods for evaluating construct validity. Our results demonstrate that a considerable number of preclinical systematic reviews investigating diverse topics have been conducted; however, their quality of reporting is inconsistent. Our study provides the justification and evidence to inform the development of guidelines for conducting and reporting preclinical systematic reviews.

[Promotion of physical activity among the elderly as an example of knowledge translation: How do scientific findings enter community practice?]

BACKGROUND

Scientific findings can be an important source of knowledge for public health stakeholders involved in promoting physical activity, but several barriers hinder their use. Knowledge translation can simplify this process, but it requires the understanding of the stakeholder's needs.

OBJECTIVES

This qualitative study aims to describe how public health stakeholders access information and scientific findings, identify possible barriers, and highlight the needs of stakeholders in terms of presentation and processing.

MATERIALS AND METHODS

Semi-structured interviews were conducted with twelve local- and state-level stakeholders from North Rhine-Westphalia, Saxony-Anhalt, and Thuringia working in the area of physical activity promotion. The interviewees were selected through purposive sampling. The interviews were evaluated using qualitative content analysis.

RESULTS

The benefits of scientific findings are emphasized by the interviewees, but a lack of resources in combination with a flood of information, high complexity, and technical jargon complicate their application. There is a need for tailored preparation in the form of summaries, filter functions, elaboration of practice-relevant elements, and ways of provision.

CONCLUSIONS

To achieve successful knowledge translation, collaboration and interactive exchange between researchers, policymakers, and practice as well as a demand-oriented processing of scientific findings are central. Networking and bundling of knowledge on a platform are important tasks for the future.

[Establishing evidence-based prevention and health promotion: criteria for evidence-based interventions and necessary organizational requirements and capacities]

Für die Umsetzung des Präventionsgesetzes in einem sich entwickelnden System Prävention und Gesundheitsförderung (PGF) ist die Anforderung der Evidenzbasierung formuliert. Vor diesem Hintergrund stellt sich die Frage, welche Schritte, Prozesse und Vorgehensweisen in diesem System benötigt werden, um der Anforderung zunehmend gerecht zu werden. Dieser Übersichtsartikel diskutiert für Deutschland, wie evidenzbasierte Maßnahmen in der Praxis operationalisiert werden können und welche organisationalen Rahmenbedingungen und Kapazitäten für ein evidenzbasiertes Handeln von AkteurInnen notwendig sind.

Aufbauend auf internationalen wissenschaftlichen Erkenntnissen und dem Memorandum Evidenzbasierte Prävention und Gesundheitsförderung der Bundeszentrale für gesundheitliche Aufklärung (BZgA) wird zunächst das Verständnis von evidenzbasierten Maßnahmen erläutert und im Weiteren werden Elemente zur Umsetzung von mehr Evidenzbasierung skizziert.

Neben der transparenten und einheitlichen Darstellung in Datenbanken und Empfehlungen ist es notwendig, bei EntscheidungsträgerInnen in Praxis und Politik ein gemeinsames Verständnis von evidenzbasierten Interventionen und von Anforderungen für eine Evaluation, die Evidenzbasierung sichert, zu schaffen. Darüber hinaus kann evidenzbasiertes Handeln von EntscheidungsträgerInnen gefördert werden durch Wertschätzung gegenüber Evidenzbasierung in ihren Organisationen, durch Gewährleistung eines regelhaften Zugangs zu Evidenzdatenbanken, durch verbesserte Kompetenzen in Bezug auf Interpretation von Evidenz und durch eine systematische Zusammenarbeit mit der Wissenschaft.

Mehr Evidenzbasierung ist eine Voraussetzung für die nachhaltige Verankerung von PGF als fünfte Säule des Gesundheitssystems.

Use of a quantitative data report in a hypothetical decision scenario for health policymaking: a computer-assisted laboratory study

BACKGROUND

Quantitative data reports are widely produced to inform health policy decisions. Policymakers are expected to critically assess provided information in order to incorporate the best available evidence into the decision-making process. Many other factors are known to influence this process, but little is known about how quantitative data reports are actually read. We explored the reading behavior of (future) health policy decision-makers, using innovative methods.

METHODS

We conducted a computer-assisted laboratory study, involving starting and advanced students in medicine and health sciences, and professionals as participants. They read a quantitative data report to inform a decision on the use of resources for long-term care in dementia in a hypothetical decision scenario. Data were collected through eye-tracking, questionnaires, and a brief interview. Eye-tracking data were used to generate 'heatmaps' and five measures of reading behavior. The questionnaires provided participants' perceptions of understandability and helpfulness as well as individual characteristics. Interviews documented reasons for attention to specific report sections. The quantitative analysis was largely descriptive, complemented by Pearson correlations. Interviews were analyzed by qualitative content analysis.

RESULTS

In total, 46 individuals participated [students (85%), professionals (15%)]. Eye-tracking observations showed that the participants spent equal time and attention for most parts of the presented report, but were less focused when reading the methods section. The qualitative content analysis identified 29 reasons for attention to a report section related to four topics. Eye-tracking measures were largely unrelated to participants' perceptions of understandability and helpfulness of the report.

CONCLUSIONS

Eye-tracking data added information on reading behaviors that were not captured by questionnaires or interviews with health decision-makers.

Cost-effectiveness of prehabilitation prior to elective surgery compared to usual preoperative care: protocol for a systematic review of economic evaluations

INTRODUCTION

Preoperative functional capacity is an important predictor of postoperative outcomes. Prehabilitation aims to optimise patients' functional capacity before surgery to improve postoperative outcomes. As prolonged hospital stay and postoperative complications present an avoidable use of healthcare resources, prehabilitation might also save costs.The aim of this systematic review is to investigate the cost-effectiveness of prehabilitation programmes for patients awaiting elective surgery compared with usual preoperative care. The results will be useful to inform decisions about the implementation of prehabilitation programmes and the design of future economic evaluations of prehabilitation programmes.

METHODS AND ANALYSIS

We will search PubMed, Embase, the Centre for Reviews and Dissemination Database, the WHO International Clinical Trials Registry Platform and ClinicalTrials.gov for full or partial economic evaluations of preoperative prehabilitation programmes conducted in any population compared with usual preoperative care. Studies will be included regardless of the type, design and perspective of the economic evaluation, and their publication year, language or status. Initial searches were performed between 30 April and 4 May 2020.Study selection, data extraction and assessment of the included studies' risk of bias and methodological quality will initially be performed by two independent reviewers and, if agreement was sufficiently high, by one reviewer. We will extract data regarding the included studies' basic characteristics, economic evaluation methods and cost-effectiveness results.A narrative synthesis will be performed. The primary endpoint will be cost-effectiveness based on cost-utility analyses. We will discuss heterogeneity between the studies and assess the risk of publication bias. The certainty of the evidence will be determined using the Grading of Recommendations, Assessment, Development and Evaluation approach.

ETHICS AND DISSEMINATION

Ethics approval is not required as the systematic review will not involve human participants. We plan to present our findings at scientific conferences, pass them on to relevant stakeholder organisations and publish them in a peer-reviewed journal.

PROSPERO REGISTRATION NUMBER

CRD42020182813.

Assessing how information is packaged in rapid reviews for policy-makers and other stakeholders: a cross-sectional study

Background

Rapid reviews (RRs) are useful products to healthcare policy-makers and other stakeholders, who require timely evidence. Therefore, it is important to assess how well RRs convey useful information in a format that is easy to understand so that decision-makers can make best use of evidence to inform policy and practice.

Methods

We assessed a diverse sample of 103 RRs against the BRIDGE criteria, originally developed for communicating clearly to support healthcare policy-making. We modified the criteria to increase assessability and to align with RRs. We identified RRs from key database searches and through searching organisations known to produce RRs. We assessed each RR on 26 factors (e.g. organisation of information, lay language use). Results were descriptively analysed. Further, we explored differences between RRs published in journals and those published elsewhere.

Results

Certain criteria were well covered across the RRs (e.g. all aimed to synthesise research evidence and all provided references of included studies). Further, most RRs provided detail on the problem or issue (96%; n = 99) and described methods to conduct the RR (91%; n = 94), while several addressed political or health systems contexts (61%; n = 63). Many RRs targeted policy-makers and key stakeholders as the intended audience (66%; n = 68), yet only 32% (n = 33) involved their tacit knowledge, while fewer (27%; n = 28) directly involved them reviewing the content of the RR. Only six RRs involved patient partners in the process. Only 23% (n = 24) of RRs were prepared in a format considered to make information easy to absorb (i.e. graded entry) and 25% (n = 26) provided specific key messages. Readability assessment indicated that the text of key RR sections would be hard to understand for an average reader (i.e. would require post-secondary education) and would take 42 (± 36) minutes to read.

Conclusions

Overall, conformity of the RRs with the modified BRIDGE criteria was modest. By assessing RRs against these criteria, we now understand possible ways in which they could be improved to better meet the information needs of healthcare decision-makers and their potential for innovation as an information-packaging mechanism. The utility and validity of these items should be further explored.

Protocol availability

The protocol, published on the Open Science Framework, is available at: osf.io/68tj7

What works, why and how? A scoping review and logic model of rural clinical placements for allied health students

Background

Allied health services are core to the improvement in health outcomes for remote and rural residents. Substantial infrastructure has been put into place to facilitate rural work-ready allied health practitioners, yet it is difficult to understand or measure how successful this is and how it is facilitated.

Methods

A scoping review and thematic synthesis of the literature using program logic was undertaken to identify and describe the contexts, mechanisms and outcomes of successful models of rural clinical placements for allied health students. This involved all empirical literature examining models of regional, rural and remote clinical placements for allied health students between 1995 and 2019.

Results

A total of 292 articles were identified; however, after removal of duplicates and article screening, 18 were included in the final synthesis. Australian papers dominated the evidence base (n = 11). Drivers for rural allied health clinical placements include: attracting allied health students to the rural workforce; increasing the number of allied health clinical placements available; exposing students to and providing skills in rural and interprofessional practice; and improving access to allied health services in rural areas. Depending on the placement model, a number of key mechanisms were identified that facilitated realisation of these drivers and therefore the success of the model. These included: support for students; engagement, consultation and partnership with key stakeholders and organisations; and regional coordination, infrastructure and support. Placement success was measured in terms of student, rural, community and/or program outcomes. Although the strength and quality of the evidence was found to be low, there is a trend for placements to be more successful when the driver for the placement is specifically reflected in the structure of the placement model and outcomes measured. This was seen most effectively in placement models that were driven by the need to meet rural community needs and upskill students in interprofessional rural practice.

Conclusion

This study identifies the factors that can be manipulated to ensure more successful models of allied health rural clinical placements and provides an evidence based framework for improved planning and evaluation.

Assessing the format and content of journal published and non-journal published rapid review reports: A comparative study

BACKGROUND

As production of rapid reviews (RRs) increases in healthcare, knowing how to efficiently convey RR evidence to various end-users is important given they are often intended to directly inform decision-making. Little is known about how often RRs are produced in the published or unpublished domains, and what and how information is structured.

OBJECTIVES

To compare and contrast report format and content features of journal-published (JP) and non-journal published (NJP) RRs.

METHODS

JP RRs were identified from key databases, and NJP RRs were identified from a grey literature search of 148 RR producing organizations and were sampled proportionate to cluster size by organization and product type to match the JP RR group. We extracted and formally compared 'how' (i.e., visual arrangement) and 'what' information was presented.

RESULTS

We identified 103 RRs (52 JP and 51 NJP) from 2016. A higher percentage of certain features were observed in JP RRs compared to NJP RRs (e.g., reporting authors; use of a traditional journal article structure; section headers including abstract, methods, discussion, conclusions, acknowledgments, conflict of interests, and author contributions; and use of figures (e.g., Study Flow Diagram) in the main document). For NJP RRs, a higher percentage of features were observed (e.g., use non-traditional report structures; bannering of executive summary sections and appendices; use of typographic cues; and including outcome tables). NJP RRs were more than double in length versus JP RRs. Including key messages was uncommon in both groups.

CONCLUSIONS

This comparative study highlights differences between JP and NJP RRs. Both groups may benefit from better use of plain language, and more clear and concise design. Alternative innovative formats and end-user preferences for content and layout should be studied further with thought given to other considerations to ensure better packaging of RR results to facilitate uptake into policy and practice.

STUDY REGISTRATION

The full protocol is available at: https://osf.io/29xvk/.

A critical interpretive synthesis of the roles of midwives in health systems

BACKGROUND

Midwives' roles in sexual and reproductive health and rights continues to evolve. Understanding the profession's role and how midwives can be integrated into health systems is essential in creating evidence-informed policies. Our objective was to develop a theoretical framework of how political system factors and health systems arrangements influence the roles of midwives within the health system.

METHODS

A critical interpretive synthesis was used to develop the theoretical framework. A range of electronic bibliographic databases (CINAHL, EMBASE, Global Health database, HealthSTAR, Health Systems Evidence, MEDLINE and Web of Science) was searched through to 14 May 2020 as were policy and health systems-related and midwifery organisation websites. A coding structure was created to guide the data extraction.

RESULTS

A total of 4533 unique documents were retrieved through electronic searches, of which 4132 were excluded using explicit criteria, leaving 401 potentially relevant records, in addition to the 29 records that were purposively sampled through grey literature. A total of 100 documents were included in the critical interpretive synthesis. The resulting theoretical framework identified the range of political and health system components that can work together to facilitate the integration of midwifery into health systems or act as barriers that restrict the roles of the profession.

CONCLUSIONS

Any changes to the roles of midwives in health systems need to take into account the political system where decisions about their integration will be made as well as the nature of the health system in which they are being integrated. The theoretical framework, which can be thought of as a heuristic, identifies the core contextual factors that governments can use to best leverage their position when working to improve sexual and reproductive health and rights.

What steps can researchers take to increase research uptake by policymakers? A case study in China

Empirical analysis of the connections between research and health policymaking is scarce in middle-income countries. In this study, we focused on a national multidrug-resistant tuberculosis (TB) healthcare provider training programme in China as a case study to examine the role that research plays in influencing health policy. We specifically focused on the factors that influence research uptake within the complex Chinese policymaking process. Qualitative data were collected from 34 participants working at multilateral organizations, funding agencies, academia, government agencies and hospitals through 14 in-depth interviews and 2 focus group discussions with 10 participants each. Themes were derived inductively from data and grouped based on the 'Research and Policy in Developing countries' framework developed by the Overseas Development Institute. We further classified how actors derive their power to influence policy decisions following the six sources of power identified by Sriram et al. We found that research uptake by policymakers in China is influenced by perceived importance of the health issues addressed in the research, relevance of the research to policymakers' information needs and government's priorities, the research quality and the composition of the research team. Our analysis identified that international donors are influential in the TB policy process through their financial power. Furthermore, the dual roles of two government agencies as both evidence providers and actors who have the power to influence policy decisions through their technical expertise make them natural intermediaries in the TB policy process. We concluded that resolving the conflict of interests between researchers and policymakers, as suggested in the 'two-communities theory', is not enough to improve evidence use by policymakers. Strategies such as framing research to accommodate the fast-changing policy environment and making alliances with key policy actors can be effective to improve the communication of research findings into the policy process, particularly in countries undergoing rapid economic and political development.

Innovations in framework synthesis as a systematic review method

Framework synthesis is one systematic review method employed to address health care practice and policy. Adapted from framework analysis methods, it has been used increasingly, using both qualitative and mixed-method systematic review methods. This article demonstrates a spectrum of approaches to framework synthesis that are dependent on the extent to which theory is tentative, emergent, refined, or established; and that stakeholder involvement may help to understand the topic's complexity where theory is more nascent. The choice of approach depends on the degree of match with existing theories and, in the absence of existing theory, the scale and heterogeneity of the literature to be managed.

Investigating the process of evidence-informed health policymaking in Bangladesh: a systematic review

Over the last four decades, Bangladesh has made considerable improvements in population health, this is in part due to the use of evidence to inform policymaking. This systematic review aims to better understand critical factors that have facilitated the diffusion of scientific evidence into multiple phases of health policymaking in Bangladesh. To do this an existing policy framework designed by Shiffman and Smith in 2007, was used to extract and synthesize data from selected policy analyses. This framework was used to ensure the content, context and actors involved with evidence-informed policymaking were considered in each case where research had helped shape a health policy. The 'PRISMA Checklist' was employed to design pre-specified eligibility criteria for the selection of information sources, search strategy, inclusion and exclusion criteria, and process of data extraction and synthesis. Through our systematic search conducted from February to May 2017, we initially identified 1859 articles; after removal of duplicates, followed by the screening of titles, abstracts and full-texts, 24 articles were included in the analysis. Health policy issues included the following topics: maternal and child health, tobacco control, reproductive health, infectious disease control and the impact and sustainability of knowledge translation platforms. Findings suggested that research evidence that could be used to meet key targets associated with the Millennium Development Goals (MDGs) were more likely to be considered as a political (and therefore policy) priority. Furthermore, avenues of engagement between research organizations and the government as well as collective action from civil-society organizations were important for the diffusion of evidence into policies. Through this article, it is apparent that the interface between evidence and policy formulation occurs when evidence is, disseminated by a cohesive policy-network with strong leadership and framed to deliver solutions for problems on both the domestic and global development agenda.

A mixed methods examination of knowledge brokers and their use of theoretical frameworks and evaluative practices

BACKGROUND

Knowledge brokering is a knowledge translation approach that includes making connections between researchers and decision-makers to facilitate the latter's use of evidence in health promotion and the provision of healthcare. Despite knowledge brokering being well-established in Canada, many knowledge gaps exist, including understanding what theoretical frameworks have been developed and which evaluative practices knowledge brokers (KBs) use.

METHODS

This study used a mixed methods design to examine how KBs in Canada (1) use frameworks, models and theories in their practice and (2) how they evaluate knowledge brokering interventions. We gathered interview and survey data from KB practitioners to better understand their perspectives on effective practices. Our analysis focused on understanding the theoretical frameworks used by KBs.

RESULTS

This study demonstrates that KBs in Canada tend not to rely on theories or models that are specific to knowledge brokering. Rather, study participants/respondents draw on (sometimes multiple) theories and models that are fundamental to the broader field of knowledge translation - in particular, the Knowledge to Action model and the Promoting Action Research in Health Sciences framework. In evaluating the impact of their own knowledge brokering practice, participants/respondents use a wide variety of mechanisms. Evaluation was often seen as less important than supporting knowledge users and/or paying clients in accessing and utilising evidence.

CONCLUSIONS

Knowledge brokering as a form of knowledge translation continues to expand, but the impact on its targeted knowledge users has yet to be clearly established. The quality of engagement between KBs and their clients might increase - the knowledge brokering can be more impactful - if KBs made efforts to describe, understand and evaluate their activities using theories or models specific to KB.

The State of Evidence-Based Policy Evaluation and its Role in Policy Formation

This paper argues that evidence-based policy has clearly made a worldwide impact, at least at the rhetorical and institutional levels, and in terms of analytical activity. The paper then addresses whether or not evidence-based policy evaluation has had an impact on policy formation and public service delivery. The paper uses a model of research-use that suggests that evidence can be used in instrumental, conceptual and symbolic ways. Taking four examples of the use of evidence in the UK over the past decade, this paper argues that evidence can be used instrumentally, conceptually and symbolically in complementary ways at different stages of the policy cycle and under different policy and political circumstances. The fact that evidence is not always used instrumentally, in the sense of “acting on research results in specific, direct ways” (Lavis et al., 2003, p. 228), does not mean that it has little or no influence. The paper ends by considering some of the obstacles to getting research evidence into policy and practice, and how these obstacles might be overcome.

Gathering Diverse Perspectives to Tackle "Wicked Problems": Racial/Ethnic Disproportionality in Educational Placement

Among students receiving behavioral health and special education services, racial/ethnic minority students are consistently overrepresented in settings separate from general classrooms. Once separated, many young people struggle to improve academically and face significant difficulty upon trying to return to a general education setting. Given the complex, ongoing, and multifaceted nature of this challenge, racial/ethnic disproportionality can be identified as a "wicked problem," for which solutions are not easily identified. Here, we describe our community-engaged research efforts, eliciting perspectives from relevant partners in an ongoing dialogue, to better integrate diverse stakeholders' perspectives when attempting to address such disparities. We conducted focus groups and qualitative interviews with members of three stakeholder groups: community-serving organizations, individuals with lived experience of behavioral health conditions, and state-level policymakers, with a shared interest in addressing racial and ethnic disparities. Participant responses illustrated the "wickedness" of this problem and highlighted the need for additional supports for students, families, and school personnel, increased collaboration across relevant systems and agencies, and reduced barriers related to funding. Overall, this methodology bridged differing perspectives to develop, in concert with our partners, a shared language of the problem and a core set of issues to consider when seeking to effect change.

Stakeholders' experiences with the evidence aid website to support 'real-time' use of research evidence to inform decision-making in crisis zones: a user testing study

Background

Humanitarian action in crisis zones is fraught with many challenges, including lack of timely and accessible research evidence to inform decision-making about humanitarian interventions. Evidence websites have the potential to address this challenge. Evidence Aid is the only evidence website designed for crisis zones that focuses on providing research evidence in the form of systematic reviews. The objective of this study is to explore stakeholders’ views of Evidence Aid, contributing further to our understanding of the use of research evidence in decision-making in crisis zones.

Methods

We designed a qualitative user-testing study to collect interview data from stakeholders about their impressions of Evidence Aid. Eligible stakeholders included those with and without previous experience of Evidence Aid. All participants were either currently working or have worked within the last year in a crisis zone. Participants were asked to perform the same user experience-related tasks and answer questions about this experience and their knowledge needs. Data were analysed using a deductive framework analysis approach drawing on Morville’s seven facets of the user experience — findability, usability, usefulness, desirability, accessibility, credibility and value.

Results

A total of 31 interviews were completed with senior decision-makers (n = 8), advisors (n = 7), field managers (n = 7), analysts/researchers (n = 5) and healthcare providers (n = 4). Participant self-reported knowledge needs varied depending on their role. Overall, participants did not identify any ‘major’ problems (highest order) and identified only two ‘big’ problems (second highest order) with using the Evidence Aid website, namely the lack of a search engine on the home page and that some full-text articles linked to/from the site require a payment. Participants identified seven specific suggestions about how to improve Evidence Aid, many of which can also be applied to other evidence websites.

Conclusions

Stakeholders in crisis zones found Evidence Aid to be useful, accessible and credible. However, they experienced some problems with the lack of a search engine on the home page and the requirement for payment for some full-text articles linked to/from the site.

The use of evidence in decision making by hospital managers in Lebanon: A cross-sectional study

BACKGROUND

Knowledge synthesis products have emerged as support agents for decision making in clinical practice and policy. However, their use for evidence-informed decision making remains limited in health care management especially in low- and middle-income countries. This study assesses the use of evidence by middle and senior managers in Lebanese hospitals.

METHODS

This multihospital cross-sectional study used a self-administered web survey of middle and senior managers. Hospitals were purposively selected, and data were analyzed using descriptive statistics and thematic analysis.

RESULTS

Hospital participation rate was 25%, while adjusted managers' response rate was 44.8%. Prevalence of using evidence was 70%, while prevalence of evidence-seeking behavior was 90%. Evidence was mainly used in design of policies, protocols, and procedures; nursing issues; or procurement decisions. Facilitators for evidence-informed decision making included upper management support and organizational culture, whereas limited resources such as funding, time, and training hindered use of evidence.

CONCLUSIONS

Findings indicate that utilization of evidence was comparable with that of high-income countries. Training and continuous education were crucial for advancing evidence-informed decision making among hospital managers. However, neither the quality nor the sources of evidence used for decision making were assessed in this study. Future studies should assess the quality and sources of evidence utilized in decision making.

A typology of systematic reviews for synthesising evidence on health care

OBJECTIVES

The objectives of this paper are to (a) Review published references to systematic reviews; (b) offer a typology of systematic reviews for synthesising evidence on health care; and (c) summarise the guides for designing, reporting and appraising the reviews.

BACKGROUND

Systematic reviews play a role in finding, synthesising, transferring and implementing evidence for healthcare policy, practice guidelines and allocation of health resources. They have been particularly successful in confirming or synthesising evidence for health care by meta-analysing aggregated data from multiple randomised controlled trials. However, concerns about the limitations of evidence from controlled trials have prompted interest in other review methods capable of locating and appraising evidence from more diverse, and possibly more realistic, healthcare situations.

METHODS

An iterative citation-tracking process with Google Search and grey literature identified 204 papers on previous typologies and methods of systematic reviews.

RESULTS AND CONCLUSIONS

There are six types of systematic reviews: narrative; meta-analysis; scoping; qualitative; umbrella; and realist. Each type has distinct objectives, characteristics and attributes, but with much overlapping of methods and guides. Sensitivity to the need for qualitative evidence on complex human responses to ill-health and health care has broadened the objectives and methods of health-related systematic reviews to find, appraise and synthesis useful evidence for practice guidelines, healthcare policy and allocation of health resources.

The governmental health policy-development process for Syrian refugees: an embedded qualitative case studies in Lebanon and Ontario

Background

The unprecedented amount of resources dedicated to humanitarian aid has led many stakeholders to demand the use of reliable evidence in humanitarian aid decisions to ensure that desired impacts are achieved at acceptable costs. However, little is known about the factors that influence the use of research evidence in the policy development in humanitarian crises. We examined how research evidence was used to inform two humanitarian policies made in response to the Syrian refugee crisis.

Methods

We identified two policies as rich potential case studies to examine the use of evidence in humanitarian aid policy decision-making: Lebanon’s 2016 Health Response Strategy and Ontario’s 2016 Phase 2: Health System Action Plan, Syrian Refugees. To study each, we used an embedded qualitative case study methodology and recruited senior decision-makers, policy advisors, and healthcare providers who were involved with the development of each policy. We reviewed publicly available documents and media articles that spoke to the factors that influence the process. We used the analytic technique of explanation building to understand the factors that influence the use of research evidence in the policy-development process in crisis zones.

Results

We interviewed eight informants working in government and six in international agencies in Lebanon, and two informants working in healthcare provider organizations and two in non-governmental organizations in Ontario, for a total of 18 key informants. Based on our interviews and documentary analysis, we identified that there was limited use of research evidence and that four broad categories of factors helped to explain the policy-development process for Syrian refugees – development of health policies without significant chance for derailment from other government bodies (Lebanon) or opposition parties (Ontario) (i.e., facing no veto points), government’s engagement with key societal actors to inform the policy-development process, the values underpinning the process, and external factors significantly influencing the policy-development process.

Conclusions

This study suggests that use of research evidence in the policy-development process for Syrian refugees was subordinate to key political factors, resulting in limited influence of research evidence in the development of both the Lebanese and Ontarian policy.

Evaluation of an HIV-specific rapid response service for community-based organisations in Ontario, Canada

Background

To support AIDS service organisations and other community-based organisations’ use of research evidence to inform HIV-related programmes, services and policies, the Ontario HIV Treatment Network (OHTN) developed a Rapid Response Service. The final product of the rapid response process at the OHTN, which is more streamlined than that of traditional systematic reviews, consists of a detailed report answering questions regarding an HIV-specific issue and how the findings apply within the local context. In 2016, the OHTN conducted an evaluation to assess the effectiveness of its Rapid Response Service. This article reports on the development of this service as well as the results of the evaluation.

Methods

All rapid responses published between January 1, 2009, and September 30, 2016, by the OHTN (n = 102) were analysed using univariate analyses. Frequency distributions were determined for the following variables for each rapid response: populations observed, topics covered, requestor affiliations and number of downloads from the OHTN’s website. Requestors of rapid responses were also interviewed regarding perceived helpfulness and utility of the service and final products, and suggestions for changes to the service. Six-month follow-up interviews were conducted to determine how affiliated organisations used the evidence from the rapid response they requested.

Results

The 102 rapid responses published covered 14 different populations of interest. Topics covered included the HIV prevention, engagement and care cascade, determinants of health, syndemics, and comorbidities. Requestor affiliations consisted of AIDS service organisations, government agencies and policy-makers, non-HIV-focused community-based organisations, and hospitals, universities or health centres. Requestors perceived most aspects of the Rapid Response Service as very helpful and most frequently suggested that the rapid responses should provide recommendations. Follow-up interviews regarding the impact of rapid responses show that rapid responses have been used to assist organisations in numerous activities.

Conclusions

Organisations that have used the OHTN’s Rapid Response Service describe it as a valuable service useful for the development of programmes and policies. Improvements in capacity-building efforts may increase its utility. Describing the findings of this evaluation may serve as a reference for similar programmes to increase the use of research evidence among public health decision-makers.

Electronic supplementary material

The online version of this article (10.1186/s12961-019-0476-4) contains supplementary material, which is available to authorized users.

Clinician barriers and facilitators to heart failure advance care plans: a systematic literature review and qualitative evidence synthesis

BACKGROUND

Clinicians hesitate to engage with advance care planning (ACP) in heart failure. We aimed to identify the disease-specific barriers and facilitators for clinicians to engage with ACP.

METHODS

We searched Medline, Embase, CINAHL, PubMed, Scopus, the British Nursing Index, the Cochrane Library, the EPOC register, ERIC, PsycINFO, the Science Citation Index and the Grey Literature from inception to July 2018. We conducted the review according to Enhancing Transparency in Reporting the Synthesis of Qualitative Research (ENTREQ) guidelines. Two reviewers independently assessed original and empirical studies according to Critical Appraisal Skills Programme criteria. The SURE framework and thematic analysis were used to identify barriers and facilitators.

RESULTS

Of 2308 articles screened, we reviewed the full text of 42 studies. Seventeen studies were included. The main barriers were lack of disease-specific knowledge about palliative care in heart failure, high emotional impact on clinicians when undertaking ACP and lack of multidisciplinary collaboration between healthcare professionals to reach consensus on when ACP is indicated. The main facilitators were being competent to provide holistic care when using ACP in heart failure, a patient taking the initiative of having an ACP conversation, and having the resources to deliver ACP at a time and place appropriate for the patient.

CONCLUSIONS

Training healthcare professionals in the delivery of ACP in heart failure might be as important as enabling patients to start an ACP conversation. This twofold approach may mitigate against the high emotional impact of ACP. Complex interventions are needed to support clinicians as well as patients to engage with ACP.

An inductive exploration of the implementation knowledge of research funders

Background

Healthcare research funders may undertake various roles to facilitate implementation of research findings. Their ability to enact such roles depends on several factors, knowledge of implementation being one essential requirement. However, previous studies do not assess the type or level of knowledge about implementation that research funders possess. This paper therefore presents findings from a qualitative, inductive study of the implementation knowledge of research funders. Three aspects of this knowledge are explored, namely how research funders define implementation, their level of self-assessed implementation knowledge and the factors influencing their self-assessment of implementation knowledge.

Methods

Research funders (n = 18) were purposefully selected from a sample of research funding organisations in Sweden (n = 10). In-depth semi-structured interviews were conducted, recorded and transcribed verbatim. An inductive method using a systematic coding procedure was employed to derive the findings.

Results

The research funders defined implementation as either an outcome or a process, with the majority believing that implementation of healthcare research results demands a process, although its complexity varied in the research funders’ view. They perceived their own level of implementation knowledge as either limited or substantial, with a majority regarding it as limited. Clinical research experience, clinical experience and task relevance were singled out as the clearest factors affecting the self-assessment of their own implementation knowledge.

Conclusions

This study, the first to focus on implementation knowledge of research funders, demonstrates that they are a category of policy-makers who may possess knowledge, based on their previous professional experience, that is comparable to some important findings from implementation research. Consequently, the findings not only pinpoint the relevance of professional experience, but also reveal a lack of awareness and knowledge of the results of implementation research among research funders in charge of healthcare research.

Electronic supplementary material

The online version of this article (10.1186/s12961-019-0472-8) contains supplementary material, which is available to authorized users.

A description of a tailored knowledge translation intervention delivered by knowledge brokers within public health departments in Canada

Background

While there is an expectation to demonstrate evidence-informed public health there is an ongoing need for capacity development. The purpose of this paper is to provide a description of a tailored knowledge translation intervention implemented by knowledge brokers (KBs), and reflections on the factors that facilitated or hindered its implementation.

Methods

The 22-month knowledge translation intervention, implemented by two KBs, sought to facilitate evidence-informed public health decision-making. Data on outcomes were collected using a knowledge, skills and behavioural assessment survey. In addition, the KBs maintained reflective journals noting which activities appeared successful or not, as well as factors related to the individual or the organisation that facilitated or hindered evidence-informed decision-making.

Results

Tailoring of the knowledge translation intervention to address the needs, preferences and structure of each organisation resulted in three unique interventions being implemented. A consistent finding across organisations was that each site needed to determine where evidence-informed decision-making ‘fit’ within pre-existing organisational processes. Components of the intervention consistent across the three organisations included one-to-one mentoring of teams through rapid evidence reviews, large group workshops and regular meetings with senior management. Components that varied included the frequency of the KB being physically onsite, the amount of time staff spent with the KB and proportion of time spent one-to-one with a KB versus in workshops. Key facilitating factors for implementation included strong leadership, influential power of champions, supportive infrastructure, committed resources and staff enthusiasm.

Conclusions

The results of this study illustrate the importance of working collaboratively with organisations to tailor knowledge translation interventions to best meet unique needs, preferences, organisational structures and contexts. Organisational factors such as leadership, champions and supportive infrastructure play a key role in determining the impact of the knowledge translation interventions. Future studies should explore how these factors can be fostered and/or developed within organisations. While KBs implemented the knowledge translation intervention in this study, more research is needed to understand the impact of all change agent roles including KBs, as well as how these roles can be maintained in the long-term if proven effective.

Guidelines on the surgical management of sleep disorders: A systematic review

OBJECTIVE

To facilitate the development of U.K. guidelines for sleep surgery and to guide sleep surgeons to existing guidelines relevant to their practice, we provide a systematic review and quality assessment of all existing guidelines on the surgical management of sleep disorders.

METHODS

Systematic review using preferred reporting items for systematic reviews and meta-analyses (PRISMA) recommendations. Medline and Embase databases were searched from inception to April 2018. Publications were included if they described a guideline for the surgical management of sleep disorders. Three assessors used the Appraisal of Guidelines for Research and Evaluation II (AGREE II) instrument to evaluate included guidelines.

RESULTS

The systematic search revealed 1,161 publications. Twenty-two guidelines from eight countries were included. Fourteen focused on adults, five on children, and three on both. The guidelines discussed nasal, tonsillar, palatal, tongue, hyoid, maxillomandibular, tracheal, bariatric, and multilevel surgeries. The mean overall AGREE II quality score of included guidelines was 3.5 (range = 2 to 5.3; maximum possible score = 7).

CONCLUSION

This article provides a summary and quality assessment of all published guidelines on the surgical management of sleep disorders. No U.K. guidelines were identified, and existing guidelines have several shortcomings. This highlights the need for robust U.K. national guidelines on sleep surgery to promote clinical and cost-effective care in this field. Our findings can be used by stakeholders as a foundation for the development of new guidelines and can be used by sleep surgeons to direct them to existing guidelines relevant to their practice, promoting evidence-based clinical care. Laryngoscope, 130:1070-1084, 2020.

Formative research to develop a school-based, community-linked physical activity role model programme for girls: CHoosing Active Role Models to INspire Girls (CHARMING)

BACKGROUND

Physical inactivity is a persistent challenge among girls. School-based physical activity (PA) interventions have shown mixed effects on girl's activity levels, with multi-component approaches involving both school and community links appearing more effective for sustainable change. The purpose of the current research was to gather views from preadolescent girls, parents, teachers and stakeholders in order to co-produce a multi-component school-based, community linked PA intervention programme.

METHODS

Focus groups were conducted in two primary schools with 34 girls aged 9-11 years and 11 parents (10 female, 1 male). In-depth interviews were conducted with four female teachers (including two head teachers). Focus groups and interviews focused on programme design (structure, content and delivery) and potential factors affecting intervention uptake and continued PA participation. A series of stakeholder engagement events occurred throughout the study period. All data were transcribed verbatim and thematically analysed in NVivo 11.

RESULTS

Girls reported that fun taster sessions delivered by role models would encourage them to participate in a school-based role model programme, with tailored taster sessions each week to enhance continued PA participation. Parents and teachers identified a number of barriers to uptake and continued PA participation, and active involvement of stakeholders facilitated the development of intervention strategies. Strategies included; single-sex after-school sessions, use of female role models, low-cost activity options and mapping community provision. Analyses revealed the importance of tailoring the programme to align with local needs, demands and provision.

CONCLUSIONS

Data show numerous barriers to intervention uptake and continued PA participation when designing a school-based, community-linked intervention. Adopting a co-production approach, this formative work highlights a number of potential strategies for overcoming these barriers. Findings from the research directed the development and implementation of the CHARMING role model intervention and informed the creation of an intervention logic model.

How are evidence generation partnerships between researchers and policy-makers enacted in practice? A qualitative interview study

Background

Evidence generation partnerships between researchers and policy-makers are a potential method for producing more relevant research with greater potential to impact on policy and practice. Little is known about how such partnerships are enacted in practice, however, or how to increase their effectiveness. We aimed to determine why researchers and policy-makers choose to work together, how they work together, which partnership models are most common, and what the key (1) relationship-based and (2) practical components of successful research partnerships are.

Methods

Semi-structured qualitative interviews were conducted with 18 key informants largely based in New South Wales, Australia, who were (1) researchers experienced in working in partnership with policy in health or health-related areas or (2) policy and programme developers and health system decision-makers experienced in working in partnership with researchers. Data was analysed thematically by two researchers.

Results

Researcher-initiated and policy agency-initiated evidence generation partnerships were common. While policy-initiated partnerships were thought to be the most likely to result in impact, researcher-initiated projects were considered important in advancing the science and were favoured by researchers due to greater perceived opportunities to achieve key academic career metrics. Participants acknowledged that levels of collaboration varied widely in research/policy partnerships from minimal to co-production. Co-production was considered a worthy goal by all, conferring a range of benefits, but one that was difficult to achieve in practice. Some participants asserted that the increased time and resources required for effective co-production meant it was best suited to evaluation and implementation projects where the tacit, experiential knowledge of policy-makers provided critical nuance to underpin study design, implementation and analysis. Partnerships that were mutually considered to have produced the desired outcomes were seen to be underpinned by a range of both relationship-based (such as shared aims and goals and trust) and practical factors (such as sound governance and processes).

Conclusions

Our findings highlight the important role of policy-makers in New South Wales in ensuring the relevance of research. There is still much to understand about how to initiate and sustain successful research/policy partnerships, particularly at the highly collaborative end.

Electronic supplementary material

The online version of this article (10.1186/s12961-019-0441-2) contains supplementary material, which is available to authorized users.

The impact of general practitioners working in or alongside emergency departments: a rapid realist review

OBJECTIVES

Worldwide, emergency healthcare systems are under intense pressure from ever-increasing demand and evidence is urgently needed to understand how this can be safely managed. An estimated 10%-43% of emergency department patients could be treated by primary care services. In England, this has led to a policy proposal and £100 million of funding (US$130 million), for emergency departments to stream appropriate patients to a co-located primary care facility so they are 'free to care for the sickest patients'. However, the research evidence to support this initiative is weak.

DESIGN

Rapid realist literature review.

SETTING

Emergency departments.

INCLUSION CRITERIA

Articles describing general practitioners working in or alongside emergency departments.

AIM

To develop context-specific theories that explain how and why general practitioners working in or alongside emergency departments affect: patient flow; patient experience; patient safety and the wider healthcare system.

RESULTS

Ninety-six articles contributed data to theory development sourced from earlier systematic reviews, updated database searches (Medline, Embase, CINAHL, Cochrane DSR & CRCT, DARE, HTA Database, BSC, PsycINFO and SCOPUS) and citation tracking. We developed theories to explain: how staff interpret the streaming system; different roles general practitioners adopt in the emergency department setting (traditional, extended, gatekeeper or emergency clinician) and how these factors influence patient (experience and safety) and organisational (demand and cost-effectiveness) outcomes.

CONCLUSIONS

Multiple factors influence the effectiveness of emergency department streaming to general practitioners; caution is needed in embedding the policy until further research and evaluation are available. Service models that encourage the traditional general practitioner approach may have shorter process times for non-urgent patients; however, there is little evidence that this frees up emergency department staff to care for the sickest patients. Distinct primary care services offering increased patient choice may result in provider-induced demand. Economic evaluation and safety requires further research.

PROSPERO REGISTRATION NUMBER

CRD42017069741.