Expanding the evidence base for global recommendations on health systems: strengths and challenges of the OptimizeMNH guidance process

"Going into the black box": a policy analysis of how the World Health Organization uses evidence to inform guideline recommendations

Background

The World Health Organization (WHO) plays a crucial role in producing global guidelines. In response to previous criticism, WHO has made efforts to enhance the process of guideline development, aiming for greater systematicity and transparency. However, it remains unclear whether these changes have effectively addressed these earlier critiques. This paper examines the policy process employed by WHO to inform guideline recommendations, using the update of the WHO Consolidated HIV Testing Services (HTS) Guidelines as a case study.

Methods

We observed guideline development meetings and conducted semi-structured interviews with key participants involved in the WHO guideline-making process. The interviews were recorded, transcribed, and analysed thematically. The data were deductively coded and analysed in line with the main themes from a published conceptual framework for context-based evidence-based decision making: introduction, interpretation, and application of evidence.

Results

The HTS guideline update was characterized by an inclusive and transparent process, involving a wide range of stakeholders. However, it was noted that not all stakeholders could participate equally due to gaps in training and preparation, particularly regarding the complexity of the Grading Recommendations Assessment Development Evaluation (GRADE) framework. We also found that WHO does not set priorities for which or how many guidelines should be produced each year and does not systematically evaluate the implementation of their recommendations. Our interviews revealed disconnects in the evidence synthesis process, starting from the development of systematic review protocols. While GRADE prioritizes evidence from RCTs, the Guideline Development Group (GDG) heavily emphasized "other" GRADE domains for which little or no evidence was available from the systematic reviews. As a result, expert judgements and opinions played a role in making recommendations. Finally, the role of donors and their presence as observers during GDG meetings was not clearly defined.

Conclusion

We found a need for a different approach to evidence synthesis due to the diverse range of global guidelines produced by WHO. Ideally, the evidence synthesis should be broad enough to capture evidence from different types of studies for all domains in the GRADE framework. Greater structure is required in formulating GDGs and clarifying the role of donors through the process.

A Perspective on Implementation Outcomes and Strategies to Promote the Uptake of COVID-19 Vaccines

Recent articles have highlighted the importance of incorporating implementation science concepts into pandemic-related research. However, limited research has been documented to date regarding implementation outcomes that may be unique to COVID-19 vaccinations and how to utilize implementation strategies to address vaccine program-related implementation challenges. To address these gaps, we formed a global COVID-19 implementation workgroup of implementation scientists who met weekly for over a year to review the available literature and learn about ongoing research during the pandemic. We developed a hierarchy to prioritize the applicability of "lessons learned" from the vaccination-related implementation literature. We identified applications of existing implementation outcomes as well as identified additional implementation outcomes. We also mapped implementation strategies to those outcomes. Our efforts provide rationale for the utility of using implementation outcomes in pandemic-related research. Furthermore, we identified three additional implementation outcomes: availability, health equity, and scale-up. Results include a list of COVID-19 relevant implementation strategies mapped to the implementation outcomes.

Evidence-informed decision about (de-)implementing return-to-work coordination to reduce sick leave: a case study

Background

Coordination of return to work (RtW) is implemented in many countries, but a Cochrane review found no considerable effect on workers’ sick leave compared to usual care. The aim of the study is to analyse how the evidence from this review can be used for decisions about (de-)implementing RtW coordination in a country-specific setting, using Finland as an example.

Methods

We conducted a systematic literature search and online survey with two groups of experts to compare interventions included in the Cochrane review to Finnish RtW practice using content analysis methods. We applied the evidence-to-decision (EtD) framework criteria to draw conclusions about (de-)implementing RtW coordination in Finland, including benefits, harms and costs of the intervention compared to usual care.

Results

We included seven documents from the literature search and received data from 10 of 42 survey participants. RtW coordination included, both in Finland and in the review, at least one face-to-face meeting between the physician and the worker, a workers’ needs assessment, and an individual RtW plan and its implementation. Usual care focuses on medical treatment and may include general RtW advice. RtW coordination would be cost-saving if it decreases sick leave with at least 2 days compared to usual care. The evidence in the Cochrane review was mainly of low certainty, and the effect sizes had relatively wide confidence intervals. Only a new, high-quality and large randomized controlled trial (RCT) can decrease the current uncertainty, but this is unlikely to happen. The EtD framework did not provide arguments for further implementation or for de-implementation of the intervention.

Conclusions

Interventions evaluated in the Cochrane review are similar to RtW coordination and usual care interventions in Finland. Considering all EtD framework criteria, including certainty of the evidence and costs, de-implementation of RtW coordination interventions in Finland seems unnecessary. Better evidence about the costs and stakeholders’ values regarding RtW coordination is needed to improve decision-making.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12961-022-00823-4.

The challenge of equipoise in trials with a surgical and non-surgical comparison: a qualitative synthesis using meta-ethnography

BACKGROUND

Randomised controlled trials in surgery can be a challenge to design and conduct, especially when including a non-surgical comparison. As few as half of initiated surgical trials reach their recruitment target, and failure to recruit is cited as the most frequent reason for premature closure of surgical RCTs. The aim of this qualitative evidence synthesis was to identify and synthesise findings from qualitative studies exploring the challenges in the design and conduct of trials directly comparing surgical and non-surgical interventions.

METHODS

A qualitative evidence synthesis using meta-ethnography was conducted. Six electronic bibliographic databases (Medline, Central, Cinahl, Embase and PsycInfo) were searched up to the end of February 2018. Studies that explored patients' and health care professionals' experiences regarding participating in RCTs with a surgical and non-surgical comparison were included. The GRADE-CERQual framework was used to assess confidence in review findings.

RESULTS

In total, 3697 abstracts and 49 full texts were screened and 26 published studies reporting experiences of patients and healthcare professionals were included. The focus of the studies (24/26) was primarily related to the challenge of recruitment. Two studies explored reasons for non-compliance to treatment allocation following randomisation. Five themes related to the challenges to these types of trials were identified: (1) radical choice between treatments; (2) patients' discomfort with randomisation: I want the best treatment for me as an individual; (3) challenge of equipoise: patients' a priori preferences for treatment; (4) challenge of equipoise: clinicians' a priori preferences for treatment and (5) imbalanced presentation of interventions.

CONCLUSION

The marked dichotomy between the surgical and non-surgical interventions was highlighted in this review as making recruitment to these types of trials particularly challenging. This review identified factors that increase our understanding of why patients and clinicians may find equipoise more challenging in these types of trials compared to other trial comparisons. Trialists may wish to consider exploring the balance of potential factors influencing patient and clinician preferences towards treatments before they start recruitment, to enable issues specific to a particular trial to be identified and addressed. This may enable trial teams to make more efficient considered design choices and benefit the delivery of such trials.

Evaluation of the reporting quality of clinical practice guidelines on gliomas using the RIGHT checklist

Background

The reporting quality of clinical practice guidelines (CPGs) for gliomas has not yet been thoroughly assessed. The International Reporting Items for Practice Guidelines in Healthcare (RIGHT) statement developed in 2016 provides a reporting framework to improve the quality of CPGs. We aimed to estimate the reporting quality of glioma guidelines using the RIGHT checklist and investigate how the reporting quality differs by selected characteristics.

Methods

We systematically searched electronic databases, guideline databases, and medical society websites to retrieve CPGs on glioma published between 2018 and 2020. We calculated the compliance of the CPGs to individual items, domains and the RIGHT checklist overall. We performed stratified analyses by publication year, country of development, reporting of funding, and impact factor (IF) of the journal.

Results

Our search revealed 20 eligible guidelines. Mean overall adherence to the RIGHT statement was 54.6%. Eight CPGs reported more than 60% of the items, and five reported less than 50%. All guidelines adhered to the items 1a, 3, 7a, 13a, while no guidelines reported the items 17 or 18b (see http://www.right-statement.org/right-statement/checklist for a description of the items). Two of the seven domains, “Basic information” and “Background”, had mean reporting rates above 60%. The “Review and quality assurance” domain had the lowest mean reporting rate, 12.5%. The reporting quality of guidelines published in 2020, guidelines developed in the United States, and guidelines that reported funding tended to be above average.

Conclusions

The reporting quality of CPGs on gliomas is low and needs improvement. Particular attention should be paid on reporting the external review and quality assurance process. The use of the RIGHT criteria should be encouraged to guide the development, reporting and evaluation of CPGs.

Users' experiences with an interactive Evidence to Decision (iEtD) framework: a qualitative analysis

BACKGROUND

Evidence to Decision (EtD) frameworks bring clarity, structure and transparency to health care decision making. The interactive Evidence to Decision (iEtD) tool, developed in the context of the DECIDE project and published by Epistemonikos, is a stand-alone online solution for producing and using EtD frameworks. Since its development, little is known about how organizations have been using the iEtD tool and what characterizes users' experiences with it. This missing information is necessary for any teams planning future developments of the iEtD tool.

METHODS

This study aimed to describe users' experiences with the iEtD and identify main barriers and facilitators related to use. We contacted all users registered in the iEtD via email and invited people who identified themselves as having used the solution to a semi-structured interview. Audio recordings were transcribed, and one researcher conducted a directed content analysis of the interviews guided by a user experience framework. Two researchers checked the content independently for accuracy.

RESULTS

Out of 860 people contacted, 81 people replied to our introductory email (response rate 9.4%). Twenty of these had used the tool in a real scenario and were invited to an interview. We interviewed all eight users that accepted this invitation (from six countries, four continents). 'Guideline development' was the iEtD use scenario they most commonly identified. Most participants reported an overall positive experience, without major difficulties navigating or using the different sections. They reported having used most of the EtD framework criteria. Participants reported tailoring their frameworks, for instance by adding or deleting criteria, translating to another language, or rewording headings. Several people preferred to produce a Word version rather than working online, due to the burden of completing the framework, or lack of experience with the tool. Some reported difficulties working with the exportable formats, as they needed considerable editing.

CONCLUSION

A very limited number of guideline developers have used the iEtD tool published by Epistemonikos since its development. Although users' general experiences are positive, our work has identified some aspects of the tool that need improvement. Our findings could be also applied to development or improvement of other solutions for producing or using EtD frameworks.

An exploration of how developers use qualitative evidence: content analysis and critical appraisal of guidelines

BACKGROUND

Clinical practice guidelines have become increasingly widely used to guide quality improvement of clinical practice. Qualitative research may be a useful way to improve the quality and implementation of guidelines. The methodology for qualitative evidence used in guidelines development is worthy of further research.

METHODS

A comprehensive search was made of WHO, NICE, SIGN, NGC, RNAO, PubMed, Embase, Web of Science, CNKI, Wanfang, CBM, and VIP from January 1, 2011 to February 25, 2020. Guidelines which met IOM criteria and were focused on clinical questions using qualitative research or qualitative evidence, were included. Four authors extracted significant information and entered this onto data extraction forms. The Appraisal of Guidelines for Research and Evaluation (AGREE II) tool was used to evaluate the guidelines' quality. The data were analyzed using SPSS version 17.0 and R version 3.3.2.

RESULTS

Sixty four guidelines were identified. The overall quality of the guidelines was high (almost over 60%). Domain 1 (Scope and Purpose) was ranked the highest with a median score of 83% (IQ 78-83). Domain 2 (Stakeholder involvement) and Domain 5 (Applicability) were ranked the lowest with median scores of 67% (IQ 67-78) and 67% (IQ 63-73) respectively. 20% guidelines used qualitative research to identify clinical questions. 86% guidelines used qualitative evidence to support recommendations (mainly based on primary studies, a few on qualitative evidence synthesis). 19% guidelines applied qualitative evidence when considering facilitators and barriers to recommendations' implementation. 52% guideline developers evaluated the quality of the primary qualitative research study using the CASP tool or NICE checklist for qualitative studies. No guidelines evaluated the quality of qualitative evidence synthesis to formulate recommendations. 17% guidelines presented the level of qualitative research using the grade criteria of evidence and recommendation in different forms such as I, III, IV, very low. 28% guidelines described the grades of the recommendations supported by qualitative and quantitative evidence. No guidelines described the grade of recommendations only supported by qualitative evidence.

CONCLUSIONS

The majority of the included guidelines were high-quality. Qualitative evidence was mainly used to identify clinical questions, support recommendations, and consider facilitators and barriers to implementation of recommendations'. However, more attention needs to be paid to the methodology. For example, no experts proficient in qualitative research were involved in guideline development groups, no assessment of the quality of qualitative evidence synthesis was included and there was lack of details reported on the level of qualitative evidence or grade of recommendations.

The role of real-world data in the development of treatment guidelines: a case study on guideline developers' opinions about using observational data on antibiotic prescribing in primary care

Background

Antimicrobial resistance (AMR) is a prominent threat to public health. Although many guidelines have been developed over the years to tackle this issue, their impact on health care practice varies. Guidelines are often based on evidence from clinical trials, but these have limitations, particularly in the breadth and generalisability of the evidence and evaluation of the guidelines’ uptake. The aim of this study was to investigate how national and local guidelines for managing common infections are developed and explore guideline committee members’ opinions about using real-world observational evidence in the guideline development process.

Methods

Six semi-structured interviews were completed with participants who had contributed to the development or adjustment of national or local guidelines on antimicrobial prescribing over the past 5 years (from the English National Institute for Health and Care Excellence (NICE)). Interviews were audio recorded and transcribed verbatim. Data was analysed thematically. This also included review of policy documents including guidelines, reports and minutes of guideline development group meetings that were available to the public.

Results

Three key themes emerged through our analysis: perception versus actual guideline development process, using other types of evidence in the guideline development process, and guidelines are not enough to change antibiotic prescribing behaviour. In addition, our study was able to provide some insight between the documented and actual guideline development process within NICE, as well as how local guidelines are developed, including differences in types of evidence used.

Conclusions

This case study indicates that there is the potential for a wider range of evidence to be included as part of the guideline development process at both the national and local levels. There was a general agreement that the inclusion of observational data would be appropriate in enhancing the guideline development process, as well providing a potential solution for monitoring guideline use in clinical practice, and improving the implementation of treatment guidelines in primary care.

Clients' perceptions and experiences of targeted digital communication accessible via mobile devices for reproductive, maternal, newborn, child, and adolescent health: a qualitative evidence synthesis

BACKGROUND

Governments and health systems are increasingly using mobile devices to communicate with patients and the public. Targeted digital client communication is when the health system transmits information to particular individuals or groups of people, based on their health or demographic status. Common types of targeted client communication are text messages that remind people to go to appointments or take their medicines. Other types include phone calls, interactive voice response, or multimedia messages that offer healthcare information, advice, monitoring, and support.

OBJECTIVES

To explore clients' perceptions and experiences of targeted digital communication via mobile devices on topics related to reproductive, maternal, newborn, child, or adolescent health (RMNCAH).

SEARCH METHODS

We searched MEDLINE (OvidSP), MEDLINE In-Process & Other Non-Indexed Citations (OvidSP), Embase (Ovid), World Health Organization Global Health Library, and POPLINE databases for eligible studies from inception to 3-6 July 2017 dependant on the database (See appendix 2).

SELECTION CRITERIA

We included studies that used qualitative methods for data collection and analysis; that explored clinets' perceptions and experiences of targeted digital communication via mobile device in the areas of RMNCAH; and were from any setting globally.

DATA COLLECTION AND ANALYSIS

We used maximum variation purposive sampling for data synthesis, employing a three-step sampling frame. We conducted a framework thematic analysis using the Supporting the Use of Research Evidence (SURE) framework as our starting point. We assessed our confidence in the findings using the GRADE-CERQual (Confidence in the Evidence from Reviews of Qualitative research) approach. We used a matrix approach to explore whether potential implementation barriers identified in our synthesis had been addressed in the trials included in the related Cochrane Reviews of effectiveness.

MAIN RESULTS

We included 35 studies, from a wide range of countries on six continents. Nineteen studies were conducted in low- and middle-income settings and sixteen in high-income settings. Some of the studies explored the views of people who had experienced the interventions, whereas others were hypothetical in nature, asking what people felt they would like from a digital health intervention. The studies covered a range of digital targeted client communication, for example medication or appointment reminders, prenatal health information, support for smoking cessation while pregnant, or general sexual health information.Our synthesis showed that clients' experiences of these types of programmes were mixed. Some felt that these programmes provided them with feelings of support and connectedness, as they felt that someone was taking the time to send them messages (moderate confidence in the evidence). They also described sharing the messages with their friends and family (moderate confidence).However, clients also pointed to problems when using these programmes. Some clients had poor access to cell networks and to the internet (high confidence). Others had no phone, had lost or broken their phone, could not afford airtime, or had changed their phone number (moderate confidence). Some clients, particularly women and teenagers, had their access to phones controlled by others (moderate confidence). The cost of messages could also be a problem, and many thought that messages should be free of charge (high confidence). Language issues as well as skills in reading, writing, and using mobile phones could also be a problem (moderate confidence).Clients dealing with stigmatised or personal health conditions such as HIV, family planning, or abortion care were also concerned about privacy and confidentiality (high confidence). Some clients suggested strategies to deal with these issues, such as using neutral language and tailoring the content, timing, and frequency of messages (high confidence).Clients wanted messages at a time and frequency that was convenient for them (moderate confidence). They had preferences for different delivery channels (e.g. short message service (SMS) or interactive voice response) (moderate confidence). They also had preferences about message content, including new knowledge, reminders, solutions, and suggestions about health issues (moderate confidence). Clients' views about who sent the digital health communication could influence their views of the programme (moderate confidence).For an overview of the findings and our confidence in the evidence, please see the 'Summary of qualitative findings' tables.Our matrix shows that many of the trials assessing these types of programmes did not try to address the problems we identified, although this may have been a reporting issue.

AUTHORS' CONCLUSIONS

Our synthesis identified several factors that can influence the successful implementation of targeted client communication programmes using mobile devices. These include barriers to use that have equity implications. Programme planners should take these factors into account when designing and implementing programmes. Future trial authors also need to actively address these factors and to report their efforts in their trial publications.

Qualitative Evidence Synthesis (QES) for Guidelines: Paper 3 - Using qualitative evidence syntheses to develop implementation considerations and inform implementation processes

BACKGROUND

This is the third in a series of three papers describing the use of qualitative evidence syntheses (QES) to inform the development of clinical and health systems guidelines. WHO has recognised the need to improve its guideline methodology to ensure that decision-making processes are transparent and evidence based, and that the resulting recommendations are relevant and applicable to end users. In addition to the standard data on effectiveness, WHO guidelines increasingly use evidence derived from QES to provide information on acceptability and feasibility and to develop important implementation considerations.

METHODS

WHO convened a group drawn from the technical teams involved in formulating recent (2010-2018) guidelines employing QES. Using a pragmatic and iterative approach that included feedback from WHO staff and other stakeholders, the group reflected on, discussed and identified key methods and research implications from designing QES and using the resulting findings in guideline development. As members of WHO guideline technical teams, our aim in this paper is to explore how we have used findings from QES to develop implementation considerations for these guidelines.

RESULTS

For each guideline, in addition to using systematic reviews of effectiveness, the technical teams used QES to gather evidence of the acceptability and feasibility of interventions and, in some cases, equity issues and the value people place on different outcomes. This evidence was synthesised using standardised processes. The teams then used the QES to identify implementation considerations combined with other sources of information and input from experts.

CONCLUSIONS

QES were useful sources of information for implementation considerations. However, several issues for further development remain, including whether researchers should use existing health systems frameworks when developing implementation considerations; whether researchers should take confidence in the evidence into account when developing implementation considerations; whether qualitative evidence that reveals implementation challenges should lead guideline panels to make conditional recommendations or only point to implementation considerations; and whether guideline users find it helpful to have challenges pointed out to them or whether they also need solutions. Finally, we need to explore how QES findings can be incorporated into derivative products to aid implementation.

Qualitative Evidence Synthesis (QES) for Guidelines: Paper 2 - Using qualitative evidence synthesis findings to inform evidence-to-decision frameworks and recommendations

BACKGROUND

WHO has recognised the need to improve its guideline methodology to ensure that guideline decision-making processes are transparent and evidence based, and that the resulting recommendations are relevant and applicable. To help achieve this, WHO guidelines now typically enhance intervention effectiveness data with evidence on a wider range of decision-making criteria, including how stakeholders value different outcomes, equity, gender and human rights impacts, and the acceptability and feasibility of interventions. Qualitative evidence syntheses (QES) are increasingly used to provide evidence on this wider range of issues. In this paper, we describe and discuss how to use the findings from QES to populate decision-making criteria in evidence-to-decision (EtD) frameworks. This is the second in a series of three papers that examines the use of QES in developing clinical and health system guidelines.

METHODS

WHO convened a writing group drawn from the technical teams involved in its recent (2010-2018) guidelines employing QES. Using a pragmatic and iterative approach that included feedback from WHO staff and other stakeholders, the group reflected on, discussed and identified key methods and research implications from designing QES and using the resulting findings in guideline development.

RESULTS

We describe a step-wise approach to populating EtD frameworks with QES findings. This involves allocating findings to the different EtD criteria (how stakeholders value different outcomes, equity, acceptability and feasibility, etc.), weaving the findings into a short narrative relevant to each criterion, and inserting this summary narrative into the corresponding 'research evidence' sections of the EtD. We also identify areas for further methodological research, including how best to summarise and present qualitative data to groups developing guidelines, how these groups draw on different types of evidence in their decisions, and the extent to which our experiences are relevant to decision-making processes in fields other than health.

CONCLUSIONS

This paper shows the value of incorporating QES within a guideline development process, and the roles that qualitative evidence can play in integrating the views and experiences of relevant stakeholders, including groups who may not be otherwise represented in the decision-making process.

Improving reporting of meta-ethnography: The eMERGe reporting guidance

AIMS

The aim of this study was to provide guidance to improve the completeness and clarity of meta-ethnography reporting.

BACKGROUND

Evidence-based policy and practice require robust evidence syntheses which can further understanding of people's experiences and associated social processes. Meta-ethnography is a rigorous seven-phase qualitative evidence synthesis methodology, developed by Noblit and Hare. Meta-ethnography is used widely in health research, but reporting is often poor quality and this discourages trust in and use of its findings. Meta-ethnography reporting guidance is needed to improve reporting quality.

DESIGN

The eMERGe study used a rigorous mixed-methods design and evidence-based methods to develop the novel reporting guidance and explanatory notes.

METHODS

The study, conducted from 2015 - 2017, comprised of: (1) a methodological systematic review of guidance for meta-ethnography conduct and reporting; (2) a review and audit of published meta-ethnographies to identify good practice principles; (3) international, multidisciplinary consensus-building processes to agree guidance content; (4) innovative development of the guidance and explanatory notes.

FINDINGS

Recommendations and good practice for all seven phases of meta-ethnography conduct and reporting were newly identified leading to 19 reporting criteria and accompanying detailed guidance.

CONCLUSION

The bespoke eMERGe Reporting Guidance, which incorporates new methodological developments and advances the methodology, can help researchers to report the important aspects of meta-ethnography. Use of the guidance should raise reporting quality. Better reporting could make assessments of confidence in the findings more robust and increase use of meta-ethnography outputs to improve practice, policy, and service user outcomes in health and other fields. This is the first tailored reporting guideline for meta-ethnography. This article is being simultaneously published in the following journals: Journal of Advanced Nursing, Psycho-oncology, Review of Education, and BMC Medical Research Methodology.

Improving reporting of meta-ethnography: the eMERGe reporting guidance

AIMS

The aim of this study was to provide guidance to improve the completeness and clarity of meta-ethnography reporting.

BACKGROUND

Evidence-based policy and practice require robust evidence syntheses which can further understanding of people's experiences and associated social processes. Meta-ethnography is a rigorous seven-phase qualitative evidence synthesis methodology, developed by Noblit and Hare. Meta-ethnography is used widely in health research, but reporting is often poor quality and this discourages trust in and use of its findings. Meta-ethnography reporting guidance is needed to improve reporting quality.

DESIGN

The eMERGe study used a rigorous mixed-methods design and evidence-based methods to develop the novel reporting guidance and explanatory notes.

METHODS

The study, conducted from 2015 to 2017, comprised of: (1) a methodological systematic review of guidance for meta-ethnography conduct and reporting; (2) a review and audit of published meta-ethnographies to identify good practice principles; (3) international, multidisciplinary consensus-building processes to agree guidance content; (4) innovative development of the guidance and explanatory notes.

FINDINGS

Recommendations and good practice for all seven phases of meta-ethnography conduct and reporting were newly identified leading to 19 reporting criteria and accompanying detailed guidance.

CONCLUSION

The bespoke eMERGe Reporting Guidance, which incorporates new methodological developments and advances the methodology, can help researchers to report the important aspects of meta-ethnography. Use of the guidance should raise reporting quality. Better reporting could make assessments of confidence in the findings more robust and increase use of meta-ethnography outputs to improve practice, policy, and service user outcomes in health and other fields. This is the first tailored reporting guideline for meta-ethnography. This article is being simultaneously published in the following journals: Journal of Advanced Nursing, Psycho-oncology, Review of Education, and BMC Medical Research Methodology.

Community and Drug Distributor Perceptions and Experiences of Mass Drug Administration for the Elimination of Lymphatic Filariasis: A Rapid Review of Qualitative Research

STUDY OBJECTIVES

This article presents findings from a rapid review of qualitative research conducted to inform decision makers about community and drug distributor perceptions and experiences of mass drug administration campaigns for the elimination of lymphatic filariasis. We focused on questions related to acceptability of the mass drug administration campaigns within these groups and their thoughts around the feasibility of planning and carrying out the campaigns.

METHODS

We carried out a systematic search in five databases to identify potential studies. We included studies that focused on community members and drug distributors and used qualitative methods for data collection and analysis. We conducted a thematic framework analysis using the Supporting the Use of Research Evidence framework. Due to time constraints, one author conducted the screening, extraction and data analysis.

FINDINGS

Studies found that communities lack knowledge and information about lymphatic filariasis and the mass drug administration campaigns and that this can have an impact on how many community members choose to take medication. Health workers often had a good understanding that lymphatic filariasis was a problem in their setting, of its cause and mode of transmission and that hydrocele and elephantiasis had the same cause. However, this knowledge was not as prevalent amongst community drug distributors who often had misconceptions surrounding the topic. Furthermore, studies found that the length, timing, level of community and health system involvement, access to care for side effects, inadequate numbers of drug distributors and supervisors and motivation of drug distributors influenced participation in mass drug administration campaigns. Finally, the inadequate training of drug distributors could influence community trust in the mass drug administration program and the drug distributor themselves if there was a perception that the person was uninformed or not trained to carry out their tasks.

Are we entering a new era for qualitative research? Using qualitative evidence to support guidance and guideline development by the World Health Organization

Qualitative approaches are one of several methodologies utilised within the social sciences. New developments within qualitative methods are widening the opportunities for using qualitative evidence to inform health policy and systems decisions. In this commentary, we discuss how, in our work with the World Health Organization (WHO), we have explored ways of broadening the types of evidence used to develop evidence-informed guidance for health systems.Health systems decisions are commonly informed by evidence on the effectiveness of health system interventions. However, decision makers and other stakeholders also typically have additional questions, including how different stakeholders value different outcomes, the acceptability and feasibility of different interventions and the impacts of these interventions on equity and human rights. Evidence from qualitative research can help address these questions, and a number of WHO guidelines are now using qualitative evidence in this way. This growing use of qualitative evidence to inform decision making has been facilitated by recent methodological developments, including robust methods for qualitative evidence syntheses and approaches for assessing how much confidence to place in findings from such syntheses. For research evidence to contribute optimally to improving and sustaining the performance of health systems, it needs to be transferred easily between different elements of what has been termed the 'evidence ecosystem'. This ecosystem includes primary and secondary evidence producers, guidance developers and those implementing and evaluating interventions to strengthen health systems. We argue that most of the elements of an ecosystem for qualitative evidence are now in place - an important milestone that suggests that we are entering a new era for qualitative research. However, a number of challenges and constraints remain. These include how to build stronger links between the communities involved in the different parts of the qualitative evidence ecosystem and the need to strengthen capacity, particularly in low and middle income countries, to produce and utilise qualitative evidence and decision products informed by such evidence. We invite others who want to support the wider use of qualitative evidence in decision processes to look for opportunities in their settings to put this into practice.

Institutionalising an evidence-informed approach to guideline development: progress and challenges at the World Health Organization

This study explored experiences, perceptions and views among World Health Organization (WHO) staff about the changes, progress and challenges brought by the guideline development reforms initiated in 2007. Thirty-five semistructured interviews were conducted with senior WHO staff. Sixteen of the interviewees had in-depth experience with WHO's formal guideline development process. Thematic analysis was conducted to identify key themes in the qualitative data, and these were interpreted in the context of the existing literature on WHO's guideline development processes. First, the reforms were seen to have transformed and improved the quality of WHO's guidelines. Second, independent evaluation and feedback by the Guidelines Review Committee (GRC) was described to have strengthened the legitimacy of WHO's recommendations. Third, WHO guideline development processes are not yet designed to systematically make use of all types of research evidence needed to inform decisions about health systems and public health interventions. For example, several interviewees expressed dissatisfaction with the insufficient attention paid to qualitative evidence and evidence from programme experience, and how the Grading of Recommendations Assessment, Development and Evaluation (GRADE) process evaluates the quality of evidence from non-randomised study designs, while others believed that GRADE was just not properly understood or applied. Fourth, some staff advocated for a more centralised quality assurance process covering all outputs from WHO's departments and scientific advisory committees, especially to eliminate strategic efforts aimed at bypassing the GRC's requirements. Overall, the 'culture change' senior WHO staff called for over 10 years ago appears to have gradually spread throughout the organisation. However, at least two major challenges remain: (1) ensuring that all issued advice benefits from independent evaluation, monitoring and feedback for quality and (2) designing guideline development processes to better acquire, assess, adapt and apply the full range of evidence that can inform recommendations on health systems and public health interventions.

The GRADE Evidence to Decision (EtD) framework for health system and public health decisions

OBJECTIVE

To describe a framework for people making and using evidence-informed health system and public health recommendations and decisions.

BACKGROUND

We developed the GRADE Evidence to Decision (EtD) framework for health system and public health decisions as part of the DECIDE project, in which we simultaneously developed frameworks for these and other types of healthcare decisions, including clinical recommendations, coverage decisions and decisions about diagnostic tests.

DEVELOPING THE FRAMEWORK

Building on GRADE EtD tables, we used an iterative approach, including brainstorming, consultation of the literature and with stakeholders, and an international survey of policy-makers. We applied the framework to diverse examples, conducted workshops and user testing with health system and public health guideline developers and policy-makers, and observed and tested its use in real-life guideline panels.

FINDINGS

All the GRADE EtD frameworks share the same basic structure, including sections for formulating the question, making an assessment and drawing conclusions. Criteria listed in the assessment section of the health system and public health framework cover the important factors for making these types of decisions; in addition to the effects and economic impact of an option, the priority of the problem, the impact of the option on equity, and its acceptability and feasibility are important considerations that can inform both whether and how to implement an option. Because health system and public health interventions are often complex, detailed implementation considerations should be made when making a decision. The certainty of the evidence is often low or very low, but decision-makers must still act. Monitoring and evaluation are therefore often important considerations for these types of decisions. We illustrate the different components of the EtD framework for health system and public health decisions by presenting their application in a framework adapted from a real-life guideline.

DISCUSSION

This framework provides a structured and transparent approach to support policy-making informed by the best available research evidence, while making the basis for decisions accessible to those whom they will affect. The health system and public health EtD framework can also be used to facilitate dissemination of recommendations and enable decision-makers to adopt, and adapt, recommendations or decisions.

Applying GRADE-CERQual to qualitative evidence synthesis findings-paper 2: how to make an overall CERQual assessment of confidence and create a Summary of Qualitative Findings table

BACKGROUND

The GRADE-CERQual (Confidence in Evidence from Reviews of Qualitative research) approach has been developed by the GRADE (Grading of Recommendations Assessment, Development and Evaluation) Working Group. The approach has been developed to support the use of findings from qualitative evidence syntheses in decision making, including guideline development and policy formulation. CERQual includes four components for assessing how much confidence to place in findings from reviews of qualitative research (also referred to as qualitative evidence syntheses): (1) methodological limitations, (2) coherence, (3) adequacy of data and (4) relevance. This paper is part of a series providing guidance on how to apply CERQual and focuses on making an overall assessment of confidence in a review finding and creating a CERQual Evidence Profile and a CERQual Summary of Qualitative Findings table.

METHODS

We developed this guidance by examining the methods used by other GRADE approaches, gathering feedback from relevant research communities and developing consensus through project group meetings. We then piloted the guidance on several qualitative evidence syntheses before agreeing on the approach.

RESULTS

Confidence in the evidence is an assessment of the extent to which a review finding is a reasonable representation of the phenomenon of interest. Creating a summary of each review finding and deciding whether or not CERQual should be used are important steps prior to assessing confidence. Confidence should be assessed for each review finding individually, based on the judgements made for each of the four CERQual components. Four levels are used to describe the overall assessment of confidence: high, moderate, low or very low. The overall CERQual assessment for each review finding should be explained in a CERQual Evidence Profile and Summary of Qualitative Findings table.

CONCLUSIONS

Structuring and summarising review findings, assessing confidence in those findings using CERQual and creating a CERQual Evidence Profile and Summary of Qualitative Findings table should be essential components of undertaking qualitative evidence syntheses. This paper describes the end point of a CERQual assessment and should be read in conjunction with the other papers in the series that provide information on assessing individual CERQual components.

Further exploration of dissemination bias in qualitative research required to facilitate assessment within qualitative evidence syntheses

OBJECTIVES

To conceptualise and discuss dissemination bias in qualitative research.

RESULTS

It is likely that the mechanisms leading to dissemination bias in quantitative research, including time lag, language, gray literature, and truncation bias also contribute to dissemination bias in qualitative research. These conceptual considerations have informed the development of a research agenda.

CONCLUSION

Further exploration of dissemination bias in qualitative research is needed, including the extent of non-dissemination and related dissemination bias, and how to assess dissemination bias within qualitative evidence syntheses. We also need to consider the mechanisms through which dissemination bias in qualitative research could occur to explore approaches for reducing it.

Integrated opioid substitution therapy and HIV care: a qualitative systematic review and synthesis of client and provider experiences

People who use drugs in many contexts have limited access to opioid substitution therapy and HIV care. Service integration is one strategy identified to support increased access. We reviewed and synthesized literature exploring client and provider experiences of integrated opioid substitution therapy and HIV care to identify acceptable approaches to care delivery. We systematically reviewed qualitative literature. We searched nine bibliographic databases, supplemented by manual searches of reference lists of articles from the database search, relevant journals, conferences, key organizations and consultation with experts. Thematic synthesis was used to develop descriptive themes in client and provider experiences. The search yielded 11 articles for inclusion, along with 8 expert and policy reports. We identify five descriptive themes: the convenience and comprehensive nature of co-located care, contrasting care philosophies and their role in shaping integration, the limits to disclosure and communication between clients and providers, opioid substitution therapy enabling HIV care access and engagement, and health system challenges to delivering integrated services. The discussion explores how integrated opioid substitution therapy and HIV care needs to adapt to specific social conditions, rather than following universal approaches. We identify priorities for future research. Acceptable integrated opioid substitution therapy and HIV care for people who use drugs and providers is most likely through co-located care and relies upon attention to stigma, supportive relationships and client centred cultures of delivery. Further research is needed to understand experiences of integrated care, particularly delivery in low and middle income settings and models of care focused on community and non-clinic based delivery.