Development of a checklist for people communicating evidence-based information about the effects of healthcare interventions: a mixed methods study

[Advancing standards for the development of evidence-based health information for consumers: Needs and priorities among members of the Network for Evidence-based Medicine]

BACKGROUND

The Good Practice Guidelines for health information, Good Health Information Austria and Guideline Evidence-based Health Information are established resources for developing evidence-based health information in the German-speaking regions. The aim of this project was to capture challenges in applying these standards in practice, identify gaps and development needs and gain insights for their further development.

METHODS

In December 2020, members of the working group for patient information and involvement of the Network for Evidence-based Medicine were invited to share their experiences and needs in applying German standards for evidence-based health information through an online survey focussing on open questions (part 1: needs assessment). The feedback was analysed using qualitative content analysis and presented in a workshop at the EbM Congress 2021 with the goal of specifying the feedback from the needs assessment and discussing ideas for the further development of the standards (part 2: specification). In the final step, a second survey was conducted in February 2023 to prioritize the identified topics by the working group members (part 3: prioritization). The results were analysed descriptively.

RESULTS

Among the 41 participants, only 23% considered the standards to be sufficient, and only 55% found their application in the development of information to be easy or rather easy. The needs assessment and workshop (n=46) helped to identify various areas of action. With regard to the application of the standards, the following challenges were identified: lack of user orientation, content gaps, methods and risk communication. Gaps in the standards were identified regarding formats, content, and the connection to healthcare provision. For the advancement of the standards ideas for additional content, stakeholder involvement, and improvement of the usability of the standards were identified. In the prioritization survey, the topic areas, "presenting benefits and harms" and "content beyond treatments (in particular, diagnostics and prognosis)" were considered to be the most important (n=36).

DISCUSSION

Among members of the working group for patient information and involvement of the German Network for Evidence-based Medicine, a high demand has been identified for the further development of standards for creating evidence-based health information. In addition to content development, the integration of existing documents and tools should also be considered, including products issued by other institutions. The success of advancing the standards also depends on improving their applicability - for example through an attractive online platform. The results are limited by the sample which only included members of the EbM Network's patient information and participation working group and a limited response rate.

CONCLUSIONS

The needs assessment showed that the currently established standards and recommendations for the development of evidence-based health information in the German-speaking regions represent important cornerstones but need to be expanded to answer more practice-oriented questions. The challenges and proposed solutions stated by the participants can help further develop the standards. The prioritization can be used to set priorities for the development of the standards, guide the order of possible work packages and allocate resources.

Public health leadership in the COVID-19 era: how does it fit? A scoping review

The COVID-19 pandemic has put a lot of pressure on all the world's health systems and public health leaders who have often found themselves unprepared to handle an emergency of this magnitude. This study aims to bring together published evidence on the qualities required to leaders to deal with a public health issue like the COVID-19 pandemic. This scoping literature review was conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews checklist. A search of relevant articles was performed in the PubMed, Scopus and Web of Science databases. A total of 2499 records were screened, and 45 articles were included, from which 93 characteristics of effective leadership were extrapolated and grouped into 6 clusters. The qualities most frequently reported in the articles were human traits and emotional intelligence (46.7%) and communication skills such as transparency and reliability (48.9%). Responsiveness and preparedness (40%), management skills (33.3%) and team working (35.6%) are considered by a significant percentage of the articles as necessary for the construction of rapid and effective measures in response to the emergency. A considerable proportion of articles also highlighted the need for leaders capable of making evidence-based decisions and driving innovation (31.1%). Although identifying leaders who possess all the skills described in this study appears complex, determining the key characteristics of effective public health leadership in a crisis, such as the COVID-19 pandemic, is useful not only in selecting future leaders but also in implementing training and education programmes for the public health workforce.

Strategies for communicating scientific evidence on healthcare to managers and the population: a scoping review

BACKGROUND

Health evidence needs to be communicated and disseminated in a manner that is clearly understood by decision-makers. As an inherent component of health knowledge translation, communicating results of scientific studies, effects of interventions and health risk estimates, in addition to understanding key concepts of clinical epidemiology and interpreting evidence, represent a set of essential instruments to reduce the gap between science and practice. The advancement of digital and social media has reshaped the concept of health communication, introducing new, direct and powerful communication platforms and gateways between researchers and the public. The objective of this scoping review was to identify strategies for communicating scientific evidence in healthcare to managers and/or population.

METHODS

We searched Cochrane Library, Embase®, MEDLINE® and other six electronic databases, in addition to grey literature, relevant websites from related organizations for studies, documents or reports published from 2000, addressing any strategy for communicating scientific evidence on healthcare to managers and/or population.

RESULTS

Our search identified 24 598 unique records, of which 80 met the inclusion criteria and addressed 78 strategies. Most strategies focused on risk and benefit communication in health, were presented by textual format and had been implemented and somehow evaluated. Among the strategies evaluated and appearing to yield some benefit are (i) risk/benefit communication: natural frequencies instead of percentages, absolute risk instead relative risk and number needed to treat, numerical instead nominal communication, mortality instead survival; negative or loss content appear to be more effective than positive or gain content; (ii) evidence synthesis: plain languages summaries to communicate the results of Cochrane reviews to the community were perceived as more reliable, easier to find and understand, and better to support decisions than the original summaries; (iii) teaching/learning: the Informed Health Choices resources seem to be effective for improving critical thinking skills.

CONCLUSION

Our findings contribute to both the knowledge translation process by identifying communication strategies with potential for immediate implementation and to future research by recognizing the need to evaluate the clinical and social impact of other strategies to support evidence-informed policies. Trial registration protocol is prospectively available in MedArxiv (doi.org/10.1101/2021.11.04.21265922).

How to present work productivity loss results from clinical trials for patients and caregivers? A mixed methods approach

OBJECTIVES

From the perspectives of patients and caregivers, the objectives were: identifying which result presentations, describing work productivity loss (WPL) outcomes, are most understandable; measuring which presentations are important to report; and investigating which WPL outcomes are viewed as important alongside clinical trials results.

METHODS

We used a four phased, sequential mixed methods design, guided by patient-oriented research engaging one patient partner. We conducted think-aloud interviews, in British Columbia/Canada, to review WPL results and our survey measuring the understandability and importance of the results, and importance of each WPL outcome. We surveyed a sample representing working Canadians. The findings were summarized and analyzed using linear and logistic regression. We conducted sub-group analyses; one was gender based. All regressions were conducted using generalized estimating equations.

RESULTS

In our qualitative phases, 20 patients and caregivers were interviewed. Participants recommended for the results to be brief, simple, and represented visually. Then, 118 patients and 120 caregivers were surveyed. The results presented in days or cost yielded the highest understandability and importance to report. All WPL outcomes were identified as important to somewhat important to report by most. The associations indicated that the more understandable the result presentation was, the more likely it was to be rated as important. Age was the only factor significantly associated with selecting days or cost as the most important result.

CONCLUSION

Presenting WPL results in days and cost, using lay terms and visual supports, were viewed as easiest to understand and most important to report in clinical trials by patients and caregivers. Our findings are supportive of clinical trials standardizing the measurement of WPL to include all of its outcomes (absenteeism, presenteeism, employment status changes and total work productivity loss), in addition to tools assessing the comprehensiveness of WPL results to be provided to patients and caregivers.

Communication of anticancer drug benefits and related uncertainties to patients and clinicians: document analysis of regulated information on prescription drugs in Europe

OBJECTIVE

To evaluate the frequency with which relevant and accurate information about the benefits and related uncertainties of anticancer drugs are communicated to patients and clinicians in regulated information sources in Europe.

DESIGN

Document content analysis.

SETTING

European Medicines Agency.

PARTICIPANTS

Anticancer drugs granted a first marketing authorisation by the European Medicines Agency, 2017-19.

MAIN OUTCOME MEASURES

Whether written information on a product addressed patients' commonly asked questions about: who and what the drug is used for; how the drug was studied; types of drug benefit expected; and the extent of weak, uncertain, or missing evidence for drug benefits. Information on drug benefits in written sources for clinicians (summaries of product characteristics), patients (patient information leaflets), and the public (public summaries) was compared with information reported in regulatory assessment documents (European public assessment reports).

RESULTS

29 anticancer drugs that received a first marketing authorisation for 32 separate cancer indications in 2017-19 were included. General information about the drug (including information on approved indications and how the drug works) was frequently reported across regulated information sources aimed at both clinicians and patients. Nearly all summaries of product characteristics communicated full information to clinicians about the number and design of the main studies, the control arm (if any), study sample size, and primary measures of drug benefit. None of the patient information leaflets communicated information to patients about how drugs were studied. 31 (97%) summaries of product characteristics and 25 (78%) public summaries contained information about drug benefits that was accurate and consistent with information in regulatory assessment documents. The presence or absence of evidence that a drug extended survival was reported in 23 (72%) summaries of product characteristics and four (13%) public summaries. None of the patient information leaflets communicated information about the drug benefits that patients might expect based on study findings. Scientific concerns about the reliability of evidence on drug benefits, which were raised by European regulatory assessors for almost all drugs in the study sample, were rarely communicated to clinicians, patients, or the public.

CONCLUSIONS

The findings of this study highlight the need to improve the communication of the benefits and related uncertainties of anticancer drugs in regulated information sources in Europe to support evidence informed decision making by patients and their clinicians.

Demands on Health Information and Clinical Practice Guidelines for Patients from the Perspective of Adults with Mental Illness and Family Members: A Qualitative Study with In-Depth Interviews

(1) Background: "Patient health information" promote health literacy. "Patient guidelines" as a sub group reflect the current evidence about illnesses and treatment options adapted to the needs of laypersons. Little is known about factors promoting and hindering their use by people affected by mental illness and their relatives. (2) Methods: Telephone interviews (N = 15; n = 4 adults affected by mental illness, n = 5 relatives, n = 6 both applicable) were conducted according to the Sørensen model of health literacy. Data were recorded, transcribed and content-analyzed following Mayring. (3) Results: Health information is used regularly by individuals affected by mental illness and their relatives, but "patient guidelines" are largely unknown. Yet, there is a great willingness to use them. Main barriers are a lack of statistical knowledge, the complexity of health-related topics and cognitive impairment sometimes accompanying mental illnesses. Target group-oriented adaptation as well as transparent and even-handed presentation of (dis-)advantages of treatment options can increase trust. (4) Conclusions: Health information and guidelines can help affected persons and relatives to make treatment decisions by conveying unbiased, up-to-date knowledge. Target group-specific adaptations should be made for psychiatric illnesses and features specific to mental illnesses compared to physical illnesses should be included. Clinical practice guidelines must be distributed more widely to increase their impact.

Bursting out of our bubble: using creative techniques to communicate within the systematic review process and beyond

BACKGROUND

Increasing pressure to publicise research findings and generate impact, alongside an expectation from funding bodies to go beyond publication within academic journals, has generated interest in alternative methods of science communication. Our aim is to describe our experience of using a variety of creative communication tools, reflect on their use in different situations, enhance learning and generate discussion within the systematic review community.

METHODS

Over the last 5 years, we have explored several creative communication tools within the systematic review process and beyond to extend dissemination beyond traditional academic mechanisms. Central to our approach is the co-production of a communication plan with potential evidence users which facilitates (i) the identification of key messages for different audiences, (ii) discussion of appropriate tools to communicate key messages and (iii) exploration of avenues to share them. We aim to involve evidence users in the production of a variety of outputs for each research project cognisant of the many ways in which individuals engage with information.

RESULTS

Our experience has allowed us to develop an understanding of the benefits and challenges of a wide range of creative communication tools. For example, board games can be a fun way of learning, may flatten power hierarchies between researchers and research users and enable sharing of large amounts of complex information in a thought provoking way, but they are time and resource intensive both to produce and to engage with. Conversely, social media shareable content can be quick and easy to produce and to engage with but limited in the depth and complexity of shareable information.

DISCUSSION

It is widely recognised that most stakeholders do not have time to invest in reading large, complex documents; creative communication tools can be a used to improve accessibility of key messages. Furthermore, our experience has highlighted a range of additional benefits of embedding these techniques within our project processes e.g. opening up two-way conversations with end-users of research to discuss the implications of findings.

Health communication in and out of public health emergencies: to persuade or to inform?

Much health communication during the COVID-19 pandemic has been designed to persuade people more than to inform them. For example, messages like “masks save lives” are intended to compel people to wear face masks, not to enable them to make an informed decision about whether to wear a face mask or to understand the justification for a mask mandate. Both persuading people and informing them are reasonable goals for health communication. However, those goals can sometimes be in conflict. In this article, we discuss potential conflicts between seeking to persuade or to inform people, the use of spin to persuade people, the ethics of persuasion, and implications for health communication in the context of the pandemic and generally. Decisions to persuade people rather than enable them to make an informed choice may be justified, but the basis for those decisions should be transparent and the evidence should not be distorted. We suggest nine principles to guide decisions by health authorities about whether to try to persuade people.

Assessing the impact of knowledge communication and dissemination strategies targeted at health policy-makers and managers: an overview of systematic reviews

Background

The use of research evidence as an input for health decision-making is a need for most health systems. There are a number of approaches for promoting evidence use at different levels of the health system, but knowledge of their effectiveness is still scarce. The objective of this overview was to evaluate the effectiveness of knowledge communication and dissemination interventions, strategies or approaches targeting policy-makers and health managers.

Methods

This overview of systematic reviews used systematic review methods and was conducted according to a predefined and published protocol. A comprehensive electronic search of 13 databases and a manual search in four websites were conducted. Both published and unpublished reviews in English, Spanish or Portuguese were included. A narrative synthesis was undertaken, and effectiveness statements were developed, informed by the evidence identified.

Results

We included 27 systematic reviews. Three studies included only a communication strategy, while eight only included dissemination strategies, and the remaining 16 included both. None of the selected reviews provided “sufficient evidence” for any of the strategies, while four provided some evidence for three communication and four dissemination strategies. Regarding communication strategies, the use of tailored and targeted messages seemed to successfully lead to changes in the decision-making practices of the target audience. Regarding dissemination strategies, interventions that aimed at improving only the reach of evidence did not have an impact on its use in decisions, while interventions aimed at enhancing users’ ability to use and apply evidence had a positive effect on decision-making processes. Multifaceted dissemination strategies also demonstrated the potential for changing knowledge about evidence but not its implementation in decision-making.

Conclusions

There is limited evidence regarding the effectiveness of interventions targeting health managers and policy-makers, as well as the mechanisms required for achieving impact. More studies are needed that are informed by theoretical frameworks or specific tools and using robust methods, standardized outcome measures and clear descriptions of the interventions. We found that passive communication increased access to evidence but had no effect on uptake. Some evidence indicated that the use of targeted messages, knowledge-brokering and user training was effective in promoting evidence use by managers and policy-makers.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12961-021-00780-4.

The effects of quality of evidence communication on perception of public health information about COVID-19: Two randomised controlled trials

Background

The quality of evidence about the effectiveness of non-pharmaceutical health interventions is often low, but little is known about the effects of communicating indications of evidence quality to the public.

Methods

In two blinded, randomised, controlled, online experiments, US participants (total n = 2140) were shown one of several versions of an infographic illustrating the effectiveness of eye protection in reducing COVID-19 transmission. Their trust in the information, understanding, feelings of effectiveness of eye protection, and the likelihood of them adopting it were measured.

Findings

Compared to those given no quality cues, participants who were told the quality of the evidence on eye protection was ‘low’, rated the evidence less trustworthy (p = .001, d = 0.25), and rated it as subjectively less effective (p = .018, d = 0.19). The same effects emerged compared to those who were told the quality of the evidence was ‘high’, and in one of the two studies, those shown ‘low’ quality of evidence said they were less likely to use eye protection (p = .005, d = 0.18). Participants who were told the quality of the evidence was ‘high’ showed no statistically significant differences on these measures compared to those given no information about evidence quality.

Conclusions

Without quality of evidence cues, participants responded to the evidence about the public health intervention as if it was high quality and this affected their subjective perceptions of its efficacy and trust in the provided information. This raises the ethical dilemma of weighing the importance of transparently stating when the evidence base is actually low quality against evidence that providing such information can decrease trust, perception of intervention efficacy, and likelihood of adopting it.

Quality of information in news media reports about the effects of health interventions: Systematic review and meta-analyses

Background Many studies have assessed the quality of news reports about the effects of health interventions, but there has been no systematic review of such studies or meta-analysis of their results. We aimed to fill this gap (PROSPERO ID: CRD42018095032). Methods We included studies that used at least one explicit, prespecified and generic criterion to assess the quality of news reports in print, broadcast, or online news media, and specified the sampling frame, and the selection criteria and technique. We assessed criteria individually for inclusion in the meta-analyses, excluding ineligible criteria and criteria with inadequately reported results. We mapped and grouped criteria to facilitate evidence synthesis. Where possible, we extracted the proportion of news reports meeting the included criterion. We performed meta-analyses using a random effects model to estimate such proportions for individual criteria and some criteria groups, and to characterise heterogeneity across studies.  Results We included 44 primary studies in the review, and 18 studies and 108 quality criteria in the meta-analyses. Many news reports gave an unbalanced and oversimplified picture of the potential consequences of interventions. A limited number mention or adequately address conflicts of interest (22%; 95% CI 7%-49%) (low certainty), alternative interventions (36%; 95% CI 26%-47%) (moderate certainty), potential harms (40%; 95% CI 23%-61%) (low certainty), or costs (18%; 95% CI 12%-28%) (moderate certainty), or quantify effects (53%; 95% CI 36%-69%) (low certainty) or report absolute effects (17%; 95% CI 4%-49%) (low certainty).  Discussion There is room for improving health news, but it is logically more important to improve the public's ability to critically appraise health information and make judgements for themselves.

Quality of information in news media reports about the effects of health interventions: Systematic review and meta-analyses

Background Many studies have assessed the quality of news reports about the effects of health interventions, but there has been no systematic review of such studies or meta-analysis of their results. We aimed to fill this gap (PROSPERO ID: CRD42018095032). Methods We included studies that used at least one explicit, prespecified and generic criterion to assess the quality of news reports in print, broadcast, or online news media, and specified the sampling frame, and the selection criteria and technique. We assessed criteria individually for inclusion in the meta-analyses, excluding inappropriate criteria and criteria with inadequately reported results. We mapped and grouped criteria to facilitate evidence synthesis. Where possible, we extracted the proportion of news reports meeting the included criterion. We performed meta-analyses using a random effects model to estimate such proportions for individual criteria and some criteria groups, and to characterise heterogeneity across studies.  Results We included 44 primary studies in the qualitative summary, and 18 studies and 108 quality criteria in the meta-analyses. Many news reports gave an unbalanced and oversimplified picture of the potential consequences of interventions. A limited number mention or adequately address conflicts of interest (22%; 95% CI 7%-49%) (low certainty), alternative interventions (36%; 95% CI 26%-47%) (moderate certainty), potential harms (40%; 95% CI 23%-61%) (low certainty), or costs (18%; 95% CI 12%-28%) (moderate certainty), or quantify effects (53%; 95% CI 36%-69%) (low certainty) or report absolute effects (17%; 95% CI 4%-49%) (low certainty).  Discussion There is room for improving health news, but it is logically more important to improve the public's ability to critically appraise health information and make judgements for themselves.

The reporting checklist for public versions of guidelines: RIGHT-PVG

BACKGROUND

Public or patient versions of guidelines (PVGs) are derivative documents that "translate" recommendations and their rationale from clinical guidelines for health professionals into a more easily understandable and usable format for patients and the public. PVGs from different groups and organizations vary considerably in terms of quality of their reporting. In order to address this issue, we aimed to develop a reporting checklist for developers of PVGs and other potential users.

METHODS

First, we collected a list of potential items through reviewing a sample of PVGs, existing guidance for developing and reporting PVGs or other similar evidence-based patient tools, as well as qualitative studies on original studies of patients' needs about the content and/or reporting of information in PVGs or similar evidence-based patient tools. Second, we conducted a two-round Delphi consultation to determine the level of consensus on the items to be included in the final reporting checklist. Third, we invited two external reviewers to provide comments on the checklist.

RESULTS

We generated the initial list of 45 reporting items based on a review of a sample of 30 PVGs, four PVG guidance documents, and 46 relevant studies. After the two-round Delphi consultation, we formed a checklist of 17 items grouped under 12 topics for reporting PVGs.

CONCLUSION

The RIGHT-PVG reporting checklist provides an international consensus on the important criteria for reporting PVGs.

A systematic review of risk communication in clinical trials: How does it influence decisions to participate and what are the best methods to improve understanding in a trial context?

Background

Effective risk communication is challenging. Ensuring potential trial participants’ understand ‘risk’ information presented to them is a key aspect of the informed consent process within clinical trials, yet minimal research has looked specifically at how to communicate probabilities to support decisions about trial participation. This study reports a systematic review of the literature focusing on presentation of probabilistic information or understanding of risk by potential trial participants.

Methods

A search strategy for risk communication in clinical trials was designed and informed by systematic reviews of risk communication in treatment and screening contexts and supplemented with trial participation terms. Extracted data included study characteristics and the main interventions/findings of each study. Explanatory studies that investigated the methods for presenting probabilistic information within participant information leaflets for a clinical trial were included, as were interventions that focused on optimising understanding of probabilistic information within the context of a clinical trial.

Results

The search strategy identified a total of 4931 studies. Nineteen papers were selected for full text screening, and seven studies included. All reported results from risk communication studies that aimed to support potential trial participants’ decision making set within hypothetical trials. Five of these were randomised comparisons of risk communication interventions, and two were prospectively designed, non-randomised studies. Study interventions focused on probability presentation, risk framing and risk interpretation with a wide variety of interventions being evaluated and considerable heterogeneity in terms of outcomes assessed. Studies show conflicting findings when it comes to how best to present information, although numerical, particularly frequency formats and some visual aids appear to have promise.

Conclusions

The evidence base surrounding risk communication in clinical trials indicates that there is as yet no clear optimal method for improving participant understanding, or clear consensus on how it affects their willingness to participate. Further research into risk communication within trials is needed to help illuminate the mechanisms underlying risk perception and understanding and provide appropriate ways to present and communicate risk in a trial context so as to further promote informed choices about participation. A key focus for future research should be to investigate the potential for learning in the evidence on risk communication from treatment and screening decisions when applied to decisions about trial participation.