Comparison of the Informed Health Choices Key Concepts Framework to other frameworks relevant to teaching and learning how to think critically about health claims and choices: a systematic review

Public participation in decisions about measures to manage the COVID-19 pandemic: a systematic review

BACKGROUND

During the COVID-19 pandemic, governments and health authorities faced tough decisions about infection prevention and control measures such as social distancing, face masks and travel. Judgements underlying those decisions require democratic input, as well as expert input. The aim of this review is to inform decisions about how best to achieve public participation in decisions about public health and social interventions in the context of a pandemic or other public health emergencies.

OBJECTIVES

To systematically review examples of public participation in decisions by governments and health authorities about how to control the COVID-19 pandemic.

DESIGN

We searched Participedia and relevant databases in August 2022. Two authors reviewed titles and abstracts and one author screened publications promoted to full text. One author extracted data from included reports using a standard data-extraction form. A second author checked 10% of the extraction forms. We conducted a structured synthesis using framework analysis.

RESULTS

We included 24 reports (18 from Participedia). Most took place in high-income countries (n=23), involved 'consulting' the public (n=17) and involved public meetings (usually online). Two initiatives reported explicit support for critical thinking. 11 initiatives were formally evaluated (only three reported impacts). Many initiatives did not contribute to a decision, and 17 initiatives did not include any explicit decision-making criteria.

CONCLUSIONS

Decisions about how to manage the COVID-19 pandemic affected nearly everyone. While public participation in those decisions had the potential to improve the quality of the judgements and decisions that were made, build trust, improve adherence and help ensure transparency and accountability, few examples of such initiatives have been reported and most of those have not been formally evaluated. Identified initiatives did point out potential good practices related to online engagement, crowdsourcing and addressing potential power imbalance. Future research should address improved reporting of initiatives, explicit decision-making criteria, support for critical thinking, engagement of marginalised groups and decision-makers and communication with the public.

PROSPERO REGISTRATION NUMBER

358991.

Key concepts for informed health choices: Where's the evidence?

Background

The Informed Health Choices (IHC) Key Concepts is a framework that provides a basis for developing educational resources and evaluating people's ability to think critically about health actions. We developed the original Key Concepts framework by reviewing texts and checklists for the public, journalists, and health professionals and collecting structured feedback from an international advisory group. We revised the original 2015 framework yearly from 2016 to 2018 based on feedback and experience using the framework. The objectives of this paper are to describe the development of the framework since 2018 and summarise their basis.

Methods

For the 2019 version, we responded to feedback on the 2018 version. For the current 2022 version, in addition to responding to feedback on the 2019 version, we reviewed the evidence base for each of the concepts. Whenever possible, we referenced systematic reviews that provide a basis for a concept. We screened all Cochrane methodology reviews and searched Epistemonikos, PubMed, and Google Scholar for methodology reviews and meta-epidemiological studies.

Results

The original framework included 32 concepts in six groups. The 2019 version and the current 2022 version include 49 concepts in the same three main groups that we have used since 2016. There are now 10 subgroups or higher-level concepts. For each concept, there is an explanation including one or more examples, the basis for the concept, and implications. Over 600 references are cited that support the concepts, and over half of the references are systematic reviews.

Conclusions

There is a large body of evidence that supports the IHC key concepts and we have received few suggestions for changes since 2019.

Evaluating health claim assessment skills of parents with preschool children: A cross-sectional study using Informed Health Choices Claim Evaluation Tool

Background

Health literacy is a global problem and is particularly relevant when making health care decisions about small children. We analysed how parents of preschool children assess health claims and explored the predictors of their assessment skill.

Methods

We conducted a survey with questions from the Claim Evaluation Tools (CET) database, part of the Informed Health Choices (IHC) project, in ten paediatric primary care practices of the Split-Dalmatia County Health Center, Split, Croatia, from 1 February to 31 March 2023. Eligible participants were parents accompanying preschool-aged children for check-ups. We also collected data on parents' and children's demographic and health characteristics (including the presence of any chronic illness in the child), visits to paediatric emergency service, hospitalisations, vaccination status, the presence of chronic illness of parents or relatives, and whether parents had to made treatment decisions for themselves and/or their family member.

Results

Overall, 402 parents of preschool children (median age 35 years (interquartile range (IQR) = 31.0-38.3)) had a median IHC CET test score of 10.0 (IQR = 8.0-11.0) out of 12 questions. The multiple regression analysis showed that female gender, higher level of education, being employed, and having a history of a visit to paediatric emergency service were significant predictors of the test score, explaining 21.9% of the variance.

Conclusions

Parents of preschool children have a very good ability for critical assessment of health-related statements in a complex health care system and an environment of generally unsatisfactory health literacy. Further studies should explore how parents understand health claims in different geographical, socio-economic and cultural setting, and explore educational interventions to increase critical thinking abilities and informed decision-making, especially among fathers, unemployed parents and those with lower levels of education.

An evidence-informed policymaking (EIPM) competency profile for the Brazilian Health System developed through consensus: process and outcomes

BACKGROUND

Evidence-informed policymaking (EIPM) requires a set of individual and organizational knowledge, skills and attitudes that should be articulated with background factors and needs. In this regard, the development of an EIPM competency profile is important to support the diagnosis, planning and implementation of EIPM.

PURPOSE

To present the process and outcomes of the development of an EIPM competency profile by an expert committee, to be applied in different contexts of the Brazilian Health System.

METHODS

A committee of experts in EIPM shared different views, experiences and opinions to develop an EIPM competency profile for Brazil. In six consensus workshops mediated by facilitators, the committee defined from macro problems to key actions and performances essential for the competency profile. The development steps consisted of: (1) Constitution of the committee, including researchers, professionals with practical experience, managers, and educators; (2) Development of a rapid review on EIPM competency profiles; (3) Agreement on commitments and responsibilities in the processes; (4) Identification and definition of macro problems relating to the scope of the competency profile; and (5) Outlining of general and specific capacities, to be incorporated into the competency profile, categorized by key actions.

RESULTS

The development of the EIPM competency profile was guided by the following macro problems: (1) lack of systematic and transparent decision-making processes in health policy management; (2) underdeveloped institutional capacity for knowledge management and translation; and (3) incipient use of scientific evidence in the formulation and implementation of health policies. A general framework of key actions and performances of the EIPM Competency Profile for Brazil was developed, including 42 specific and general key actions distributed by area of activity (Health Management, Scientific Research, Civil Society, Knowledge Translation, and Cross-sectional areas).

CONCLUSIONS

The competency profile presented in this article can be used in different contexts as a key tool for the institutionalization of EIPM.

Strategies for communicating scientific evidence on healthcare to managers and the population: a scoping review

BACKGROUND

Health evidence needs to be communicated and disseminated in a manner that is clearly understood by decision-makers. As an inherent component of health knowledge translation, communicating results of scientific studies, effects of interventions and health risk estimates, in addition to understanding key concepts of clinical epidemiology and interpreting evidence, represent a set of essential instruments to reduce the gap between science and practice. The advancement of digital and social media has reshaped the concept of health communication, introducing new, direct and powerful communication platforms and gateways between researchers and the public. The objective of this scoping review was to identify strategies for communicating scientific evidence in healthcare to managers and/or population.

METHODS

We searched Cochrane Library, Embase®, MEDLINE® and other six electronic databases, in addition to grey literature, relevant websites from related organizations for studies, documents or reports published from 2000, addressing any strategy for communicating scientific evidence on healthcare to managers and/or population.

RESULTS

Our search identified 24 598 unique records, of which 80 met the inclusion criteria and addressed 78 strategies. Most strategies focused on risk and benefit communication in health, were presented by textual format and had been implemented and somehow evaluated. Among the strategies evaluated and appearing to yield some benefit are (i) risk/benefit communication: natural frequencies instead of percentages, absolute risk instead relative risk and number needed to treat, numerical instead nominal communication, mortality instead survival; negative or loss content appear to be more effective than positive or gain content; (ii) evidence synthesis: plain languages summaries to communicate the results of Cochrane reviews to the community were perceived as more reliable, easier to find and understand, and better to support decisions than the original summaries; (iii) teaching/learning: the Informed Health Choices resources seem to be effective for improving critical thinking skills.

CONCLUSION

Our findings contribute to both the knowledge translation process by identifying communication strategies with potential for immediate implementation and to future research by recognizing the need to evaluate the clinical and social impact of other strategies to support evidence-informed policies. Trial registration protocol is prospectively available in MedArxiv (doi.org/10.1101/2021.11.04.21265922).

Piloting the informed health choices resources in Barcelona primary schools: A mixed methods study

INTRODUCTION

The main objective of the Informed Health Choices (IHC) project is to teach people to assess treatment claims and make informed health choices. For this purpose, the IHC learning resources were developed for primary school children. The aim of this study is to explore students' and teachers' experience when using the IHC resources in primary schools in Barcelona (Spain).

METHODS

We conducted a mixed methods study for piloting the IHC resources in a convenience sample of primary schools in Barcelona. The intervention included a workshop with teachers, and nine lessons with students. We collected data using multiple approaches. We performed quantitative and qualitative analyses, and integrated the findings in a joint display. Finally, we formulated recommendations for using the IHC resources in this setting.

RESULTS

Two schools, with a total of 143 students in 4th and 5th grade and six teachers, participated in the study. One school followed the suggested IHC teaching plan and competed all the lessons; the other school modified the plan substantially and did not complete all the lessons. Overall, students and teachers from both schools understood, were interested in, and were able to apply the content of the lessons. During the lessons, the textbook was useful for students; nevertheless, for the teachers, the usefulness of the IHC resources was variable. Teachers adapted the IHC resources to increase student participation and used Information and Communications Technologies tools. We observed more facilitators than barriers to teach the lessons. The teachers suggested some ideas to improve the lessons based on activities they developed and implemented. The integration analysis showed great convergence of the quantitative and qualitative findings. We propose seven recommendations for using the IHC resources in this setting.

CONCLUSIONS

Students and teachers from primary schools in Barcelona showed a positive experience when using IHC resources; however, these resources should be adapted to promote classroom participation.

Developing a questionnaire to evaluate the health information literacy in China

Introduction

Health information literacy is critical for individuals to obtain, understand, screen, and apply health information. However, there is currently no specific tool available to evaluate all four dimensions of health information literacy in China. Public health emergencies can present an opportunity to evaluate and monitor the health information literacy level of residents. Therefore, this study aimed to develop a questionnaire to evaluate the level of health information literacy and to measure the reliability and validity.

Methods

The development process of the questionnaire consisted of the determination of questionnaire items, expert consultation, and validation. Based on the National Residents Health Literacy Monitoring Questionnaire (2020) and the 2019 Informed Health Choices key concepts, the researchers drafted the questionnaire, including all four dimensions of health information literacy. Experts in relevant fields were invited to evaluate the draft questionnaire, and revisions were made accordingly. Finally, the reliability and validity of the finalized version were examined in Gansu Province, China.

Results

The research team preliminarily formulated 14 items encompassing the four dimensions of health information literacy. After consulting with 28 experts, modifications were made. A convenience sample of 185 Chinese residents was invited to participate. Cronbach's alpha coefficient was 0.715 and McDonald's omega was 0.739 for internal consistency, and the test-retest intra-class correlation coefficient after 4 weeks was 0.906, indicating that the questionnaire content and measurement structure was relatively stable.

Conclusion

This questionnaire is the first evidence-based assessment tool developed for monitoring health information literacy in China, and it has shown good reliability and validity. It can help to monitor the health information literacy levels of Chinese residents, promote evidence-based decision-making, and guide interventions to improve health information literacy.

Competency profiles for evidence-informed policy-making (EIPM): a rapid review

BACKGROUND

Evidence-informed policy-making (EIPM) requires a set of individual and organizational capacities, linked with background factors and needs. The identification of essential knowledge, skills and attitudes for EIPM can support the development of competency profiles and their application in different contexts.

PURPOSE

To identify elements of competency (knowledge, skills and attitudes) for EIPM, according to different professional profiles (researcher, health professional, decision-maker and citizen).

METHODS

Rapid umbrella review. A structured search was conducted and later updated in two comprehensive repositories (BVSalud and PubMed). Review studies with distinctive designs were included, published from 2010 onwards, without language restrictions. Assessment of the methodological quality of the studies was not performed. A meta-aggregative narrative synthesis was used to report the findings.

RESULTS

Ten reviews were included. A total of 37 elements of competency were identified, eight were categorized as knowledge, 19 as skills and 10 as attitudes. These elements were aggregated into four competency profiles: researcher, health professional, decision-maker and citizen. The competency profiles included different sets of EIPM-related knowledge, skills and attitudes.

STRENGTHS AND LIMITATIONS

This study is innovative because it aggregates different profiles of competency from a practical perspective, favouring the application of its results in different contexts to support EIPM. Methodological limitations are related to the shortcuts adopted in this review: complementary searches of the grey literature were not performed, and the study selection and data extraction were not conducted in duplicate.

FINAL CONSIDERATIONS

CONCLUSIONS AND IMPLICATIONS OF THE FINDINGS: EIPM requires the development of individual and organizational capacities. This rapid review contributes to the discussion on the institutionalization of EIPM in health systems. The competency profiles presented here can support discussions about the availability of capacity and the need for its development in different contexts.

Evaluating student's ability to assess treatment claims: validating a German version of the Claim Evaluation Tools

BACKGROUND

The Claim Evaluation Tools measure the ability to assess claims about treatment effects. The aim of this study was to adapt the German item sets to the target group of secondary school students (aged 11 to 16 years, grade 6 to 10) and to validate them accordingly. The scale's reliability and validity using Rasch's probabilistic test theory should be determined.

METHODS

We conducted a sequential mixed-method study comprising three stages: contextualisation and adaption of the items (stage 1), piloting of the item sets using qualitative interviews (stage 2) and a construct validation by testing the unidimensional Rasch scalability for each item set after data collection in one secondary school in Germany and two secondary schools in Austria. We explored summary and individual fit statistics and performed a distractor analysis (stage 3).

RESULTS

Secondary school students (n = 6) and their teachers (n = 5) participated in qualitative interviews in Germany. The qualitative interviews identified the need for minor modifications (e.g. reducing thematic repetitions, changing the order of the items). The data of 598 German and Austrian secondary school students were included to test for Rasch scalability. Rasch analyses showed acceptable overall model fit. Distractor analyses suggested that model fit could be improved by simplifying the text in the scenarios, removing and editing response options of some items.

CONCLUSION

After the revision of some items, the questionnaires are suitable to evaluate secondary school students' ability to assess health claims. A future goal is to increase the pool of items being translated and tested.

Prioritising Informed Health Choices Key Concepts for those impacted by cancer: a protocol

Background: Few areas of health have been as insidiously influenced by misinformation as cancer. Thus, interventions that can help people impacted by cancer reduce the extent to which they are victims of misinformation are necessary. The Informed Health Choices (IHC) initiative has developed Key Concepts that can be used in the development of interventions for evaluating the trustworthiness of claims about the effects of health treatments. We are developing an online education programme called Informed Health Choices-Cancer (IHC-C) based on the IHC Key Concepts. We will provide those impacted by cancer with the knowledge and skills necessary to think critically about the reliability of health information and claims and make informed choices. Methods: We will establish a steering group (SG) of 12 key stakeholders, including oncology specialists and academics. In addition, we will establish a patient and public involvement (PPI) panel of 20 people impacted by cancer. After training the members on the Key Concepts and the prioritisation process, we will conduct a two-round prioritisation process. In the first round, 12 SG members and four PPI panel members will prioritise Key Concepts for inclusion. In the second round, the remaining 16 PPI members will undertake the prioritisation based on the prioritised Key Concepts from the first round. Participants in both rounds will use a structured judgement form to rate the importance of the Key Concepts for inclusion in the online IHC-C programme. A consensus meeting will be held, where members will reach a consensus on the Key Concepts to be included and rank the order in which the prioritised Key Concepts will be addressed in the IHC-C programme. Conclusions: At the end of this process, we will identify which Key Concepts should be included and the order in which they should be addressed in the IHC-C programme.

Teaching and learning how to make informed health choices: Protocol for a context analysis in Spanish primary schools

Introduction The Informed Health Choices (IHC) project developed learning resources to teach primary school children (10 to 12-year-olds) to assess treatment claims and make informed health choices. The aim of our study is to explore the educational context for teaching and learning critical thinking about health in Spanish primary schools. Methods During the 2020-2021 school year, we will conduct 1) a systematic assessment of educational documents and resources, and 2) semi-structured interviews with key education and health stakeholders. In the systematic assessment of educational documents and resources, we will include state and autonomous communities' curriculums, school educational projects, and commonly used textbooks and other health teaching materials. In the semi-structured interviews, we will involve education and health policy makers, developers of learning resources, developers of health promotion and educational interventions, head teachers, teachers, families, and paediatric primary care providers. We will design and pilot a data extraction form and a semi-structured interview guide to collect the data. We will perform a quantitative and a qualitative analysis of the data to explore how critical thinking about health is being taught and learned in Spanish primary schools. Conclusion We will identify opportunities for and barriers to teaching and learning critical thinking about health in Spanish primary schools. We will formulate recommendations-for both practice and research purposes-on how to use, adapt (if needed), and implement the IHC resources in this context.

Quality of information in news media reports about the effects of health interventions: Systematic review and meta-analyses

Background Many studies have assessed the quality of news reports about the effects of health interventions, but there has been no systematic review of such studies or meta-analysis of their results. We aimed to fill this gap (PROSPERO ID: CRD42018095032). Methods We included studies that used at least one explicit, prespecified and generic criterion to assess the quality of news reports in print, broadcast, or online news media, and specified the sampling frame, and the selection criteria and technique. We assessed criteria individually for inclusion in the meta-analyses, excluding ineligible criteria and criteria with inadequately reported results. We mapped and grouped criteria to facilitate evidence synthesis. Where possible, we extracted the proportion of news reports meeting the included criterion. We performed meta-analyses using a random effects model to estimate such proportions for individual criteria and some criteria groups, and to characterise heterogeneity across studies.  Results We included 44 primary studies in the review, and 18 studies and 108 quality criteria in the meta-analyses. Many news reports gave an unbalanced and oversimplified picture of the potential consequences of interventions. A limited number mention or adequately address conflicts of interest (22%; 95% CI 7%-49%) (low certainty), alternative interventions (36%; 95% CI 26%-47%) (moderate certainty), potential harms (40%; 95% CI 23%-61%) (low certainty), or costs (18%; 95% CI 12%-28%) (moderate certainty), or quantify effects (53%; 95% CI 36%-69%) (low certainty) or report absolute effects (17%; 95% CI 4%-49%) (low certainty).  Discussion There is room for improving health news, but it is logically more important to improve the public's ability to critically appraise health information and make judgements for themselves.

Quality of information in news media reports about the effects of health interventions: Systematic review and meta-analyses

Background Many studies have assessed the quality of news reports about the effects of health interventions, but there has been no systematic review of such studies or meta-analysis of their results. We aimed to fill this gap (PROSPERO ID: CRD42018095032). Methods We included studies that used at least one explicit, prespecified and generic criterion to assess the quality of news reports in print, broadcast, or online news media, and specified the sampling frame, and the selection criteria and technique. We assessed criteria individually for inclusion in the meta-analyses, excluding inappropriate criteria and criteria with inadequately reported results. We mapped and grouped criteria to facilitate evidence synthesis. Where possible, we extracted the proportion of news reports meeting the included criterion. We performed meta-analyses using a random effects model to estimate such proportions for individual criteria and some criteria groups, and to characterise heterogeneity across studies.  Results We included 44 primary studies in the qualitative summary, and 18 studies and 108 quality criteria in the meta-analyses. Many news reports gave an unbalanced and oversimplified picture of the potential consequences of interventions. A limited number mention or adequately address conflicts of interest (22%; 95% CI 7%-49%) (low certainty), alternative interventions (36%; 95% CI 26%-47%) (moderate certainty), potential harms (40%; 95% CI 23%-61%) (low certainty), or costs (18%; 95% CI 12%-28%) (moderate certainty), or quantify effects (53%; 95% CI 36%-69%) (low certainty) or report absolute effects (17%; 95% CI 4%-49%) (low certainty).  Discussion There is room for improving health news, but it is logically more important to improve the public's ability to critically appraise health information and make judgements for themselves.

Learning to make informed health choices: Protocol for a pilot study in schools in Barcelona

Introduction: The Informed Health Choices (IHC) project has developed learning resources to teach primary school children (10 to 12-year-olds) to assess treatment claims and make informed health choices. The aim of our study is to explore both the students' and teachers' experience when using these resources in the context of Barcelona (Spain). Methods: During the 2019-2020 school year, we will conduct a pilot study with 4 th and 5 th-year primary school students (9 to 11-year-olds) from three schools in Barcelona. The intervention in the schools will include: 1) a workshop with the teachers, and 2) lessons to the students. The data collection will include: 1) initial assessment of the resources by the teachers, 2) non-participatory observations during the lessons, 3) semi-structured interviews with the students after a lesson, 4) assessment of the lessons by the teachers, 5) treatment claim assessment by the students, and 6) final assessment of the resources by the teachers. We will use ad hoc questionnaires and guides to register the data. We will perform a quantitative and qualitative analysis of the data to explore understandability, desirability, suitability, usefulness, facilitators and barriers of the resources. The most relevant results will be discussed and some recommendations on how to use, how to adapt (if needed), and how to implement the IHC resources to this context will be agreed. We will publish the study results in a peer-reviewed journal and in several Internet resources (web pages, electronic bulletins, and social media), and we will present them to the different users of interest in conferences, workshops, and meetings. Ethical considerations: The study protocol has obtained an approval exemption from the Ethics Committee of the Hospital de la Santa Creu i Sant Pau (Barcelona, Spain).

Learning to make informed health choices: Protocol for a pilot study in schools in Barcelona

Introduction: The Informed Health Choices (IHC) project has developed learning resources to teach primary school children (10 to 12-year-olds) to assess treatment claims and make informed health choices. The aim of our study is to explore both the students' and teachers' experience when using these resources in the context of Barcelona (Spain). Methods: During the 2019-2020 school year, we will conduct a pilot study with 4 th and 5 th-year primary school students (9 to 11-year-olds) from three schools in Barcelona. The intervention in the schools will include: 1) assessment of the IHC resources by the teachers before the lessons, 2) non-participatory observations during the lessons, 3) semi-structured interviews with the students after a lesson, 4) assessment of the lessons by the teachers after a lesson, 5) treatment claim assessment by the students at the end of the lessons, and 6) assessment of the IHC resources by the teachers at the end of the lessons. We will use ad hoc questionnaires and guides to register the data. We will perform a quantitative and qualitative analysis of the data to explore understandability, desirability, suitability, usefulness, facilitators and barriers of the resources. The most relevant results will be discussed and some recommendations on how to use, how to adapt (if needed), and how to implement the IHC resources to this context will be agreed. The findings of the contextualization activities could inform the design of a cluster-randomised trial, to determine the effectiveness of the IHC resources in this context prior to scaling-up its use. Ethical considerations: The study protocol has obtained an approval exemption from the Ethics Committee of the Hospital de la Santa Creu i Sant Pau (Barcelona, Spain).

Learning to make informed health choices: Protocol for a pilot study in schools in Barcelona

Introduction: The Informed Health Choices (IHC) project has developed learning resources to teach primary school children (10 to 12-year-olds) to assess treatment claims and make informed health choices. The aim of our study is to explore both the students' and teachers' experience when using these resources in the context of Barcelona (Spain). Methods: During the 2019-2020 school year, we will conduct a pilot study with 4 th and 5 th-year primary school students (9 to 11-year-olds) from three schools in Barcelona. The intervention in the schools will include: 1) a workshop with the teachers, and 2) lessons to the students. The data collection will include: 1) assessment of the IHC resources by the teachers before the lessons, 2) non-participatory observations during the lessons, 3) semi-structured interviews with the students after a lesson, 4) assessment of the lessons by the teachers after a lesson, 5) treatment claim assessment by the students at the end of the lessons, and 6) assessment of the IHC resources by the teachers at the end of the lessons. We will use ad hoc questionnaires and guides to register the data. We will perform a quantitative and qualitative analysis of the data to explore understandability, desirability, suitability, usefulness, facilitators and barriers of the resources. The most relevant results will be discussed and some recommendations on how to use, how to adapt (if needed), and how to implement the IHC resources to this context will be agreed. The findings of the contextualization activities could inform the design of a cluster-randomised trial, to determine the effectiveness of the IHC resources in this context prior to scaling-up its use. Ethical considerations: The study protocol has obtained an approval exemption from the Ethics Committee of the Hospital de la Santa Creu i Sant Pau (Barcelona, Spain).