Informed health choices intervention to teach primary school children in low-income countries to assess claims about treatment effects: process evaluation

What is the effect of the Informed Health Choices secondary school intervention on the ability of students in Rwanda to think critically about health choices after one-year follow-up? A cluster-randomized trial

AIM

The aim of this study was to evaluate the effects of the Informed Health Choices secondary school intervention on the ability of students in Rwanda to think critically and make informed health choices after 1 year.

METHODS

This was a two-arm cluster-randomized trial conducted in 84 lower secondary schools from 10 districts representing five provinces of Rwanda. We used stratified randomization to allocate schools 1:1 to the intervention or control arm. One class in each intervention school had ten 40-min lessons taught by a trained teacher in addition to the usual curriculum. Control schools followed the usual curriculum. The primary outcome was a passing score (≥ 9 out of 18 questions answered correctly) for students on the Critical Thinking about Health Test completed 1 year after the intervention. We conducted an intention to treat analysis using generalized linear mixed models, accounting for the cluster design using random intercepts.

RESULTS

After 1 year, 35 of 42 teachers (83.3%) and 1181 of 1556 students (75.9%) in the control arm completed the test. In the intervention arm, 35 of 42 teachers (83.3%) and 1238 of 1572 students (78.8%) completed the test. The proportion of students who had a passing score in the intervention arm was 625/1238 (50.5%) compared to 230/1181 (19.5%) in the control arm (adjusted odds ratio 7.6 [95% CI: 4.6-12.6], p < 0.0001). The adjusted difference in the proportion of students with a passing score was 32.2% (95% CI 24.5-39.8%).

CONCLUSIONS

The IHC secondary school intervention was effective after 1 year. However, the size of the effect was smaller than immediately after the intervention (adjusted difference 32.2% vs 37.2%) due to decay in the proportion of students in intervention schools with a passing score (50.5% vs 58.2%).

TRIAL REGISTRATION

Pan African Clinical Trial Registry (PCTR), trial identifier: PACTR202203880375077. Registered on February 15, 2022.

Process Evaluation of Teaching Critical Thinking About Health Using the Informed Health Choices Intervention in Rwanda: A Mixed Methods Study

INTRODUCTION

We evaluated the Informed Health Choices secondary school intervention in a cluster randomized trial in Rwanda. The intervention was effective in helping students to think critically about health. In parallel to the trial, we conducted a process evaluation to assess factors affecting the implementation, impacts, and scale-up of the intervention.

METHODS

We used a mixed methods approach that included quantitative and qualitative methods. We collected quantitative data from teachers to evaluate the teacher training and each lesson. We conducted focus group discussions with students (n=10) and their parents/guardians (n=5). We conducted lesson observations (n=16) and key informant interviews with teachers (n=10) and school administrators (n=10) from intervention schools and policymakers (n=2). We analyzed the quantitative data using descriptive statistics. We used framework analysis and thematic content analysis to analyze the qualitative data.

RESULTS

Teachers noted that the teacher training supported their delivery of the intervention and that they made only small adaptations to fit student, teacher, or contextual needs. Students reported obtaining important skills, including recognizing health claims, understanding the need for research, and "thinking twice" before deciding. Participants saw the design of the intervention, students' and teachers' motivation, and school and home support as key facilitators for the implementation and impact of the intervention. Implementation barriers identified included the content of the lessons not being included in national examinations, competing priorities, and time constraints. Participants identified several factors that could facilitate intervention scale-up, including the need for the skills taught in the lessons and compatibility of the intervention with the national curriculum.

CONCLUSION

We found that it was feasible to implement the intervention in Rwandan secondary schools and that students benefited from the intervention. Scaling up the intervention will likely require addressing the barriers identified in this study.

Process Evaluations for the Scale-Up of Complex Interventions - a Scoping Review

Introduction

Complex health interventions (CHIs) are common in (public) health and social care practice and policy. A process evaluation (PE) is an essential part of designing and testing CHIs and questions what is implemented, the mechanisms of change, and how context affects implementation. The scale-up of CHIs is challenging and heterogeneous, making the accompanying PE unique to the nature of the inquiry.

Methods

We conducted a scoping review to describe the current practice of conducting PEs alongside or following the scale-up of CHI. Eight primary data sources were searched and data extracted on study characteristics, intervention characteristics, methods used in relation to the PE, and stakeholders included.

Results

We reviewed 10,538 records and included 56 studies. Seven common thematic areas emerged in which CHIs were being scaled-up. The use of scale-up specific frameworks was rare, and common outcomes of the process evaluation focussed on barriers and facilitators in relation to the context; often obtained "once-off" using qualitative and quantitative data sources. Scale-up strategies reported were: supporting increased coverage, comprehensiveness, and institutionalisation; often simultaneously.

Conclusion

Variations in the conduct of process evaluations during the scale-up phase of complex health interventions may reflect differences in context, conceptual challenges, the multi-dimensional nature of scale-up, and the point of engagement with the health care system (e.g., community-level). Ideally, a process evaluation is a recurrent continuous process, leveraging a systems-driven understanding and triangulation of qualitative and quantitative data, that takes place alongside the scale-up project to inform real-world adaptations of scale-up strategies and (untoward) mechanisms of impact when applicable.

Teaching critical thinking about health information and choices in secondary schools: human-centred design of digital resources

Background

Learning to thinking critically about health information and choices can protect people from unnecessary suffering, harm, and resource waste. Earlier work revealed that children can learn these skills, but printing costs and curricula compatibility remain important barriers to school implementation. We aimed to develop a set of digital learning resources for students to think critically about health that were suitable for use in Kenyan, Rwandan, and Ugandan secondary schools.

Methods

We conducted work in two phases collaborating with teachers, students, schools, and national curriculum development offices using a human-centred design approach. First, we conducted context analyses and an overview of teaching strategies, prioritised content and collected examples. Next, we developed lessons and guidance iteratively, informed by data from user-testing, individual and group interviews, and school pilots.

Results

Final resources include online lesson plans, teachers' guide, and extra resources, with lesson plans in two modes, for use in a classroom equipped with a blackboard/flip-chart and a projector. The resources are accessible offline for use when electricity or Internet is lacking. Teachers preferred the projector mode, as it provided structure and a focal point for class attention. Feedback was largely positive, with teachers and students appreciating the learning and experiencing it as relevant. Four main challenges included time to teach lessons; incorrect comprehension; identifying suitable examples; and technical, logistical, and behavioural challenges with a student-computer mode that we piloted. We resolved challenges by simplifying and combining lessons; increasing opportunities for review and assessment; developing teacher training materials, creating a searchable set of examples; and deactivating the student-computer mode.

Conclusion

Using a human-centred design approach, we created digital resources for teaching secondary school students to think critically about health actions and for training teachers. Be smart about your health resources are open access and can be translated or adapted to other settings.

Teaching methods for critical thinking in health education of children up to high school: A scoping review

According to the World Health Organization, the improvement of people's health literacy is one of the fundamental public health challenges in the 21st century. The key issue in teaching health literacy is to develop critical thinking skills. As health literacy and critical thinking should be developed at school age, we reviewed teaching methods or educational interventions used in empirical studies focused on the development of critical thinking regarding health and implemented by teachers in preschools, primary schools, or secondary schools. We searched seven databases (Medline, Embase, Web of Science, ERIC, ProqQuest, PsycArticles, and CINAHL) from inception to 20 September 2023 for any type of empirical studies. Due to the heterogeneity in interventions and inadequate reporting of results, a descriptive synthesis of studies was performed in addition to quantitative analysis. Of the 15919 initial records, 115 studies were included in the review. Most of the educational interventions focused on lifestyle-related health issues such as substance use, sexual and reproductive health, and nutrition. The popularity of health issues changed over time and depended on the geographical context. Six dimensions that differentiated the teaching methods were identified: central teaching component, central educator, pupils' activity level, teaching context, educational materials, and significance of critical thinking. Many educational interventions did not address the development of critical thinking skills in a comprehensive manner, and the significance of critical thinking varied greatly. Interventions in which critical thinking had high and very high significance applied mainly problem-solving methods and involved pupils' activity. The evidence on the effectiveness of the teaching methods that develop critical thinking is limited because most articles failed to provide detailed information on the teaching methods or did not examine their effects. We recommend that a checklist is developed to facilitate a detailed description of health educational interventions and thus promoting their replicability. Study registration: The protocol of the review was registered in the OSF Registries on 13 January 2022 (doi: https://doi.org/10.17605/OSF.IO/46TEZ).

Key concepts for informed health choices: Where's the evidence?

Background

The Informed Health Choices (IHC) Key Concepts is a framework that provides a basis for developing educational resources and evaluating people's ability to think critically about health actions. We developed the original Key Concepts framework by reviewing texts and checklists for the public, journalists, and health professionals and collecting structured feedback from an international advisory group. We revised the original 2015 framework yearly from 2016 to 2018 based on feedback and experience using the framework. The objectives of this paper are to describe the development of the framework since 2018 and summarise their basis.

Methods

For the 2019 version, we responded to feedback on the 2018 version. For the current 2022 version, in addition to responding to feedback on the 2019 version, we reviewed the evidence base for each of the concepts. Whenever possible, we referenced systematic reviews that provide a basis for a concept. We screened all Cochrane methodology reviews and searched Epistemonikos, PubMed, and Google Scholar for methodology reviews and meta-epidemiological studies.

Results

The original framework included 32 concepts in six groups. The 2019 version and the current 2022 version include 49 concepts in the same three main groups that we have used since 2016. There are now 10 subgroups or higher-level concepts. For each concept, there is an explanation including one or more examples, the basis for the concept, and implications. Over 600 references are cited that support the concepts, and over half of the references are systematic reviews.

Conclusions

There is a large body of evidence that supports the IHC key concepts and we have received few suggestions for changes since 2019.

Effects of the Informed Health Choices secondary school intervention on the ability of students in Rwanda to think critically about health choices: A cluster-randomized trial

AIM

The aim of this trial was to evaluate the effects of the Informed Health Choices intervention on the ability of students in Rwandan to think critically and make Informed Health Choices.

METHODS

We conducted a two-arm cluster-randomized trial in 84 lower secondary schools from 10 districts representing five provinces of Rwanda. We used stratified randomization to allocate schools to the intervention or control. One class in each intervention school had ten 40-min lessons taught by a trained teacher in addition to the usual curriculum. Control schools followed the usual curriculum. The primary outcome was a passing score (≥ 9 out of 18 questions answered correctly) for students on the Critical Thinking about Health Test completed within 2 weeks after the intervention. We conducted an intention-to-treat analysis using generalized linear mixed models, accounting for the cluster design using random intercepts.

RESULTS

Between February 25 and March 29, 2022, we recruited 3,212 participants. We assigned 1,572 students and 42 teachers to the intervention arm and 1,556 students and 42 teachers to the control arm. The proportion of students who passed the test in the intervention arm was 915/1,572 (58.2%) compared to 302/1,556 (19.4%) in the control arm, adjusted odds ratio 10.6 (95% CI: 6.3-17.8), p < 0.0001, adjusted difference 37.2% (95% CI: 29.5%-45.0%).

CONCLUSIONS

The intervention is effective in helping students think critically about health choices. It was possible to improve students' ability to think critically about health in the context of a competence-based curriculum in Rwanda, despite challenging postpandemic conditions.

Use of the informed health choices educational intervention to improve secondary students' ability to think critically about health interventions in Uganda: A cluster-randomized trial

AIM

The aim was to evaluate the effect of the Informed Health Choices (IHC) educational intervention on secondary students' ability to assess health-related claims and make informed choices.

METHODS

In a cluster-randomized trial, we randomized 80 secondary schools (students aged 13-17 years) in Uganda to the intervention or control (usual curriculum). The intervention included a 2-day teacher training workshop, 10 lessons accessed online by teachers and delivered in one school term. The lesson plans were developed for classrooms equipped with a blackboard or a blackboard and projector. The lessons addressed nine prioritized concepts. We used two multiple-choice questions for each concept to evaluate the students' ability to assess claims and make informed choices. The primary outcome was the proportion of students with a passing score (≥9 of 18 questions answered correctly).

RESULTS

Eighty schools consented and were randomly allocated. A total of 2477 students in the 40 intervention schools and 2376 students in the 40 control schools participated in this trial. In the intervention schools, 1364 (55%) of students that completed the test had a passing score compared with 586 (25%) of students in the control schools (adjusted difference 33%, 95% CI 26%-39%).

CONCLUSIONS

The IHC secondary school intervention improved students' ability to think critically and make informed choices. Well-designed digital resources may improve access to educational material, even in schools without computers or other information and communication technology (ICT). This could facilitate scaling-up use of the resources and help to address inequities associated with limited ICT access.

Effects of the Informed Health Choices secondary school intervention: A prospective meta-analysis

AIM

The aim of this prospective meta-analysis was to synthesize the results of three cluster-randomized trials of an intervention designed to teach lower-secondary school students (age 14-16) to think critically about health choices.

METHODS

We conducted the trials in Kenya, Rwanda, and Uganda. The intervention included a 2- to 3-day teacher training workshop, digital resources, and ten 40-min lessons. The lessons focused on nine key concepts. We did not intervene in control schools. The primary outcome was a passing score on a test (≥9 of 18 multiple-choice questions answered correctly). We performed random effects meta-analyses to estimate the overall adjusted odds ratios. Secondary outcomes included effects of the intervention on teachers.

RESULTS

Altogether, 244 schools (11,344 students) took part in the three trials. The overall adjusted odds ratio was 5.5 (95% CI: 3.0-10.2; p < 0.0001) in favor of the intervention (high certainty evidence). This corresponds to 33% (95% CI: 25-40%) more students in the intervention schools passing the test. Overall, 3397 (58%) of 5846 students in intervention schools had a passing score. The overall adjusted odds ratio for teachers was 13.7(95% CI: 4.6-40.4; p < 0.0001), corresponding to 32% (95% CI: 6%-57%) more teachers in the intervention schools passing the test (moderate certainty evidence). Overall, 118 (97%) of 122 teachers in intervention schools had a passing score.

CONCLUSIONS

The intervention led to a large improvement in the ability of students and teachers to think critically about health choices, but 42% of students in the intervention schools did not achieve a passing score.

Piloting the informed health choices resources in Barcelona primary schools: A mixed methods study

INTRODUCTION

The main objective of the Informed Health Choices (IHC) project is to teach people to assess treatment claims and make informed health choices. For this purpose, the IHC learning resources were developed for primary school children. The aim of this study is to explore students' and teachers' experience when using the IHC resources in primary schools in Barcelona (Spain).

METHODS

We conducted a mixed methods study for piloting the IHC resources in a convenience sample of primary schools in Barcelona. The intervention included a workshop with teachers, and nine lessons with students. We collected data using multiple approaches. We performed quantitative and qualitative analyses, and integrated the findings in a joint display. Finally, we formulated recommendations for using the IHC resources in this setting.

RESULTS

Two schools, with a total of 143 students in 4th and 5th grade and six teachers, participated in the study. One school followed the suggested IHC teaching plan and competed all the lessons; the other school modified the plan substantially and did not complete all the lessons. Overall, students and teachers from both schools understood, were interested in, and were able to apply the content of the lessons. During the lessons, the textbook was useful for students; nevertheless, for the teachers, the usefulness of the IHC resources was variable. Teachers adapted the IHC resources to increase student participation and used Information and Communications Technologies tools. We observed more facilitators than barriers to teach the lessons. The teachers suggested some ideas to improve the lessons based on activities they developed and implemented. The integration analysis showed great convergence of the quantitative and qualitative findings. We propose seven recommendations for using the IHC resources in this setting.

CONCLUSIONS

Students and teachers from primary schools in Barcelona showed a positive experience when using IHC resources; however, these resources should be adapted to promote classroom participation.

Prioritisation of Informed Health Choices (IHC) key concepts to be included in lower secondary school resources: A consensus study

BACKGROUND

The Informed Health Choices Key Concepts are principles for thinking critically about healthcare claims and deciding what to do. The Key Concepts provide a framework for designing curricula, learning resources, and evaluation tools.

OBJECTIVES

To prioritise which of the 49 Key Concepts to include in resources for lower secondary schools in East Africa.

METHODS

Twelve judges used an iterative process to reach a consensus. The judges were curriculum specialists, teachers, and researchers from Kenya, Uganda, and Rwanda. After familiarising themselves with the concepts, they pilot-tested draft criteria for selecting and ordering the concepts. After agreeing on the criteria, nine judges independently assessed all 49 concepts and reached an initial consensus. We sought feedback on the draft consensus from other stakeholders, including teachers. After considering the feedback, nine judges independently reassessed the prioritised concepts and reached a consensus. The final set of concepts was determined after user-testing prototypes and pilot-testing the resources.

RESULTS

The first panel of judges prioritised 29 concepts. Based on feedback from teachers, students, curriculum specialists, and members of the research team, two concepts were dropped. A second panel of nine judges prioritised 17 of the 27 concepts that emerged from the initial prioritisation and feedback. Based on feedback on prototypes of lessons and pilot-testing a set of 10 lessons, we determined that it was possible to introduce nine concepts in 10 single-period (40-minute) lessons. We included eight of the 17 prioritised concepts and one additional concept.

CONCLUSION

Using an iterative process with explicit criteria, we prioritised nine concepts as a starting point for students to learn to think critically about healthcare claims and choices.

Development of a video-based evidence synthesis knowledge translation resource: Drawing on a user-centred design approach

Objectives

We aimed to develop a video animation knowledge translation (KT) resource to explain the purpose, use and importance of evidence synthesis to the public regarding healthcare decision-making.

Methods

We drew on a user-centred design approach to develop a spoken animated video (SAV) by conducting two cycles of idea generation, prototyping, user testing, analysis, and refinement. Six researchers identified the initial key messages of the SAV and informed the first draft of the storyboard and script. Seven members of the public provided input on this draft and the key messages through think-aloud interviews, which we used to develop an SAV prototype. Seven additional members of the public participated in think-aloud interviews while watching the video prototype. All members of the public also completed a questionnaire on perceived usefulness, desirability, clarity and credibility. We subsequently synthesised all data to develop the final SAV.

Results

Researchers identified the initial key messages as 1) the importance of evidence synthesis, 2) what an evidence synthesis is and 3) how evidence synthesis can impact healthcare decision-making. Members of the public rated the initial video prototype as 9/10 for usefulness, 8/10 for desirability, 8/10 for clarity and 9/10 for credibility. Using their guidance and feedback, we produced a three-and-a-half-minute video animation. The video was uploaded on YouTube, has since been translated into two languages, and viewed over 12,000 times to date.

Conclusions

Drawing on user-centred design methods provided a structured and transparent approach to the development of our SAV. Involving members of the public enhanced the credibility and usefulness of the resource. Future work could explore involving the public from the outset to identify key messages in developing KT resources explaining methodological topics. This study describes the systematic development of a KT resource with limited resources and provides transferrable learnings for others wishing to do similar.

Feasibility of contextualizing the Informed Health Choices learning resources in Italy: A pilot study in a primary school in Florence

Background The Informed Health Choices (IHC) project team developed learning resources for primary school children to teach critical thinking about treatments claims and health choices and evaluated their effect in a randomized controlled trial of 120 schools in Uganda. Children taught with these resources showed a better ability to think critically about treatments claims and health choices than children not taught with these resources. Teams in multiple countries are contextualising the IHC resources for use in other languages and settings; in this pilot we describe contextualization for use in Italian primary school.  Methods After translating the IHC resources to Italian and holding an introductory workshop with participating schoolteachers, we piloted the resources with two classes of a primary school in Florence over nine lessons. Our aims were: 1) to assess the feasibility of introducing the IHC curriculum in Italian primary school; 2) to evaluate students' ability to assess health claims and make informed health choices; to explore 3) students' and 4) teachers' experiences with the IHC learning resources; 5) to identify barriers and facilitators to implementation of IHC learning resources in Italian primary school. To assess these objectives, we used qualitative and quantitative methods. Results Both qualitative and quantitative analyses consistently showed that the IHC learning resources had a positive impact on the objectives examined. The resources integrated well into the Italian primary school curriculum. Both students and teachers considered these resources comprehensible, appealing in design and content, and stimulating for the development of a critical attitude. The only barrier teachers and students expressed was using the resources in a remote learning context.  Conclusions Findings from our contextualisation of IHC learning resources in Italian primary school indicate that these resources are well-suited for Italian teachers and students in a primary school context and compatible with the Italian primary school curriculum.

Contextualizing critical thinking about health using digital technology in secondary schools in Kenya: a qualitative analysis

BACKGROUND

Good health decisions depend on one's ability to think critically about health claims and make informed health choices. Young people can learn these skills through school-based interventions, but learning resources need to be low-cost and built around lessons that can fit into existing curricula. As a first step to developing and evaluating digital learning resources that are feasible to use in Kenyan secondary schools, we conducted a context analysis to explore interest in critical thinking for health, map where critical thinking about health best fits in the curriculum, explore conditions for introducing new learning resources, and describe the information and communication technology (ICT) infrastructure available for teaching and learning.

METHODS

We employed a qualitative descriptive approach. We interviewed 15 key informants, carried out two focus group discussions, observed ICT conditions in five secondary schools, reviewed seven documents, and conducted an online catalog of ICT infrastructure in all schools (n=250) in Kisumu County. Participants included national curriculum developers, national ICT officers, teachers, and national examiners. We used a framework analysis approach to analyze data and report findings.

FINDINGS

Although critical thinking is a core competence in the curriculum, critical thinking about health is not currently taught in Kenyan secondary schools. Teachers, health officials, and curriculum developers recognized the importance of teaching critical thinking about health in secondary schools. Stakeholders agreed that Informed Health Choices learning resources could be embedded in nine subjects. The National Institute of Curriculum Development regulates resources for learning; the development of new resources requires collaboration and approval from this body. Most schools do not use ICT for teaching, and for those few that do, the use is limited. Implementation of Kenya's ICT policy framework for schools faces several challenges which include inadequate ICT infrastructure, poor internet connectivity, and teachers' lack of training and experience.

CONCLUSION

Teaching critical thinking about health is possible within the current Kenyan lower secondary school curriculum, but the learning resources will need to be designed for inclusion in and across existing subjects. The National ICT Plan and Vision for 2030 provides an opportunity for scale-up and integration of technology in teaching and learning environments, which can enable future use of digital resources in schools. However, given the current ICT condition in schools in the country, digital learning resources should be designed to function with limited ICT infrastructure, unstable Internet access, and for use by teachers with low levels of experience using digital technology.

Learning to think critically about health using digital technology in Ugandan lower secondary schools: A contextual analysis

INTRODUCTION

The world is awash with claims about the effects of health interventions. Many of these claims are untrustworthy because the bases are unreliable. Acting on unreliable claims can lead to waste of resources and poor health outcomes. Yet, most people lack the necessary skills to appraise the reliability of health claims. The Informed Health Choices (IHC) project aims to equip young people in Ugandan lower secondary schools with skills to think critically about health claims and to make good health choices by developing and evaluating digital learning resources. To ensure that we create resources that are suitable for use in Uganda's secondary schools and can be scaled up if found effective, we conducted a context analysis. We aimed to better understand opportunities and barriers related to demand for the resources, how the learning content overlaps with existing curriculum and conditions in secondary schools for accessing and using digital resources, in order to inform resource development.

METHODS

We used a mixed methods approach and collected both qualitative and quantitative data. We conducted document analyses, key informant interviews, focus group discussions, school visits, and a telephone survey regarding information communication and technology (ICT). We used a nominal group technique to obtain consensus on the appropriate number and length of IHC lessons that should be planned in a school term. We developed and used a framework from the objectives to code the transcripts and generated summaries of query reports in Atlas.ti version 7.

FINDINGS

Critical thinking is a key competency in the lower secondary school curriculum. However, the curriculum does not explicitly make provision to teach critical thinking about health, despite a need acknowledged by curriculum developers, teachers and students. Exam oriented teaching and a lack of learning resources are additional important barriers to teaching critical thinking about health. School closures and the subsequent introduction of online learning during the COVID-19 pandemic has accelerated teachers' use of digital equipment and learning resources for teaching. Although the government is committed to improving access to ICT in schools and teachers are open to using ICT, access to digital equipment, unreliable power and internet connections remain important hinderances to use of digital learning resources.

CONCLUSIONS

There is a recognized need for learning resources to teach critical thinking about health in Ugandan lower secondary schools. Digital learning resources should be designed to be usable even in schools with limited access and equipment. Teacher training on use of ICT for teaching is needed.

Quality of information in news media reports about the effects of health interventions: Systematic review and meta-analyses

Background Many studies have assessed the quality of news reports about the effects of health interventions, but there has been no systematic review of such studies or meta-analysis of their results. We aimed to fill this gap (PROSPERO ID: CRD42018095032). Methods We included studies that used at least one explicit, prespecified and generic criterion to assess the quality of news reports in print, broadcast, or online news media, and specified the sampling frame, and the selection criteria and technique. We assessed criteria individually for inclusion in the meta-analyses, excluding ineligible criteria and criteria with inadequately reported results. We mapped and grouped criteria to facilitate evidence synthesis. Where possible, we extracted the proportion of news reports meeting the included criterion. We performed meta-analyses using a random effects model to estimate such proportions for individual criteria and some criteria groups, and to characterise heterogeneity across studies.  Results We included 44 primary studies in the review, and 18 studies and 108 quality criteria in the meta-analyses. Many news reports gave an unbalanced and oversimplified picture of the potential consequences of interventions. A limited number mention or adequately address conflicts of interest (22%; 95% CI 7%-49%) (low certainty), alternative interventions (36%; 95% CI 26%-47%) (moderate certainty), potential harms (40%; 95% CI 23%-61%) (low certainty), or costs (18%; 95% CI 12%-28%) (moderate certainty), or quantify effects (53%; 95% CI 36%-69%) (low certainty) or report absolute effects (17%; 95% CI 4%-49%) (low certainty).  Discussion There is room for improving health news, but it is logically more important to improve the public's ability to critically appraise health information and make judgements for themselves.

Quality of information in news media reports about the effects of health interventions: Systematic review and meta-analyses

Background Many studies have assessed the quality of news reports about the effects of health interventions, but there has been no systematic review of such studies or meta-analysis of their results. We aimed to fill this gap (PROSPERO ID: CRD42018095032). Methods We included studies that used at least one explicit, prespecified and generic criterion to assess the quality of news reports in print, broadcast, or online news media, and specified the sampling frame, and the selection criteria and technique. We assessed criteria individually for inclusion in the meta-analyses, excluding inappropriate criteria and criteria with inadequately reported results. We mapped and grouped criteria to facilitate evidence synthesis. Where possible, we extracted the proportion of news reports meeting the included criterion. We performed meta-analyses using a random effects model to estimate such proportions for individual criteria and some criteria groups, and to characterise heterogeneity across studies.  Results We included 44 primary studies in the qualitative summary, and 18 studies and 108 quality criteria in the meta-analyses. Many news reports gave an unbalanced and oversimplified picture of the potential consequences of interventions. A limited number mention or adequately address conflicts of interest (22%; 95% CI 7%-49%) (low certainty), alternative interventions (36%; 95% CI 26%-47%) (moderate certainty), potential harms (40%; 95% CI 23%-61%) (low certainty), or costs (18%; 95% CI 12%-28%) (moderate certainty), or quantify effects (53%; 95% CI 36%-69%) (low certainty) or report absolute effects (17%; 95% CI 4%-49%) (low certainty).  Discussion There is room for improving health news, but it is logically more important to improve the public's ability to critically appraise health information and make judgements for themselves.

Teaching and learning how to make informed health choices: Protocol for a context analysis in Spanish primary schools

Introduction The Informed Health Choices (IHC) project developed learning resources to teach primary school children (10 to 12-year-olds) to assess treatment claims and make informed health choices. The aim of our study is to explore the educational context for teaching and learning critical thinking about health in Spanish primary schools. Methods During the 2020-2021 school year, we will conduct 1) a systematic assessment of educational documents and resources, and 2) semi-structured interviews with key education and health stakeholders. In the systematic assessment of educational documents and resources, we will include state and autonomous communities' curriculums, school educational projects, and commonly used textbooks and other health teaching materials. In the semi-structured interviews, we will involve education and health policy makers, developers of learning resources, developers of health promotion and educational interventions, head teachers, teachers, families, and paediatric primary care providers. We will design and pilot a data extraction form and a semi-structured interview guide to collect the data. We will perform a quantitative and a qualitative analysis of the data to explore how critical thinking about health is being taught and learned in Spanish primary schools. Conclusion We will identify opportunities for and barriers to teaching and learning critical thinking about health in Spanish primary schools. We will formulate recommendations-for both practice and research purposes-on how to use, adapt (if needed), and implement the IHC resources in this context.

Teaching and learning how to make informed health choices: Protocol for a context analysis in Spanish primary schools

Introduction The Informed Health Choices (IHC) project developed learning resources to teach primary school children (10 to 12-year-olds) to assess treatment claims and make informed health choices. The aim of our study is to explore the educational context for teaching and learning critical thinking about health in Spanish primary schools. Methods During the 2020-2021 school year, we will conduct 1) a systematic assessment of educational documents and resources, and 2) semi-structured interviews with key education and health stakeholders. In the systematic assessment of educational documents and resources, we will include state and autonomous communities' curriculums, school educational projects, and commonly used textbooks and other health teaching materials. In the semi-structured interviews, we will involve education and health policy makers, developers of learning resources, developers of health promotion and educational interventions, head teachers, teachers, families, and paediatric primary care providers. We will design and pilot a data extraction form and a semi-structured interview guide to collect the data. We will perform a quantitative and a qualitative analysis of the data to explore how critical thinking about health is being taught and learned in Spanish primary schools. Conclusion We will identify opportunities for and barriers to teaching and learning critical thinking about health in Spanish primary schools. We will formulate recommendations-for both practice and research purposes-on how to use, adapt (if needed), and implement the IHC resources in this context.

Teaching critical thinking about health using digital technology in lower secondary schools in Rwanda: A qualitative context analysis

Introduction

Adolescents encounter misleading claims about health interventions that can affect their health. Young people need to develop critical thinking skills to enable them to verify health claims and make informed choices. Schools could teach these important life skills, but educators need access to suitable learning resources that are aligned with their curriculum. The overall objective of this context analysis was to explore conditions for teaching critical thinking about health interventions using digital technology to lower secondary school students in Rwanda.

Methods

We undertook a qualitative descriptive study using four methods: document review, key informant interviews, focus group discussions, and observations. We reviewed 29 documents related to the national curriculum and ICT conditions in secondary schools. We conducted 8 interviews and 5 focus group discussions with students, teachers, and policy makers. We observed ICT conditions and use in five schools. We analysed the data using a framework analysis approach.

Results

Two major themes found. The first was demand for teaching critical thinking about health. The current curriculum explicitly aims to develop critical thinking competences in students. Critical thinking and health topics are taught across subjects. But understanding and teaching of critical thinking varies among teachers, and critical thinking about health is not being taught. The second theme was the current and expected ICT conditions. Most public schools have computers, projectors, and internet connectivity. However, use of ICT in teaching is limited, due in part to low computer to student ratios.

Conclusions

There is a need for learning resources to develop critical thinking skills generally and critical thinking about health specifically. Such skills could be taught within the existing curriculum using available ICT technologies. Digital resources for teaching critical thinking about health should be designed so that they can be used flexibly across subjects and easily by teachers and students.

App-Based Salt Reduction Intervention in School Children and Their Families (AppSalt) in China: Protocol for a Mixed Methods Process Evaluation

Background

The app-based salt reduction intervention program in school children and their families (AppSalt) is a multicomponent mobile health (mHealth) intervention program, which involves multiple stakeholders, including students, parents, teachers, school heads, and local health and education authorities. The complexity of the AppSalt program highlights the need for process evaluation to investigate how the implementation will be achieved at different sites.

Objective

This paper presents a process evaluation protocol of the AppSalt program, which aims to monitor the implementation of the program, explain its causal mechanisms, and provide evidence for scaling up the program nationwide.

Methods

A mixed methods approach will be used to collect data relating to five process evaluation dimensions: fidelity, dose delivered, dose received, reach, and context. Quantitative data, including app use logs, activity logs, and routine monitoring data, will be collected alongside the intervention process to evaluate the quantity and quality of intervention activities. The quantitative data will be summarized as medians, means, and proportions as appropriate. Qualitative data will be collected through semistructured interviews of purposely selected intervention participants and key stakeholders from local health and education authorities. The thematic analysis technique will be used for analyzing the qualitative data with the support of NVivo 12. The qualitative data will be triangulated with the quantitative data during the interpretation phase to explain the 5 process evaluation dimensions.

Results

The intervention activities of the AppSalt program were initiated at 27 primary schools in three cities since October 2018. We have completed the 1-year intervention of this program. The quantitative data for this study, including app use log, activity logs, and the routine monitoring data, were collected and organized during the intervention process. After completing the intervention, we conducted semistructured interviews with 32 students, 32 parents, 9 teachers, 9 school heads, and 8 stakeholders from local health and education departments. Data analysis is currently underway.

Conclusions

Using mHealth technology for salt reduction among primary school students is an innovation in China. The findings of this study will help researchers understand the implementation of the AppSalt program and similar mHealth interventions in real-world settings. Furthermore, this process evaluation will be informative for other researchers and policy makers interested in replicating the AppSalt program and designing their salt reduction intervention.

International Registered Report Identifier (IRRID)

DERR1-10.2196/19430

Can learning about trials be child's play? A qualitative exploration of the 'Schools Teaching Awareness of Randomised Trials' (START) initiative

Background

The Health Research Board-Trials Methodology Research Network (HRB-TMRN) celebrates International Clinical Trials Day with the help of the younger members of our community through the Network’s ‘Schools Teaching Awareness of Randomised Trials (START)’ initiative. START seeks to increase public awareness of randomised trials in Ireland. Launched in 2016, it asks children (8–12 years old) to conduct and report their very own fun randomised trial. The study reported in this paper sought to explore children and teachers perceptions and experiences of the START initiative.

Methods

We conducted eight, one-to one interviews with teachers and eight focus groups with 61 children who took part in the 2018 START initiative. Interviews and focus groups were recorded and transcribed and the data analysed using template analysis.

Results

The findings of this study highlight the benefits of participating in START and the areas of the initiative that required further attention. Teachers and children recalled the benefits of experiential learning associated with START and learning by doing encouraged a fun way of engaging with trial processes. By recalling all aspects of planning, conducting and reporting their trial, the children in this study demonstrated their awareness of the trial processes. The teachers suggested that START provides a valuable framework to contribute to key aspects of the primary school curriculum in Ireland. The experiences of these participants also provided recommendation for improving the programme for future START participants.

Conclusions

Increasing public awareness and understanding of randomised trials can help increase public engagement in trials. By educating children about the importance of trials and supporting them to ‘learn by doing’ by carrying out their own trial, the START initiative can contribute substantially to children’s awareness and understanding of trial processes. Given that children are the public, the patients and the researchers of the future, initiatives such as START deserve attention.

Development of the informed health choices resources in four countries to teach primary school children to assess claims about treatment effects: a qualitative study employing a user-centred approach

Background

People of all ages are flooded with health claims about treatment effects (benefits and harms of treatments). Many of these are not reliable, and many people lack skills to assess their reliability. Primary school is the ideal time to begin to teach these skills, to lay a foundation for continued learning and enable children to make well-informed health choices, as they grow older. However, these skills are rarely being taught and yet there are no rigorously developed and evaluated resources for teaching these skills.

Objectives

To develop the Informed Health Choices (IHC) resources (for learning and teaching people to assess claims about the effects of treatments) for primary school children and teachers.

Methods

We prototyped, piloted, and user-tested resources in four settings that included Uganda, Kenya, Rwanda, and Norway. We employed a user-centred approach to designing IHC resources which entailed multiple iterative cycles of development (determining content scope, generating ideas, prototyping, testing, analysing and refining) based on continuous close collaboration with teachers and children.

Results

We identified 24 Key Concepts that are important for children to learn. We developed a comic book and a separate exercise book to introduce and explain the Key Concepts to the children, combining lessons with exercises and classroom activities. We developed a teachers' guide to supplement the resources for children.

Conclusion

By employing a user-centred approach to designing resources to teach primary children to think critically about treatment claims and choices, we developed learning resources that end users experienced as useful, easy to use and well-suited to use in diverse classroom settings.

Effects of the Informed Health Choices podcast on the ability of parents of primary school children in Uganda to assess the trustworthiness of claims about treatment effects: one-year follow up of a randomised trial

INTRODUCTION

Earlier, we designed and evaluated an educational mass media intervention for improving people's ability to think more critically and to assess the trustworthiness of claims (assertions) about the benefits and harms (effects) of treatments. The overall aims of this follow-up study were to evaluate the impact of our intervention 1 year after it was administered, and to assess retention of learning and behaviour regarding claims about treatments.

METHODS

We randomly allocated consenting parents to listen to either the Informed Health Choices podcast (intervention) or typical public service announcements about health issues (control) over 7-10 weeks. Each intervention episode explained how the trustworthiness of treatment claims can be assessed by using relevant key concepts of evidence-informed decision-making. Participants listened to two episodes per week, delivered by research assistants. We evaluated outcomes immediately, and a year after the intervention. Primary outcomes were mean score and the proportion with a score indicating a basic ability to apply the key concepts (> 11 out of 18 correct answers) on a tool measuring people's ability to critically appraise the trustworthiness of treatment claims. Skills decay/retention was estimated by calculating the relative difference between the follow-up and initial results in the intervention group, adjusting for chance. Statistical analyses were performed using R (R Core Team, Vienna, Austria; version 3.4.3).

RESULTS

After 1 year, the mean score for parents in the intervention group was 58.9% correct answers, compared to 52.6% in the control (adjusted mean difference of 6.7% (95% CI 3.3% to 10.1%)). In the intervention group, 47.2% of 267 parents had a score indicating a basic ability to assess treatment claims compared to 39.5% of 256 parents in the control (adjusted difference of 9.8% more parents (95% CI 0.9% to 18.9%). These represent relative reductions of 29% in the mean scores and 33% in the proportion of parents with a score indicating a basic ability to assess the trustworthiness of claims about treatment effects.

CONCLUSIONS

Although listening to the Informed Health Choices podcast initially led to a large improvement in the ability of parents to assess claims about the effects of treatments, our findings show that these skills decreased substantially over 1 year. More active practice could address the substantial skills decay observed over 1 year.

TRIAL REGISTRATION

Pan African Clinical Trial Registry (www.pactr.org), PACTR201606001676150. Registered on 12 June 2016.

Effects of the Informed Health Choices primary school intervention on the ability of children in Uganda to assess the reliability of claims about treatment effects, 1-year follow-up: a cluster-randomised trial

INTRODUCTION

We evaluated an intervention designed to teach 10- to 12-year-old primary school children to assess claims about the effects of treatments (any action intended to maintain or improve health). We report outcomes measured 1 year after the intervention.

METHODS

In this cluster-randomised trial, we included primary schools in the central region of Uganda that taught year 5 children (aged 10 to 12 years). We randomly allocated a representative sample of eligible schools to either an intervention or control group. Intervention schools received the Informed Health Choices primary school resources (textbooks, exercise books and a teachers' guide). The primary outcomes, measured at the end of the school term and again after 1 year, were the mean score on a test with two multiple-choice questions for each of the 12 concepts and the proportion of children with passing scores.

RESULTS

We assessed 2960 schools for eligibility; 2029 were eligible, and a random sample of 170 were invited to recruitment meetings. After recruitment meetings, 120 eligible schools consented and were randomly assigned to either the intervention group (n = 60 schools; 76 teachers and 6383 children) or the control group (n = 60 schools; 67 teachers and 4430 children). After 1 year, the mean score in the multiple-choice test for the intervention schools was 68.7% compared with 53.0% for the control schools (adjusted mean difference 16.7%; 95% CI, 13.9 to 19.5; P < 0.00001). In the intervention schools, 3160 (80.1%) of 3943 children who completed the test after 1 year achieved a predetermined passing score (≥ 13 of 24 correct answers) compared with 1464 (51.5%) of 2844 children in the control schools (adjusted difference, 39.5%; 95% CI, 29.9 to 47.5).

CONCLUSION

Use of the learning resources led to a large improvement in the ability of children to assess claims, which was sustained for at least 1 year.

TRIAL REGISTRATION

Pan African Clinical Trial Registry (www.pactr.org), PACTR201606001679337. Registered on 13 June 2016.

Informed Health Choices media intervention for improving people's ability to critically appraise the trustworthiness of claims about treatment effects: a mixed-methods process evaluation of a randomised trial in Uganda

We developed the Informed Health Choices podcast to improve people's ability to assess claims about the effects of treatments. We evaluated the effects of the podcast in a randomised trial.

OBJECTIVES

We conducted this process evaluation to assess the fidelity of the intervention, identify factors that affected the implementation and impact of the intervention and could affect scaling up, and identify potential adverse and beneficial effects.

SETTING

The study was conducted in central Uganda in rural, periurban and urban settings.

PARTICIPANTS

We collected data on parents who were in the intervention arm of the Informed Health Choices study that evaluated an intervention to improve parents' ability to assess treatment effects.

PROCEDURES

We conducted 84 semistructured interviews during the intervention, 19 in-depth interviews shortly after, two focus group discussions with parents, one focus group discussion with research assistants and two in-depth interviews with the principal investigators. We used framework analysis to manage qualitative data, assessed the certainty of the findings using the GRADE-CERQual (Grading of Recommendations, Assessment, Development and Evaluations-Confidence in the Evidence from Reviews of Qualitative Research) approach, and organised findings in a logic model.

OUTCOMES

Proportion of participants listening to all episodes; factors influencing the implementation of the podcast; ways to scale up and any adverse and beneficial effects.

RESULTS

All participants who completed the study listened to the podcast as intended, perhaps because of the explanatory design and recruitment of parents with a positive attitude. This was also likely facilitated by the podcast being delivered by research assistants, and providing the participants with MP3 players. The podcast was reportedly clear, understandable, credible and entertaining, which motivated them to listen and eased implementation. No additional adverse effects were reported.

CONCLUSIONS

Participants experienced the podcast positively and were motivated to engage with it. These findings help to explain the short-term effectiveness of the intervention, but not the decrease in effectiveness over the following year.

Learning to make informed health choices: Protocol for a pilot study in schools in Barcelona

Introduction: The Informed Health Choices (IHC) project has developed learning resources to teach primary school children (10 to 12-year-olds) to assess treatment claims and make informed health choices. The aim of our study is to explore both the students' and teachers' experience when using these resources in the context of Barcelona (Spain). Methods: During the 2019-2020 school year, we will conduct a pilot study with 4 th and 5 th-year primary school students (9 to 11-year-olds) from three schools in Barcelona. The intervention in the schools will include: 1) a workshop with the teachers, and 2) lessons to the students. The data collection will include: 1) initial assessment of the resources by the teachers, 2) non-participatory observations during the lessons, 3) semi-structured interviews with the students after a lesson, 4) assessment of the lessons by the teachers, 5) treatment claim assessment by the students, and 6) final assessment of the resources by the teachers. We will use ad hoc questionnaires and guides to register the data. We will perform a quantitative and qualitative analysis of the data to explore understandability, desirability, suitability, usefulness, facilitators and barriers of the resources. The most relevant results will be discussed and some recommendations on how to use, how to adapt (if needed), and how to implement the IHC resources to this context will be agreed. We will publish the study results in a peer-reviewed journal and in several Internet resources (web pages, electronic bulletins, and social media), and we will present them to the different users of interest in conferences, workshops, and meetings. Ethical considerations: The study protocol has obtained an approval exemption from the Ethics Committee of the Hospital de la Santa Creu i Sant Pau (Barcelona, Spain).

Learning to make informed health choices: Protocol for a pilot study in schools in Barcelona

Introduction: The Informed Health Choices (IHC) project has developed learning resources to teach primary school children (10 to 12-year-olds) to assess treatment claims and make informed health choices. The aim of our study is to explore both the students' and teachers' experience when using these resources in the context of Barcelona (Spain). Methods: During the 2019-2020 school year, we will conduct a pilot study with 4 th and 5 th-year primary school students (9 to 11-year-olds) from three schools in Barcelona. The intervention in the schools will include: 1) assessment of the IHC resources by the teachers before the lessons, 2) non-participatory observations during the lessons, 3) semi-structured interviews with the students after a lesson, 4) assessment of the lessons by the teachers after a lesson, 5) treatment claim assessment by the students at the end of the lessons, and 6) assessment of the IHC resources by the teachers at the end of the lessons. We will use ad hoc questionnaires and guides to register the data. We will perform a quantitative and qualitative analysis of the data to explore understandability, desirability, suitability, usefulness, facilitators and barriers of the resources. The most relevant results will be discussed and some recommendations on how to use, how to adapt (if needed), and how to implement the IHC resources to this context will be agreed. The findings of the contextualization activities could inform the design of a cluster-randomised trial, to determine the effectiveness of the IHC resources in this context prior to scaling-up its use. Ethical considerations: The study protocol has obtained an approval exemption from the Ethics Committee of the Hospital de la Santa Creu i Sant Pau (Barcelona, Spain).

Learning to make informed health choices: Protocol for a pilot study in schools in Barcelona

Introduction: The Informed Health Choices (IHC) project has developed learning resources to teach primary school children (10 to 12-year-olds) to assess treatment claims and make informed health choices. The aim of our study is to explore both the students' and teachers' experience when using these resources in the context of Barcelona (Spain). Methods: During the 2019-2020 school year, we will conduct a pilot study with 4 th and 5 th-year primary school students (9 to 11-year-olds) from three schools in Barcelona. The intervention in the schools will include: 1) a workshop with the teachers, and 2) lessons to the students. The data collection will include: 1) assessment of the IHC resources by the teachers before the lessons, 2) non-participatory observations during the lessons, 3) semi-structured interviews with the students after a lesson, 4) assessment of the lessons by the teachers after a lesson, 5) treatment claim assessment by the students at the end of the lessons, and 6) assessment of the IHC resources by the teachers at the end of the lessons. We will use ad hoc questionnaires and guides to register the data. We will perform a quantitative and qualitative analysis of the data to explore understandability, desirability, suitability, usefulness, facilitators and barriers of the resources. The most relevant results will be discussed and some recommendations on how to use, how to adapt (if needed), and how to implement the IHC resources to this context will be agreed. The findings of the contextualization activities could inform the design of a cluster-randomised trial, to determine the effectiveness of the IHC resources in this context prior to scaling-up its use. Ethical considerations: The study protocol has obtained an approval exemption from the Ethics Committee of the Hospital de la Santa Creu i Sant Pau (Barcelona, Spain).