Assessing the usefulness of systematic reviews for policymakers in public health: A case study of overweight and obesity prevention interventions

Combining public health evidence, policy experience and communications expertise to inform preventive health: reflections on a novel method of knowledge synthesis

Knowledge synthesis methods help summarize evidence and utilize content expertise to draw out key messages to aid knowledge mobilization and translation. Systems thinking and coproduction can support this by facilitating a multiperspective view and ensuring that knowledge is mobilized and translated in a useful and meaningful way for policy-makers and practitioners. In this paper, we describe the development of a knowledge synthesis approach that utilizes coproduction with policy-makers to combine the findings of a programme of research with policy knowledge to support decision-makers working in chronic disease prevention. The process developed by The Australian Prevention Partnership Centre combined the expertise of research, policy and science communications experts. We reflect on how we used coproduction processes to embed policy-makers as partners in the evidence synthesis process via research-policy dialogues, and embedded science communication into the development and presentation of the findings. This differs from a more common approach of researchers generating evidence for policy with limited input from policy-makers themselves. By collaborating with policy-makers and using coproduction, we can better inform policy-relevant research and generate policy-relevant knowledge. We describe the development of our knowledge synthesis approach using two case studies: the first drawing on a body of work in public health law, and the second on a body of work focused on the first 2000 days of life. We consider how these case studies demonstrate the value of working with policy partners as part of a knowledge synthesis process, and discuss how this process could be adapted and used in future.

Interventions to prevent obesity in school-aged children 6-18 years: An update of a Cochrane systematic review and meta-analysis including studies from 2015-2021

BACKGROUND

Childhood obesity remains a global public health priority due to the enormous burden it generates. Recent surveillance data suggests there has been a sharp increase in the prevalence of childhood obesity during the COVID-19 pandemic. The Cochrane review of childhood obesity prevention interventions (0-18 years) updated to 2015 is the most rigorous and comprehensive review of randomised controlled trials (RCTs) on this topic. A burgeoning number of high quality studies have been published since that are yet to be synthesised.

METHODS

An update of the Cochrane systematic review was conducted to include RCT studies in school-aged children (6-18 years) published to 30 June 2021 that assessed effectiveness on child weight (PROSPERO registration: CRD42020218928). Available cost-effectiveness and adverse effect data were extracted. Intervention effects on body mass index (BMI) were synthesised in random effects meta-analyses by setting (school, after-school program, community, home), and meta-regression examined the association of study characteristics with intervention effect.

FINDINGS

Meta-analysis of 140 of 195 included studies (183,063 participants) found a very small positive effect on body mass index for school-based studies (SMD -0·03, 95%CI -0·06,-0·01; trials = 93; participants = 131,443; moderate certainty evidence) but not after-school programs, community or home-based studies. Subgroup analysis by age (6-12 years; 13-18 years) found no differential effects in any setting. Meta-regression found no associations between study characteristics (including setting, income level) and intervention effect. Ten of 53 studies assessing adverse effects reported presence of an adverse event. Insufficient data was available to draw conclusions on cost-effectiveness.

INTERPRETATION

This updated synthesis of obesity prevention interventions for children aged 6-18 years, found a small beneficial impact on child BMI for school-based obesity prevention interventions. A more comprehensive assessment of interventions is required to identify mechanisms of effective interventions to inform future obesity prevention public health policy, which may be particularly salient in for COVID-19 recovery planning.

FUNDING

This research was funded by the National Health and Medical Research Council (NHMRC), Australia (Application No APP1153479).

Cost comparison of five Australasian obesity prevention interventions for children aged from birth to two years

BACKGROUND

In the absence of rigorous evidence of cost-effectiveness for early childhood obesity prevention interventions, the next-best option may be for decision-makers to consider the relevant costs of interventions when allocating resources.

OBJECTIVES

This study aimed to estimate systematically the cost of five obesity prevention interventions in children aged 0-2 years, undertaken in research settings in Australia and New Zealand.

METHODS

A standardised costing protocol informed the costing methodology, ensuring comparability of results across interventions. Micro-costing was undertaken, with intervention costs defined from the funder perspective and valued in 2018 Australian dollars using unit costs from the trials or market rates.

RESULTS

Interventions varied widely in their resource use. The total cost per participant ranged from $80 for the CHAT SMS intervention arm (95% UI $77-$82) to $1135 for the Healthy Beginnings intervention (95% UI $1059-$1189). Time costs of personnel delivering interventions contributed >50% of total intervention costs for all included studies.

CONCLUSIONS

An understanding of the costs associated with intervention delivery modes is important, alongside effectiveness. Telephone delivery may include unexpected costs associated with connection to intervention participants at convenient times. A SMS-based intervention had the lowest delivery cost in this study.

Use and value of systematic reviews in English local authority public health: a qualitative study

Background

Responsibility for public health in England transferred from the National Health Service to local authorities in 2013, representing a different decision-making environment. Systematic reviews are considered the gold standard of evidence for clinical decision-making but little is known about their use in local government public health. This study aimed to explore the extent to which public health decision-makers in local authorities engage with systematic reviews and how they do so.

Methods

Semi-structured interviews were conducted with senior public health practitioners (n = 14) in Yorkshire and the Humber local authorities. Sampling was purposive and involved contacting Directors of Public Health directly and snowballing through key contacts. Face-to-face or telephone interviews were digitally recorded, transcribed verbatim and analysed using the Framework Method.

Results

Public health practitioners described using systematic reviews directly in decision-making and engaging with them more widely in a range of ways, often through a personal commitment to professional development. They saw themselves as having a role to advocate for the use of rigorous evidence, including systematic reviews, in the wider local authority. Systematic reviews were highly valued in principle and public health practitioners had relevant skills to find and appraise them. However, the extent of use varied by individual and local authority and was limited by the complexity of decision-making and various barriers. Barriers included that there were a limited number of systematic reviews available on certain public health topics, such as the wider determinants of health, and that the narrow focus of reviews was not reflective of complex public health decisions facing local authorities. Reviews were used alongside a range of other evidence types, including grey literature. The source of evidence was often considered an indicator of quality, with specific organisations, such as Public Health England, NICE and Cochrane, particularly trusted.

Conclusions

Research use varies and should be considered within the specific decision-making and political context. There is a need for systematic reviews to be more reflective of the decisions facing local authority public health teams.

A Systematic Search and Review of Adult-Targeted Overweight and Obesity Prevention Mass Media Campaigns and Their Evaluation: 2000-2017

Mass media campaigns are a commonly used strategy in public health. However, no review has assessed whether the design and evaluation of overweight and obesity campaigns meets best practice recommendations. This study aimed to fill this gap. We systematically searched five databases for peer-reviewed articles describing adult-targeted obesity mass media campaigns published between 2000 and 2017, complemented by reference list searches and contact with authors and agencies responsible for the campaigns. We extracted data on campaign design, implementation, and evaluation from eligible publications and conducted a qualitative review of 29 publications reporting on 14 campaigns. We found a need for formative research with target audiences to ensure campaigns focus on the most salient issues. Further, we noted that most campaigns targeted individual behaviors, despite calls for campaigns to also focus upstream and to address social determinants of obesity. Television was the dominant communication channel but, with the rapid advance of digital media, evaluation of other channels, such as social media, is increasingly important. Finally, although evaluation methods varied in quality, the evidence suggests that campaigns can have an impact on intermediate outcomes, such as knowledge and attitudes. However, evidence is still limited as to whether campaigns can influence behavior change.

A scoping approach to systematically review published reviews: Adaptations and recommendations

Knowledge translation is a central focus of the health research community, which includes strategies to synthesize published research to support uptake within health care practice and policy arenas. Within the literature concerning review methodologies, a new discussion has emerged concerning methods that review and synthesize published review articles. In this paper, our multidisciplinary team from family medicine, nursing, dental hygiene, kinesiology, occupational therapy, physiology, population health, clinical psychology, and library sciences contributes to this discussion by sharing our experiences in conducting 3 scoping reviews of published review studies. A brief discussion of Cochrane Collaboration overview reviews and Joanna Briggs Institute umbrella reviews foreshadows a discussion of insights from our experiences of conducting the 3 scoping reviews of published reviews. We address 6 adaptations along with our recommendations for each, which may guide other researchers with designing scoping review approaches to synthesize published reviews. The ability of researchers to publish research findings is growing, and our ability to effectively transfer findings into useful evidence for health care practice and policy is imperative to our work.

Knowledge mobilisation for policy development: implementing systems approaches through participatory dynamic simulation modelling

BACKGROUND

Evidence-based decision-making is an important foundation for health policy and service planning decisions, yet there remain challenges in ensuring that the many forms of available evidence are considered when decisions are being made. Mobilising knowledge for policy and practice is an emergent process, and one that is highly relational, often messy and profoundly context dependent. Systems approaches, such as dynamic simulation modelling can be used to examine both complex health issues and the context in which they are embedded, and to develop decision support tools.

OBJECTIVE

This paper reports on the novel use of participatory simulation modelling as a knowledge mobilisation tool in Australian real-world policy settings. We describe how this approach combined systems science methodology and some of the core elements of knowledge mobilisation best practice. We describe the strategies adopted in three case studies to address both technical and socio-political issues, and compile the experiential lessons derived. Finally, we consider the implications of these knowledge mobilisation case studies and provide evidence for the feasibility of this approach in policy development settings.

CONCLUSION

Participatory dynamic simulation modelling builds on contemporary knowledge mobilisation approaches for health stakeholders to collaborate and explore policy and health service scenarios for priority public health topics. The participatory methods place the decision-maker at the centre of the process and embed deliberative methods and co-production of knowledge. The simulation models function as health policy and programme dynamic decision support tools that integrate diverse forms of evidence, including research evidence, expert knowledge and localised contextual information. Further research is underway to determine the impact of these methods on health service decision-making.

Information sources for obesity prevention policy research: a review of systematic reviews

BACKGROUND

Systematic identification of evidence in health policy can be time-consuming and challenging. This study examines three questions pertaining to systematic reviews on obesity prevention policy, in order to identify the most efficient search methods: (1) What percentage of the primary studies selected for inclusion in the reviews originated in scholarly as opposed to gray literature? (2) How much of the primary scholarly literature in this topic area is indexed in PubMed/MEDLINE? (3) Which databases index the greatest number of primary studies not indexed in PubMed, and are these databases searched consistently across systematic reviews?

METHODS

We identified systematic reviews on obesity prevention policy and explored their search methods and citations. We determined the percentage of scholarly vs. gray literature cited, the most frequently cited journals, and whether each primary study was indexed in PubMed. We searched 21 databases for all primary study articles not indexed in PubMed to determine which database(s) indexed the highest number of these relevant articles.

RESULTS

In total, 21 systematic reviews were identified. Ten of the 21 systematic reviews reported searching gray literature, and 12 reviews ultimately included gray literature in their analyses. Scholarly articles accounted for 577 of the 649 total primary study papers. Of these, 495 (76%) were indexed in PubMed. Google Scholar retrieved the highest number of the remaining 82 non-PubMed scholarly articles, followed by Scopus and EconLit. The Journal of the American Dietetic Association was the most-cited journal.

CONCLUSIONS

Researchers can maximize search efficiency by searching a small yet targeted selection of both scholarly and gray literature resources. A highly sensitive search of PubMed and those databases that index the greatest number of relevant articles not indexed in PubMed, namely multidisciplinary and economics databases, could save considerable time and effort. When combined with a gray literature search and additional search methods, including cited reference searching and consulting with experts, this approach could help maintain broad retrieval of relevant studies while improving search efficiency. Findings also have implications for designing specialized databases for public health research.

Obesity prevention advocacy in Australia: an analysis of policy impact on autonomy

OBJECTIVE

To explore obesity policy options recommended by stakeholders and identify their impact on individual autotomy.

METHODS

Qualitative and quantitative methods were used. A content analysis of submissions to the Australian Government's Inquiry into Obesity was conducted. Each recommendation was categorised by its impact on autonomy, according to existing frameworks. Chi-square test for independence was used to explore the association between autonomy and stakeholder support defined as frequency of recommendation.

RESULTS

The extent of support for a policy option was significantly associated with impact on autonomy (p<0.001). Options that reduce autonomy were least frequently recommended in every setting; but more likely in schools (27%) than other settings (<1%). Recommendations to provide incentives (9%) were more common than disincentives (2%) or restrictions (3%), and those that enhance autonomy were most widely recommended (46%).

CONCLUSIONS

Stakeholders advocated policy options that enhance individual autonomy to a greater extent than those that diminish autonomy. Implications for public health: Targeting obesity policy options that enhance rather than diminish autonomy may be more politically acceptable across most settings, with the exception of schools where more restrictive policy options are appropriate. Re-framing options accordingly may improve leadership by government in obesity policy.

Modified Policy-Delphi study for exploring obesity prevention priorities

INTRODUCTION

Until now, industry and government stakeholders have dominated public discourse about policy options for obesity. While consumer involvement in health service delivery and research has been embraced, methods which engage consumers in health policy development are lacking. Conflicting priorities have generated ethical concern around obesity policy. The concept of 'intrusiveness' has been applied to policy decisions in the UK, whereby ethical implications are considered through level of intrusiveness to choice; however, the concept has also been used to avert government regulation to address obesity. The concept of intrusiveness has not been explored from a stakeholder's perspective. The aim is to investigate the relevance of intrusiveness and autonomy to health policy development, and to explore consensus on obesity policy priorities of under-represented stakeholders.

METHODS AND ANALYSIS

The Policy-Delphi technique will be modified using the James Lind Alliance approach to collaborative priority setting. A total of 60 participants will be recruited to represent three stakeholder groups in the Australian context: consumers, public health practitioners and policymakers. A three-round online Policy-Delphi survey will be undertaken. Participants will prioritise options informed by submissions to the 2009 Australian Government Inquiry into Obesity, and rate the intrusiveness of those proposed. An additional round will use qualitative methods in a face-to-face discussion group to explore stakeholder perceptions of the intrusiveness of options. The novelty of this methodology will redress the balance by bringing the consumer voice forward to identify ethically acceptable obesity policy options.

ETHICS AND DISSEMINATION

Ethical approval was granted by the Bond University Health Research Ethics Committee. The findings will inform development of a conceptual framework for analysing and prioritising obesity policy options, which will be relevant internationally and to ethical considerations of wider public health issues. The findings will be disseminated through peer-reviewed publications, conference presentations and collaborative platforms of policy and science.

An assessment of health research impact in Iran

Background

In recent years, Iran has made significant developments in the field of health sciences. However, the question is whether this considerable increase has affected public health. The research budget has always been negligible and unsustainable in developing countries. Hence, using the Payback Framework, we conducted this study to evaluate the impact of health research in Iran.

Methods

By using a cross-sectional method and two-stage stratified cluster sampling, the projects were randomly selected from six medical universities. A questionnaire was designed according to the Payback Framework and completed by the principle investigators of the randomly selected projects.

Results

The response rate was 70.4%. Ten point twenty-four percent (10.24%) of the studies had been ordered by a knowledge user organization. The average number of articles published in journals per project was 0.96, and half of the studies had no articles published in Scopus. The results of 12% of the studies had been used in systematic review articles and the same proportion had been utilized in clinical or public health guidelines. The results of 5.3% of the studies had been implemented in the Health Ministry’s policymaking. 62% of the studies were expected to affect health directly, 38% of them had been implemented, and among the latter 60% had achieved the expected results. Concerning the economic impacts, the most common expected impact was the reduction of ‘days of work missed because of illness or disability’ and impact on personal and health system costs. About 36% of these studies had been implemented, and 61% had achieved the expected impact.

Conclusion

In most aspects, the status of research impact needs improvement. A comparison of Iran’s ranking of knowledge creation and knowledge impact in the Global Innovation Index confirms these findings. The most important problems identified were, not conducting research based on national needs, and the lack of implementation of research results.

Electronic supplementary material

The online version of this article (doi:10.1186/s12961-016-0129-9) contains supplementary material, which is available to authorized users.

The effectiveness of workplace dietary interventions: protocol for a systematic review and meta-analysis

BACKGROUND

The lack of evidence of the role of workplaces as settings for behaviour change delivery and the failure to recognise and address the complexity of the work environment has been acknowledged. This systematic review and meta-analysis will identify the effectiveness of dietary interventions in the workplace facilitating an understanding of what works, why and how by identifying key components of and examining the theoretical models of behaviour change underpinning successful dietary interventions in the workplace.

METHODS/DESIGN

We will conduct searches in MEDLINE, EMBASE, CINAHL, PsycINFO, CENTRAL and PubMed for studies that assess dietary interventions based within workplace settings in any country, of any length of time or duration of follow-up. We will include all randomised controlled trials (RCTs), non-randomised controlled trials (NRCTs), controlled before-after studies (CBAs) and interrupted time series (ITS) studies with a control group. Risk of bias of included studies will be assessed using a tool adapted from the Cochrane Public Health Review Group's recommended Effective Public Health Practice Project Quality Assessment Tool for Quantitative Studies. Meta-analysis will be conducted if appropriate, or a narrative synthesis will be conducted following the ESRC Narrative Synthesis Guidance.

DISCUSSION

This paper outlines the study protocol for a systematic review and meta-analysis that will identify, critically appraise, and summarise the relevant evidence on the effectiveness and implications of interventions to promote healthier dietary behaviours in the workplace. This review will give an overview of the evidence and provide a guide for development of interventions promoting dietary behaviour change in workplaces.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO CRD42015015175.