Evidence for public health policy on inequalities: 1: the reality according to policymakers

Systematic reviews of health effects of social interventions: 2. Best available evidence: how low should you go?

STUDY OBJECTIVE

There is little guidance on how to select the best available evidence of health effects of social interventions. The aim of this paper was to assess the implications of setting particular inclusion criteria for evidence synthesis.

DESIGN

Analysis of all relevant studies for one systematic review, followed by sensitivity analysis of the effects of selecting studies based on a two dimensional hierarchy of study design and study population.

SETTING

Case study of a systematic review of the effectiveness of interventions in promoting a population shift from using cars towards walking and cycling.

MAIN RESULTS

The distribution of available evidence was skewed. Population level interventions were less likely than individual level interventions to have been studied using the most rigorous study designs; nearly all of the population level evidence would have been missed if only randomised controlled trials had been included. Examining the studies that were excluded did not change the overall conclusions about effectiveness, but did identify additional categories of intervention such as health walks and parking charges that merit further research, and provided evidence to challenge assumptions about the actual effects of progressive urban transport policies.

CONCLUSIONS

Unthinking adherence to a hierarchy of study design as a means of selecting studies may reduce the value of evidence synthesis and reinforce an "inverse evidence law" whereby the least is known about the effects of interventions most likely to influence whole populations. Producing generalisable estimates of effect sizes is only one possible objective of evidence synthesis. Mapping the available evidence and uncertainty about effects may also be important.

Researchers and policymakers: travelers in parallel universes

Public policy, in the form of laws, guidelines, and regulations, has a profound effect on our daily lives and health status. Reasons for a lack of consistent and systematic translation of public health research into public policy is examined, including differences in decision-making processes, poor timing, ambiguous findings, the need to balance objectivity and advocacy, personal demands of the process, information overload, lack of relevant data, and the mismatch of randomized thinking with nonrandom problems. Next, several actions are suggested that should help bridge the chasm between science and policy, such as greater involvement in the process, better understanding of political decision making, building of effective teams, and development of political champions. Scientists are obligated not only to discover new knowledge but also to ensure that discoveries are applied to improve health.

Developing an evidence base for policies and interventions to address health inequalities: the analysis of "public health regimes"

Systematic reviews have become an important methodology in the United Kingdom by which research informs health policy, and their use now extends beyond evidence-based medicine to evidence-based public health and, particularly, health inequalities policies. This article reviews the limitations of systematic reviews as stand-alone tools for this purpose and suggests a complementary approach to make better use of the evidence. That is, systematic reviews and other sources of evidence should be incorporated into a wider analytical framework, the public health regime (defined here as the specific legislative, social, political, and economic structures that have an impact on both public health and the appropriateness and effectiveness of public health interventions adopted). At the national level this approach would facilitate analysis at all levels of the policy framework, countering the current focus on individual interventions. It could also differentiate at the international level between those policies and interventions that are effective in different contexts and are therefore potentially generalizable and those that depend on particular conditions for success.

Healthy settings: challenges to generating evidence of effectiveness

This paper starts by briefly reviewing the history, theory and practice of the settings approach to promoting public health--highlighting its ecological perspective, its understanding of settings as dynamic open systems and its primary focus on whole system organization development and change. It goes on to outline perceived benefits and consider why, almost 20 years after the Ottawa Charter advocated the approach, there remains a relatively poorly developed evidence base of effectiveness. Identifying three key challenges--relating to the construction of the evidence base for health promotion, the diversity of conceptual understandings and real-life practice and the complexity of evaluating ecological whole system approaches--it suggests that these have resulted in an ongoing tendency to evaluate only discrete projects in settings, thus failing to capture the 'added value' of whole system working. It concludes by exploring the potential value of theory-based evaluation and identifying key issues that will need to be addressed in moving forward--funding evaluation within and across settings; ensuring links between evidence, policy and practice; and clarifying and articulating the theories that underpin the settings approach generically and inform the approach as applied within particular settings.

Evaluating the effect of policies and interventions to address inequalities in health: lessons from a Dutch programme

OBJECTIVES

Many initiatives have been taken in European countries that are designed to reduce inequalities in health. However, the effects of only a very few of these initiatives have been assessed. The main aim of a Dutch research and development programme was to systematically investigate and evaluate interventions aimed at reducing inequalities in health. In this paper, we report on this investigation, and draw lessons from the methodology used to evaluate such interventions.

APPROACH

The programme included 12 evaluation studies, focusing on the wider determinants of inequalities in health (n = 2), behavioural determinants (n = 4), working conditions (n = 3) and health care (n = 3).

RESULTS

An experimental design was applied in two evaluation studies. The studies provided evidence of a positive effect. A quasi-experimental design appeared to be the only attainable option in seven studies. Five of these provided sufficient evidence for a positive effect, but two interventions appeared not to be successful. In three studies, no experimental or quasi-experimental design could be applied.

CONCLUSIONS

The programme showed that it is possible to apply experimental or quasi-experimental studies to complex public health interventions. The Programme Committee steering the programme considered that the evidence generated by the experimental and quasi-experimental studies justified the implementation of the interventions on a wider scale, accompanied by further evaluation studies. Further development of the methodology of public health interventions is necessary. These include non-experimental designs such as international comparisons and time trend studies, especially in order to be able to evaluate broader policy measures.

Evidence for public health policy on inequalities: 2: assembling the evidence jigsaw

STUDY OBJECTIVE

To garner research leaders' perceptions and experiences of the types of evidence that influence policy on health inequalities, and their reflections on how the flow of such research evidence could be increased.

DESIGN, SETTING, AND PARTICIPANTS

Qualitative two day residential workshop with senior research leaders, most of whom were currently involved in evaluations of the health effects of major policies. In four in depth sessions, facilitated by the authors in turn, focused questions were presented to participants to reveal their views and experiences concerning evidence synthesis for policy on inequalities. These were analysed thematically.

MAIN RESULTS

Five types of evidence for policy on health inequalities were felt to be particularly persuasive with policymakers: observational evidence showing the existing of a problem; narrative accounts of the impacts of policies from the household perspective; controlled evaluations; natural policy experiments; and historical evidence. Methods of improving the availability and use of these sources of information were put forward.

CONCLUSIONS

This paper and its companion have considered the current evidence base for policies to reduce health inequalities, and how this could be improved. There is striking congruence between the views of the researchers in this study and policy advisers in paper 1, suggesting that a common understanding may be emerging. The findings suggest significant potential for rapid progress to be made in developing both evidence based policy, and policy relevant evidence to tackle inequalities in health.