Criteria for evaluating evidence on public health interventions

THE PROMISE AND PITFALLS OF SYSTEMATIC REVIEWS

The systematic review "movement" that has transformed medical journal reports of clinical trials and reviews of clinical trials has taken hold in public health, with the most recent milestone, the publication of the first edition of The Guide to Community Health Services in 2005. In this paper we define and distinguish current terms, point out important resources for systematic reviews, describe the impact of systematic review on the quality of primary studies and summaries of the evidence, and provide perspectives on the promise of systematic reviews for shaping the agenda for public health research. Several pitfalls are discussed, including a false sense of rigor implied by the terms "systematic review" and "meta-analysis" and substantial variation in the validity of claims that a particular intervention is "evidence based," and the difficulty of translating conclusions from systematic reviews into public health advocacy and practice.

Systematic reviews of health effects of social interventions: 2. Best available evidence: how low should you go?

STUDY OBJECTIVE

There is little guidance on how to select the best available evidence of health effects of social interventions. The aim of this paper was to assess the implications of setting particular inclusion criteria for evidence synthesis.

DESIGN

Analysis of all relevant studies for one systematic review, followed by sensitivity analysis of the effects of selecting studies based on a two dimensional hierarchy of study design and study population.

SETTING

Case study of a systematic review of the effectiveness of interventions in promoting a population shift from using cars towards walking and cycling.

MAIN RESULTS

The distribution of available evidence was skewed. Population level interventions were less likely than individual level interventions to have been studied using the most rigorous study designs; nearly all of the population level evidence would have been missed if only randomised controlled trials had been included. Examining the studies that were excluded did not change the overall conclusions about effectiveness, but did identify additional categories of intervention such as health walks and parking charges that merit further research, and provided evidence to challenge assumptions about the actual effects of progressive urban transport policies.

CONCLUSIONS

Unthinking adherence to a hierarchy of study design as a means of selecting studies may reduce the value of evidence synthesis and reinforce an "inverse evidence law" whereby the least is known about the effects of interventions most likely to influence whole populations. Producing generalisable estimates of effect sizes is only one possible objective of evidence synthesis. Mapping the available evidence and uncertainty about effects may also be important.

Process evaluation in randomised controlled trials of complex interventions

Most randomised controlled trials focus on outcomes, not on the processes involved in implementing an intervention. Using an example from school based health promotion, this paper argues that including a process evaluation would improve the science of many randomised controlled trials

From rhetoric to reality: barriers faced by Health For All initiatives

The last two decades have witnessed an upsurge in the development and implementation of 'Health For All-type' initiatives in many parts of the world. However, despite the popularity of the approach, barriers and constraints to the fulfillment of their remit still persist, making it difficult for them to achieve the potential originally envisaged. Drawing upon considerable empirical work while evaluating the European Healthy City projects and English Health Action Zones, this paper explores the differences between barriers and constraints and then focuses on barriers as they manifest themselves in England. It distinguishes between cultural barriers, stemming from different philosophical, organisational, and professional/experiential cultures, and political barriers, stemming from both party political and realpolitik concerns. It discusses how these barriers often operate together, compounding their individual impacts, with detrimental effects for Health For All initiatives. Consequently, while the prevailing rhetoric appears to promote an alternative, and more appropriate, vision of how health can be maintained and enhanced, these barriers effectively function to sustain the hegemony of the status quo which was, and is, based on a different and outdated vision. We argue that acknowledging the continuous persistence of these barriers is an essential first step towards turning the prevailing health-related rhetoric into reality.

Systematic reviews of health effects of social interventions: 1. Finding the evidence: how far should you go?

STUDY OBJECTIVE

There is little guidance on how to identify useful evidence about the health effects of social interventions. The aim of this study was to assess the value of different ways of finding this type of information.

DESIGN

Retrospective analysis of the sources of studies for one systematic review.

SETTING

Case study of a systematic review of the effectiveness of interventions in promoting a population shift from using cars towards walking and cycling.

MAIN RESULTS

Only four of the 69 relevant studies were found in a "first-line" health database such as Medline. About half of all relevant studies were found through the specialist Transport database. Nine relevant studies were found through purposive internet searches and seven relevant studies were found by chance. The unique contribution of experts was not to identify additional studies, but to provide more information about those already found in the literature.

CONCLUSIONS

Most of the evidence needed for this review was not found in studies indexed in familiar literature databases. Applying a sensitive search strategy across multiple databases and interfaces is very labour intensive. Retrospective analysis suggests that a more efficient method might have been to search a few key resources, then to ask authors and experts directly for the most robust reports of studies identified. However, internet publications and serendipitous discoveries did make a significant contribution to the total set of relevant evidence. Undertaking a comprehensive search may provide unique evidence and insights that would not be obtained using a more focused search.

School-based interventions for spinal pain: a systematic review

STUDY DESIGN

Systematic review.

OBJECTIVES

To establish the effectiveness of school-based spinal health interventions in terms of: 1) improving knowledge about the spine/spinal care; 2) changing spinal care behaviors; and 3) decreasing the prevalence of spinal pain.

SUMMARY OF BACKGROUND DATA

Spinal pain is a significant problem in children and adolescents that has been addressed through school-based spinal health interventions. No systematic review has been carried out on this topic to date.

METHODS

A systematic literature review sought studies that evaluated school-based spinal health interventions. Using clearly defined study inclusion criteria, 11 databases were searched from their inception to March 2004. To identify further literature, three relevant journals were hand searched, reference lists were checked, and authors of included papers were contacted. Two reviewers independently appraised the quality of identified papers and extracted data regarding intervention and study characteristics, statistical analyses performed, and study results. Data were examined using a narrative synthesis of results, and the outcomes of interest were considered individually (knowledge, behaviors, pain prevalence).

RESULTS

Twelve papers were included in this review; all papers received a "weak" quality rating. Results of these studies indicate that school-based spinal health interventions may be effective in increasing spinal care knowledge and decreasing the prevalence of spinal pain. However, overall the evidence is inconclusive regarding spinal care behaviors.

CONCLUSIONS

The poor quality of the reviewed studies limits the conclusions that can be made regarding the effectiveness of school-based spinal health interventions.

Developing an evidence base for policies and interventions to address health inequalities: the analysis of "public health regimes"

Systematic reviews have become an important methodology in the United Kingdom by which research informs health policy, and their use now extends beyond evidence-based medicine to evidence-based public health and, particularly, health inequalities policies. This article reviews the limitations of systematic reviews as stand-alone tools for this purpose and suggests a complementary approach to make better use of the evidence. That is, systematic reviews and other sources of evidence should be incorporated into a wider analytical framework, the public health regime (defined here as the specific legislative, social, political, and economic structures that have an impact on both public health and the appropriateness and effectiveness of public health interventions adopted). At the national level this approach would facilitate analysis at all levels of the policy framework, countering the current focus on individual interventions. It could also differentiate at the international level between those policies and interventions that are effective in different contexts and are therefore potentially generalizable and those that depend on particular conditions for success.

Healthy settings: challenges to generating evidence of effectiveness

This paper starts by briefly reviewing the history, theory and practice of the settings approach to promoting public health--highlighting its ecological perspective, its understanding of settings as dynamic open systems and its primary focus on whole system organization development and change. It goes on to outline perceived benefits and consider why, almost 20 years after the Ottawa Charter advocated the approach, there remains a relatively poorly developed evidence base of effectiveness. Identifying three key challenges--relating to the construction of the evidence base for health promotion, the diversity of conceptual understandings and real-life practice and the complexity of evaluating ecological whole system approaches--it suggests that these have resulted in an ongoing tendency to evaluate only discrete projects in settings, thus failing to capture the 'added value' of whole system working. It concludes by exploring the potential value of theory-based evaluation and identifying key issues that will need to be addressed in moving forward--funding evaluation within and across settings; ensuring links between evidence, policy and practice; and clarifying and articulating the theories that underpin the settings approach generically and inform the approach as applied within particular settings.

Criteria for the systematic review of health promotion and public health interventions

Systematic reviews of public health interventions are fraught with challenges. Complexity is inherent; this may be due to multi-component interventions, diverse study populations, multiple outcomes measured, mixed study designs utilized and the effect of context on intervention design, implementation and effectiveness. For policy makers and practitioners to use systematic reviews to implement effective public health programmes, systematic reviews must include this information, which seeks to answer the questions posed by decision makers, including recipients of programmes. This necessitates expanding the traditional evaluation of evidence to incorporate the assessment of theory, integrity of interventions, context and sustainability of the interventions and outcomes. Unfortunately however, the critical information required for judging both the quality of a public health intervention and whether or not an intervention is worthwhile or replicable is missing from most public health intervention studies. When the raw material is not available in primary studies the systematic review process becomes even more challenging. Systematic reviews, which highlight these critical gaps, may act to encourage better reporting in primary studies. This paper provides recommendations to reviewers on the issues to address within a public health systematic review and, indirectly, provides advice to researchers on the reporting requirements of primary studies for the production of high quality systematic reviews.

Making evaluation an integral part of injury programme implementation

Uncertainty about the nature of evaluation can lead to injury prevention programme evaluations being poorly resourced and poorly conducted. The aim of this paper is to demystify programme evaluation and, by stressing its function, offer a perspective on evaluation that may encourage its more widespread integration in the general activity of injury prevention and control. Programme evaluation is best understood simply as the process of getting answers to essential questions about a programme. Methodology used in evaluation needs to be based in empirical science but is otherwise unrestricted except by the chosen question and the practical circumstances relating to the programme and the community in which it is implemented. Discussion about which methodology is appropriate for evaluation research is (unwittingly) a debate about 'which questions should you be asking?'. If the right people ask the right (and properly formulated) questions and build the means of obtaining the information to answer these questions (using appropriate methodology) into the conduct of the programme then evaluation will no longer be a problem but an essential component of the overall effort to reduce the community burden of injury.

A realist study of the mechanisms of cardiac rehabilitation

AIM

The aim of this paper is to report patients' experiences of cardiac rehabilitation and perceptions of the mechanisms and contexts influencing its long-term effectiveness.

BACKGROUND

Cardiac rehabilitation programmes for the secondary prevention of coronary heart disease are common. The effects of these programmes, however, can be inconsistent and little is known of the personal and contextual factors that influence service effectiveness.

METHOD

Forty-seven participants with a formal diagnosis of coronary heart disease who had attended a programme of cardiac rehabilitation in Scotland 3 years previously were included in focus groups to discuss their perceptions and experiences (30 males and 17 females). The data were generated in 2002 and analysed using the realist approach of Pawson and Tilley (1997).

RESULTS

Participants' accounts indicated that the didactic content of cardiac rehabilitation was not strongly linked to longer-term health behaviour change. The main positive effects of cardiac rehabilitation were related to the effect of participation on mediating social and body-focused mechanisms that were triggered when the rehabilitation setting was perceived to be safe. Social mechanisms identified included social comparisons, camaraderie, and social capital. Body-focused mechanisms included greater knowledge of personal physical boundaries and a greater trust in the heart-diseased body. Collectively, these mechanisms had a positive effect on confidence that was perceived as being imperative to maintain health behaviour change.

CONCLUSIONS

More support is required to promote health behaviour change after the completion of cardiac rehabilitation. Use of community-based exercise services and conventional or web-based support groups for coronary heart disease patients should be encouraged, as these appear to extend the positive health effects of the mechanisms that promote behaviour change. At the completion of cardiac rehabilitation programmes, patients should be referred to safe and appropriate community-based exercise services. Further research is needed to examine the effects on health outcomes of mechanisms and contexts related to cardiac rehabilitation.

Applicability and transferability of interventions in evidence-based public health

The context in which public health programmes operate can play an important role in influencing their implementation and effectiveness. An intervention that has been shown to be effective in one setting may turn out to be ineffective somewhere else, even supposing it can be implemented there. Therefore, systematic reviews of public health interventions should appraise the applicability of the intervention process and the transferability of the intervention effectiveness to other localities. However, applicability and transferability appraisal is seldom reported in systematic reviews of public health and health promotion interventions. This paper aims to introduce an innovative approach to bridging this gap. A list of attributes that may impact on applicability and transferability can be developed, based on knowledge of the proposed intervention. Then the applicability and transferability of the intervention to the local setting can be rated, and given a score, based on knowledge of the local setting. This approach provides a useful tool for evaluating public health interventions and provides a reliable basis for informed decision making in resource-poor settings, where rigorous primary studies are lacking and where very limited resources put a high demand on evidence-based approaches to health promotion.

Challenging the framework for evidence in palliative care research

This paper examines the debate about best evidence within the public health literature and proposes that similar arguments and concerns exist with respect to use of current evidence-based approaches to implementing research and evaluating the literature in palliative care. Whilst randomized controlled trials (RCTs) remain the gold standard and are appropriate in many instances of palliative care research, there is a need for an alternate research design framework that incorporates contextual and compositional effects pertinent to palliative care research. A framework, entitled Equity-Based Evidence, is discussed as an approach to evidence-based knowledge development in palliative care.

Evaluating the effect of policies and interventions to address inequalities in health: lessons from a Dutch programme

OBJECTIVES

Many initiatives have been taken in European countries that are designed to reduce inequalities in health. However, the effects of only a very few of these initiatives have been assessed. The main aim of a Dutch research and development programme was to systematically investigate and evaluate interventions aimed at reducing inequalities in health. In this paper, we report on this investigation, and draw lessons from the methodology used to evaluate such interventions.

APPROACH

The programme included 12 evaluation studies, focusing on the wider determinants of inequalities in health (n = 2), behavioural determinants (n = 4), working conditions (n = 3) and health care (n = 3).

RESULTS

An experimental design was applied in two evaluation studies. The studies provided evidence of a positive effect. A quasi-experimental design appeared to be the only attainable option in seven studies. Five of these provided sufficient evidence for a positive effect, but two interventions appeared not to be successful. In three studies, no experimental or quasi-experimental design could be applied.

CONCLUSIONS

The programme showed that it is possible to apply experimental or quasi-experimental studies to complex public health interventions. The Programme Committee steering the programme considered that the evidence generated by the experimental and quasi-experimental studies justified the implementation of the interventions on a wider scale, accompanied by further evaluation studies. Further development of the methodology of public health interventions is necessary. These include non-experimental designs such as international comparisons and time trend studies, especially in order to be able to evaluate broader policy measures.

The general practice research database: role in pharmacovigilance

The General Practice Research Database (GPRD) is the world's largest computerised database of anonymised longitudinal clinical records from primary care. The database already has an international reputation in the field of drug safety signal evaluation where the results of GPRD-based pharmacoepidemiological studies have been used to inform regulatory pharmacovigilance decision making. The characteristics and richness of the data are such that the GPRD is likely to prove a key data resource for the proactive pharmacovigilance anticipated in risk management and pharmacovigilance plans. An update of recent developments to the database and new data available from it -- including spontaneously recorded suspected adverse drug reactions -- is presented in the article, with a description of how the data can be used to support a variety of pharmacovigilance applications. The possibility of using the GPRD in signal detection and assessment of the impact of pharmacovigilance activities in the future is also discussed.

Utilización de proyectos de trabajo en una organización de salud del sector público en Argentina: paradojas, dilemas y oportunidades

El objetivo del artículo es presentar una experiencia de aplicación de un instrumento de gestión en una organización de salud del ámbito público en Argentina y reflexionar acerca de las interacciones entre sus componentes, potencialidades y dificultades, y las complicaciones para su relato, análisis y fundamentación. Se trata de un estudio descriptivo. Se analizaron registros, formularios y documentos institucionales, se consultó a informantes claves y se realizó una revisión bibliográfica. Se analizó la introducción de la planificación personal desde la perspectiva del desarrollo de los trabajadores y de la mejora del desempeño de las organizaciones. Como principal hallazgo se señala que los proyectos de trabajo resultaron orientadores del esfuerzo individual hacia los objetivos institucionales y proveyeron elementos para el diseño organizacional. Contaron con diferentes grados de adhesión y seguimiento. También generaron conflictos, desconfianza y resistencias. Finalmente, puede decirse que en organizaciones de la administración pública es posible gerenciar por resultados y poner en equilibrio confianza y control. Su alcance refleja las tensiones con las presiones externas, las luchas internas de poder, la resistencia a la supervisión y monitoreo, la rigidez de normativas centralizadas y la tendencia a convertir instrumentos de gestión en meros elementos de control y/o rutinarios.

Medication reviews in the community: results of a randomized, controlled effectiveness trial

AIMS

To examine the effectiveness of a multidisciplinary service model delivering medication review to patients at risk of medication misadventure in the community.

METHODS

The study was carried out in three Australian states; Queensland, New South Wales and Western Australia, and conducted as a randomized, controlled effectiveness trial with the general practitioner (GP) as the unit of randomization. In total, 92 GPs, 53 pharmacists and 400 patients enrolled in the study. The multidisciplinary service model consisted of GP education, patient home visits, pharmacist medication reviews, primary healthcare team conferences, GP implementation of action plans in consultation with patients, and follow-up surgery visits for monitoring. Effectiveness was assessed using the four clinical value compass domains of (i) functional status, (ii) clinical outcomes, (iii) satisfaction and (iv) costs. The domains of functional status (assessed by the health-related quality of life measure SF-36 subscales) and clinical outcomes (as assessed by adverse drug events (ADEs), number of GP visits, hospital services and severity of illness) were measured at baseline and endpoint. Satisfaction was measured by success in implementation and by participant satisfaction at endpoint, and costs (as assessed using medication and healthcare service costs, less intervention costs) were measured preintervention and during the trial. In addition, process evaluation was conducted for intervention patients, in which problems and recommendations from the medication reviews were described.

RESULTS

The model was successfully implemented with 92% of intervention GPs suggesting that the model had improved the care of participating patients, a view shared by 94% of pharmacists. In addition, positive trends in clinical outcomes (ADEs and severity of illness) and costs (an ongoing trend towards reduction in healthcare service costs) were evident, although the trial was limited to a 6-month intervention time. No differences between intervention and control groups were identified for the health-related quality of life domain. The cost-effectiveness ratio for the intervention based on cost savings, reduced adverse events and improved health outcomes was small. The most common problems identified in the medication reviews were potential adverse drug reactions, suboptimal monitoring and adherence/lack of concordance issues. In total, 54.4% of recommendations were enacted, and 23.9% were implemented precisely as recommended in the medication review. Follow-up evaluation showed that 70.9% of actions had a positive outcome, 15.7% no effect and 3.7% had a negative outcome.

CONCLUSIONS

Most studies emphasize efficacy and the best achievable clinical outcomes rather than whether an intervention will be effective in practice. The current trial showed that three of the four domains in the clinical value compass showed trends of improvement or were indeed improved in the relatively short follow-up period of the trial, suggesting that a service based on this model could achieve similar benefits in practice. A domiciliary medication review programme similar to this model has now been implemented into national Australian practice, where GPs and pharmacists are reimbursed by the Australian government for the provision of these services.

Hospital based alternatives to acute paediatric admission: a systematic review

AIMS

To synthesise published evidence of the impacts of introducing hospital based alternatives to acute paediatric admission.

METHODS

Systematic review of studies of interventions for children with acute medical problems. Main outcome measures were: admission or discharge, unscheduled returns to hospital, satisfaction of parents and general practitioners, effects on health service activity, and costs.

RESULTS

Twenty five studies were included: one randomised controlled trial, 23 observational or cross-sectional studies, and one qualitative study. Many studies were of uncertain quality or were open to significant potential bias. About 40% of children attending acute assessment units in paediatric departments, and over 60% of those attending acute assessment units in A&E departments, do not require inpatient admission. There is little evidence of serious clinical consequences in children discharged from these units, although up to 7% may subsequently return to hospital. There is some evidence that users are satisfied with these services and that they are associated with reductions in inpatient activity levels and certain hospital costs. Evidence about the impact of urgent outpatient clinics is very limited.

CONCLUSIONS

Current evidence supports a view that acute paediatric assessment services are a safe, efficient, and acceptable alternative to inpatient admission, but this evidence is of limited quantity and quality. Further research is required to confirm that this type of service reorganisation does not disadvantage children and their families, particularly where inpatient services are withdrawn from a hospital.

Obesity prevention: a proposed framework for translating evidence into action

Obesity as a major public health and economic problem has risen to the top of policy and programme agendas in many countries, with prevention of childhood obesity providing a particularly compelling mandate for action. There is widespread agreement that action is needed urgently, that it should be comprehensive and sustained, and that it should be evidence-based. While policy and programme funding decisions are inevitably subject to a variety of historical, social, and political influences, a framework for defining their evidence base is needed. This paper describes the development of an evidence-based, decision-making framework that is particularly relevant to obesity prevention. Building upon existing work within the fields of public health and health promotion, the Prevention Group of the International Obesity Task Force (IOTF) developed a set of key issues and evidence requirements for obesity prevention. These were presented and discussed at an IOTF workshop in April 2004 and were then further developed into a practical framework. The framework is defined by five key policy and programme issues that form the basis of the framework. These are: (i) building a case for action on obesity; (ii) identifying contributing factors and points of intervention; (iii) defining the opportunities for action; (iv)evaluating potential interventions; and (v) selecting a portfolio of specific policies, programmes, and actions. Each issue has a different set of evidence requirements and analytical outputs to support policy and programme decision-making. Issue 4 was identified as currently the most problematic because of the relative lack of efficacy and effectiveness studies. Compared with clinical decision-making where the evidence base is dominated by randomized controlled trials with high internal validity, the evidence base for obesity prevention needs many different types of evidence and often needs the informed opinions of stakeholders to ensure external validity and contextual relevance.

Methodological challenges posed by economic evaluations of early childhood intervention programmes

Early childhood intervention programmes have emerged in recent years with the aim of fostering the cognitive and social-emotional functioning and physical health of preschool children and enhancing their emerging competencies. This article presents a structured critical appraisal of economic evaluations of early childhood intervention programmes. It highlights a range of methodological issues in the field. These include: the fidelity of the evaluation process; the selection of the appropriate comparison group given the complexity of care routinely provided; the appropriate perspective and coverage of the study; methodological concerns relating to cost and benefit measurement and valuation; analytical requirements relating to the form of sensitivity analysis and the decision rules adopted by decision makers; and the interpretation of the results in the light of contextual factors. It is concluded that more transparent methodological guidance is required for analysts conducting economic evaluations of early childhood intervention programmes in particular and of public health interventions in general. Greater multidisciplinary collaboration between social scientists should also enhance the development of ground-breaking methods in this field.

Expert searching in public health

OBJECTIVE

The article explores the characteristics of public health information needs and the resources available to address those needs that distinguish it as an area of searching requiring particular expertise.

METHODS

Public health searching activities from reference questions and literature search requests at a large, urban health department library were reviewed to identify the challenges in finding relevant public health information.

RESULTS

The terminology of the information request frequently differed from the vocabularies available in the databases. Searches required the use of multiple databases and/or Web resources with diverse interfaces. Issues of the scope and features of the databases relevant to the search questions were considered.

CONCLUSION

Expert searching in public health differs from other types of expert searching in the subject breadth and technical demands of the databases to be searched, the fluidity and lack of standardization of the vocabulary, and the relative scarcity of high-quality investigations at the appropriate level of geographic specificity. Health sciences librarians require a broad exposure to databases, gray literature, and public health terminology to perform as expert searchers in public health.

Methods for exploring implementation variation and local context within a cluster randomised community intervention trial

Insignificant or modest findings in intervention trials may be attributable to poorly designed or theorised interventions, poorly implemented interventions, or inadequate evaluation methods. The pre-existing context may also account for the effects observed. A combination of qualitative and quantitative methods is outlined that will permit the determination of how context level factors might modify intervention effectiveness, within a cluster randomised community intervention trial to promote the health of mothers with new babies. The methods include written and oral narratives, key informant interviews, impact logs, and inter-organisational network analyses. Context level factors, which may affect intervention uptake, success, and sustainability are the density of inter-organisational ties within communities at the start of the intervention, the centrality of the primary care agencies expected to take a lead with the intervention, the extent of context-level adaptation of the intervention, and the amount of local resources contributed by the participating agencies. Investigation of how intervention effects are modified by context is a new methodological frontier in community intervention trial research.

Evidence for public health policy on inequalities: 1: the reality according to policymakers

OBJECTIVE

To explore with UK and international policy advisors how research evidence influences public health policy making, and how its relevance and utility could be improved, with specific reference to the evidence on the production and reduction of health inequalities.

DESIGN, SETTING, AND PARTICIPANTS

Qualitative residential workshop involving senior policy advisors with a substantive role in policy development across a range of sectors (mainly public health, but also including education, social welfare, and health services). In four in depth sessions, facilitated by the authors, focused questions were presented to participants. Their responses were then analysed thematically to identify key themes, relating to the availability and utility of existing evidence on health inequalities.

MAIN RESULTS

The lack of an equity dimension in much aetiological and evaluative research was highlighted by participants. Much public health research was also felt to have weak underlying theoretical underpinnings. As well as evaluations of the effectiveness and cost-effectiveness of policy and other interventions, they identified a need for predictive research, and for methodological research to further develop methods for assessing the impact on health of clusters of interventions.

CONCLUSIONS

This study reinforces the view that there is a lack of information on the effectiveness and cost-effectiveness of policies, and it uncovered additional gaps in the health inequalities evidence base. A companion paper discusses researchers' views of how the production of more relevant public health evidence can be stimulated.

Back pain and health policy research: the what, why, how, who, and when

STUDY DESIGN

A background literature, supported by discussion and outcomes on the subject of Health Policy and Back Pain, from the Fifth International Forum on Low Back Pain Research in Primary Care, in Montreal in May 2002.

SUMMARY OF BACKGROUND DATA

A multitude of randomized controlled trials and systematic reviews have been completed in the field of back pain research. There has been limited health policy research in the field of back pain but a greater amount of health policy research in other medical fields.

METHODS

The focus of the workshop was on the contribution health policy could make in the area of back pain, the methodologies that are appropriate to research in back pain, and the barriers to back pain health policy research. The workshop was supported by the workshop coordinators' literature review.

RESULTS

There was consensus about the lack of improved outcomes from randomized controlled trials and individual treatments and general agreement on the importance supporting current research initiatives with health policy research. That policy-makers were developing policy in this area was agreed, and study methodology to support evidence based policy development was explored.

CONCLUSIONS

Health policy research is a relatively underdeveloped area of research in back pain. Back pain as a public health problem may be supported by a broader research approach and a collaborative association with policy-makers in this area.

Assessing the impact of humanitarian assistance in the health sector

There have been significant improvements in the design and management of humanitarian aid responses in the last decade. In particular, a significant body of knowledge has been accumulated about public health interventions in emergencies, following calls for developing the evidence base of humanitarian health interventions. Several factors have prompted this, such as the increased volume of humanitarian assistance with subsequent higher levels of scrutiny on aid spending, and greater pressure for improving humanitarian aid quality and performance. However, documentation of the ability of humanitarian interventions to alleviate suffering and curb mortality remains limited. This paper argues that epidemiological studies can potentially be a useful tool for measuring the impact of health interventions in humanitarian crises. Survey methods or surveillance systems are mainly used for early warning or needs assessment and their potential for assessing the impact of aid programmes is underutilised.

Promoting walking and cycling as an alternative to using cars: systematic review

OBJECTIVES

To assess what interventions are effective in promoting a population shift from using cars towards walking and cycling and to assess the health effects of such interventions.

DATA SOURCES

Published and unpublished reports in any language identified from electronic databases, bibliographies, websites, and reference lists.

REVIEW METHODS

Systematic search and appraisal to identify experimental or observational studies with a prospective or controlled retrospective design that evaluated any intervention applied to an urban population or area by measuring outcomes in members of the local population.

RESULTS

22 studies met the inclusion criteria. We found some evidence that targeted behaviour change programmes can change the behaviour of motivated subgroups, resulting (in the largest study) in a shift of around 5% of all trips at a population level. Single studies of commuter subsidies and a new railway station also showed positive effects. The balance of best available evidence about publicity campaigns, engineering measures, and other interventions suggests that they have not been effective. Participants in trials of active commuting experienced short term improvements in certain measures of health and fitness, but we found no good evidence on effects on health of any effective intervention at population level.

CONCLUSIONS

The best available evidence of effectiveness in promoting a modal shift is for targeted behaviour change programmes, but the social distribution of their effects is unclear and some other types of intervention have yet to be rigorously evaluated.

A glossary for evidence based public health

This glossary seeks to define and explain some of the main concepts underpinning evidence based public health. It draws on the published literature, experience gained over several years analysis of the topic, and discussions with public health colleagues, including researchers, practitioners, policy makers, and students.

Evidence based practice in population health: a regional survey to inform workforce development and organisational change

STUDY OBJECTIVE

To assess current capacity to implement evidence based practice (EBP) in population health.

DESIGN

Postal survey of a regional population health workforce in Sydney, Australia.

SETTING

Division of Population Health, South Western Sydney Area Health Service.

PARTICIPANTS

104 population health staff (response rate: 73%).

MAIN RESULTS

In the sample of regional population health practitioners, views about the current promotion of EBP were positive. Non-medical respondents with less that Masters degree were more likely to report "high self assessed need" to increase their capacity in EBP (p = 0.022). Confidence in understanding of EBP terminology was not associated with seniority but with highest level of education reached (p<0.000) and having medical qualifications (p<0.000). Occupational category was not associated with respondents' self assessed "need for evidence", "need for EBP skills" or "need to increase their capacity in EBP" in their current position. The proportion of participants "strongly" supporting implementation of a colorectal cancer screening programme whose benefit was expressed as relative risk reduction was greater than that so supporting a programme whose benefit was expressed as number needed to screen (p = 0.008). Most respondents referred to their immediate managers when seeking support for EBP.

CONCLUSIONS

The findings provide a quantitative baseline for capacity building through workplace programmes. Managerial commitment has been increased and performance development is now underway.

On the nature of programs: health promotion programs as action

Assuming that sound methodological indications regarding the evaluation of health promotion programs should be rooted in a critical reflection on the nature of health promotion programs, this paper reviews two traditional ontological perspectives at the basis of most scientific activity. While empirical realism conceptualises programs as natural objects, idealism and relativism strictly confine programs in the realm of representations and models. Both ontological perspectives however are unsatisfying for health promotion programs. It is suggested that critical realism which proposes a three-layer ontology offers a better framework for conceiving health promotion programs. In this ontology, the nature of programs lies in actions undertaken to create the conditions by which social causal mechanisms are triggered. Ultimately locating programs in the realm of practice.

The future of health behavior change research: what is needed to improve translation of research into health promotion practice?

BACKGROUND

It is well documented that the results of most behavioral and health promotion studies have not been translated into practice.

PURPOSE

In this article, reasons for this gap, focusing on study design characteristics as a central contributing barrier, are discussed.

METHODS

Four reviews of recent controlled studies in work sites, health care, school, and community settings are briefly discussed and summarized. Their implications for future research and for closing the gap between research and practice are then discussed.

RESULTS

These reviews come to consistent conclusions regarding key internal and external validity factors that have and have not been reported. It is very clear that moderating variables and generalization issues have not been included or reported in the majority of investigations, and that as a consequence little is known about the representatives or the robustness of results from current studies.

CONCLUSIONS

To significantly improve the current state of affairs, substantial changes will be required on the part of researchers, funding agencies, and review and editorial boards. In conclusion, recommendations for each of these entities are provided.

Putting public health evidence into practice: increasing the prevalence of working smoke alarms in disadvantaged inner city housing

STUDY OBJECTIVES

The UK government recommends that local authorities install battery operated smoke alarms to prevent fire related injury. However, a randomised controlled trial of smoke alarm installation in local authority housing found a low level of working alarms at follow up. Qualitative work, which accompanied the trial explored barriers and levers to the use of this public health intervention.

DESIGN

Semi-structured group and individual interviews were conducted with a sample of the adult participants in a randomised controlled trial of free smoke alarm installation. Group interviews and "draw and write" exercises were conducted with children at a local primary school.

PARTICIPANTS

A sample of trial participants and primary school children in the trial neighbourhood.

SETTING

An inner city housing estate in central London.

MAIN RESULTS

The main barrier to smoke alarm use was the distress caused by false alarms. Although trial participants considered themselves to be at high risk for fires and would recommend smoke alarms to others, respondents' reports on the distress caused by false alarms suggest that people balance immediate and longer term risks to their health and wellbeing when they disable alarms.

CONCLUSIONS

This study identified some of the reasons for the low level of functioning smoke alarms, and problems experienced with alarms. The results have implications for the implementation of this public health intervention. The effectiveness of smoke alarm installation could be improved if alarm manufacturers and those responsible for implementation programmes considered ways of tackling the issues raised in this study.

Evaluating the health effects of social interventions

Is no evidence better than any evidence when controlled studies are unethical?

The future of research that promotes cancer screening

The authors draw on the lessons highlighted in preceding articles in the current supplement to provide recommendations for cancer screening intervention research and to highlight some of the many questions that will require further investigation.

Evidence, hierarchies, and typologies: horses for courses

Debate is ongoing about the nature and use of evidence in public health decision making, and there seems to be an emerging consensus that the "hierarchy of evidence" may be difficult to apply in other settings. It may be unhelpful however to simply abandon the hierarchy without having a framework or guide to replace it. One such framework is discussed. This is based around a matrix, and emphasises the need to match research questions to specific types of research. This emphasis on methodological appropriateness, and on typologies rather than hierarchies of evidence may be helpful in organising and appraising public health evidence.

Coping as a mediator of psychosocial impediments to optimal management and control of asthma

Adherence to asthma medication regimens by asthma patients is often poor and contributes to the continued and substantial burden of asthma in the community. There is evidence of increased rates of behavioural problems, anxiety and depression in people with moderate-to-severe asthma and these factors may interfere with adherence and contribute to poor asthma control. An alternative explanation is that the relationship between feelings of anxiety and depression, and adherence to the treatment regimen may be more accurately predicted from the coping styles used, rather than the experience of asthma itself. The objective of this paper was to review evidence for associations between coping strategies used by asthma patients, asthma management and health outcomes. The Medline and PsychInfo databases were searched for articles containing the terms "asthma" and "coping". Patients with asthma tended to use different strategies for coping with stress and illness compared to healthy participants and individuals with other chronic illnesses. Emotion-focussed coping strategies such as denial were commonly used by patients with poor medication adherence, those who attended emergency departments for asthma, were admitted to hospital for asthma, or suffered near-fatal asthma attacks. Interventions to improve coping strategies have been effective in reducing symptoms and psychological distress. The availability of coping resources to patients and/or their caregivers and the coping strategies that are used are likely to mediate the influence of psychosocial factors on the management of asthma. Further studies exploring the ways in which individuals cope with asthma will improve our understanding of the mechanisms linking psychological and social status to asthma morbidity and mortality.

The prevention of depression using the Internet

Efficacy trials suggest that depression is preventable in children and adults. However, current depression prevention interventions are not deliverable to the community en masse. The Internet offers an opportunity to deliver tailored prevention interventions such as those based on cognitive behavioural therapy (CBT) to a large audience, cost-effectively, while preserving intervention fidelity and anonymity. The Internet offers distinct advantages for data collection, which can be used to help refine intervention programs. There are no published randomised controlled trials of the effectiveness of the Internet in delivering depression prevention programs. The feasibility and potential effectiveness of the Internet is indicated by research demonstrating the successful delivery of CBT by computer, the use of the Internet in the delivery of CBT treatment, and the effective prevention of obesity and the promotion of exercise using Internet technologies. Possible limitations to public health interventions using the Internet include selective access, the inability to promote the sites to potential users and the issue of uptake once users access the sites. Randomised controlled trials of CBT delivered by the Internet are required.