Criteria for evaluating evidence on public health interventions

Explaining Changes in Walking and Bicycling Behavior: Challenges for Transportation Research

As issues of traffic congestion, obesity, and environmental conservation receive increased attention globally and in the US, focus turns to the role that walking and cycling can play in mitigating such problems. This enthusiasm has created a need for evidence on the degree to which policies to increase walking and cycling travel have worked. This paper outlines the important challenges researchers face in their attempts to produce credible evidence on walking and cycling interventions. It closes by discussing matters to consider in such research endeavors, including the importance of clear conceptualization, sound research design, measurement innovations, and strategic sampling.

Reviewing evidence on complex social interventions: appraising implementation in systematic reviews of the health effects of organisational-level workplace interventions

BACKGROUND

The reporting of intervention implementation in studies included in systematic reviews of organisational-level workplace interventions was appraised. Implementation is taken to include such factors as intervention setting, resources, planning, collaborations, delivery and macro-level socioeconomic contexts. Understanding how implementation affects intervention outcomes may help prevent erroneous conclusions and misleading assumptions about generalisability, but implementation must be adequately reported if it is to be taken into account.

METHODS

Data on implementation were obtained from four systematic reviews of complex interventions in workplace settings. Implementation was appraised using a specially developed checklist and by means of an unstructured reading of the text.

RESULTS

103 studies were identified and appraised, evaluating four types of organisational-level workplace intervention (employee participation, changing job tasks, shift changes and compressed working weeks). Many studies referred to implementation, but reporting was generally poor and anecdotal in form. This poor quality of reporting did not vary greatly by type or date of publication. A minority of studies described how implementation may have influenced outcomes. These descriptions were more usefully explored through an unstructured reading of the text, rather than by means of the checklist.

CONCLUSIONS

Evaluations of complex interventions should include more detailed reporting of implementation and consider how to measure quality of implementation. The checklist helped us explore the poor reporting of implementation in a more systematic fashion. In terms of interpreting study findings and their transferability, however, the more qualitative appraisals appeared to offer greater potential for exploring how implementation may influence the findings of specific evaluations. Implementation appraisal techniques for systematic reviews of complex interventions require further development and testing.

A family-based intervention to promote healthy lifestyles in an aboriginal community in Canada

CONTEXT

Obesity is a major public health problem in North America, particularly in Aboriginal people.

OBJECTIVE

To determine if a household-based lifestyle intervention is effective at reducing energy intake and increasing physical activity among Aboriginal families after 6 months.

DESIGN, PARTICIPANTS, AND INTERVENTION

Randomized, open trial of 57 Aboriginal households recruited between May 2004 and April 2005 from the Six Nations Reserve in Ohsweken, Canada. Aboriginal Health Counsellors made regular home visits to assist families in setting dietary and physical activity goals. Additional interventions included provision of filtered water, a physical activity program for children, and educational events about healthy lifestyles.

RESULTS

57 households involving 174 individuals were randomized to intervention or usual care. Intervention households decreased consumption of fats, oils and sweets compared to usual care households (-4.9 servings per day vs. -3 servings/day, p=0.006), and this was associated with a reduction in trans fatty acids (-0.2 vs. +0.6 grams/day, p=0.02). Water consumption increased (+0.3 vs. -0.1 servings/day, p<0.04) and soda pop consumption decreased (-0.3 vs. -0.1 servings/day, p=0.02) in intervention households compared to usual care. A trend toward increased knowledge about healthy dietary practices in children, increased leisure-time activity and decreased sedentary behaviours was observed, although these differences were not statistically significant.

CONCLUSION

A household-based intervention is associated with some positive changes in dietary practices and activity patterns. A larger and longer-term intervention which addresses both individual change and structural barriers in the community is needed.

Organizational determinants of patients' experiences of care for breast, lung and colorectal cancers

Organizational characteristics in English NHS hospitals and the experiences of patients with three common cancers - breast, colorectal and lung - were examined using secondary data analyses. Two specific measures of satisfaction, Respect and Dignity, reflecting inpatient care, and Communication reflecting hospital outpatient care, were drawn from a national survey of cancer patients after first hospital treatment. They were compared at hospital level with hospital cancer service standards, and measures of hospital provision, each drawn from national surveys. Respect and Dignity was greater in hospitals with fewer complaints, slower admission procedures and a greater proportion of medicine consultants, for breast and colorectal cancers only. For breast cancer alone, Respect and Dignity was greater in hospitals achieving more participation in meetings by lead team members at the cancer unit level. For lung cancer alone, there were tumour-specific team organizational measures (relating to outpatient assessment) associated with Communication. However, the majority of recorded standards did not show associations, and there were occasional negative associations (dissatisfaction). The impact of organizational factors on patients may be examined through observational studies when experimental designs are not possible. Understanding how organizational factors affect quality of care for cancer patients can contribute to planning and management of cancer services.

Effects of fully-established Sure Start Local Programmes on 3-year-old children and their families living in England: a quasi-experimental observational study

BACKGROUND

Sure Start Local Programmes (SSLPs) are area-based interventions to improve services for young children and their families in deprived communities, promote health and development, and reduce inequalities. We therefore investigated whether SSLPs affect the wellbeing of 3-year-old children and their families.

METHODS

In a quasi-experimental observational study, we compared 5883 3-year-old children and their families from 93 disadvantaged SSLP areas with 1879 3-year-old children and their families from 72 similarly deprived areas in England who took part in the Millennium Cohort Study. We studied 14 outcomes-children's immunisations, accidents, language development, positive and negative social behaviours, and independence; parenting risk; home-learning environment; father's involvement; maternal smoking, body-mass index, and life satisfaction; family's service use; and mother's rating of area.

FINDINGS

After we controlled for background factors, we noted beneficial effects associated with the programmes for five of 14 outcomes. Children in the SSLP areas showed better social development than those in the non-SSLP areas, with more positive social behaviour (mean difference 0.45, 95% CI 0.09 to 0.80, p=0.01) and greater independence (0.32, 0.18 to 0.47, p<0.0001). Families in SSLP areas showed less negative parenting (-0.90, -1.11 to -0.69, p<0.0001) and provided a better home-learning environment (1.30, 0.75 to 1.86, p<0.0001). These families used more services for supporting child and family development than those not living in SSLP areas (0.98, 0.86 to 1.09, p<0.0001). Effects of SSLPs seemed to apply to all subpopulations and SSLP areas.

INTERPRETATION

Children and their families benefited from living in SSLP areas. The contrast between these and previous findings on the effect of SSLPs might indicate increased exposure to programmes that have become more effective. Early interventions can improve the life chances of young children living in deprived areas.

Beyond evidence--to ethics: a decision-making framework for health promotion, public health and health improvement

Echoing the rise of 'evidence-based medicine', the concept of evidence-based policy and practice in the inter-related fields of health promotion, public health and health improvement has attracted increasing attention over the past two decades. More recently, again with roots traceable to biomedical thinking, there has been growing interest in ethics in relation to these fields. This paper links these two topical themes in a practical way. It explores the extent to which policies and activities 'on the ground' can and should be based on evidence, and considers the relative places of evidence and ethics in decision-making. It goes on to present the 'decision-making triangle', a framework that gives primacy to a set of ethical principles--with available evidence and plausible theory being used to inform the application of these. After introducing the concept of 'ethical logic modelling', the paper concludes by suggesting an 'ethical imperative' for health promotion, public health and health improvement: to make decisions based on the explicit application of ethical principles, using available evidence and theory appropriately.

The psychosocial and health effects of workplace reorganisation. 2. A systematic review of task restructuring interventions

OBJECTIVE

To systematically review the health and psychosocial effects (with reference to the demand-control-support model) of changes to the work environment brought about by task structure work reorganisation, and to determine whether those effects differ for different socioeconomic groups.

DESIGN

Systematic review (QUORUM) of experimental and quasi-experimental studies (any language) reporting health and psychosocial effects of such interventions.

DATA SOURCES

Seventeen electronic databases (medical, social science and economic), bibliographies and expert contacts.

RESULTS

Nineteen studies were reviewed. Some task-restructuring interventions failed to alter the psychosocial work environment significantly, and so could not be expected to have a measurable effect on health. Those that increased demand and decreased control tended to have an adverse effect on health, while those that decreased demand and increased control resulted in improved health, although some effects were minimal. Increases in workplace support did not appear to mediate this relationship.

CONCLUSION

This systematic review suggests that task-restructuring interventions that increase demand or decrease control adversely affect the health of employees, in line with observational research. It lends support to policy initiatives such as the recently enforced EU directive on participation at work, which aims to increase job control and autonomy.

Effectiveness of the Austrian disease-management-programme for type 2 diabetes: study protocol of a cluster-randomized controlled trial

Background

Due to its rising prevalence type 2 diabetes plays an important role concerning population health in Austria and other western countries. In various studies deficiencies in the care of diabetic patients have been revealed. These deficiencies may be overcome by disease-management-programmes (DMPs), but international experience shows that the effectiveness of DMPs is inconsistent. In particular large programmes designed by state-affiliated public health insurances have not been evaluated in randomized controlled trials (RCTs). We are therefore conducting a large scale RCT of the Austrian DMP for type 2 diabetic patients in the province of Salzburg to evaluate the programme regarding its effects on metabolic control, guideline adherent care and the quality of life of diabetic patients.

Methods/Design

The study is open for participation to all GPs and internists in the province of Salzburg. Physicians are randomized before recruitment of patients with the districts of Salzburg as clusters of randomisation. A total of over 1200 patients with type 2 diabetes will then be recruited. In the intervention group the DMP is applied for one year. Controls receive usual care. Endpoints are a decrease in HbA1c in the intervention group > 0,5% compared to controls, a higher percentage of patients with required diagnostic measures according to guidelines, improved cardiovascular risk profile and higher quality of life scores within one year.

Current status of the study

98 Physicians agreed to participate in the study. 96 of them recruited 1494 patients, 654 in the intervention and 840 in the control group.

Trail Registration

This trial has been registered with Current Controlled Trials Ltd. (ISRCTN27414162).

Failure to improve appropriateness of referrals to adult community mental health services--lessons from a multi-site cluster randomized controlled trial

BACKGROUND

Non-clinical factors impact on decisions about whether to refer a patient from primary care to specialist mental health services. The aim of this study was to investigate whether introducing a standardized assessment of severity improves agreement on referrals.

METHODS

Multi-site mixed-method cluster randomized controlled trial, investigating GP referrals from 73 practices (408 839 patients) to 11 community mental health teams (CMHTs). Intervention group GPs were asked to complete a Threshold Assessment Grid (TAG) rating of mental health problem severity. CMHTs rated referral appropriateness.

RESULTS

Two hundred and eighty-one GPs made 1061 mental health referrals. The intervention was only partly implemented with 25% of intervention group GPs completing TAGs. No difference was found in appropriateness (OR 1.18, 95% CI 0.91-1.53) or secondary outcomes. Post-referral primary care contact rates were higher for the intervention group (IRR 1.36, 95% CI 1.07-1.73). Qualitative data identified professional and organizational barriers to implementation.

CONCLUSIONS

Asking GPs to complete a TAG when referring to CMHTs did not improve primary-secondary care agreement on referrals. Low implementation means that uncertainty remains about whether introducing a severity-focussed measure into the referral process is beneficial. Introducing local protocols to manage demand at this interface may not be successful and more attention needs to be paid to human and organizational factors in managing interfaces between services.

Knowledge integration: conceptualizing communications in cancer control systems

OBJECTIVE

This paper was prepared by the National Cancer Institute of Canada (NCIC) Working Group on Translational Research and Knowledge Transfer. The goal was to nurture common ground upon which to build a platform for translating what we know about cancer into what we do in practice and policy.

METHODS

Methods included expert panels, literature review, and concept mapping, to develop a framework that built on earlier cancer control conceptualizations of communications that have guided researchers and end users.

RESULTS

The concept of 'knowledge integration' is used to describe the resulting refinement and the nature of evidence necessary for decision-making to at the systems level. Current evidence for knowledge integration in cancer control is presented across the levels of individual, organizational and systems level interventions and across basic, clinical and population science knowledge bases.

CONCLUSION

A systems-oriented approach to integrating evidence into action assists organizations to conduct research and policy and practice.

PRACTICE IMPLICATIONS

Practitioners can use this framework to understand the challenges of implementing and evaluating cancer control strategies.

Is reporting on interventions a weak link in understanding how and why they work? A preliminary exploration using community heart health exemplars

BACKGROUND

The persistent gap between research and practice compromises the impact of multi-level and multi-strategy community health interventions. Part of the problem is a limited understanding of how and why interventions produce change in population health outcomes. Systematic investigation of these intervention processes across studies requires sufficient reporting about interventions. Guided by a set of best processes related to the design, implementation, and evaluation of community health interventions, this article presents preliminary findings of intervention reporting in the published literature using community heart health exemplars as case examples.

METHODS

The process to assess intervention reporting involved three steps: selection of a sample of community health intervention studies and their publications; development of a data extraction tool; and data extraction from the publications. Publications from three well-resourced community heart health exemplars were included in the study: the North Karelia Project, the Minnesota Heart Health Program, and Heartbeat Wales.

RESULTS

Results are organized according to six themes that reflect best intervention processes: integrating theory, creating synergy, achieving adequate implementation, creating enabling structures and conditions, modifying interventions during implementation, and facilitating sustainability. In the publications for the three heart health programs, reporting on the intervention processes was variable across studies and across processes.

CONCLUSION

Study findings suggest that limited reporting on intervention processes is a weak link in research on multiple intervention programs in community health. While it would be premature to generalize these results to other programs, important next steps will be to develop a standard tool to guide systematic reporting of multiple intervention programs, and to explore reasons for limited reporting on intervention processes. It is our contention that a shift to more inclusive reporting of intervention processes would help lead to a better understanding of successful or unsuccessful features of multi-strategy and multi-level interventions, and thereby improve the potential for effective practice and outcomes.

[Association between preventive care provided in public dental services and caries prevalence]

OBJECTIVE

To assess the association between preventive care provided in public dental services and young people's oral health.

METHODS

Oral health data on 4,033 young people aged 15 to 19 years living in 85 municipalities of the state of Rio Grande do Sul, Southern Brazil, were obtained from the national oral health survey "Saúde Bucal Brasil 2003" for the period 2002-2003. The following variables were studied: age, gender, income, education, time elapsed since last dental visit, reason for dental visit, and water fluoridation. Data on dental care services were obtained from the national database of public health services. Statistical analysis was performed using multilevel logistic regression.

RESULTS

Youngsters from the 21 municipalities with the lowest preventive care (scaling + fluoride + sealants) rates per 100 inhabitants were 2.27 (95% CI: 1.45;3.56) more likely to have non-filled dental cavities than those from the 21 municipalities with the highest care rates. After adjustment for a number of individual and contextual factors this likelihood decreased to 1.76 (95% CI: 1.13;2.72). The variance attributable to variables at municipal level was 14.1% for the empty model and decreased to 10.5% for the fully adjusted model.

CONCLUSIONS

Rio Grande do Sul public dental services may have contributed for the reduction in the number of non-filled cavities in young people. However, it was not possible to detect the impact of this service on total dental caries experience.

What happened to posteroventral pallidotomy for Parkinson's disease and dystonia?

Fifteen years after its resurrection, pallidotomy for Parkinson's disease (PD) and dystonia has once again been supplanted, this time by deep brain stimulation (DBS). Did this occur because pallidotomy was not effective or safe, or because DBS was found to be more effective and safer? This review focuses on the evidence-and its quality-supporting the effectiveness and safety of pallidotomy for PD and dystonia, and the comparative effectiveness and safety of DBS of the subthalamic nucleus (STN) and globus pallidus pars interna (GPi). Discussed first are the determinants of "level 1" recommendations, including the confounding effects on interpretation of randomized clinical trials (RCTs) that fail to control for patient bias (i.e., placebo effects). Although several RCTs have been performed comparing unilateral pallidotomy to medical therapy, GPi DBS, or STN DBS for PD, none controlled for patient bias. Comparison of these trials to estimate the placebo effect, and examination of retrospective case series, suggests that the true effectiveness of unilateral pallidotomy is 20% to 30% reduction of 'off' total motor UPDRS scores, which is similar to the effects of unilateral GPi DBS or STN DBS, but less than bilateral STN DBS. At experienced centers, safety of unilateral pallidotomy appears equivalent to unilateral DBS, but bilateral DBS is likely safer than bilateral pallidotomy. Whereas there have been no RCTs of pallidotomy for dystonia, two double-blind, sham-controlled RCTs of bilateral GPi DBS were performed. Nevertheless, limited uncontrolled series suggest that bilateral pallidotomy is similar to GPi DBS in effectiveness and safety for dystonia. Thus, pallidotomy was not rejected because of lack of effectiveness or safety, and it remains a viable alternative in situations where DBS is not available or not feasible.

Indigenous women and smoking during pregnancy: knowledge, cultural contexts and barriers to cessation

Despite active tobacco control efforts in Australia, smoking prevalence remains disproportionately high in pregnant Indigenous women. This study investigated the place of smoking in pregnancy and attitudes towards smoking within the broader context of Indigenous lives. Focus groups and in-depth interviews were used to collect data from 40 women, and ten Aboriginal Health Workers (AHWs) in Perth, Western Australia. The research process and interpretation was assisted by working with an Indigenous community reference group. Results demonstrated the impact of contextual factors in smoking maintenance, and showed that smoking cessation even in pregnancy was not a priority for most women, given the considerable social and economic pressures that they face in their lives. Overwhelmingly, smoking was believed to reduce stress and to provide opportunities for relaxation. Pregnancy did not necessarily influence attitudes to cessation, though women's understanding of the consequences of smoking during pregnancy was low. Reduction of cigarette intake during pregnancy was seen as an acceptable and positive behaviour change. The AHWs saw their role to be primarily one of support and were conscious of the importance of maintaining positive relationships. As a result, they were often uncomfortable with raising the issue of smoking cessation with pregnant women. The stories of Indigenous women and AHWs provided important insight into smoking during pregnancy and the context in which it occurs.

Fighting anaemia and malnutrition in Hebron (Palestine): impact evaluation of a humanitarian project

Iron supplementation in children is recommended by WHO when prevalence of anaemia is above 40%. In Palestine, due to the emergency situation caused by the outburst of the second intifada, the nutritional situation of children is worsening and iron deficiency anaemia represent one of the most serious problem. An emergency project which included universal iron supplementation (for 12,300 children), free treatment for common diseases and food supplementation to vulnerable families (for 3,275 children found anaemic or/and malnourished) and community health education was implemented in part of Hebron governorate. To evaluate project's impact we performed two random surveys: one before and one after the project. Before the project 30.1% of children (CI 95%=24.5-35.6%) were found anaemic (<11 g/dl), while after the prevalence was 18.8% (CI 95%=14.8-22.8%; decrease of 38%, p=0.001). Levels and reduction in anaemia prevalence were different according to geographical areas: where prevention activities had been implemented in previous years, prevalence of anaemia was 17.1% and remained stable at lower levels (14.7%). In other areas it sharply declined from 47.4 to 21.7% (p<0.0005): the declined was present only for children who received iron supplementation. Comparison of anthropometrics index before versus after the project showed that low "weight for age" declined from 10.9 to 3.8% (p=0.0006), low "weight for height" declined from 6.0 to 1.4% after (p=0.0025). With simple epidemiological tools we could demonstrate and measure the effectiveness of our interventions on the health status of the general population: a 50% reduction of anaemia and a 70% reduction of global acute malnutrition.

Developing the role of pharmacists as members of community mental health teams: perspectives of pharmacists and mental health professionals

BACKGROUND

People living with bipolar mood disorder and psychotic illnesses in Australia primarily access public-sector mental health care through community mental health teams (CMHTs). Adverse drug events are common among clients of CMHTs taking psychotropic medications.

OBJECTIVE

The study aimed to investigate and describe a potential role for pharmacists as members of CMHTs.

METHODS

Five study pharmacists were employed 1 day per week to work with 5 mental health teams over a 24-week period. The pharmacists conducted both client and team specific activities designed to optimize the use of medications. The pharmacists recorded their professional activities in diaries. Audiotaped focus groups were conducted with the pharmacists (n=1) and mental health team staff (n=3) at the conclusion of the study. Pharmacists' diaries and the transcripts of focus groups were thematically content analyzed.

RESULTS

Study pharmacists were perceived as valuable sources of unbiased and evidence-based drug information for both mental health team staff and their clients and caregivers. Mental health team staff particularly appreciated the provision of information about nonpsychotropic medications. Pharmacists' participation in clinical team meetings was used as an opportunity to present medication review findings and recommendations. Liaising between medication prescribers and dispensers working in primary and secondary care settings was deemed an important additional role; however, participating only 1 day per week was considered to delay rapport building and the establishment of collaborative working relationships with mental health team staff.

CONCLUSIONS

Including pharmacists as members of CMHTs addressed an unmet need for pharmaceutical services among clients and staff of CMHTs. Pharmacists' contributions were welcomed by mental health team staff. The study raised the issue of whether pharmacists should be considered as essential and legitimate members of interdisciplinary CMHTs.

A process for systematically reviewing the literature: providing the research evidence for public health nursing interventions

BACKGROUND

Several groups have outlined methodologies for systematic literature reviews of the effectiveness of interventions. The Effective Public Health Practice Project (EPHPP) began in 1998. Its mandate is to provide research evidence to guide and support the Ontario Ministry of Health in outlining minimum requirements for public health services in the province. Also, the project is expected to disseminate the results provincially, nationally, and internationally. Most of the reviews are relevant to public health nursing practice.

AIMS

This article describes four issues related to the systematic literature reviews of the effectiveness of public health nursing interventions: (1) the process of systematically reviewing the literature, (2) the development of a quality assessment instrument, (3) the results of the EPHPP to date, and (4) some results of the dissemination strategies used.

METHODS

The eight steps of the systematic review process including question formulation, searching and retrieving the literature, establishing relevance criteria, assessing studies for relevance, assessing relevant studies for methodological quality, data extraction and synthesis, writing the report, and dissemination are outlined. Also, the development and assessment of content and construct validity and intrarater reliability of the quality assessment questionnaire used in the process are described.

RESULTS

More than 20 systematic reviews have been completed. Content validity was ascertained by the use of a number of experts to review the questionnaire during its development. Construct validity was demonstrated through comparisons with another highly rated instrument. Intrarater reliability was established using Cohen's Kappa. Dissemination strategies used appear to be effective in that professionals report being aware of the reviews and using them in program planning/policymaking decisions.

CONCLUSIONS

The EPHPP has demonstrated the ability to adapt the most current methods of systematic literature reviews of effectiveness to questions related to public health nursing. Other positive outcomes from the process include the development of a critical mass of public health researchers and practitioners who can actively participate in the process, and the work on dissemination has been successful in attracting external funds. A program of research in this area is being developed.

Evaluating the unintended health consequences of poverty alleviation strategies: or what is the relevance of Mohammed Yunus to public health?

Public health researchers are increasingly shifting their attention away from merely documenting those factors that determine health--a solid evidence base on health determinants now exists--to improving our understanding of how various interventions influence population health. This paper argues for greater investigations of the potential unintended health benefits associated with participation in a poverty alleviation strategy (PAS) in low-income countries. We focus on microcredit, a PAS that has been spreading across the developing world. Microcredit aims to address the "credit gap" between the poor and the better off by offering an alternative for the poor to acquire loans: small groups are formed and loans are allocated to members based on group solidarity instead of formal collateral. We argue that microcredit corresponds with activities that will help build up health capital (e.g., greater access to resources) and describe the main pathways from microcredit participation to health. We advocate that microcredit and other potential pro-health PAS be included among the range of interventions considered by public health researchers in improving the health of the poor.

The delivery and organization of public health in England: setting the research agenda

The publication of Best Research for Best Health highlighted the need for the NHS Service and Delivery Organisation (SDO) Programme to develop a programme of research on public health service delivery and organization. This paper reviews the need for this research by reviewing recent policy documents and public health research activity. The paper outlines the role of the SDO Programme in relation to other current developments and discusses how these respond to the need to develop research on public health and what the research priorities are.

Critical appraisal for public health: A new checklist

OBJECTIVES

There have been a number of attempts to develop critical appraisal tools, but few have had a public health focus. This paper describes a new checklist with public health aspects.

STUDY DESIGN

Review of previous appraisal instruments and pilot test of new checklist.

METHODS

Criteria of particular reference to public health practice were added to well-established appraisal criteria. The checklist was piloted with 21 public health professionals, research staff or postgraduate students.

RESULTS

The checklist is organized using the 'ask', 'collect', 'understand' and 'use' categories of the Population Health Evidence Cycle. Readers are asked to assess validity, completeness and transferability of the data as they relate to: the study question; key aspects of the methodology; possible public health implications of the key results; and the implications for implementation in their own public health practice. Of the 21 public health professionals that piloted the checklist, 20 said that they found the checklist useful and 18 would use it or recommend it in the future. Participants were prepared to commit to the majority of the questions, and there was good agreement with a consensus of 'correct' answers.

CONCLUSIONS

The public health critical appraisal checklist adds public health aspects that were missing from previous critical appraisal tools.

Macro practice teaching and curriculum development from an evidence-based perspective

This article contributes to the discourse around evidence-based practice (EBP) as an organizing principle and guiding framework for macro-practice education as it has developed in the George Warren Brown School of Social Work at Washington University in St. Louis. In examining the first five years of implementing evidence-based education at the macro level, some lessons learned are provided. This learning has opened the door for continuing the dialogue on surmounting the challenges around training macro social workers in evidence-based practices. The overarching challenge in integrating EBP into the curriculum lies in the complexity of the multi-dimensional conceptualization of evidence.

Comprehensive quality assessment of healthy school interventions

OBJECTIVES

The number of healthy school interventions of unknown quality overwhelms schools. Quality is a construct that is differently interpreted by teachers and health promoters. The schoolBeat checklist for quality assessment of healthy school interventions incorporates the quality perceptions of both professional groups. To support quality improvements - and thus effectiveness - in school health promotion, this study evaluates the schoolBeat checklist.

METHODS

Twenty-nine healthy school interventions were assessed in the Netherlands, each by two health promoters and two teachers-individually and at a consensus meeting. Generalizability coefficients were calculated for the nine specific quality criteria.

RESULTS

The mean consensus score differs from the mean average individual score for two out of nine criteria. To obtain a threshold Generalizability coefficient of 0.70, the number of assessors required per criterion ranges from 1.6 to 10.8, with an average of 4.7.

CONCLUSION

Quality assessment procedures of healthy school interventions using the schoolBeat checklist require about four experienced assessors from each professional domain to facilitate reliable quality scores based on individual assessment only. Publicly available quality scores enable the inclusion of high quality interventions in school policies in order to increase the impact of school health.

The psychosocial and health effects of workplace reorganisation. 1. A systematic review of organisational-level interventions that aim to increase employee control

OBJECTIVE

Systematic review of the health and psychosocial effects of increasing employee participation and control through workplace reorganisation, with reference to the "demand-control-support" model of workplace health.

DESIGN

Systematic review (QUORUM) of experimental and quasi-experimental studies (any language) reporting health and psychosocial effects of such interventions.

DATA SOURCES

Electronic databases (medical, social science and economic), bibliographies and expert contacts.

RESULTS

We identified 18 studies, 12 with control/comparison groups (no randomised controlled trials). Eight controlled and three uncontrolled studies found some evidence of health benefits (especially beneficial effects on mental health, including reduction in anxiety and depression) when employee control improved or (less consistently) demands decreased or support increased. Some effects may have been short term or influenced by concurrent interventions. Two studies of participatory interventions occurring alongside redundancies reported worsening employee health.

CONCLUSIONS

This systematic review identified evidence suggesting that some organisational-level participation interventions may benefit employee health, as predicted by the demand-control-support model, but may not protect employees from generally poor working conditions. More investigation of the relative impacts of different interventions, implementation and the distribution of effects across the socioeconomic spectrum is required.

Obesity prevention programs demand high-quality evaluations

Obesity prevention programs are at last underway or being planned in Australia and New Zealand. However, it is imperative that they are well-evaluated so that they can contribute to continuous program improvement and add much-needed evidence to the international literature on what works and does not work to prevent obesity. Three critical components of program evaluation are especially at risk when the funding comes from service delivery rather than research sources. These are: the need for comparison groups; the need for measured height and weight; and the need for sufficient process and context information. There is an important opportunity to build collaborative mechanisms across community-based obesity prevention sites to enhance the program and evaluation quality and to accelerate knowledge translation into practice and policy.

Current methodological approaches to the study of sleep disturbances and quality of life in adults with cancer: a systematic review

In recent years, sleep disturbances and the health-related quality of life (QOL) experienced by adults with cancer, during and after cancer treatment, have received increasing attention in the scientific literature. The purpose of this paper was to systematically review current methodological approaches to the study of sleep disturbances and QOL in adults with cancer. Databases were searched to identify longitudinal studies of adults with cancer that measured sleep disturbances and QOL in the past 10 years. The review was focused in five primary areas: trends in publication, measurement of sleep and QOL, study design, changes in sleep disturbances and QOL, and the level of this evidence. Of the 40 studies that met the authors' criteria for inclusion, 75% were descriptive in design and 25% were intervention studies. Studies on sleep and QOL among cancer patients have become more common since 2000, include a range of sample sizes and settings, use a variety of measures of sleep and QOL, and examine patients undergoing many types of cancer therapies. No programs of research have been developed in sleep disturbances and QOL in adults with cancer. The 'evidence' that can be drawn from such studies is obviously weak. Current approaches usually describe changes over time, but have not described whether a relationship exists between sleep disturbances and QOL in adults with cancer. Directions for future research are suggested.

Limitations of the randomized controlled trial in evaluating population-based health interventions

Population- and systems-based interventions need evaluation, but the randomized controlled trial (RCT) research design has significant limitations when applied to their complexity. After some years of being largely dismissed in the ranking of evidence in medicine, alternatives to the RCT have been debated recently in public health and related population and social service fields to identify the trade-offs in their use when randomization is impractical or unethical. This review summarizes recent debates and considers the pragmatic and economic issues associated with evaluating whole-population interventions while maintaining scientific validity and credibility.

Putting evidence into practice: evidence-based interventions for fatigue during and following cancer and its treatment

Cancer-related fatigue has a significant impact on patients' physical and psychosocial functioning, symptom distress, and quality of life, yet it remains under-recognized and undertreated. The Oncology Nursing Society's Putting Evidence Into Practice initiative sought to improve patient outcomes relative to this important problem by critically examining and summarizing the evidence base for interventions to prevent and manage fatigue during and following treatment. This article critically reviews and summarizes the available empirical evidence regarding interventions for cancer-related fatigue. In addition to offering patients and clinicians a tool to facilitate effective management of the distressing symptom, this evidence-based review identifies gaps in knowledge and research opportunities.

Obesity prevention: necessary and possible. A structured approach for effective planning

Obesity is a serious public health problem that has important social, economic and health consequences. The prevalence of obesity is rising rapidly throughout the world in both rich and poor countries, and it affects all sections of society. There are several important reasons for addressing the prevention of obesity, rather than its treatment or management. The prevention of weight gain (or the reversal of small gains) and the maintenance of a healthy weight are likely to be easier, less expensive and potentially more effective than the treatment of obesity after it has fully developed. A structured planning framework for the identification of potential interventions for the promotion of healthy weight and the prevention of weight gain is clearly required. However, detailed reviews of the scientific literature have revealed that the body of research is too small to provide firm guidance on consistently-effective interventions for adults or children. Ultimately, a broader approach to evidence of effectiveness needs to be adopted. The present paper proposes a structured planning approach that utilises the portfolio model and allows the selection of interventions to be based on the best available evidence, while not excluding untried but promising strategies.

Improving the use of research evidence in guideline development: 7. Deciding what evidence to include

BACKGROUND

The World Health Organization (WHO), like many other organisations around the world, has recognised the need to use more rigorous processes to ensure that health care recommendations are informed by the best available research evidence. This is the seventh of a series of 16 reviews that have been prepared as background for advice from the WHO Advisory Committee on Health Research to WHO on how to achieve this.

OBJECTIVES

We reviewed the literature on what constitutes "evidence" in guidelines and recommendations.

METHODS

We searched PubMed and three databases of methodological studies for existing systematic reviews and relevant methodological research. We did not conduct systematic reviews ourselves. Our conclusions are based on the available evidence, consideration of what WHO and other organisations are doing and logical arguments. KEY QUESTION AND ANSWERS: We found several systematic reviews that compared the findings of observational studies with randomised trials, a systematic review of methods for evaluating bias in non-randomised trials and several descriptive studies of methods used in systematic reviews of population interventions and harmful effects. What types of evidence should be used to address different types of questions? The most important type of evidence for informing global recommendations is evidence of the effects of the options (interventions or actions) that are considered in a recommendation. This evidence is essential, but not sufficient for making recommendations about what to do. Other types of required evidence are largely context specific. The study designs to be included in a review should be dictated by the interventions and outcomes being considered. A decision about how broad a range of study designs to consider should be made in relationship to the characteristics of the interventions being considered, what evidence is available, and the time and resources available. There is uncertainty regarding what study designs to include for some specific types of questions, particularly for questions regarding population interventions, harmful effects and interventions where there is only limited human evidence. Decisions about the range of study designs to include should be made explicitly. Great caution should be taken to avoid confusing a lack of evidence with evidence of no effect, and to acknowledge uncertainty. Expert opinion is not a type of study design and should not be used as evidence. The evidence (experience or observations) that is the basis of expert opinions should be identified and appraised in a systematic and transparent way.

Evaluating the effectiveness of public health interventions: the role and activities of the Cochrane Collaboration

Public health decision makers, funders, practitioners, and the public are increasingly interested in the evidence that underpins public health decision making. Decisions in public health cover a vast range of activities. With the ever increasing global volume of primary research, knowledge and changes in thinking and approaches, quality systematic reviews of all the available research that is relevant to a particular practice or policy decision are an efficient way to synthesise and utilise research efforts. The Cochrane Collaboration includes an organised entity that aims to increase the quality and quantity of public health systematic reviews, through a range of activities. This paper aims to provide a glossary of the terms and activities related to public health and the Cochrane Collaboration.

Pragmatic randomised controlled trials in parenting research: the issue of intention to treat

STUDY OBJECTIVE

To evaluate trials of parenting programmes, regarding their use of intention to treat (ITT).

DESIGN

Individual trials included in two relevant Cochrane systematic reviews were scrutinised by two independent reviewers. Data on country of origin, target audience, trial type, treatment violations, use of ITT, and the management of missing data were extracted.

MAIN RESULTS

Thirty trial reports were reviewed. Three reported the use of an ITT approach to data analysis. Nineteen reported losing subjects to follow up although the implications of this were rarely considered. Insufficient detail in reports meant it was difficult to identify study drop outs, the nature of treatment violations, and those failing to provide outcome assessments. In two trials, study drop outs were considered as additional control groups, violating the basic principle of ITT.

CONCLUSIONS

It is recommended that future trial reports adhere to CONSORT guidelines. In particular ITT should be used for the main analyses, with strategies for managing treatment violations and handling missing data being reported a priori. Those conducting trials need to acknowledge the social nature of these programmes can sometimes result in erratic parent attendance and participation, which would only increase the chances of missing data. The use of approaches that can limit the proportion of missing data is therefore recommended.

Dissemination of a community-based physical activity project: The case of 10,000 steps

This paper describes the use of a web-site for the dissemination of the community-based '10,000 steps' program which was originally developed and evaluated in Rockhampton, Queensland in 2001-2003. The website provides information and interactive activities for individuals, and promotes resources and programs for health promotion professionals. The dissemination activity was assessed in terms of program adoption and implementation. In a 2-year period (May 2004-March 2006) more than 18,000 people registered as users of the web-site (logging more than 8.5 billion steps) and almost 100 workplaces and 13 communities implemented aspects of the 10,000 steps program. These data support the use of the internet as an effective means of disseminating ideas and resources beyond the geographical borders of the original project. Following this preliminary dissemination, there remains a need for the systematic study of different dissemination strategies, so that evidence-based physical activity programs can be translated into more widespread public health practice.

Natural experiments: an underused tool for public health?

Policymakers and public health researchers alike have demanded better evidence of the effects of interventions on health inequalities. These calls have been repeated most recently in the UK in the final Wanless report, which spoke of the "almost complete lack of an evidence base on the cost-effectiveness of public health interventions", and pointed more generally to the limited evidence base for public health policy and practice. Wanless and others have suggested that the gaps may be partially filled by exploiting the opportunities offered by "natural experiments", such as changes in employment opportunities, housing provision, or cigarette pricing. Natural experiments have an important contributions to make within the health inequalities agenda. First, they can play an important role in investigating the determinants of health inequalities. Second, they can assist in the identification of effective interventions, an area where it is widely acknowledged that the evidence-base is currently sparsely populated. This paper discusses some of the benefits and limitations of using this type of evidence, drawing on two ongoing quasi-experimental studies as examples.

Desempenho do PSF no Sul e no Nordeste do Brasil: avaliação institucional e epidemiológica da Atenção Básica à Saúde

A pesquisa, desenvolvida dentro dos Estudos de Linha de Base do Proesf analisou o desempenho do Programa Saúde da Família (PSF) em 41 municípios dos Estados de Alagoas, Paraíba, Pernambuco, Piauí, Rio Grande do Norte, Rio Grande do Sul e Santa Catarina. Utilizou delineamento transversal, com grupo de comparação externo (atenção básica tradicional). Entrevistou 41 presidentes de Conselhos Municipais de Saúde, 29 secretários municipais de Saúde e 32 coordenadores de Atenção Básica. Foram caracterizados a estrutura e o processo de trabalho em 234 Unidades Básicas de Saúde (UBS), incluindo 4.749 trabalhadores de saúde; 4.079 crianças; 3.945 mulheres; 4.060 adultos e 4.006 idosos. O controle de qualidade alcançou 6% dos domicílios amostrados. A cobertura do PSF de 1999 a 2004 cresceu mais no Nordeste do que no Sul. Menos da metade dos trabalhadores ingressaram por concurso público e o trabalho precário foi maior no PSF do que em UBS tradicionais. Os achados sugerem um desempenho da Atenção Básica à Saúde (ABS) ainda distante das prescrições do SUS. Menos da metade da demanda potencial utilizou a UBS de sua área de abrangência. A oferta de ações de saúde, a sua utilização e o contato por ações programáticas foram mais adequados no PSF.

Evaluating health effects of transport interventions methodologic case study

BACKGROUND

There is little evidence about the effects of environmental interventions on population levels of physical activity. Major transport projects may promote or discourage physical activity in the form of walking and cycling, but researching the health effects of such "natural experiments" in transport policy or infrastructure is challenging.

METHODS

Case study of attempts in 2004-2005 to evaluate the effects of two major transport projects in Scotland: an urban congestion charging scheme in Edinburgh, and a new urban motorway (freeway) in Glasgow.

RESULTS

These interventions are typical of many major transport projects. They are unique to their context. They cannot easily be separated from the other components of the wider policies within which they occur. When, where, and how they are implemented are political decisions over which researchers have no control. Baseline data collection required for longitudinal studies may need to be planned before the intervention is certain to take place. There is no simple way of defining a population or area exposed to the intervention or of defining control groups. Changes in quantitative measures of health-related behavior may be difficult to detect.

CONCLUSIONS

Major transport projects have clear potential to influence population health, but it is difficult to define the interventions, categorize exposure, or measure outcomes in ways that are likely to be seen as credible in the field of public health intervention research. A final study design is proposed in which multiple methods and spatial levels of analysis are combined in a longitudinal quasi-experimental study.

The limitations of 'evidence-based' public health

This paper examines how the concept of the 'evidence-based' approach has transferred from clinical medicine to public health and has been applied to health promotion and policy making. In policy making evidence has always been interpreted broadly to cover all types of reasoned enquiry and after some debate the same is now true for health promotion. Taking communities rather than individuals as the unit of intervention and the importance of context means that frequently randomized controlled trials are not appropriate for study of public health interventions. Further, the notion of a 'best solution' ignores the complexity of the decision making process. Evidence 'enlightens' policy makers shaping how policy problems are framed rather than providing the answer to any particular problem. There are lessons from the way that evidence-based policy is being applied in public health that could usefully be taken back into medicine.

Do urban regeneration programmes improve public health and reduce health inequalities? A synthesis of the evidence from UK policy and practice (1980-2004)

OBJECTIVES

To synthesise data on the impact on health and key socioeconomic determinants of health and health inequalities reported in evaluations of national UK regeneration programmes.

DATA SOURCES

Eight electronic databases were searched from 1980 to 2004 (IBSS, COPAC, HMIC, IDOX, INSIDE, Medline, Urbadisc/Accompline, Web of Knowledge). Bibliographies of located documents and relevant web sites were searched. Experts and government departmental libraries were also contacted.

REVIEW METHODS

Evaluations that reported achievements drawing on data from at least two target areas of a national urban regeneration programme in the UK were included. Process evaluations and evaluations reporting only business outcomes were excluded. All methods of evaluation were included. Impact data on direct health outcomes and direct measures of socioeconomic determinants of health were narratively synthesised.

RESULTS

19 evaluations reported impacts on health or socioeconomic determinants of health; data from 10 evaluations were synthesised. Three evaluations reported health impacts; in one evaluation three of four measures of self reported health deteriorated, typically by around 4%. Two other evaluations reported overall reductions in mortality rates. Most socioeconomic outcomes assessed showed an overall improvement after regeneration investment; however, the effect size was often similar to national trends. In addition, some evaluations reported adverse impacts.

CONCLUSION

There is little evidence of the impact of national urban regeneration investment on socioeconomic or health outcomes. Where impacts have been assessed, these are often small and positive but adverse impacts have also occurred. Impact data from future evaluations are required to inform healthy public policy; in the meantime work to exploit and synthesise "best available" data is required.

Heatwaves and public health in Europe

Public health measures need to be implemented to prevent heat-related illness and mortality in the community and in institutions that care for elderly or vulnerable people. Heat health warning systems (HHWS) link public health actions to meteorological forecasts of dangerous weather. Such systems are being implemented in Europe in the absence of strong evidence of the effectiveness of specific measures in reducing heatwave mortality or morbidity. Passive dissemination of heat avoidance advice is likely to be ineffective given the current knowledge of high-risk groups. HHWS should be linked to the active identification and care of high-risk individuals. The systems require clear lines of responsibility for the multiple agencies involved (including the weather service, and the local health and social care agencies). Other health interventions are necessary in relation to improved housing, and the care of the elderly at home and vulnerable people in institutions. European countries need to learn from each other how to prepare for and effectively cope with heatwaves in the future. Including evaluation criteria in the design of heatwave early warning systems will help ensure effective and efficient system operation.