Methods for reviewing economic evaluations of community preventive services: a cart without a horse?

Systematic review and quality assessment of economic evaluation studies of injury prevention

OBJECTIVE

To review and assess the quality of economic evaluation studies on injury prevention measures.

DESIGN

Systematic review.

DATA SOURCES

Electronic databases searched included Medline (Pubmed), EMBASE, Web of Science, PsycINFO, and Safetylit.

INCLUSION CRITERIA

Empirical studies published in English in international peer-reviewed journals in the period 1998-2009. The subject of the study was economic evaluation of prevention of unintentional injury. Cost-effectiveness (CEA), cost-benefit (CBA) and cost utility (CUA) analyses were included.

METHODS

Methodological details, study designs, and analysis and interpretation of results of the included articles were reviewed and extracted into summary tables. Study quality was judged using the criteria recommended by the Panel on cost-effectiveness in health and medicine and the British Medical Journal (BMJ) checklist for economic evaluations.

RESULTS

Forty-eight studies met the inclusion criteria of our review. Interventions assessed most frequently were hip protectors and exercise programs for the elderly. A wide variety of methodological approaches was found, including differences in type of economic evaluation, perspective, time horizon, study design, cost categories, effect outcomes, and adjustments for timing and uncertainty used. The majority of studies performed a cost-effectiveness analysis from a societal perspective with a time horizon of one to five years, in which the effect was expressed in terms of injuries prevented and only direct health care costs were included. Most studies deviated from one or more of the Panel recommendations or BMJ guidelines; e.g. not adopting the societal perspective, not including all relevant costs, no incremental analysis.

CONCLUSIONS

This review has shown that approaches to economic evaluation of injury prevention vary widely and most studies do not fulfill methodological rigour. Improving quality and harmonization of economic evaluation studies in the field of injury prevention is needed. One way of achieving this would be to establish international guidelines on economic evaluation for injury prevention interventions, based on established economic evaluation checklists, to assist researchers in the design and reporting of economic evaluations.

Overview of methods in economic analyses of behavioral interventions to promote oral health

BACKGROUND

Broad adoption of interventions that prove effective in randomized clinical trials or comparative effectiveness research may depend to a great extent on their costs and cost-effectiveness (CE). Many studies of behavioral health interventions for oral health promotion and disease prevention lack robust economic assessments of costs and CE.

OBJECTIVE

To describe methodologies employed to assess intervention costs, potential savings, net costs, CE, and the financial sustainability of behavioral health interventions to promote oral health.

METHODS

We provide an overview of terminology and strategies for conducting economic evaluations of behavioral interventions to improve oral health based on the recommendations of the Panel of Cost-Effectiveness in Health and Medicine. To illustrate these approaches, we summarize methodologies and findings from a limited number of published studies. The strategies include methods for assessing intervention costs, potential savings, net costs, CE, and financial sustainability from various perspectives (e.g., health-care provider, health system, health payer, employer, society). Statistical methods for estimating short-term and long-term economic outcomes and for examining the sensitivity of economic outcomes to cost parameters are described.

DISCUSSION

Through the use of established protocols for evaluating costs and savings, it is possible to assess and compare intervention costs, net costs, CE, and financial sustainability. The addition of economic outcomes to outcomes reflecting effectiveness, appropriateness, acceptability, and organizational sustainability strengthens evaluations of oral health interventions and increases the potential that those found to be successful in research settings will be disseminated more broadly.

Measuring the costs of outreach motivational interviewing for smoking cessation and relapse prevention among low-income pregnant women

Background

Economic theory provides the philosophical foundation for valuing costs in judging medical and public health interventions. When evaluating smoking cessation interventions, accurate data on costs are essential for understanding resource consumption. Smoking cessation interventions, for which prior data on resource costs are typically not available, present special challenges. We develop a micro-costing methodology for estimating the real resource costs of outreach motivational interviewing (MI) for smoking cessation and relapse prevention among low-income pregnant women and report results from a randomized controlled trial (RCT) employing the methodology. Methodological standards in cost analysis are necessary for comparison and uniformity in analysis across interventions. Estimating the costs of outreach programs is critical for understanding the economics of reaching underserved and hard-to-reach populations.

Methods

Randomized controlled trial (1997-2000) collecting primary cost data for intervention. A sample of 302 low-income pregnant women was recruited from multiple obstetrical sites in the Boston metropolitan area. MI delivered by outreach health nurses vs. usual care (UC), with economic costs as the main outcome measures.

Results

The total cost of the MI intervention for 156 participants was $48,672 or $312 per participant. The total cost of $311.8 per participant for the MI intervention compared with a cost of $4.82 per participant for usual care, a difference of $307 ([CI], $289.2 to $322.8). The total fixed costs of the MI were $3,930 and the total variable costs of the MI were $44,710. The total expected program costs for delivering MI to 500 participants would be 147,430, assuming no economies of scale in program delivery. The main cost components of outreach MI were intervention delivery, travel time, scheduling, and training.

Conclusion

Grounded in economic theory, this methodology systematically identifies and measures resource utilization, using a process tracking system and calculates both component-specific and total costs of outreach MI. The methodology could help improve collection of accurate data on costs and estimates of the real resource costs of interventions alongside clinical trials and improve the validity and reliability of estimates of resource costs for interventions targeted at underserved and hard-to-reach populations.

Economic evaluations of smoking cessation and relapse prevention programs for pregnant women: a systematic review

OBJECTIVE

Smoking cessation and relapse prevention during and after pregnancy reduces the risk of adverse maternal and infant health outcomes, but the economic evaluations of such programs have not been systematically reviewed. This study aims to critically assess economic evaluations of smoking cessation and relapse prevention programs for pregnant women.

METHODS

All relevant English-language articles were identified using PubMed (January 1966-2003), the British National Health Service Economic Evaluation Database, and reference lists of key articles. Economic evaluations of smoking cessation and relapse prevention among pregnant women were reviewed. Fifty-one articles were retrieved, and eight articles were included and evaluated. A single reviewer extracted methodological details, study designs, and outcomes into summary tables. All studies were reviewed, and study quality was judged using the criteria recommended by the Panel on Cost-Effectiveness in Health and Medicine and the British Medical Journal (BMJ) checklist for economic evaluations.

RESULTS

The search retrieved 51 studies. No incremental cost-effectiveness studies or cost-utility studies were found. A narrative synthesis was conducted on the eight studies thatmet the inclusion criteria. Roughly one-third employed cost-benefit analyses (CBA). Those conducting CBA have found favorable benefit-cost ratios of up to 3:1; for every dollar invested $3 are saved in downstream health-related costs.

CONCLUSIONS

CBA suggests favorable cost-benefit ratios for smoking cessation among pregnant women, although currently available economic evaluations of smoking cessation and relapse prevention programs for pregnant women provide limited evidence on cost-effectiveness to determine optimal resource allocation strategies. Although none of these studies had been performed in accordance with Panel recommendations or BMJ guidelines, they are, however, embryonic elements of a more systematic framework. Existing analyses suggest that the return on investment will far outweigh the costs for this critical population. There is significant potential to improve the quality of economic evaluations of such programs; therefore, additional analyses are needed. The article concludes with ideas on how to design and conduct an economic evaluation of such programs in accordance with accepted quality standards.

Political will: a bridge between public health knowledge and action

We propose a new model of the public health policy cycle: the Bridges From Knowledge to Action model. Many prevention initiatives require policy change to achieve broad implementation. Political will, society's commitment to support or alter prevention initiatives, is essential for securing the resources for policy change. We focus on the role of political will in developing and implementing public health policy that integrates scientific evidence and community participation.

Developing and using the Guide to Community Preventive Services: lessons learned about evidence-based public health

The Guide to Community Preventive Services (Community Guide) is being developed under the leadership of the independent, nonfederal Task Force on Community Preventive Services. The Task Force makes recommendations for the use of public health programs and policies based on scientific evidence about what practices have worked to improve health. The Community Guide thoroughly searches scientific literature for topic-relevant studies, evaluates their quality according to established criteria, and makes recommendations based on the overall strength of the body of evidence and the size and variability of reported effects. In addition, the Community Guide identifies promising interventions that have not been adequately researched, thus helping to inform the public health research agenda. The continuously updated and expanded body of recommendations and research agenda formulated by this rigorous process have been published on the Internet and in various publications since 1999 and constitute a highly valued and objective evidence-based resource for guiding current and future public health activities. More remains to be learned, however, on how best to disseminate Community Guide findings to key target audiences and encourage their use to inform practice, policy, and additional research.

Evaluating the evidence of effectiveness for preventive interventions: using a registry system to influence policy through science

In order to deal with the serious problems of drug use, delinquency, and mental health problems in this country, policy makers, communities, practitioners, and advocates need to identify prevention programs that have empirical support and at the same time meet their own community needs regarding acceptability, cost, and training, and supervision. An "evidence-based" approach to prevention, one that identifies the scientific knowledge about what prevention programs work, for whom, and under what circumstances, should play a critical part in this process. The premise of this evidence-based approach is that the wide adoption of prevention programs that have been found to be successful in rigorous evaluations, will lead to successful population-based prevention strategies. A web-based registry, or database that describes the available scientific information on individual preventive trials, would be of high potential use to policy makers engaged in prevention planning for their communities. This presentation describes a model registry based on descriptions of 167 preventive trials aimed at children aged 0-6. Empirical findings are presented regarding two critical components of the registry, the measurement of the quality of the trial and the strength of evidence for a beneficial effect produced by each intervention. Implications for the use of such a registry are also discussed.