Policy evaluation research. Measuring the independent variables

Comparison of Laws that Impact Volunteer Response in Three Disaster-Prone Southeastern States

OBJECTIVE

As natural disasters become more frequent and severe, the ability of impacted states to rapidly scale up their capacity to respond and facilitate recovery through volunteers is ever more crucial. However, the legal landscape that may facilitate or hamper augmentation through volunteers has yet to be fully explored.

DESIGN

This study utilized policy surveillance methodology to code relevant laws identified in searches using the Emergency Law Inventory database and Westlaw.

SETTING

We assessed laws in three natural disaster-prone states: North Carolina, South Carolina, and Georgia.

MAIN OUTCOME MEASURES

We assessed the presence of liability protection, license reciprocity, and scope of practice laws for health care volunteers.

RESULTS

Of the three assessed legal domains, liability protection for health care volunteers across all states contained the most robust language. Liability protections apply unless the health care volunteers act with reckless disregard, willful misconduct, or gross negligence. However, nuance exists based on which organizations (state vs charitable) volunteers are affiliated with to qualify for coverage. License reciprocity for out-of-state specific health care professions entering the impacted states was available across all three states. However, only Georgia enacted federal model legislation that provides additional flexibility regarding credentialing and liability protections. Lastly, very few laws addressed the scope of practice in impacted states.

CONCLUSIONS

Comparisons of laws that impact the ability of health care volunteers to respond and recover from disasters are scarce. More research needs to be performed to better understand the laws that impact volunteers in emergencies, particularly as the public health system is overtasked when providing services to impacted communities.

Scan of physical activity policy actions in Europe: Lessons learned from populating the MOVING database

Adequate levels of physical activity are important for population health. Policy databases can track, monitor, and compare the development and implementation of physical activity policy actions and are populated by different methods. The new MOVING database, developed through the Confronting Obesity: Co-creating Policy with Youth (CO-CREATE) project, collates governmental policy actions designed to increase physical activity and is populated by an in-depth scan of implemented national policy actions. This paper presents lessons learned from conducting the policy scan across 27 European countries. Policy actions were identified using a structured search protocol from preselected sources, assessed against pre-specified inclusion criteria and verified by an in-country expert. 625 eligible national implemented policy actions were identified. Challenges included policy actions falling out of scope, a lack of available information on policy actions, difficulty in identifying policy actions using specific search terms, and increased resource requirements for translation of policy actions into English. The scan indicated improvements, which informed protocol modifications. Identifying the challenges and opportunities around conducting a policy scan is necessary to understand and assess the reliability, validity, and utility of a policy database. The policy scan will help to deliver a comprehensive picture of physical activity policy actions across Europe.

Improving measures of access to legal abortion: A validation study triangulating multiple data sources to assess a global indicator

BACKGROUND

Global mechanisms have been established to monitor and facilitate state accountability regarding the legal status of abortion. However, there is little evidence describing whether these mechanisms capture accurate data. Moreover, it is uncertain whether the "legal status of abortion" is a valid proxy measure for access to safe abortion, pursuant to the global goals of reducing preventable maternal mortality and advancing reproductive rights. Therefore, this study sought to assess the accuracy of reported monitoring data, and to determine whether evidence supports the consistent application of domestic law by health care professionals such that legality of abortion functions as a valid indicator of access.

METHODS AND FINDINGS

We conducted a validation study using three countries as illustrative case examples: Argentina, Ghana, and India. We compared data reported by two global monitoring mechanisms (Countdown to 2030 and the Global Abortion Policies Database) against domestic source documents collected through in-depth policy review. We then surveyed health care professionals authorized to perform abortions about their knowledge of abortion law in their countries and their personal attitudes and practices regarding provision of legal abortion. We compared professionals' responses to the domestic legal frameworks described in the source documents to establish whether professionals consistently applied the law as written. This analysis revealed weaknesses in the criterion validity and construct validity of the "legal status of abortion" indicator. We detected discrepancies between data reported by the global monitoring and accountability mechanisms and the domestic policy reviews, even though all referenced the same source documents. Further, provider surveys unearthed important context-specific barriers to legal abortion not captured by the indicator, including conscientious objection and imposition of restrictions at the provider's discretion.

CONCLUSIONS

Taken together, these findings denote weaknesses in the indicator "legal status of abortion" as a proxy for access to safe abortion, as well as inaccuracies in data reported to global monitoring mechanisms. This information provides important groundwork for strengthening indicators for monitoring access to abortion and for renewed advocacy to assure abortion rights worldwide.

Global Policy Surveillance: Creating and Using Comparative National Data on Health Law and Policy

Throughout the world, laws play an important role in shaping population health. Law making is an intervention with measurable effects yet often unfolds without evaluation or monitoring. Policy surveillance-the systematic, scientific collection and analysis of laws of public health significance-can help bridge this gap by capturing important features of law in numeric form in structured longitudinal data sets.Currently deployed primarily in high-income countries, methods for cross-national policy surveillance hold significant promise, particularly given the growing quality and accessibility of global health data. Global policy surveillance can enable comparative research on the implementation and health impact of laws, their spread, and their political determinants. Greater transparency of status and trends in law supports health policy advocacy and promotes public accountability. Collecting, coding, and analyzing laws across countries presents numerous challenges-especially in low-resource settings.With insights from comparative politics and law, we suggest methods to address those challenges. We describe how longitudinal legal data have been used in limited, but important, ways for cross-national analysis and propose incorporating global policy surveillance into core global public health practice.

Anti-poverty policy and health: Attributes and diffusion of state earned income tax credits across U.S. states from 1980 to 2020

PURPOSE

The U.S. federal Earned Income Tax Credit (EITC) is often considered the most effective antipoverty program for families in the U.S., leading to a variety of improved outcomes such as educational attainment, work incentives, economic activity, income, and health benefits for mothers, infants and children. State EITC supplements to the federal credit can significantly enhance the magnitude of this intervention. In this paper we advance EITC and health research by: 1) describing the diffusion of state EITC policies over 40 years, 2) presenting patterns in important EITC policy dimensions across space and time, and 3) disseminating a robust data set to advance future research by policy analysts and scientists.

METHODS

We used current public health law research methods to systematically collect, conduct textual legal analysis, and numerically code all EITC legislative changes from 1980 through 2020 in the 50 states and Washington, D.C.

RESULTS

First, the pattern of diffusion across states and time shows initial introductions during the 1990s in the Midwest, then spreading to the Northeast, with more recent expansions in the West and South. Second, differences by state and time of important policy dimensions are evident, including size of credit and refundability. Third, state EITC benefits vary considerably by household structure.

CONCLUSION

Continued research on health outcomes is warranted to capture the full range of potential beneficial effects of EITCs on family and child wellbeing. Lawyers and policy analysts can collaborate with epidemiologists and economists on other high-quality empirical studies to assess the many dimensions of policy and law that potentially affect the social determinants of health.

Creating legal data for public health monitoring and evaluation: delphi standards for policy surveillance

Surveillance in public health is the means by which people who are responsible for preventing or controlling threats to health get the timely, ongoing, and reliable information they need about the occurrence, antecedents, time course, geographic spread, consequences, and nature of these threats among the populations they serve. “Policy surveillance” is the ongoing, systematic collection, analysis, and dissemination of information about laws and other policies of health importance.

Improving state health policy assessment: an agenda for measurement and analysis

We examine the scope of inquiry into the measurement and assessment of the state public health policy environment. We argue that there are gains to be made by looking systematically at policies both within and across health domains. We draw from the public health and public policy literature to develop the concepts of interdomain and intradomain policy comprehensiveness and illustrate how these concepts can be used to enhance surveillance of the current public health policy environment, improve understanding of the adoption of new policies, and enhance evaluations of the impact of such policies on health outcomes.

Measuring law for evaluation research

Evaluations that combine social science and law have tremendous potential to illuminate the effects of governmental policies and yield insights into how effectively policy makers' efforts achieve their aims. This potential is infrequently achieved, however, because such interdisciplinary research contains often overlooked substantive and methodological challenges. This article offers detailed guidance for conducting successful multidisciplinary evaluations that use legal data. It addresses major issues that commonly arise and offers practical solutions based both on the authors' extensive experience and recommended best practices developed in concert with the Robert Wood Johnson Foundation's Public Health Law Research Program.

User-Centered Policy Evaluations of Section 508 of the Rehabilitation Act

The author examines user-centered evaluations of e-government Web sites for compliance with a policy related to persons with disabilities: the requirements of Section 508 of the Rehabilitation Act. Although Section 508 requires that federal e-government sites offer equal access to all users, research indicates that inaccessibility is still prevalent. User-centered evaluation approaches offer a vital way to discover areas of inaccessibility on Web sites related to the requirements of Section 508. Following an overview of accessibility, Section 508, and e-government sites, the goals of evaluation and various approaches to evaluating e-government sites are analyzed. The author then focuses on methods and issues in user-centered evaluations of e-government that measure accessibility, and considerations for universal design and future studies. Sample instruments and example data from a 2006 study of e-government accessibility are included to illustrate methods and issues. Evaluating e-government Web sites can ultimately prove quite informative in ascertaining compliance with laws intended to promote accessibility.

Alcohol and malt liquor availability and promotion and homicide in inner cities

We investigated the role of the alcohol environment in explaining disparities in homicide rates among minorities in 10 cities in the United States using 2003 data from the Malt Liquor and Homicide study. We hypothesized that (a) higher concentrations of African Americans would be associated with higher homicide rates, as well as higher alcohol and malt liquor availability and promotion, and (b) the relationship between neighborhood racial/ethnic concentration and homicide would be attenuated by the greater alcohol and malt liquor availability and promotion in African American neighborhoods. Hypotheses were tested using separate Poisson, linear, and logistic regression models that corrected for spatial autocorrelation. Census block groups served as the unit of analysis (n = 450). We found that higher concentrations of African Americans were associated with higher homicide rates as well as greater alcohol availability, especially malt liquor availability. The promotion of malt liquor on storefronts was also significantly greater in African American than in other neighborhoods. However, none of the measures representing alcohol or malt liquor availability and promotion variables changed the effect of neighborhood racial/ethnic concentration on homicide. Limitations and implications of our findings are discussed.

Effects of legal BAC limits on fatal crash involvement: analyses of 28 states from 1976 through 2002

PROBLEM

Hundreds of laws have been implemented in the United States over the past few decades designed to reduce alcohol-impaired driving and the crashes that often result. One approach has been to lower the legally allowable alcohol concentration for drivers. We examined the effects of changes in legal BAC limit in 28 U.S. states from January, 1976 to December, 2002.

METHOD

An interrupted time-series quasi-experimental design was used, incorporating non-alcohol-related crashes as comparisons. Four outcome measures of alcohol-related crash involvement were examined: single-vehicle nighttime, BAC=0.01-0.07, BAC=0.08-0.14, and BAC>/=0.15. Missing BAC test result data were handled by using multiple imputations. Analyses involved estimation of state-specific ARIMA models, controlling for other factors affecting overall crash rates and other major DUI policy changes. Inverse variance weighting methods were used to pool results across states for the most precise underlying estimate of effect of legal BAC limits.

RESULTS

Considerable state by state variability in estimated effects was observed, but results from the pooled analyses were clear and consistent. Changes in legal BAC limits significantly affected alcohol-related fatal crash involvement for both the SVN and BAC test result measures, and the laws affected drivers at all drinking levels.

SUMMARY

An estimated 360 deaths are prevented each year in the United States as a result of the move from a 0.10 to 0.08 legal limit in recent years, and an additional 538 lives could be saved each year if the United States reduced the limit to 0.05, consistent with limits in most countries worldwide.

IMPACT ON INDUSTRY

Given the significant effects of lower legal BAC limits on fatal crash involvement, businesses should support implementation of laws that further reduce the legal BAC limit for all drivers. Furthermore, all companies should set higher standards for employees, such as a zero allowable BAC limit for driving during work time.

General deterrence effects of U.S. statutory DUI fine and jail penalties: long-term follow-up in 32 states

INTRODUCTION

We examined effects of state statutory changes in DUI fine or jail penalties for firsttime offenders from 1976 to 2002.

METHODS

A quasi-experimental time-series design was used (n=324 monthly observations). Four outcome measures of drivers involved in alcohol-related fatal crashes are: single-vehicle nighttime, low BAC (0.01-0.07g/dl), medium BAC (0.08-0.14g/dl), high BAC (>/=0.15g/dl). All analyses of BAC outcomes included multiple imputation procedures for cases with missing data. Comparison series of non-alcohol-related crashes were included to efficiently control for effects of other factors. Statistical models include state-specific Box-Jenkins ARIMA models, and pooled general linear mixed models.

RESULTS

Twenty-six states implemented mandatory minimum fine policies and 18 states implemented mandatory minimum jail penalties. Estimated effects varied widely from state to state. Using variance weighted meta-analysis methods to aggregate results across states, mandatory fine policies are associated with an average reduction in fatal crash involvement by drivers with BAC>/=0.08g/dl of 8% (averaging 13 per state per year). Mandatory minimum jail policies are associated with a decline in single-vehicle nighttime fatal crash involvement of 6% (averaging 5 per state per year), and a decline in low-BAC cases of 9% (averaging 3 per state per year). No significant effects were observed for the other outcome measures.

CONCLUSIONS

The overall pattern of results suggests a possible effect of mandatory fine policies in some states, but little effect of mandatory jail policies.

Effects of Drivers' License Suspension Policies on Alcohol-Related Crash Involvement: Long-Term Follow-Up in Forty-Six States

BACKGROUND

We evaluated the effects of driving under the influence (DUI) mandatory preconviction and postconviction drivers' license suspension laws in each of 46 U.S. states using 1 to 2 decades of long-term follow-up data on fatal car crashes. State-specific results were combined using meta-analytic techniques, and provide a direct test of the concept of celerity--time between offending behavior and consequent punishment--from deterrence theory.

METHODS

Four key outcome measures of monthly fatal alcohol-related crash involvement were examined using data from January 1976 through December 2002: single-vehicle nighttime, breath or blood alcohol concentration (BAC)=0.01 to 0.07, BAC=0.08 to 0.14, and BAC>or=0.15 g/dL. Missing BAC test data for some individual cases were filled using multiple imputation methods, and consequent increases in standard errors were incorporated into subsequent analyses. Separate ARIMA models were estimated for each state, including controls for state-specific levels of crash involvement over time due to other factors and effects of other major DUI countermeasures. Estimates were pooled across states using inverse variance weighting methods.

RESULTS

Administrative or preconviction drivers license suspension policies have statistically significant and substantively important effects in reducing alcohol-related fatal crash involvement by 5%, representing at least 800 lives saved per year in the United States. Moreover, these laws have similar effects on drivers at all drinking levels--from lower-risk drivers below the legal alcohol limit to drivers at extreme levels of intoxication. In clear contrast, postconviction license suspension policies have no discernable effects.

CONCLUSIONS

The effectiveness of a deterrence policy appears to be more strongly affected by celerity-the speed by which punishment is applied after the offending behavior--than by the high severity of the penalty. This finding could be fruitfully applied to other areas of alcohol control policy and laws and regulations in general.

Effects of safety belt laws on safety belt use by American High School Seniors, 1986-2000

PROBLEM

This manuscript evaluates the effects of enactment of state laws that required safety belt use in various U.S. states between 1986 and 2000.

METHOD

Safety belt use was assessed using nationally representative cross-sectional samples of high school seniors; evaluation of the effects of laws used data from over 2,000 high school seniors before and about 3,300 after the laws took effect in 20 states.

RESULTS

Belt use was found to increase significantly between 1986 and 2000, and the laws contributed significantly to that increase. Increases were similar for students differing by gender, race/ethnicity, parent education, grades, truancy, evenings out per week, miles driven per week, and an index of illicit drug use.

DISCUSSION

The data show that although the laws have increased belt use, use is not universal and continued efforts are needed.

IMPACT ON RESEARCH, PRACTICE, AND POLICY

This study shows that many teenagers fail to use belts when there is a secondary use law; an implication is that primary laws would be more efficacious in increasing use among this vulnerable population.

Alcohol dependence and the price of alcoholic beverages

This study estimates the impact of the price of alcoholic beverages on latent dimensions of current alcohol dependence and abuse. A three-part econometric model is used to estimate the impact of price on three latent dimensions (factors). For heavier drinking, the estimated price elasticity is -1.325 (P = 0.027); for physical and other consequences of drinking, -1.895 (P = 0.003); for increased salience of drinking, -0.411 (P = 0.339). For a single latent factor characterized simply as dependence/abuse, estimated price elasticity is -1.487 (P = 0.012). These results suggest that higher prices for alcohol reduce important dimensions of current alcohol dependence and abuse.

Disseminating science-based prevention: lessons learned from CSAP's CAPTs

A wide variety of prevention approaches that reduce substance use and associated problems have been developed and tested. But successes have been limited in promoting the use of these scientific advances by the policy makers, practitioners, and concerned citizens. The Center for Substance Abuse Prevention's six regional Centers for the Application of Prevention Technologies (CSAP's CAPTs) are a major mechanism by which CSAP brings research to practice. This article synthesizes the issues that the CAPTs have faced, the solutions they have developed, and conclusions concerning the work that still needs to be done to increase the application of science-based approaches to prevention. These discussions highlight the particular importance of addressing issues related to the larger systems in which prevention programs and strategies operate.