Methodological Challenges and Proposed Solutions for Evaluating Opioid Policy Effectiveness

High-Dose Opioid Prescribing in Individuals with Acute Pain: Assessing the Effects of US State Opioid Policies

BACKGROUND

How state opioid policy environments with multiple concurrent policies affect opioid prescribing to individuals with acute pain is unknown.

OBJECTIVE

To examine how prescription drug monitoring programs (PDMPs), pain management clinic regulations, initial prescription duration limits, and mandatory continued medical education affected total and high-dose prescribing.

DESIGN

A county-level multiple-policy difference-in-difference event study framework.

SUBJECTS

A total of 2,425,643 individuals in a large national commercial insurance deidentified claims database (aged 12-64 years) with acute pain diagnoses and opioid prescriptions from 2007 to 2019.

MAIN MEASURES

The total number of acute pain opioid treatment episodes and number of episodes containing high-dose (> 90 morphine equivalent daily dosage (MEDD)) prescriptions.

KEY RESULTS

Approximately 7.5% of acute pain episodes were categorized as high-dose episodes. Prescription duration limits were associated with increases in the number of total episodes; no other policy was found to have a significant impact. Beginning five quarters after implementation, counties in states with pain management clinic regulations experienced a sustained 50% relative decline in the number of episodes containing > 90 MEDD prescriptions (95 CIs: (Q5: - 0.506, - 0.144; Q12: - 1.000, - 0.290)). Mandated continuing medical education regarding the treatment of pain was associated with a 50-75% relative increase in number of high-dose episodes following the first year-and-a-half of enactment (95 CIs: (Q7: 0.351, 0.869; Q12: 0.413, 1.107)). Initial prescription duration limits were associated with an initial relative reduction of 25% in high-dose prescribing, with the effect increasing over time (95 CI: (Q12: - 0.967, - 0.335). There was no evidence that PDMPs affected high-dose opioids dispensed to individuals with acute pain. Other high-risk prescribing indicators were explored as well; no consistent policy impacts were found.

CONCLUSIONS

State opioid policies may have differential effects on high-dose opioid dispensing in individuals with acute pain. Policymakers should consider effectiveness of individual policies in the presence of other opioid policies to address the ongoing opioid crisis.

A Longitudinal Multivariable Analysis: State Policies and Opioid Dispensing in Medicare Beneficiaries Undergoing Surgery

BACKGROUND

States have implemented policies to decrease clinically unnecessary opioid prescribing, but few studies have examined how state policies affect opioid dispensing rate trends for surgical patients.

OBJECTIVE

To examine trends in the perioperative opioid dispensing rates for fee-for-service Medicare beneficiaries and the effects of select state policies.

DESIGN AND PARTICIPANTS

A retrospective cohort study using 2006 to 2018 Medicare claims data for individuals undergoing surgical procedures for which opioid analgesic treatment is common.

EXPOSURES

State policies mandating prescription drug monitoring program (PDMP; PDMP policies) use, initial opioid prescription duration limit (duration limit policies), and mandated continuing medical education (CME; CME pain policies) on pain management.

MAIN MEASURES

Opioid dispensing rates, days' supply, and the daily morphine milligram equivalent dose (MMED).

KEY RESULTS

The percentage of Medicare beneficiaries dispensed opioids in the perioperative period increased from 2007 to 2018; MMED and days' supply decreased over the same period, with significant variation by age, sex, and race. None of the three state policies affected the likelihood of Medicare beneficiaries being dispensed perioperative opioids. However, CME pain policies and duration limit policies were associated with decreased days' supply and decreased MMED in the several years following implementation, respectively.

CONCLUSION

While we observed a slight increase in the rate of Medicare beneficiaries dispensed opioids perioperatively and a substantial decrease in MMED and days' supply for those receiving opioids, state policies examined had relatively modest effects on the main measures. Our findings suggest that these state policies may have a limited impact on opioid dispensing for a patient population that is commonly dispensed opioid analgesics to help control surgical pain, and as a result may have little direct effect on clinical outcomes for this population. Changes in opioid dispensing for this population may be the result of broader societal trends than such state policies.

Drug-related physician continuing medical education requirements, 2010-2020

INTRODUCTION

The crisis of drug-related harm in the United States continues to worsen. While prescription-related overdoses have fallen dramatically, they are still far above pre-2010 levels. Physicians can reduce the risk of overdose and other drug-related harms by improving opioid prescribing practices and ensuring that patients are able to easily access medications for substance use disorder treatment. Most physicians received little or no training in those subjects in medical school. It is possible that continuing medical education can improve physician knowledge of appropriate prescribing and substance use disorder treatment and patient outcomes.

METHODS

Descriptive legal review. Laws in all 50 states and the District of Columbia were searched for provisions that require all or most physicians to receive either one-time or continuing medical education regarding controlled substance prescribing, pain management, or substance use disorder treatment.

RESULTS

There has been a rapid increase in the number of states with relevant requirements, from three states at the end of 2010 to 42 at the end of 2020. The frequency and duration of required education varied substantially across states. In all states, the number of hours required in relevant topics is a small fraction of overall required continuing education, an average of 1 h per year. Despite recent shifts in the substances driving overdose, most requirements remain focused on opioids.

CONCLUSION

While most states have now adopted continuing education requirements regarding controlled substance prescribing, pain management, or substance use disorder treatment, these requirements comprise a small component of the required post-training education requirements. Research is needed to determine whether this training translates into reductions in drug-related harm.

Punitive legal responses to prenatal drug use in the United States: A survey of state policies and systematic review of their public health impacts

BACKGROUND

Punitive legal responses to prenatal drug use may be associated with unintended adverse health consequences. However, in a rapidly shifting policy climate, current information has not been summarized. We conducted a survey of U.S. state policies that utilize criminal or civil legal system penalties to address prenatal drug use. We then systematically identified empirical studies evaluating these policies and summarized their potential public health impacts.

METHODS

Using existing databases and original statutory research, we surveyed current U.S. state-level prenatal drug use policies authorizing explicit criminalization, involuntary commitment, civil child abuse substantiation, and parental rights termination. Next, we systematically identified quantitative associations between these policies and health outcomes, restricting to U.S.-based peer-reviewed research, published January 2000-December 2022. Results described study characteristics and synthesized the evidence on health-related harms and benefits associated with punitive policies. Validity threats were described narratively.

RESULTS

By 2022, two states had adopted policies explicitly authorizing criminal prosecution, and five states allowed pregnancy-specific and drug use-related involuntary civil commitment. Prenatal drug use was grounds for substantiating civil child abuse and terminating parental rights in 22 and five states, respectively. Of the 16 review-identified articles, most evaluated associations between punitive policies generally (k = 12), or civil child abuse policies specifically (k = 2), and multiple outcomes, including drug treatment utilization (k = 6), maltreatment reporting and foster care entry (k = 5), neonatal drug withdrawal syndrome (NDWS, k = 4) and other pregnancy and birth-related outcomes (k = 3). Most included studies reported null associations or suggested increases in adverse outcome following punitive policy adoption.

CONCLUSIONS

Nearly half of U.S. states have adopted policies that respond to prenatal drug use with legal system penalties. While additional research is needed to clarify whether such approaches engender overt health harms, current evidence indicates that punitive policies are not associated with public health benefits, and therefore constitute ineffective policy.

Scaling Interventions to Manage Chronic Disease: Innovative Methods at the Intersection of Health Policy Research and Implementation Science

Policy implementation is a key component of scaling effective chronic disease prevention and management interventions. Policy can support scale-up by mandating or incentivizing intervention adoption, but enacting a policy is only the first step. Fully implementing a policy designed to facilitate implementation of health interventions often requires a range of accompanying implementation structures, like health IT systems, and implementation strategies, like training. Decision makers need to know what policies can support intervention adoption and how to implement those policies, but to date research on policy implementation is limited and innovative methodological approaches are needed. In December 2021, the Johns Hopkins ALACRITY Center for Health and Longevity in Mental Illness and the Johns Hopkins Center for Mental Health and Addiction Policy convened a forum of research experts to discuss approaches for studying policy implementation. In this report, we summarize the ideas that came out of the forum. First, we describe a motivating example focused on an Affordable Care Act Medicaid health home waiver policy used by some US states to support scale-up of an evidence-based integrated care model shown in clinical trials to improve cardiovascular care for people with serious mental illness. Second, we define key policy implementation components including structures, strategies, and outcomes. Third, we provide an overview of descriptive, predictive and associational, and causal approaches that can be used to study policy implementation. We conclude with discussion of priorities for methodological innovations in policy implementation research, with three key areas identified by forum experts: effect modification methods for making causal inferences about how policies' effects on outcomes vary based on implementation structures/strategies; causal mediation approaches for studying policy implementation mechanisms; and characterizing uncertainty in systems science models. We conclude with discussion of overarching methods considerations for studying policy implementation, including measurement of policy implementation, strategies for studying the role of context in policy implementation, and the importance of considering when establishing causality is the goal of policy implementation research.

Identifying Optimal Methods for Addressing Confounding Bias When Estimating the Effects of State-level Policies

BACKGROUND

Policy evaluation studies that assess how state-level policies affect health-related outcomes are foundational to health and social policy research. The relative ability of newer analytic methods to address confounding, a key source of bias in observational studies, has not been closely examined.

METHODS

We conducted a simulation study to examine how differing magnitudes of confounding affected the performance of 4 methods used for policy evaluations: (1) the two-way fixed effects difference-in-differences model; (2) a 1-period lagged autoregressive model; (3) augmented synthetic control method; and (4) the doubly robust difference-in-differences approach with multiple time periods from Callaway-Sant'Anna. We simulated our data to have staggered policy adoption and multiple confounding scenarios (i.e., varying the magnitude and nature of confounding relationships).

RESULTS

Bias increased for each method: (1) as confounding magnitude increases; (2) when confounding is generated with respect to prior outcome trends (rather than levels), and (3) when confounding associations are nonlinear (rather than linear). The autoregressive model and augmented synthetic control method had notably lower root mean squared error than the two-way fixed effects and Callaway-Sant'Anna approaches for all scenarios; the exception is nonlinear confounding by prior trends, where Callaway-Sant'Anna excels. Coverage rates were unreasonably high for the augmented synthetic control method (e.g., 100%), reflecting large model-based standard errors and wide confidence intervals in practice.

CONCLUSIONS

In our simulation study, no single method consistently outperformed the others, but a researcher's toolkit should include all methodologic options. Our simulations and associated R package can help researchers choose the most appropriate approach for their data.

A Dynamic Spatial Factor Model to Describe the Opioid Syndemic in Ohio

BACKGROUND

The opioid epidemic has been ongoing for over 20 years in the United States. As opioid misuse has shifted increasingly toward injection of illicitly produced opioids, it has been associated with HIV and hepatitis C transmission. These epidemics interact to form the opioid syndemic.

METHODS

We obtain annual county-level counts of opioid overdose deaths, treatment admissions for opioid misuse, and newly diagnosed cases of acute and chronic hepatitis C and newly diagnosed HIV from 2014 to 2019. Aligned with the conceptual framework of syndemics, we develop a dynamic spatial factor model to describe the opioid syndemic for counties in Ohio and estimate the complex synergies between each of the epidemics.

RESULTS

We estimate three latent factors characterizing variation of the syndemic across space and time. The first factor reflects overall burden and is greatest in southern Ohio. The second factor describes harms and is greatest in urban counties. The third factor highlights counties with higher than expected hepatitis C rates and lower than expected HIV rates, which suggests elevated localized risk for future HIV outbreaks.

CONCLUSIONS

Through the estimation of dynamic spatial factors, we are able to estimate the complex dependencies and characterize the synergy across outcomes that underlie the syndemic. The latent factors summarize shared variation across multiple spatial time series and provide new insights into the relationships between the epidemics within the syndemic. Our framework provides a coherent approach for synthesizing complex interactions and estimating underlying sources of variation that can be applied to other syndemics.

A Bayesian Spatio-temporal Model to Optimize Allocation of Buprenorphine in North Carolina

The opioid epidemic is an ongoing public health crisis. In North Carolina, overdose deaths due to illicit opioid overdose have sharply increased over the last 5-7 years. Buprenorphine is a U.S. Food and Drug Administration approved medication for treatment of opioid use disorder and is obtained by prescription. Prior to January 2023, providers had to obtain a waiver and were limited in the number of patients that they could prescribe buprenorphine. Thus, identifying counties where increasing buprenorphine would yield the greatest overall reduction in overdose death can help policymakers target certain geographical regions to inform an effective public health response. We propose a Bayesian spatiotemporal model that relates yearly, county-level changes in illicit opioid overdose death rates to changes in buprenorphine prescriptions. We use our model to forecast the statewide count and rate of illicit opioid overdose deaths in future years, and we use nonlinear constrained optimization to identify the optimal buprenorphine increase in each county under a set of constraints on available resources. Our model estimates a negative relationship between death rate and increasing buprenorphine after accounting for other covariates, and our identified optimal single-year allocation strategy is estimated to reduce opioid overdose deaths by over 5.

An integrated abundance model for estimating county-level prevalence of opioid misuse in Ohio

Opioid misuse is a national epidemic and a significant drug related threat to the United States. While the scale of the problem is undeniable, estimates of the local prevalence of opioid misuse are lacking, despite their importance to policy-making and resource allocation. This is due, in part, to the challenge of directly measuring opioid misuse at a local level. In this paper, we develop a Bayesian hierarchical spatio-temporal abundance model that integrates indirect county-level data on opioid-related outcomes with state-level survey estimates on prevalence of opioid misuse to estimate the latent county-level prevalence and counts of people who misuse opioids. A simulation study shows that our integrated model accurately recovers the latent counts and prevalence. We apply our model to county-level surveillance data on opioid overdose deaths and treatment admissions from the state of Ohio. Our proposed framework can be applied to other applications of small area estimation for hard to reach populations, which is a common occurrence with many health conditions such as those related to illicit behaviors.

Use of an Inverted Synthetic Control Method to Estimate Effects of Recent Drug Overdose Good Samaritan Laws, Overall and by Black/White Race/Ethnicity

Overdose Good Samaritan laws (GSLs) aim to reduce mortality by providing limited legal protections when a bystander to a possible drug overdose summons help. Most research into the impact of these laws is dated or potentially confounded by coenacted naloxone access laws. Lack of awareness and trust in GSL protections, as well as fear of police involvement and legal repercussions, remain key deterrents to help-seeking. These barriers may be unequally distributed by race/ethnicity due to racist policing and drug policies, potentially producing racial/ethnic disparities in the effectiveness of GSLs for reducing overdose mortality. We used 2015-2019 vital statistics data to estimate the effect of recent GSLs on overdose mortality, overall (8 states) and by Black/White race/ethnicity (4 states). Given GSLs' near ubiquity, few unexposed states were available for comparison. Therefore, we generated an "inverted" synthetic control method (SCM) to compare overdose mortality in new-GSL states with that in states that had GSLs throughout the analytical period. The estimated relationships between GSLs and overdose mortality, both overall and stratified by Black/White race/ethnicity, were consistent with chance. An absence of effect could result from insufficient protection provided by the laws, insufficient awareness of them, and/or reticence to summon help not addressable by legal protections. The inverted SCM may be useful for evaluating other widespread policies.

When Effects Cannot be Estimated: Redefining Estimands to Understand the Effects of Naloxone Access Laws

Violations of the positivity assumption (also called the common support condition) challenge health policy research and can result in significant bias, large variance, and invalid inference. We define positivity in the single- and multiple-timepoint (i.e., longitudinal) health policy evaluation setting, and discuss real-world threats to positivity. We show empirical evidence of the practical positivity violations that can result when attempting to estimate the effects of health policies (in this case, Naloxone Access Laws). In such scenarios, an alternative is to estimate the effect of a shift in law enactment (e.g., the effect if enactment had been delayed by some number of years). Such an effect corresponds to what is called a modified treatment policy, and dramatically weakens the required positivity assumption, thereby offering a means to estimate policy effects even in scenarios with serious positivity problems. We apply the approach to define and estimate the longitudinal effects of Naloxone Access Laws on opioid overdose rates.

A modified Delphi process to identify experts' perceptions of the most beneficial and harmful laws to reduce opioid-related harm

BACKGROUND

States have enacted multiple types of laws, with a variety of constituent provisions, in response to the opioid epidemic, often simultaneously. This temporal proximity and variation in state-to-state operationalization has resulted in significant challenges for empirical research on their effects. Thus, expert consensus can be helpful to classify laws and their provisions by their degree of helpfulness and impact.

METHODS

We conducted a four-stage modified policy Delphi process to identify the top 10 most helpful and 5 most harmful provisions from eight opioid-related laws. This iterative consultation with six types of opioid experts included a preliminary focus group (n=12), two consecutive surveys (n=56 and n=40, respectively), and a final focus group feedback session (n=5).

RESULTS

On a scale of very harmful (0) to very helpful (4), overdose Good Samaritan laws received the highest average helpfulness rating (3.62, 95% CI: 3.48-3.75), followed by naloxone access laws (3.37, 95% CI: 3.22-3.51), and pain management clinic laws (3.08, 95% CI: 2.89-3.26). Drug-induced homicide (DIH) laws were rated the most harmful (0.88, 95% CI: 0.66-1.11). Impact ratings aligned similarly, although Medicaid laws received the second highest overall impact rating (3.71, 95% CI: 3.45, 3.97). The two most helpful provisions were naloxone standing orders (3.94, 95% CI: 3.86-4.02) and Medicaid coverage of medications for opioid use disorder (MOUD) (3.89, 95% CI: 3.82). Mandatory minimum DIH laws were the most harmful provision (0.73, 95% CI 0.53-0.93); followed by requiring prior authorization for Medicaid coverage of MOUD (1.00 95% CI: 0.72-1.27).

CONCLUSION

Overall, experts rated laws and provisions that facilitated harm reduction efforts and access to MOUD as most helpful. Laws and provisions rated as most harmful criminalized substance use and placed restrictions on access to MOUD. These ratings provide a foundation for evaluating the overall overdose policy environment for each state.

Effects of State Opioid Prescribing Laws on Use of Opioid and Other Pain Treatments Among Commercially Insured U.S. Adults

BACKGROUND

There is concern that state laws to curb opioid prescribing may adversely affect patients with chronic noncancer pain, but the laws' effects are unclear because of challenges in disentangling multiple laws implemented around the same time.

OBJECTIVE

To study the association between state opioid prescribing cap laws, pill mill laws, and mandatory prescription drug monitoring program query or enrollment laws and trends in opioid and guideline-concordant nonopioid pain treatment among commercially insured adults, including a subgroup with chronic noncancer pain conditions.

DESIGN

Thirteen treatment states that implemented a single law of interest in a 4-year period and unique groups of control states for each treatment state were identified. Augmented synthetic control analyses were used to estimate the association between each state law and outcomes.

SETTING

United States, 2008 to 2019.

PATIENTS

7 694 514 commercially insured adults aged 18 years or older, including 1 976 355 diagnosed with arthritis, low back pain, headache, fibromyalgia, and/or neuropathic pain.

MEASUREMENTS

Proportion of patients receiving any opioid prescription or guideline-concordant nonopioid pain treatment per month, and mean days' supply and morphine milligram equivalents (MME) of prescribed opioids per day, per patient, per month.

RESULTS

Laws were associated with small-in-magnitude and non-statistically significant changes in outcomes, although CIs around some estimates were wide. For adults overall and those with chronic noncancer pain, the 13 state laws were each associated with a change of less than 1 percentage point in the proportion of patients receiving any opioid prescription and a change of less than 2 percentage points in the proportion receiving any guideline-concordant nonopioid treatment, per month. The laws were associated with a change of less than 1 in days' supply of opioid prescriptions and a change of less than 4 in average monthly MME per day per patient prescribed opioids.

LIMITATIONS

Results may not be generalizable to non-commercially insured populations and were imprecise for some estimates. Use of claims data precluded assessment of the clinical appropriateness of pain treatments.

CONCLUSION

This study did not identify changes in opioid prescribing or nonopioid pain treatment attributable to state laws.

PRIMARY FUNDING SOURCE

National Institute on Drug Abuse.

The effect of state policies on rates of high-risk prescribing of an initial opioid analgesic

BACKGROUND

Multiple state policies, such as prescription drug monitoring programs (PDMPs) and duration limits, have been implemented to decrease high-risk opioid prescribing. Studies demonstrate that many policies decrease certain opioid prescribing behaviors, but few examine their intended effects on the targeted high-risk prescribing practices, nor disentangle the effects of concurrent state or federal policies likely to influence those practices.

METHODS

Forty-one million initial prescriptions for new opioid episodes from 2007 to 2018 were identified using national pharmacy claims. We identified high-risk initial prescriptions, defined as >7 days' supply, average daily MME >90, or concurrent with benzodiazepines and estimated three multivariable logistic regression models to assess the association between policies and outcomes controlling for patient, prescriber, and county characteristics.

RESULTS

Initial prescriptions for >7 days declined from 23.8% in 2007 to 14.9% in 2018, associated with mandatory and interoperable PDMPs and prescription duration limits but not other policies examined. Initial prescriptions with daily MME > 90 declined from 13.2% to 1.9%, associated with pain management clinic laws but not consistently with other policies. Initial prescriptions concurrent with benzodiazepines declined only modestly from 6.9% to 6.5%, associated with pain management clinic laws but not other policies examined.

CONCLUSIONS

The opioid policy environment has changed rapidly with a range of different policies being implemented addressing high-risk prescribing. PDMP laws mandating prescriber use and pain clinic laws both appear efficacious but decrease different types of high-risk opioid prescribing. New policies should be considered in light of the prevalence of the problem being addressed.

Moving beyond the classic difference-in-differences model: a simulation study comparing statistical methods for estimating effectiveness of state-level policies

Background

Reliable evaluations of state-level policies are essential for identifying effective policies and informing policymakers’ decisions. State-level policy evaluations commonly use a difference-in-differences (DID) study design; yet within this framework, statistical model specification varies notably across studies. More guidance is needed about which set of statistical models perform best when estimating how state-level policies affect outcomes.

Methods

Motivated by applied state-level opioid policy evaluations, we implemented an extensive simulation study to compare the statistical performance of multiple variations of the two-way fixed effect models traditionally used for DID under a range of simulation conditions. We also explored the performance of autoregressive (AR) and GEE models. We simulated policy effects on annual state-level opioid mortality rates and assessed statistical performance using various metrics, including directional bias, magnitude bias, and root mean squared error. We also reported Type I error rates and the rate of correctly rejecting the null hypothesis (e.g., power), given the prevalence of frequentist null hypothesis significance testing in the applied literature.

Results

Most linear models resulted in minimal bias. However, non-linear models and population-weighted versions of classic linear two-way fixed effect and linear GEE models yielded considerable bias (60 to 160%). Further, root mean square error was minimized by linear AR models when we examined crude mortality rates and by negative binomial models when we examined raw death counts. In the context of frequentist hypothesis testing, many models yielded high Type I error rates and very low rates of correctly rejecting the null hypothesis (< 10%), raising concerns of spurious conclusions about policy effectiveness in the opioid literature. When considering performance across models, the linear AR models were optimal in terms of directional bias, root mean squared error, Type I error, and correct rejection rates.

Conclusions

The findings highlight notable limitations of commonly used statistical models for DID designs, which are widely used in opioid policy studies and in state policy evaluations more broadly. In contrast, the optimal model we identified--the AR model--is rarely used in state policy evaluation. We urge applied researchers to move beyond the classic DID paradigm and adopt use of AR models.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-021-01471-y.

Davis C, Rudolph KE, Keyes KM, Hasin DS, Cerdá M. Prescription Opioid Laws and Opioid Dispensing in US Counties: Identifying Salient Law Provisions With Machine Learning

BACKGROUND

Hundreds of laws aimed at reducing inappropriate prescription opioid dispensing have been implemented in the United States, yet heterogeneity in provisions and their simultaneous implementation have complicated evaluation of impacts. We apply a hypothesis-generating, multistage, machine-learning approach to identify salient law provisions and combinations associated with dispensing rates to test in future research.

METHODS

Using 162 prescription opioid law provisions capturing prescription drug monitoring program (PDMP) access, reporting and administration features, pain management clinic provisions, and prescription opioid limits, we used regularization approaches and random forest models to identify laws most predictive of county-level and high-dose dispensing. We stratified analyses by overdose epidemic phases-the prescription opioid phase (2006-2009), heroin phase (2010-2012), and fentanyl phase (2013-2016)-to further explore pattern shifts over time.

RESULTS

PDMP patient data access provisions most consistently predicted high-dispensing and high-dose dispensing counties. Pain management clinic-related provisions did not generally predict dispensing measures in the prescription opioid phase but became more discriminant of high dispensing and high-dose dispensing counties over time, especially in the fentanyl period. Predictive performance across models was poor, suggesting prescription opioid laws alone do not strongly predict dispensing.

CONCLUSIONS

Our systematic analysis of 162 law provisions identified patient data access and several pain management clinic provisions as predictive of county prescription opioid dispensing patterns. Future research employing other types of study designs is needed to test these provisions' causal relationships with inappropriate dispensing and to examine potential interactions between PDMP access and pain management clinic provisions. See video abstract at, http://links.lww.com/EDE/B861.

The Revolution Will Be Hard to Evaluate: How Co-Occurring Policy Changes Affect Research on the Health Effects of Social Policies

Extensive empirical health research leverages variation in the timing and location of policy changes as quasi-experiments. Multiple social policies may be adopted simultaneously in the same locations, creating co-occurrence that must be addressed analytically for valid inferences. The pervasiveness and consequences of co-occurring policies have received limited attention. We analyzed a systematic sample of 13 social policy databases covering diverse domains including poverty, paid family leave, and tobacco use. We quantified policy co-occurrence in each database as the fraction of variation in each policy measure across different jurisdictions and times that could be explained by covariation with other policies. We used simulations to estimate the ratio of the variance of effect estimates under the observed policy co-occurrence to variance if policies were independent. Policy co-occurrence ranged from very high for state-level cannabis policies to low for country-level sexual minority-rights policies. For 65% of policies, greater than 90% of the place-time variation was explained by other policies. Policy co-occurrence increased the variance of effect estimates by a median of 57-fold. Co-occurring policies are common and pose a major methodological challenge to rigorously evaluating health effects of individual social policies. When uncontrolled, co-occurring policies confound one another, and when controlled, resulting positivity violations may substantially inflate the variance of estimated effects. Tools to enhance validity and precision for evaluating co-occurring policies are needed.

What to Do When Everything Happens at Once: Analytic Approaches to Estimate the Health Effects of Co-Occurring Social Policies

Social policies have great potential to improve population health and reduce health disparities. Increasingly, those doing empirical research have sought to quantify the health effects of social policies by exploiting variation in the timing of policy changes across places. Multiple social policies are often adopted simultaneously or in close succession in the same locations, creating co-occurrence that must be handled analytically for valid inferences. Although this is a substantial methodological challenge for researchers aiming to isolate social policy effects, only in a limited number of studies have researchers systematically considered analytic solutions within a causal framework or assessed whether these solutions are being adopted. We designated 7 analytic solutions to policy co-occurrence, including efforts to disentangle individual policy effects and efforts to estimate the combined effects of co-occurring policies. We used an existing systematic review of social policies and health to evaluate how often policy co-occurrence is identified as a threat to validity and how often each analytic solution is applied in practice. Of the 55 studies, only in 17 (31%) did authors report checking for any co-occurring policies, although in 36 studies (67%), at least 1 approach was used that helps address policy co-occurrence. The most common approaches were adjusting for measures of co-occurring policies; defining the outcome on subpopulations likely to be affected by the policy of interest (but not other co-occurring policies); and selecting a less-correlated measure of policy exposure. As health research increasingly focuses on policy changes, we must systematically assess policy co-occurrence and apply analytic solutions to strengthen studies on the health effects of social policies.