Why We Should Not Be Indifferent to Specification Choices for Difference-in-Differences

Staggered interventions with no control groups

The limitations of the two-way fixed effects for the impact evaluation of interventions that occur at different times for each group have meant that 'staggered interventions' have been highlighted in recent years in the econometric literature and, more recently, in epidemiology. Although many alternative strategies (such as staggered difference-in-differences) have been proposed, the focus has predominantly been on scenarios in which one or more control groups are available. However, control groups are often unavailable, due to limitations in the available data or because all units eventually receive the intervention. In this context, interrupted time series (ITS) designs can constitute an appropriate alternative. The extent to which common model specifications for ITS analyses are limited in the case of staggered interventions remains an underexplored area in the methodological literature. In this work, we aim to demonstrate that standard ITS model specifications typically yield biased results for staggered interventions and we propose alternative model specifications that were inspired by recent developments in the difference-in-differences literature to propose adapted analytical strategies.

Evaluating the Effect of Financial Penalty on Hospital-Acquired Infections

Purpose

This study explores the effects of CMS reimbursement financial penalties from the Hospital-Acquired Condition Reduction Program (HACRP) on hospital-acquired infections (HAI) in hospitals across the United States.

Methods

Hospital-level data for 2896 hospitals in the United States were evaluated using multiple linear regression models with random effects analysis through a difference-in-differences study design to examine HAIs under the HACRP between hospitals that were financially penalized or not from calendar years 2013 to 2020.

Results

This study showed significant differences from the pre-program Total HAC scores to the most recent reviewed year, validating the efficacy of the HACRP, and showing a reduction of overall HAIs over the years evaluated in the study. The multiple linear regression model with random effects analysis produced a significant (p < 0.001) interaction term between hospitals expected to be penalized in 2013 and each year evaluated in the study (-0.412 estimate) confirming decreases in HAI scores, and overall decreases in HAIs across the years of the study. Notably, 98% of hospitals in the worst-performing, expected to be financially penalized quartile from 2013, were found to have decreased their HAIs in their facilities, while only 38.8% of hospital in the performing, non-penalized quartiles showed decreases in HAIs across their facilities, by 2020.

Conclusion

Our research indicates that implementing financial disincentives through reimbursement reductions could potentially decrease the incidence of HAIs. Our study further suggests that incorporating financial penalties and incentives for HAIs annually across all hospitals may lead to significant reductions in HAIs throughout the US healthcare system.

Advances in Difference-in-differences Methods for Policy Evaluation Research

Difference-in-differences (DiD) is a powerful, quasi-experimental research design widely used in longitudinal policy evaluations with health outcomes. However, DiD designs face several challenges to ensuring reliable causal inference, such as when policy settings are more complex. Recent economics literature has revealed that DiD estimators may exhibit bias when heterogeneous treatment effects, a common consequence of staggered policy implementation, are present. To deepen our understanding of these advancements in epidemiology, in this methodologic primer, we start by presenting an overview of DiD methods. We then summarize fundamental problems associated with DiD designs with heterogeneous treatment effects and provide guidance on recently proposed heterogeneity-robust DiD estimators, which are increasingly being implemented by epidemiologists. We also extend the discussion to violations of the parallel trends assumption, which has received less attention. Last, we present results from a simulation study that compares the performance of several DiD estimators under different scenarios to enhance understanding and application of these methods.

The association between comprehensive medication review and medication adherence among medicare beneficiaries with chronic obstructive pulmonary disease

Background

Medicare Part D plans are required to provide Medication therapy management (MTM) services to eligible beneficiaries to optimize medication utilization. Comprehensive medication review (CMR) is a core element of the MTM program. Despite the availability of advanced medical treatment for patients with chronic obstructive pulmonary disease (COPD), medication adherence to maintenance medications poses a continued challenge for patients with COPD.

Objective

To examine the effects of CMR on medication adherence among patients with COPD.

Methods

Medicare data for 2016-2017 linked to Area Health Resource Files were analyzed. The study population was Medicare beneficiaries with COPD. The intervention group consisted of beneficiaries who received CMR in 2017 but not in 2016. Patients who were eligible for MTM services but did not receive these services in 2016 or 2017 made up the control group. Propensity score matching was used to select an intervention and control group with balanced characteristics. The study outcome was adherence to COPD medications with the proportion of days covered at or above 80%. A difference-in-differences approach was adopted in the logistic regression analyses with an interaction term between the status of CMR receipt and the year 2017.

Results

The study sample included 25,564 patients with COPD. The proportions of adherent patients were similar in the control group in both years but increased significantly from 60.08% in 2016 to 69.38% in 2017 in the intervention group (P < .001). The odds of medication adherence in the intervention group increased from 2016 to 2017 by 59% more than in the control group (adjusted odds ratio = 1.59, 95% confidence interval = 1.48-1.71).

Conclusions

Receiving CMR was associated with improved adherence to COPD medications among Medicare beneficiaries. Policymakers should ensure that Medicare beneficiaries with COPD receive CMR.

Decision Support and Behavioral Health for Reducing High-Dose Opioids in Comorbid Chronic Pain, Depression and Anxiety: Stepped-Wedge Cluster Randomized Trial

BACKGROUND

High prevalence of depression or anxiety with opioid use for chronic pain complicates co-management and may influence prescribing behaviors.

OBJECTIVE

Compare clinical effectiveness of electronic medical record clinical decision support (EMR-CDS) versus additional behavioral health (BH) care management for reducing rates of high-dose opioid prescriptions.

DESIGN

Type 2 effectiveness-implementation hybrid stepped-wedge cluster randomized trial in 35 primary care clinics within a health system in LA, USA.

PARTICIPANTS

Patients aged 18+ receiving chronic opioid therapy for non-cancer pain with depression or anxiety and matched controls.

INTERVENTION

EMR-CDS included opioid risk mitigation procedures. BH care included cognitive behavioral therapy; depression or anxiety medication adjustments; and case management.

MAIN MEASURES

Outcomes of interest included difference-in-difference (DID) estimate of changes in probability for prescribing high-dose morphine equivalent daily dose (MEDD ≥50 mg/day and MEDD ≥90), average MEDD, and rates of hospitalization, emergency department use, and opioid risk mitigation.

KEY RESULTS

Most participants were female with 3+ pain syndromes. Data analysis included 632 patients. Absolute risk differences for MEDD≥50 and ≥90 decreased post-index compared to pre-index (DID of absolute risk difference [95%CI]: -0.036 [-0.089, 0.016] and -0.029 [-0.060, 0.002], respectively). However, these differences were not statistically significant. The average MEDD decreased at a higher rate for the BH group compared to EMR-CDS only (DID rate ratio [95%CI]: 0.85 [0.77, 0.93]). There were no changes in hospitalization and emergency department utilization. The BH group had higher probabilities of new specialty referrals and prescriptions for naloxone and antidepressants.

CONCLUSIONS

Incorporation of a multidisciplinary behavioral health care team into primary care did not decrease high-dose prescribing; however, it improved adherence to clinical guideline recommendations for managing chronic opioid therapy for non-cancer pain.

TRIAL REGISTRATION

ClinicalTrials.gov ID NCT03889418.

Do studies evaluating early-life policy interventions fully adhere to the critical conditions of difference-in-differences? A systematic review

OBJECTIVES

To assess the reporting and methodological quality of early-life policy intervention papers that applied difference-in-differences (DiD) analysis.

STUDY DESIGN

Systematic review.

DATA SOURCES

Papers applying DiD of early-life policy interventions in high-income countries as identified by searching Medline, Embase and Scopus databases up to December, 2022.

STUDY ELIGIBILITY CRITERIA, PARTICIPANTS AND INTERVENTIONS

Studies evaluating policy interventions targeting expectant mothers, infants or children up to two years old and conducted in high income countries were included. We focused on seven critical conditions of DiD as proposed in a comprehensive checklist: data requirements, parallel trends, no-anticipation, standard statistical assumptions, common shocks, group composition and spillover.

RESULTS

The DiD included studies (n=19) evaluating early-life policy interventions in childhood development (n=4), healthcare utilisation and providers (n=4), nutrition programmes (n=3) and economic policies such as prenatal care expansion (n=8). Although none of the included studies met all critical conditions, the most reported and adhered to critical conditions were data requirements (n=18), standard statistical assumptions (n=11) and the parallel trends assumption (n=9). No-anticipation and spillover were explicitly reported and adhered to in two studies and one study, respectively.

CONCLUSIONS

This review highlights current deficiencies in the reporting and methodological quality of studies using DiD to evaluate early-life policy interventions. As the validity of study conclusions and consequent implications for policy depend on the extent to which critical conditions are met, this shortcoming is concerning. We recommend that researchers use the described checklist to improve the transparency and validity of their evaluations. The checklist should be further refined by adding order of importance or knock-out criteria and may also help facilitate uniform terminology. This will hopefully encourage reliable DiD evaluations and thus contribute to better policies relating to expectant mothers, infants and children.

Medicaid Expansion and Perinatal Health Outcomes: A Quasi-Experimental Study

OBJECTIVE

There has been little evidence of the impact of preventive services during pregnancy covered under the Affordable Care Act (ACA) on birthing parent and infant outcomes. To address this gap, this study examines the association between Medicaid expansion under the ACA and birthing parent and infant outcomes of low-income pregnant people.

METHODS

This study used individual-level data from the 2004-2017 annual waves of the Pregnancy Risk Assessment Monitoring System (PRAMS). PRAMS is a surveillance project of the Centers for Disease Control and Prevention and health departments that annually includes a representative sample of 1,300 to 3,400 births per state, selected from birth certificates. Birthing parents' outcomes of interest included timing of prenatal care, gestational diabetes, hypertensive disorders of pregnancy, cigarette smoking during pregnancy, and postpartum care. Infant outcomes included initiation and duration of breastfeeding, preterm birth, and birth weight. The association between ACA Medicaid expansion and the birthing parent and infant outcomes were examined using difference-in-differences estimation.

RESULTS

There was no association between Medicaid expansion and the outcomes examined after correcting for multiple testing. This finding was robust to several sensitivity analyses.

CONCLUSIONS FOR PRACTICE

Study findings suggest that expanded access to more complete insurance benefits with limited cost-sharing for pregnant people, a group that already had high rates of insurance coverage, did not impact the birthing parents' and infant health outcomes examined.

Effects of Advanced Team-Based Care on Care Processes and Health Measures in a Federally Qualified Health Center

In a federally qualified health center, we assess a novel primary care delivery model, advanced team-based care (aTBC), that embeds care team members in patient visits. Using a difference-in-differences research design, we measure visit intensity, compliance with preventive care recommendations, and health outcomes among patients in the aTBC model compared with patients in a traditional team-based delivery model. We find increases in receipt of some recommended preventive care and in visit intensity, but no change in health outcomes. The aTBC model may improve some dimensions of care quality for low-income, vulnerable populations.

Does uptake of specialty care affect HRQoL development in COPD patients beneficially? A difference-in-difference analysis linking claims and survey data

BACKGROUND

There is an evidence gap on whether the choice of specialty care beneficially affects health-related quality of life (HRQoL) in patients with chronic obstructive pulmonary disease (COPD). This study analyzes how newly initiated pulmonologist care affects the generic and disease-specific HRQoL in COPD patients over a period of 1 year.

METHODS

We linked claims data with data from two survey waves to investigate the longitudinal effect of specialty care on HRQoL using linear Difference-in-Difference models based on 1:3 propensity score matched data. Generic HRQoL was operationalized by EQ-5D-5L visual analog scale (VAS), and disease-specific HRQoL by COPD assessment test (CAT). Subgroup analyses examined COPD patients with low (GOLD AB) and high (GOLD CD) exacerbation risk.

RESULTS

In contrast to routine care patients, pulmonologists' patients (n = 442) experienced no significant deterioration in HRQoL (VAS - 0.0, p = 0.9870; CAT + 0.5, p = 0.0804). Models unveiled a small comparative advantage of specialty care on HRQoL (mean change: CAT - 0.8, VAS + 2.9), which was especially pronounced for GOLD AB (CAT - 0.7; VAS + 3.1).

CONCLUSION

The uptake of pulmonologist care had a statistically significant, but not clinically relevant, beneficial impact on the development of HRQoL by slowing down overall HRQoL deterioration within 1 year. Including specialty care more appropriately in COPD management, especially at lower disease stages (GOLD AB), could thus improve patients' health outcome.

Postpartum Expansion of Emergency Medicaid is Associated with Increased Receipt of Recommended Glycemic Screening and Care

Oregon expanded Emergency Medicaid coverage to 60 days of postpartum care in 2018, facilitating ongoing care for conditions such as gestational diabetes. We linked Medicaid claims and birth certificates from 2010 to 2019 in Oregon and South Carolina, which did not expand postpartum care. We used a difference-in-difference design to measure the effects of postpartum care coverage among Emergency Medicaid recipients with gestational diabetes. Primary outcomes were receipt of recommended glucose tolerance testing and new diagnosis of Type 2 diabetes. Our sample included 2,270 live births among a predominantly multiparous, Latina population. Postpartum coverage was associated with a significant increase in receipt of a recommended glucose tolerance test (23.1 percentage points, 95% CI 16.9-29.3) and in diagnosis of Type 2 diabetes (4.6 percentage points, 95% CI 3.3-65.9). Expansion of postpartum coverage increased recommended screenings and care among Emergency Medicaid enrollees with pregnancies complicated by gestational diabetes.

Effects of 24/7 palliative care consultation availability on the use of emergency department and emergency medical services resources from non-oncological patients: a before-and-after observational cohort study

OBJECTIVES

The non-oncological population is relatively under-represented among end-of-life (EOL) patients managed by palliative care (PC) services, and the effects of different PC delivery models are understudied in this population.This retrospective observational study on routinely collected data aimed at evaluating the effects of the extension from workday-only to 24/7 mixed hands-on and advisory home PC service on emergency department (ED) access and emergency medical services (EMS) interventions needed by non-oncological patients during their last 90 days of life, and their probability to die in hospital.

METHODS

A before-and-after design was adopted comparing preimplementation and postimplementation periods (2018-2019 and 2021-22).We used a difference-in-differences approach to estimate changes in ED access and EMS intervention rates in the postintervention period through binomial negative regression. The oncological population, always exposed to 24/7 PC, was used as a control. A robust Poisson regression model was adopted to investigate the differences regarding hospital mortality. The analyses were adjusted for age, sex and disease grouping by the system involved. Results were reported as incidence rate ratios (IRRs) and ORs.

RESULTS

A total of 2831 patients were enrolled in the final analysis.After the implementation of 24/7 home PC, both ED admissions (IRR=0.390, p<0.001) and EMS interventions (IRR=0.413, p<0.001) dropped, as well as the probability to die in hospital (OR=0.321, p<0.001).

CONCLUSIONS

The adoption of a 24/7 mixed hands-on and advisory model of home PC could have relevant effects in terms of ED access and EMS use by non-oncological EOL patients under PC.

TRIAL REGISRATION NUMBER

NCT05640076.

Does Anesthesia Quality Improvement Participation Lead to Incremental Savings in a Surgical Quality Collaborative Population? A Retrospective Observational Study

BACKGROUND

The Anesthesiology Performance Improvement and Reporting Exchange (ASPIRE) Collaborative Quality Initiative (CQI) was launched as a partnership among hospitals to measure quality, review evidence-based practices, and improve anesthesia-related outcomes. Cost savings and improved patient outcomes have been associated with surgical CQI participation, but the impact of an anesthesia CQI on health care cost has not been thoroughly assessed. In this study, we evaluated whether participation in an anesthesia CQI led to health care savings. We hypothesized that ASPIRE participation is associated with reduced total episode payments for payers and major, high-volume procedures included in the Michigan Value Collaborative (MVC) registry.

METHODS

In this retrospective observational study, we compared MVC episode payment data from Group 1 ASPIRE hospitals, the first cluster of 8 Michigan hospitals to join ASPIRE in January 2015, to non-ASPIRE matched control hospitals. MVC computes price-standardized, risk-adjusted payments for patients insured by Blue Cross Blue Shield of Michigan Preferred Provider Organization, Blue Care Network Health Maintenance Organization, and Medicare Fee-for-Service plans. Episodes from 2014 comprised the pre-ASPIRE time period, and episodes from June 2016 to July 2017 constituted the post-ASPIRE time period. We performed a difference-in-differences analysis to evaluate whether ASPIRE implementation was associated with greater reduction in total episode payments compared to the change in the control hospitals during the same time periods.

RESULTS

We found a statistically significant reduction in total episode (-$719; 95% CI [-$1340 to -$97]; P = .023) payments at the 8 ASPIRE hospitals (N = 17,852 cases) compared to the change observed in 8 matched non-ASPIRE hospitals (N = 12,987 cases) for major, high-volume surgeries, including colectomy, colorectal cancer resection, gastrectomy, esophagectomy, pancreatectomy, hysterectomy, joint replacement (knee and hip), and hip fracture repair. In secondary analyses, 30-day postdischarge (-$354; 95% CI [-$582 to -$126]; P = .002) payments were also significantly reduced in ASPIRE hospitals compared to non-ASPIRE controls. Subgroup analyses revealed a significant reduction in total episode payments for joint replacements (-$860; 95% CI [-$1222 to -$499]; P < .001) at ASPIRE-participating hospitals. Sensitivity analyses including patient-level covariates also showed consistent results.

CONCLUSIONS

Participation in an anesthesiology CQI, ASPIRE, is associated with lower total episode payments for selected major, high-volume procedures. This analysis supports that participation in an anesthesia CQI can lead to reduced health care payments.

Identifying Optimal Methods for Addressing Confounding Bias When Estimating the Effects of State-level Policies

BACKGROUND

Policy evaluation studies that assess how state-level policies affect health-related outcomes are foundational to health and social policy research. The relative ability of newer analytic methods to address confounding, a key source of bias in observational studies, has not been closely examined.

METHODS

We conducted a simulation study to examine how differing magnitudes of confounding affected the performance of 4 methods used for policy evaluations: (1) the two-way fixed effects difference-in-differences model; (2) a 1-period lagged autoregressive model; (3) augmented synthetic control method; and (4) the doubly robust difference-in-differences approach with multiple time periods from Callaway-Sant'Anna. We simulated our data to have staggered policy adoption and multiple confounding scenarios (i.e., varying the magnitude and nature of confounding relationships).

RESULTS

Bias increased for each method: (1) as confounding magnitude increases; (2) when confounding is generated with respect to prior outcome trends (rather than levels), and (3) when confounding associations are nonlinear (rather than linear). The autoregressive model and augmented synthetic control method had notably lower root mean squared error than the two-way fixed effects and Callaway-Sant'Anna approaches for all scenarios; the exception is nonlinear confounding by prior trends, where Callaway-Sant'Anna excels. Coverage rates were unreasonably high for the augmented synthetic control method (e.g., 100%), reflecting large model-based standard errors and wide confidence intervals in practice.

CONCLUSIONS

In our simulation study, no single method consistently outperformed the others, but a researcher's toolkit should include all methodologic options. Our simulations and associated R package can help researchers choose the most appropriate approach for their data.

The impact and implementation of an mHealth intervention to improve infant and young child feeding in Senegal: IIMAANJE protocol for a cluster randomized control trial

Behavior change communication (BCC) strategies have the potential to improve infant feeding and nutrition outcomes among infants and young children in low- and middle-income countries. More recently, there has been a shift toward the adoption of mHealth interventions-the use of mobile phones to transmit health-related information or direct care-to promote recommended BCC strategies among the caregivers of infants and young children. In Senegal, most infants and young children are not fed according to recommended practices leading to a high prevalence of undernutrition and micronutrient deficiencies. The aims of this cluster randomized control trial, using an effectiveness-implementation (type 1) hybrid design, were to: (1) determine the impact of an mHealth IYCF intervention on IYCF practices and nutrition outcomes; and (2) examine the implementation, costs, and opportunities for scaling up the mHealth messaging intervention. The trial was conducted in three regions in Senegal (Thies, Fatick, Diourbel) with 488 mother, father and children (6-23 months) triads. The intervention included 8 scripted messages, that underwent cognitive testing prior to the intervention implementation, and 8 unscripted messages from positive deviants. One voice message and one text message were sent each week to members of our experimental group for a 16-week period. The impact of the intervention was assessed through a household survey, 24-h dietary recall, and hemoglobin measurements before and after the intervention implementation. The primary outcomes were minimal acceptable diet (MAD) and anemia. We also included a total of 54 participants in nine focus groups held with mothers and fathers and semi-structured interviews with Badienou Gox (i.e., community health workers) (n = 6) and national partners and program implementers (n = 6) to examine the intervention implementation process. The study was registered prior to data collection on Clinicaltrials.gov (Identifier: NCT05374837).

Opportunities and challenges in using instrumental variables to study causal effects in nonrandomized stress and trauma research

OBJECTIVE

Researchers are often interested in assessing the causal effect of an exposure on an outcome when randomization is not ethical or feasible. Estimating causal effects by controlling for confounders can be unconvincing because important potential confounders remain unmeasured. Study designs leveraging instrumental variables (IVs) offer alternatives to confounder-control methods but are rarely used in stress and trauma research.

METHOD

We review the conceptual foundations and implementation of IV methods. We discuss strengths and limitations of IV approaches, contrasting with confounder-control methods, and illustrate the relevance of IVs for stress and trauma research.

RESULTS

IV approaches leverage an external or exogenous source of variation in the exposure. Instruments are variables that meet three conditions: relevance (variation in the IV is associated with variation in the chance of exposure), exclusion (the IV only affects the outcome through the exposure), and exchangeability (no unmeasured confounding of the IV-outcome relationship). Interpreting estimates from IV analyses requires an additional assumption, such as monotonicity (the instrument does not change the chance of exposure in different directions for any two individuals). Valid IVs circumvent the need to correctly identify, measure, and control for all confounders of the exposure-outcome relationship. The primary challenge is identifying a valid instrument.

CONCLUSIONS

IV approaches have strengths and weaknesses compared with confounder-control approaches. IVs offers a promising complementary study design to improve evidence about the causal effects of exposures on outcomes relevant to stress and trauma. Collaboration with scientists who are experienced with identifying and analyzing IVs will support this work. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Back to basics: A mediation analysis approach to addressing the fundamental questions of integrated care evaluations

Health systems around the world are aiming to improve the integration of health and social care services to deliver better care for patients. Existing evaluations have focused exclusively on the impact of care integration on health outcomes and found little effect. That suggests the need to take a step back and ask whether integrated care programmes actually lead to greater clinical integration of care and indeed whether greater integration is associated with improved health outcomes. We propose a mediation analysis approach to address these two fundamental questions when evaluating integrated care programmes. We illustrate our approach by re-examining the impact of an English integrated care program on clinical integration and assessing whether greater integration is causally associated with fewer admissions for ambulatory care sensitive conditions. We measure clinical integration using a concentration index of outpatient referrals at the general practice level. While we find that the scheme increased integration of primary and secondary care, clinical integration did not mediate a decrease in unplanned hospital admissions. Our analysis emphasizes the need to better understand the hypothesized causal impact of integration on health outcomes and demonstrates how mediation analysis can inform future evaluations and program design.

Earlier smartphone acquisition negatively impacts language proficiency, but only for heavy media users. Results from a longitudinal quasi-experimental study

There is a growing debate about the proper age at which teens should be given permission to own a personal smartphone. While experts in different disciplines provide parents and educators with conflicting guidelines, the age of first smartphone acquisition is constantly decreasing and there is still limited evidence on the impact of anticipating the age of access on learning outcomes. Drawing on two-wave longitudinal data collected on a sample of 1672 students in 2013 (at grade 5) and 2016 (at grade 8), this study evaluates whether obtaining the first personal smartphone at 10 or 11 years old, during the transition to lower secondary school (early owning), affected their language proficiency trends compared to receiving it from the age of 12 onwards (late owning). Results indicate an overall null effect of smartphone early owning on adolescents' language proficiency trajectories, while a negative effect is found on those who were already heavy screen media users before receiving the device.

The challenge of dementia prevention trials and the role of quasi-experimental studies

Observational studies have shown consistently that modifiable risk factors during life are associated with increased dementia risk in old age but randomized controlled trials (RCTs) on dementia prevention evaluating the treatment of these risk factors did not find consistent effects on cognitive outcomes. The discrepancy in findings is potentially attributable to inherent differences between the two study designs. Although RCTs are the gold standard for establishing causality, designing and conducting an RCT for dementia prevention is complex. Quasi-experimental studies (QESs) may contribute to investigating causality without randomization. QESs use variation in exposure to a risk factor or intervention in an observational setting to deduct causal effects. Design-specific approaches are used to control for confounding, the main caveat of QESs. In this article we address the challenges, opportunities, and limitations of QESs for research into dementia prevention. HIGHLIGHTS: Despite consistent associations between modifiable risk factors and dementia, the mostly neutral effects of randomized controlled trials (RCTs) challenge the causality of these associations. RCTs in the field of dementia prevention are often problematic due to ethical, practical, or financial constraints, and their results may have limited generalizability. Four quasi-experimental study (QES) designs may be suitable to study causality between risk factors and dementia; we critically appraise these study designs for dementia-prevention studies. We describe how specific QES designs can be used to study the effects of risk-factor modification for 12 known risk factors for dementia.

Impact of performance-based budgeting on quality outcomes in U.S. military health care facilities

BACKGROUND

Performance-based budgeting (PBB) is a variation of pay for performance that has been used in government hospitals but could be applicable to any integrated system. It works by increasing or decreasing funding based on preestablished performance thresholds, which incentivizes organizations to improve performance. In late 2006, the U.S. Army implemented a PBB program that tied hospital-level funding decisions to performance on key cost and quality-related metrics.

PURPOSE

The aim of this study was to estimate the impact of PBB on quality improvement in U.S. Army health care facilities.

APPROACH

This study used a retrospective difference-in-differences analysis of data from two Defense Health Agency data repositories. The merged data set encompassed administrative, demographic, and performance information about 428 military health care facilities. Facility-level performance data on quality indicators were compared between 187 Army PBB facilities and a comparison group of 241 non-PBB Navy and Air Force facilities before and after program implementation.

RESULTS

The Army's PBB programs had a positive impact on quality performance. Relative to comparison facilities, facilities that participated in PBB programs increased performance for over half of the indicators under investigation. Furthermore, performance was either sustained or continued to improve over 5 years for five of the six performance indicators examined long term.

CONCLUSION

Study findings indicate that PBB may be an effective policy mechanism for improving facility-level performance on quality indicators.

PRACTICE IMPLICATIONS

This study adds to the extant literature on pay for performance by examining the specific case of PBB. It demonstrates that quality performance can be influenced internally through centralized budgeting processes. Though specific to military hospitals, the findings might have applicability to other public and private sector hospitals who wish to incentivize performance internally in their organizational subunits through centralized budgeting processes.

Is the Centralization of Complex Surgical Procedures an Unintended Spillover Effect of Global Capitation? - Insights from the Maryland Global Budget Revenue Program

OBJECTIVE

To determine if global budget revenue (GBR) models incent the centralization of complex surgical care.

SUMMARY BACKGROUND

In 2014, Maryland initiated a statewide GBR model. While prior research has shown improvements in cost and outcomes for surgical care post-GBR implementation, the mechanism remains unclear.

METHODS

Utilizing state inpatient databases, we compared the proportion of adults undergoing elective complex surgeries (gastrectomy, pneumonectomy/lobectomy, proctectomies, and hip/knee revision) at high-concentration hospitals (HCHs) in Maryland and control states. Annual concentration, per procedure, was defined as hospital volume divided by state volume. HCHs were defined as hospitals with a concentration at least at the 75 th percentile in 2010. We estimated the difference-in-differences (DiD) of the probability of patients undergoing surgery at HCHs before and after GBR implementation.

FINDINGS

Our sample included 122,882 surgeries. Following GBR implementation, all procedures were increasingly performed at HCHs in Maryland. States satisfied the parallel trends assumption for the centralization of gastrectomy and pneumonectomy/lobectomy. Post-GBR, patients were more likely to undergo gastrectomy (DiD: 5.5 p.p., 95% CI [2.2, 8.8]) and pneumonectomy/lobectomy (DiD: 12.4 p.p., 95% CI [10.0, 14.8]) at an HCH in Maryland compared with control states. For our hip/knee revision analyses, we assumed persistent counterfactuals and noted a positive DiD post-GBR implementation (DiD: 4.8 p.p., 95% CI [1.3, 8.2]). No conclusion could be drawn for proctectomy due to different pre-GBR trends.

CONCLUSIONS

GBR implementation is associated with increased centralization for certain complex surgeries. Future research is needed to explore the impact of centralization on patient experience and access.

Association between better night-shift nurse staffing and surgical outcomes: A retrospective cohort study using a nationwide inpatient database in Japan

INTRODUCTION

Although many studies have investigated the relationship between patient outcomes and the level of nurse staffing, little is known about the association between increased night-shift nurse staffing and patient outcomes. In the Japanese universal health insurance system, a new scheme of additional financial incentives for acute care hospitals was launched in 2012 to increase the number of nurses during the night shift in general wards. The objective of this study was to investigate whether an additional financial incentive to increase night-shift nurse staffing in general wards was associated with better patient outcomes.

DESIGN

Adoption of the above-mentioned scheme of additional financial incentives was used as a natural experiment, and the difference-in-differences method was conducted to evaluate the effect of the scheme. The study was performed using a nationwide inpatient database and hospital information in Japan.

METHODS

To conduct a difference-in-differences analysis, first, hospitals with and without increased night-shift nurse staffing were matched using propensity score matching. A patient-level difference-in-differences analysis was then conducted. The intervention group comprised the hospitals that adopted the new scheme of additional financial incentives. The outcome measures were in-hospital mortality, failure to rescue, and length of hospital stay.

RESULTS

Subjects were 403,971 adult patients who underwent planned major surgeries in Japanese acute care hospitals from April 2012 to March 2018. The adjusted difference-in-differences estimates were not significant for in-hospital mortality (odds ratio: 0.83; 95% confidence interval: 0.68 to 1.01; p = 0.07) or failure to rescue (odds ratio: 0.92; 95% confidence interval: 0.73 to 1.14; p = 0.44). The adjusted difference-in-differences estimate for length of hospital stay was significant (percent change: -3.2%; 95% confidence interval: -6.1 to -0.3%; p = 0.029), indicating that the adoption of the scheme was associated with a decreased length of hospital stay.

CONCLUSIONS

Increased night-shift nurse staffing was not associated with a decrease in in-hospital mortality or failure to rescue, but it was associated with a reduction in the length of hospital stay. It may be necessary to consider changes in policy content to make the policy more effective. The findings of this study are potentially useful for medical policymakers considering nurse staffing to decrease the length of stay, which may decrease costs.

CLINICAL RELEVANCE

This study showed that increased night-shift nurse staffing was not associated with a decrease in in-hospital mortality or failure to rescue, but it was associated with a reduction in the length of hospital stay. The examination of the effectiveness of increasing nurse staffing during a specific shift in acute care hospitals is potentially useful for health policymakers worldwide in their considerations of future nurse staffing policies.

Mitigating hospital-onset Clostridioides difficile: The impact of an optimized environmental hygiene program in eight hospitals

OBJECTIVE

To evaluate the impact of a standardized, process-validated intervention utilizing daily hospital-wide patient-zone sporicidal disinfectant cleaning on incidence density of healthcare-onset Clostridioides difficile infection (HO-CDI) standardized infection ratios (SIRs).

DESIGN

Multi-site, quasi-experimental study, with control hospitals and a nonequivalent dependent variable.

SETTING

The study was conducted across 8 acute-care hospitals in 6 states with stable endemic HO-CDI SIRs.

METHODS

Following an 18-month preintervention control period, each site implemented a program of daily hospital-wide sporicidal disinfectant patient zone cleaning. After a wash-in period, thoroughness of disinfection cleaning (TDC) was monitored prospectively and optimized with performance feedback utilizing a previously validated process improvement program. Mean HO-CDI SIRs were calculated by quarter for the pre- and postintervention periods for both the intervention and control hospitals. We used a difference-in-differences analysis to estimate the change in the average HO-CDI SIR and HO-CAUTI SIR for the pre- and postintervention periods.

RESULTS

Following the wash-in period, the TDC improved steadily for all sites and by 18 months was 93.6% for the group. The mean HO-CDI SIRs decreased from 1.03 to 0.6 (95% CI, 0.13-0.75; P = .009). In the adjusted difference-in-differences analysis in comparison to controls, there was a 0.55 reduction (95% CI, -0.77 to -0.32) in HO-CDI (P < .001) or a 50% relative decrease from baseline.

CONCLUSIONS

This study represents the first multiple-site, quasi-experimental study with control hospitals and a nonequivalent dependent variable to evaluate a 4-component intervention on HO-CDI. Following ongoing improvement in cleaning thoroughness, there was a sustained 50% decrease in HO-CDI SIRs compared to controls.

A natural experiment: Assessment of Danish high-school students' alcohol drinking patterns from 2014 and 2019 after the introduction of a common alcohol policy

AIM

In this study, using national survey data from 2014 and 2019, we tested the hypothesis that students at schools who introduced a common alcohol policy in 2017 drank less than students at schools without a common policy.

METHODS

We used survey data from 41 high schools that participated in the Danish National Youth Study in 2014 and 2019. We perceived the introduction of a common high-school alcohol policy in 2017 among local groups of high schools as a natural experiment and assessed it using difference-in-difference analyses. We assessed drinking patterns from 2014 and 2019 among students at schools with and without a common alcohol policy combined and stratified by sex in negative binominal regression and logistic regression models. Drinking patterns were measured as average weekly alcohol use, average alcohol intake at last school party, proportion of non-drinkers and frequent binge drinkers.

RESULTS

Drinking patterns were similar among students in schools with and without a common alcohol policy. For example, among students at schools with a common alcohol policy, the average alcohol intake at the last school party among drinkers was 8.7 units in 2014 and 8.5 units in 2019, whereas average alcohol intake among students at schools without a common alcohol policy was 8.8 units in 2014 and 8.9 units in 2019 (p=0.413).

CONCLUSIONS

No statistically significant effects were observed following the introduction of a common alcohol policy on students' drinking patterns, and alcohol consumption among high-school students was stable and remained high in 2014 and in 2019.

How Do Primary Care Providers Respond to Reimbursement Cuts? Evidence From the Termination of the Primary Care Incentive Program

The Primary Care Incentive Payment Program (PCIP) provided a 10% bonus payment for Evaluation and Management (E&M) visits for eligible primary care providers (PCPs) from 2011 to 2015. Using a 2012 to 2017 sample of continuously eligible PCPs (the treatment group) and ineligible specialists with historically similar provision of billed services (the control group), this study is the first to examine how PCPs responded to the program’s termination. Using inverse probability of treatment weighted difference-in-differences models that control for inter-temporal changes in provider-specific beneficiary characteristics, individual provider fixed effects, and zip code by year fixed effects, it finds that providers responded to the removal of the 10% bonus payments by increasing their billing of bonus payment eligible E&M relative value units (RVUs) by 3.7%. This response is consistent with supplier-induced demand and suggests a 46% offsetting response consistent with actuarial assumptions by the Centers for Medicare & Medicaid Services when assessing reimbursement reductions.

Resilience in maternal and child nutrition outcomes in a refugee-hosting community in Cameroon: A quasi-experimental study

Refugees may be perceived as a burden to their host communities, and nutrition insecurity is a critical area of contention. We explored the relationship between refugee presence and a host community's resilience in nutrition outcomes in Cameroon. We also tested an analytical framework for evaluating community resilience during shocks. We used data from repeated cross-sectional Demographic and Health Surveys in Cameroon (2004 and 2011), data on refugee movement, and data on extreme climatic events, epidemics, and conflicts from multiple sources. Outcome variables were maternal underweight, maternal anaemia, and child underweight, anaemia, stunting and wasting. The exposure variable was residence within an area in which refugees settled. We used a genetic matching algorithm to select controls from the rest of the country after excluding areas experiencing concurrent shocks. We used a difference-in-differences analysis to compare outcomes between the exposed and control areas. The 2004 survey comprised 10,656 women and 8,125 children, while the 2011 survey comprised 15,426 women and 11,732 children. Apart from anaemia which showed a decreasing trend in both the refugee-hosting community and the rest of the country, all other indicators (wasting, underweight and stunting) showed increasing trends in the refugee-hosting community but decreasing trends in the rest of the country. The matched control group showed a similar trend of decreasing trend for all the indicators. Controlled comparisons showed no evidence of an association between changes in nutrition outcomes and the presence of refugees. These findings contest a common perception that refugees negatively impact hosting communities. The difference-in-differences analysis and an improved matching technique offer a method for exploring the resilience of communities to shocks.

Does the abolition of copayment increase ambulatory care utilization?: a quasi-experimental study in Germany

Due to a problematic situation with public finances, Germany introduced a copayment scheme for ambulatory care visits in 2004. In 2012, Germany achieved a balanced budget, and copayment was abolished on the 1st of January 2013. This policy change offers a rare opportunity to explore the impact of the abolition of copayment, compared to the much more frequently studied introduction of copayment. We therefore investigate the development of ambulatory care and inpatient care utilization following this policy change among people over 50 in Germany, as well as the heterogeneous impacts among vulnerable people, such as the low-income population, the chronically ill and the elderly over the age of 65. We use data from the Survey of Health, Ageing and Retirement in Europe and adopt a difference-in-differences approach with matching. We found that the abolition of copayment only caused an increase in ambulatory care use in the shorter term, while leading to a significant reduction in the longer term. In addition, we find a negative effect on inpatient care use, i.e., the hospitalization offset effect. Finally, we demonstrate that vulnerable people were more sensitive to the abolition of copayment.

Assessing real-world effectiveness of therapies: what is the impact of incretin-based treatments on hospital use for patients with type 2 diabetes?

BACKGROUND

Managing type 2 diabetes represents a major public health concern due to its important and increasing prevalence. Our study investigates the impact of taking incretin-based medication on the risk of being hospitalized and the length of hospital stay for individuals with type 2 diabetes.

METHOD

We use claim panel data from 2011 to 2015 and provide difference-in-differences (DID) estimations combined with matching techniques to better ensure the treatment and control groups' comparability. Our propensity score selects individuals according to their probability of taking an incretin-based treatment in 2013 (N = 2,116). The treatment group includes individuals benefiting from incretin-based treatments from 2013 to 2015 and is compared to individuals not benefiting from such a treatment but having a similar probability of taking it.

RESULTS

After controlling for health-related and socio-economic variables, we show that benefiting from an incretin-based treatment does not significantly impact the probability of being hospitalized but does significantly decrease the annual number of days spent in the hospital by a factor rate of 0.621 compared with the length of hospital stays for patients not benefiting from such a treatment.

CONCLUSION

These findings highlight the potential implications for our health care system in case of widespread use of these drugs among patients with severe diabetes.

The consequence of financial incentives for not prescribing antibiotics: a Japan's nationwide quasi-experiment

BACKGROUND

For addressing antibiotic overuse, Japan designed a health care policy in which eligible medical facilities could claim a financial reward when antibiotics were not prescribed for early-stage respiratory and gastrointestinal infections. The policy was introduced in a pilot manner in paediatric clinics in April 2018.

METHODS

We conducted a quasi-experimental, propensity score-matched, difference-in-differences (DID) design to determine whether the nationwide financial incentives for appropriate non-prescribing of antibiotics as antimicrobial stewardship [800 JPY (≈7.3 US D) per case] were associated with changes in prescription patterns, including antibiotics, and health care use in routine paediatric health care settings at a national level. Data consisted of 9 253 261 cases of infectious diseases in 553 138 patients treated at 10 180 eligible or ineligible facilities.

RESULTS

A total of 2959 eligible facilities claimed 316 770 cases for financial incentives and earned 253 million JPY (≈2.29 million USD). Compared with ineligible facilities, the introduction of financial incentives in the eligible facilities was associated with an excess reduction in antibiotic prescriptions [DID estimate, -228.6 days of therapy (DOTs) per 1000 cases (95% CI, -272.4 to -184.9), which corresponded to a relative reduction of 17.8% (95% CI, 14.8 to 20.7)]. The introduction was also associated with excess reductions in drugs for respiratory symptoms [DID estimates, -256.9 DOTs per 1000 cases (95% CI, -379.3 to -134.5)] and antihistamines [DID estimate, -198.5 DOTs per 1000 cases (95% CI, -282.1 to -114.9)]. There was no excess in out-of-hour visits [DID estimate, -4.43 events per 1000 cases (95% CI, -12.8 to 3.97)] or hospitalizations [DID estimate, -0.08 events per 1000 cases (95% CI, -0.48 to 0.31)].

CONCLUSIONS

Our findings suggest that financial incentives to medical facilities for not prescribing antibiotics were associated with reductions in prescriptions for antibiotics without adverse health care consequences. Japan's new health policy provided us with policy options for immediately reducing inappropriate antibiotic prescriptions by relatively small financial incentives.

Impact of the Affordable Care Act's Physician Payments Sunshine Act on branded statin prescribing

OBJECTIVE

To evaluate the impact of the Affordable Care Act's Physician Payments Sunshine Act (PPSA), which mandates disclosure of industry payments to physicians, on physician prescribing of branded statins.

DATA SOURCES

Administrative claims data from 2011 to 2015 from three large national commercial insurers were provided by the Health Care Cost Institute.

STUDY DESIGN

We adopted a difference-in-differences and event study design, leveraging the control group of physicians in two states, MA and VT, which implemented state laws on disclosure of industry payments prior to the national PPSA. To further address potential confounding caused by differences in prescribing patterns across states, our analytical sample includes physicians practicing in border counties between the treatment (NH, NY, and RI) and control (MA and VT) states.

DATA COLLECTION

We restricted our sample to physicians who had at least 50 new-fill prescription claims for statins during the five-year study period, with at least one new-fill prescription claim each year.

PRINCIPAL FINDINGS

We found that the PPSA led to a 7% (p < 0.001) reduction in monthly new prescriptions of brand-name statin over the study period, with little change in generic prescribing. The reduction in branded prescriptions was concentrated among physicians with the highest tercile of drug spending pre-PPSA, with a decrease of 15% (p < 0.001) in new branded statin prescriptions. The decline was most prominent after mandated reporting of industry payments began before the payment data was published.

CONCLUSIONS

The PPSA may have achieved its intended effect of reducing branded prescriptions at least in the short run, particularly among physicians most likely to have engaged in excessive or low-value prescribing of branded drugs.

A Statewide Approach to Reducing Re-excision Rates for Women With Breast-conserving Surgery

OBJECTIVE

Test the effectiveness of benchmarked performance reports based on existing discharge data paired with a statewide intervention to implement evidence-based strategies on breast re-excision rates.

BACKGROUND

Breast-conserving surgery (BCS) is a common breast cancer surgery performed in a range of hospital settings. Studies have demonstrated variations in post-BCS re-excision rates, identifying it as a high-value improvement target.

METHODS

Wisconsin Hospital Association discharge data (2017-2019) were used to compare 60-day re-excision rates following BCS for breast cancer. The analysis estimated the difference in the average change preintervention to postintervention between Surgical Collaborative of Wisconsin (SCW) and nonparticipating hospitals using a logistic mixed-effects model with repeated measures, adjusting for age, payer, and hospital volume, including hospitals as random effects. The intervention included 5 collaborative meetings in 2018 to 2019 where surgeon champions shared guideline updates, best practices/challenges, and facilitated action planning. Confidential benchmarked performance reports were provided.

RESULTS

In 2017, there were 3692 breast procedures in SCW and 1279 in nonparticipating hospitals; hospital-level re-excision rates ranged from 5% to >50%. There was no statistically significant baseline difference in re-excision rates between SCW and nonparticipating hospitals (16.1% vs. 17.1%, P =0.47). Re-excision significantly decreased for SCW but not for nonparticipating hospitals (odds ratio=0.69, 95% confidence interval=0.52-0.91).

CONCLUSIONS

Benchmarked performance reports and collaborative quality improvement can decrease post-BCS re-excisions, increase quality, and decrease costs. Our study demonstrates the effective use of administrative data as a platform for statewide quality collaboratives. Using existing data requires fewer resources and offers a new paradigm that promotes participation across practice settings.

Study protocol: The Maternal Health Multilevel Intervention for Racial Equity (Maternal Health MIRACLE) Project

PURPOSE

To test the effectiveness and cost-effectiveness of a multilevel intervention for population-level African American (AA) severe maternal morbidity and mortality.

BACKGROUND

Severe maternal morbidity and mortality in the U.S. disproportionately affect AA women. Inequities occur at many levels, including community, provider, and health system levels.

DESIGN

Intervention. Throughout the two intervention counties, we will expand access to enhanced prenatal care services using telehealth and flexible scheduling (community level), provide actionable maternal health-focused anti-racism training (provider level), and implement equity-focused community care maternal safety bundles (health system level). Partnership. Interventions were developed/co-developed by intervention county partners, including AA women, enhanced prenatal care staff, and health providers. For equity, 46% of project direct cost dollars go to our partners. Most study investigators are female (75%) and/or AA (38%). Partners are overwhelmingly AA women. Sample, measures, analyses. We use a quasi-experimental difference-in-differences with propensity scores approach to compare pre (2016-2019) to post (2022-2025) changes in outcomes for Medicaid-insured women in intervention counties to similar women in the other Michigan, USA, counties. The sample includes all Medicaid-insured deliveries in Michigan during these years (n ~ 540,000), with women observed during pregnancy, at birth, and up to 1 year postpartum. Measures are taken from a linked dataset that includes Medicaid claims and vital records.

CONCLUSION

This study is among the first to examine effects of any multilevel intervention on AA severe maternal morbidity and mortality. It features a rigorous quasi-experimental design, multilevel multi-partner county-wide interventions developed by community partners, and assessment of intervention effects using population-level data.

Ambulance diversion and ED destination by race/ethnicity: evaluation of Massachusetts' ambulance diversion ban

Background

The impact of ambulance diversion on potentially diverted patients, particularly racial/ethnic minority patients, is largely unknown. Treating Massachusetts’ 2009 ambulance diversion ban as a natural experiment, we examined if the ban was associated with increased concordance in Emergency Medical Services (EMS) patients of different race/ethnicity being transported to the same emergency department (ED).

Methods

We obtained Medicare Fee for Service claims records (2007–2012) for enrollees aged 66 and older. We stratified the country into patient zip codes and identified zip codes with sizable (non-Hispanic) White, (non-Hispanic) Black and Hispanic enrollees. For a stratified random sample of enrollees from all diverse zip codes in Massachusetts and 18 selected comparison states, we identified EMS transports to an ED. In each zip code, we identified the most frequent ED destination of White EMS-transported patients (“reference ED”). Our main outcome was a dichotomous indicator of patient EMS transport to the reference ED, and secondary outcome was transport to an ED serving lower-income patients (“safety-net ED”). Using a difference-in-differences regression specification, we contrasted the pre- to post-ban changes in each outcome in Massachusetts with the corresponding change in the comparison states.

Results

Our study cohort of 744,791 enrollees from 3331 zip codes experienced 361,006 EMS transports. At baseline, the proportion transported to the reference ED was higher among White patients in Massachusetts and comparison states (67.2 and 60.9%) than among Black (43.6 and 46.2%) and Hispanic (62.5 and 52.7%) patients. Massachusetts ambulance diversion ban was associated with a decreased proportion transported to the reference ED among White (− 2.7 percentage point; 95% CI, − 4.5 to − 1.0) and Black (− 4.1 percentage point; 95% CI, − 6.2 to − 1.9) patients and no change among Hispanic patients. The ban was associated with an increase in likelihood of transport to a safety-net ED among Hispanic patients (3.0 percentage points, 95% CI, 0.3 to 5.7) and a decreased likelihood among White patients (1.2 percentage points, 95% CI, − 2.3 to − 0.2).

Conclusion

Massachusetts ambulance diversion ban was associated with a reduction in the proportion of White and Black EMS patients being transported to the most frequent ED destination for White patients, highlighting the role of non-proximity factors in EMS transport destination.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-022-08358-8.

Impact of a Provider Tele-mentoring Learning Model on the Care of Medicaid-enrolled Patients With Diabetes

BACKGROUND

Project ECHO (Extension for Community Healthcare Outcomes), a tele-mentoring program for health care providers, has been shown to improve provider-reported outcomes, but there is insufficient research on patient-level outcomes.

OBJECTIVES

To evaluate the impact of primary care provider (PCP) participation in Project ECHO on the care of Medicaid enrollees with diabetes.

RESEARCH DESIGN

New Jersey Medicaid claims and encounter data and difference-in-differences models were used to compare utilization and spending between Medicaid patients seen by PCPs participating in a Project ECHO program to those of matched nonparticipating PCPs.

SUBJECTS

A total of 1776 adult Medicaid beneficiaries (318 with diabetes), attributed to 25 participating PCPs; and 9126 total (1454 diabetic) beneficiaries attributed to 119 nonparticipating PCPs.

MEASURES

Utilization and spending for total inpatient, diabetes-related inpatient, emergency department, primary care, and endocrinologist services; utilization of hemoglobin A1c tests, eye exams, and diabetes prescription medications among diabetics, and total Medicaid spending.

RESULTS

Participation in Project ECHO was associated with decreases of 44.3% in inpatient admissions (P=0.001) and 61.9% in inpatient spending (P=0.021) among treatment relative to comparison patients. Signs of most other outcome estimates were consistent with hypothesized program effects but without statistical significance. Sensitivity analyses largely confirmed these findings.

CONCLUSIONS

We find evidence that Project ECHO participation was associated with large and statistically significant reductions of inpatient hospitalization and spending. The study was observational and limited by a small sample of participating PCPs. This study demonstrates the feasibility and potential value of quasi-experimental evaluation of Project ECHO patient outcomes using claims data.

Medicaid Expansion and Not-For-Profit Hospitals' Financial Status: National and State-Level Estimates Using IRS and CMS Data, 2011-2016

Several studies have shown that Medicaid expansion has improved hospital financial performance. All of these studies have either used data from the Internal Revenue Service (IRS) or the Centers for Medicare and Medicaid Services (CMS), and none of them has examined the state-level impact of expansion on hospital finances. Using data for not-for-profit hospitals from both IRS and CMS for 2011-2016, we described the difference in costs related to uncompensated care and Medicaid shortfalls. We then estimated the impact of Medicaid expansion on hospitals' financial status nationally and by state. Nationally, the estimated net effect of expansion reduced not-for-profit hospital costs by 2 percentage points based on IRS data and 0.83 percentage points based on CMS data. Across expansion states, the estimated net effects varied widely with approximately a 10-fold difference for hospitals based on IRS data and a 2-fold difference based on CMS data. Future studies should further explore the differences across IRS and CMS data.

Impact of United States 2017 Immigration Policy changes on missed appointments at two Massachusetts Safety-Net Hospitals

INTRODUCTION

Studies have shown mixed findings regarding the impact of immigration policy changes on immigrants' utilization of primary care.

METHODS

We used a difference-in-differences analysis to compare changes in missed primary care appointments over time across two groups: patients who received care in Spanish, Portuguese, or Haitian Creole, and non-Hispanic, white patients who received care in English.

RESULTS

After adjustment for age, sex, race, insurance, hospital system, and presence of chronic conditions, immigration policy changes were associated with an absolute increase in the missed appointment prevalence of 0.74 percentage points (95% confidence interval: 0.34, 1.15) among Spanish, Portuguese and Haitian-Creole speakers. We estimated that missed appointments due to immigration policy changes resulted in lost revenue of over $185,000.

CONCLUSIONS

We conclude that immigration policy changes were associated with a significant increase in missed appointments among patients who receive medical care in languages other than English.

Association of the Delaware Contraceptive Access Now Initiative with Postpartum LARC Use

Objectives

Although multi-component policy interventions can be important tools to increase access to contraception, we know little about how they may change contraceptive use among postpartum women. We estimate the association of the Delaware Contraceptive Access Now (DelCAN) initiative with use of postpartum Long-Acting Reversible Contraception (LARC). DelCAN included Medicaid payment reform for immediate postpartum LARC use, provider training and technical assistance in LARC provision, and a public awareness campaign.

Methods

We used a difference-in-differences design and data from the 2012 to 2017 pregnancy risk assessment monitoring system to compare changes in postpartum LARC use in Delaware versus 15 comparison states, and differences in such changes by women’s Medicaid enrollment.

Results

Relative to the comparison states, postpartum LARC use in Delaware increased by 5.26 percentage points (95% CI 2.90–7.61, P < 0.001) during the 2015–2017 DelCAN implementation period. This increase was the largest among Medicaid-covered women, and grew over the first three implementation years. By the third year of the DelCAN initiative (2017), the relative increase in postpartum LARC use for Medicaid women exceeded that for non-Medicaid women by 7.24 percentage points (95% CI 0.12–14.37, P = 0.046).

Conclusions for Practice

The DelCAN initiative was associated with increased LARC use among postpartum women in Delaware. During the first 3 years of the initiative, LARC use increased progressively and to a greater extent among Medicaid-enrolled women. Comprehensive initiatives that combine Medicaid payment reforms, provider training, free contraceptive services, and public awareness efforts may reduce unmet demand for highly effective contraceptives in the postpartum months.

Supplementary Information

The online version contains supplementary material available at 10.1007/s10995-022-03433-2.

The Effect of the Medicare Advantage Value-Based Insurance Design Model Test on Utilization in 2017

In 2015, the Centers for Medicare and Medicaid Services announced the Medicare Advantage (MA) Value-Based Insurance Design (VBID) model test, which allows MA insurers to use flexible benefit design strategies, such as reduced cost-sharing, to encourage beneficiaries with chronic disease to use high-value care. During the first year of implementation (2017), nine MA insurers offered VBID in 45 health plans to a total of 96 053 eligible beneficiaries. We used MA encounter data to estimate the impact of VBID on health services utilization in 2017 using a difference-in-differences research design. We found that VBID increased use of 10 out of 18 targeted services, and led to general increases in primary care visits, specialty care visits, and drug fills across eligible beneficiaries. The model was also associated with increases in ambulatory care sensitive inpatient and emergency department visits, an unanticipated effect that may be temporary. Overall, our findings suggest that VBID successfully increased the use of high-value services among eligible MA beneficiaries, an important first step along the pathway to better chronic disease management, lower spending, and improved beneficiary health.

An online advertising intervention to increase adherence to stay-at-home-orders during the COVID-19 pandemic: An efficacy trial monitoring individual-level mobility data

The COVID-19 pandemic has led public health departments to issue several orders and recommendations to reduce COVID-19-related morbidity and mortality. However, for various reasons, including lack of ability to sufficiently monitor and influence behavior change, adherence to these health orders and recommendations has been suboptimal. Starting April 29, 2020, during the initial stay-at-home orders issued by various state governors, we conducted an intervention that sent online website and mobile application advertisements to people's mobile phones to encourage them to adhere to stay-at-home orders. Adherence to stay-at-home orders was monitored using individual-level cell phone mobility data, from April 29, 2020 through May 10, 2020. Mobile devices across 5 regions in the United States were randomly-assigned to either receive advertisements from our research team advising them to stay at home to stay safe (intervention group) or standard advertisements from other advertisers (control group). Compared to control group devices that received only standard corporate advertisements (i.e., did not receive public health advertisements to stay at home), the (intervention group) devices that received public health advertisements to stay at home demonstrated objectively-measured increased adherence to stay at home (i.e., smaller radius of gyration, average travel distance, and larger stay-at-home ratios). Results suggest that 1) it is feasible to use mobility data to assess efficacy of an online advertising intervention, and 2) online advertisements are a potentially effective method for increasing adherence to government/public health stay-at-home orders.

Readmissions performance and penalty experience of safety-net hospitals under Medicare's Hospital Readmissions Reduction Program

BACKGROUND

The Hospital Readmissions Reduction Program (HRRP), established by the Centers for Medicare and Medicaid Services (CMS) in March 2010, introduced payment-reduction penalties on acute care hospitals with higher-than-expected readmission rates for acute myocardial infarction (AMI), heart failure, and pneumonia. There is concern that hospitals serving large numbers of low-income and uninsured patients (safety-net hospitals) are at greater risk of higher readmissions and penalties, often due to factors that are likely outside the hospital's control. Using publicly reported data, we compared the readmissions performance and penalty experience among safety-net and non-safety-net hospitals.

METHODS

We used nationwide hospital level data for 2009-2016 from the Centers for Medicare and Medicaid Services (CMS) Hospital Compare program, CMS Final Impact Rule, and the American Hospital Association Annual Survey. We identified as safety-net hospitals the top quartile of hospitals in terms of the proportion of patients receiving income-based public benefits. Using a quasi-experimental difference-in-differences approach based on the comparison of pre- vs. post-HRRP changes in (risk-adjusted) 30-day readmission rate in safety-net and non-safety-net hospitals, we estimated the change in readmissions rate associated with HRRP. We also compared the penalty frequency among safety-net and non-safety-net hospitals.

RESULTS

Our study cohort included 1915 hospitals, of which 479 were safety-net hospitals. At baseline (2009), safety-net hospitals had a slightly higher readmission rate compared to non-safety net hospitals for all three conditions: AMI, 20.3% vs. 19.8% (p value< 0.001); heart failure, 25.2% vs. 24.2% (p-value< 0.001); pneumonia, 18.7% vs. 18.1% (p-value< 0.001). Beginning in 2012, readmission rates declined similarly in both hospital groups for all three cohorts. Based on difference-in-differences analysis, HRRP was associated with similar change in the readmissions rate in safety-net and non-safety-net hospitals for AMI and heart failure. For the pneumonia cohort, we found a larger reduction (0.23%; p < 0.001) in safety-net hospitals. The frequency of readmissions penalty was higher among safety-net hospitals. The proportion of hospitals penalized during all four post-HRRP years was 72% among safety-net and 59% among non-safety-net hospitals.

CONCLUSIONS

Our results lend support to the concerns of disproportionately higher risk of performance-based penalty on safety-net hospitals.

Association of Medicaid Expansion With Neuraxial Labor Analgesia Use in the United States: A Retrospective Cross-Sectional Analysis

BACKGROUND

The Affordable Care Act has been associated with increased Medicaid coverage for childbirth among low-income US women. We hypothesized that Medicaid expansion was associated with increased use of labor neuraxial analgesia.

METHODS

We performed a cross-sectional analysis of US women with singleton live births who underwent vaginal delivery or intrapartum cesarean delivery between 2009 and 2017. Data were sourced from births in 26 US states that used the 2003 Revised US Birth Certificate. Difference-in-difference linear probability models were used to compare changes in the prevalence of neuraxial labor analgesia in 15 expansion and 11 nonexpansion states before and after Medicaid expansion. Models were adjusted for potential maternal and obstetric confounders with standard errors clustered at the state level.

RESULTS

The study sample included 5,703,371 births from 15 expansion states and 5,582,689 births from 11 nonexpansion states. In the preexpansion period, the overall rate of neuraxial analgesia in expansion and nonexpansion states was 73.2% vs 76.3%. Compared with the preexpansion period, the rate of neuraxial analgesia increased in the postexpansion period by 1.7% in expansion states (95% CI, 1.6-1.8) and 0.9% (95% CI, 0.9-1.0) in nonexpansion states. The adjusted difference-in-difference estimate comparing expansion and nonexpansion states was 0.47% points (95% CI, -0.63 to 1.57; P = .39).

CONCLUSIONS

Medicaid expansion was not associated with an increase in the rate of neuraxial labor analgesia in expansion states compared to the change in nonexpansion states over the same time period. Increasing Medicaid eligibility alone may be insufficient to increase the rate of neuraxial labor analgesia.

Increased Intraoperative Faculty Entrustment and Resident Entrustability Does Not Compromise Patient Outcomes After General Surgery Procedures

BACKGROUND

Intraoperative resident autonomy has been compromised secondary to expectations for increased supervision without defined parameters for safe progressive independence, diffusion of training experience, and more to learn with less time. Surgical residents who are insufficiently entrusted during training attain less autonomy, confidence, and even clinical competency, potentially affecting future patient outcomes.

OBJECTIVE

To determine if OpTrust, an educational intervention for increasing intraoperative faculty entrustment and resident entrustability, negatively impacts patient outcomes after general surgery procedures.

METHODS

Surgical faculty and residents received OpTrust training and instruction to promote intraoperative faculty entrustment and resident entrustability. A post-intervention OpTrust cohort was compared to historical and pre-intervention OpTrust cohorts. Multivariable logistic and negative binomial regression was used to evaluate the impact of the OpTrust intervention and time on patient outcomes.

SETTING

Single tertiary academic center.

PARTICIPANTS

General surgery faculty and residents.

MAIN OUTCOMES AND MEASURES

Thirty-day postoperative outcomes, including mortality, any complication, reoperation, readmission, and length of stay.

RESULTS

A total of 8890 surgical procedures were included. After risk adjustment, overall patient outcomes were similar. Multivariable regression estimating the effect of the OpTrust intervention and time revealed similar patient outcomes with no increased risk (P > 0.05) of mortality {odds ratio (OR), 2.23 [95% confidence interval (CI), 0.87-5.6]}, any complication [OR, 0.98 (95% CI, 0.76-1.3)], reoperation [OR, 0.65 (95% CI, 0.42-1.0)], readmission [OR, 0.82 (95% CI, 0.57-1.2)], and length of stay [OR, 0.99 (95% CI, 0.86-1.1)] compared to the historic and pre-intervention OpTrust cohorts.

CONCLUSIONS

OpTrust, an educational intervention to increase faculty entrustment and resident entrustability, does not compromise postoperative patient outcomes. Integrating faculty and resident development to further enhance entrustment and entrustability through OpTrust may help facilitate increased resident autonomy within the safety net of surgical training without negatively impacting clinical outcomes.

The Impact of a Population-Based System of Care Intervention on Enhanced Prenatal Care and Service Utilization Among Medicaid-Insured Pregnant Women

INTRODUCTION

Enhanced prenatal/postnatal care home visiting programs for Medicaid-insured women have significant positive impacts on care and health outcomes. However, enhanced prenatal care participation rates are typically low, enrolling <30% of eligible women. This study investigates the impacts of a population-based systems approach on timely enhanced prenatal care participation and other healthcare utilization.

METHODS

This quasi-experimental, population-based, difference-in-differences study used linked birth certificates, Medicaid claims, and enhanced prenatal care data from complete statewide Medicaid birth cohorts (2009 to 2015), and was analyzed in 2019-2020. The population-based system intervention included cross-agency leadership and work groups, delivery system redesign with clinical-community linkages, increased enhanced prenatal care-Community Health Worker care, and patient empowerment. Outcomes included enhanced prenatal care participation and early participation, prenatal care adequacy, emergency department contact, and postpartum care.

RESULTS

Enhanced prenatal care (7.4 percentage points, 95% CI=6.3, 8.5) and first trimester enhanced prenatal care (12.4 percentage points, 95% CI=10.2, 14.5) increased among women served by practices with established clincial-community linkages, relative to that among the comparator group. First trimester enhanced prenatal care improved in the county (17.9, 95% CI=15.7, 20.0), emergency department contact decreased in the practices (-11.1, 95% CI= -12.3, -9.9), and postpartum care improved in the county (7.1, 95% CI=6.0, 8.2). Enhanced prenatal care participation for Black women served by the practices improved (4.4, 95% CI=2.2, 6.6) as well as early enhanced prenatal care (12.3, 95% CI=9.0, 15.6) and use of postpartum care (10.4, 95% CI=8.3, 12.4).

CONCLUSIONS

A population systems approach improved selected enhanced prenatal care participation and service utilization for Medicaid-insured women in a county population, those in practices with established clinical-community linkages, and Black women.